skills.rest
golemcloud
golemcloudOFFICIAL
Explore all skills by this author →
WebsiteGitHub

golem-skill-harness

Runs coding agents against YAML scenarios to test Golem skill activation and behavior.

1.5k|212|Updated Nov 24, 2023
Search Tags:#webassembly#Testing#ci#coding agents#golem#yaml scenarios#skill harness
One-click install
npx skills add https://github.com/golemcloud/golem --skill golem-skill-harness
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: golem-skill-harness
Source: https://github.com/golemcloud/golem/tree/main/.agents/skills/golem-skill-harness
Command: npx skills add https://github.com/golemcloud/golem --skill golem-skill-harness

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Validating that Golem skills actually activate and guide coding agents correctly is hard to do manually across multiple agents, languages, and scenarios. This Skill provides the operational knowledge to develop, run, and debug YAML-based test scenarios with the TypeScript skill harness in golem-skills/tests/harness/.

Core Features & Use Cases

  • Scenario Authoring: Write current-schema YAML scenarios with steps like prompt, create_project, shell, invoke, http, and mcp_call, plus assertions such as stdout_contains, body_json, and result_json.
  • Agent and Language Matrices: Run scenarios against Amp, Claude Code, OpenCode, Codex, or Gemini across TypeScript, Rust, Scala, and MoonBit, with per-language conditional values.
  • Skill Activation Verification: Track skill activations via native driver events, filesystem watchers, and atime snapshots, and pair expectedSkills with build, deploy, and invocation assertions.
  • Use Case: After editing a catalog skill in golem-skills/skills/, rebuild the CLI, regenerate docs, then dry-run and execute a scenario for one agent and language to confirm the skill activates and the agent produces a deployable Golem app.

Quick Start

Ask the agent to dry-run a specific harness scenario and then execute it for one agent and language, for example by running npx tsx src/run.ts with the scenario, agent, and language flags from the golem-skills/tests/harness directory.

Frequently Asked Questions about golem-skill-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run a Golem skill harness scenario?▼

Run scenarios from golem-skills/tests/harness with npx tsx src/run.ts, passing --agent, --language, and --scenario flags. Use --dry-run first to validate the YAML schema and regexes before executing against a real agent.

How do I write a skill harness scenario YAML file?▼

Define a name, optional settings and prerequisites, and a steps array where each step has exactly one action such as prompt, create_project, shell, invoke, http, or mcp_call. Add expect assertions like stdout_contains or body_json and optional verify blocks for build and deploy checks.

Which agents and languages does the Golem skill harness support?▼

The harness supports Amp, Claude Code, OpenCode, Codex, and Gemini as agents, and TypeScript, Rust, Scala, and MoonBit as languages. Each requires its own runtime credentials and language toolchain, such as wasm32-wasip2 for Rust or Java 17 and sbt for Scala.

Why does the skill harness fail to start the Golem server?▼

The harness always starts its own server on port 9881 and refuses to run if a healthy server already exists there. Ensure port 9881 is free and that a built golem binary exists at target/release/golem or target/debug/golem under GOLEM_PATH.

Why is skill activation not detected during a scenario run?▼

Activation detection relies on native driver events first, then filesystem watchers and atime snapshots. Check native tool events, the .agents/skills tree, atime behavior, and Linux inotifywait; note that Codex falls back to filesystem tracking and macOS starts no fswatch process.

Can the harness change the Golem server port?▼

No. Although the scenario schema exposes settings.golem_server.router_port, server startup is fixed on port 9881, so changing that field has no effect. The runner always starts on 9881 and requires it to be free.

Related Skills

Explore More Software Engineering Skills→
easyspace-aieasyspace-ai

find-skills

Search, verify, and install agent skills via the Skills CLI.

Community
Intermediate
Hack23Hack23
9

automated-content-generation

Generate template-based content with multi-language support and RSS feeds.

Official
Advanced
MicrockMicrock
322

testing

Run DBHub unit and integration tests across multiple databases with Testcontainers.

Community
Advanced
golemcloudgolemcloud
1.5k

sdk-development

Build and test Golem Rust and TypeScript SDKs against the main platform.

Official
Intermediate