skills.rest
PolyphonyRequiem
PolyphonyRequiemCOMMUNITY
Explore all skills by this author →
GitHub

polyphony-harness

Validate Conductor SDLC workflow paths by executing scripted harness scenarios.

Updated Apr 29, 2026
Search Tags:#conductor#workflow testing#scenario harness#deterministic routing#fake provider#script shim#ci pester
One-click install
npx skills add https://github.com/PolyphonyRequiem/polyphony --skill polyphony-harness
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: polyphony-harness
Source: https://github.com/PolyphonyRequiem/polyphony/tree/main/.github/skills/polyphony-harness
Command: npx skills add https://github.com/PolyphonyRequiem/polyphony --skill polyphony-harness

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It makes it practical to validate Conductor workflow path behavior end-to-end without relying on real LLM calls, real Azure DevOps state, or nondeterministic external systems—so regressions are caught early in CI.

Core Features & Use Cases

  • Workflow path coverage for scenarios: Runs scenarios defined under tests/harness/scenarios/<name>/scenario.yaml and asserts the workflow reaches the expected terminal node.
  • Deterministic LLM boundary via FakeProvider: Replays scripted agent outputs in the exact call order the workflow expects, including both structured JSON and regex-matched unstructured text.
  • Deterministic script interception via .NET shim: Matches script node calls (scripted commands like polyphony/twig/gh with argv prefix rules) and returns scenario-defined stdout/exit codes.
  • Gate-seam simplification for repeatability: Uses --skip-gates to auto-select the first human gate option, enabling stable scenario routing while gate API work is pending.

Quick Start

Ask the AI to “run and debug scenario close_out_happy_path in the polyphony workflow harness using the Python driver with verbose output so I can see why the workflow did not reach workflow_completed.”

Frequently Asked Questions about polyphony-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test Conductor workflow routing without using real LLMs?▼

Workflow path testing without real LLMs uses a FakeProvider to replay scripted agent outputs in exact call order and a .NET shim to intercept script node calls, ensuring deterministic end-to-end path coverage in CI.

How does deterministic LLM boundary testing work for scenario harnesses?▼

Deterministic LLM boundary testing works by configuring a FakeProvider that replays scripted agent outputs in the exact call order the workflow expects. It handles both structured JSON and regex-matched unstructured text to validate routing paths.

Do I need a scenario.yaml file to run workflow testing scenarios?▼

Yes, you need a scenario.yaml file to define the workflow path, FakeProvider agent_outputs playback order, and optional cli_scripts interception rules. It must be placed under tests/harness/scenarios/<name>/ to run correctly.

What is the best way to automate human gate selections during CI workflow testing?▼

The best way to automate human gate selections during CI workflow testing is to run the harness with the --skip-gates flag. This auto-selects the first human gate option, enabling stable scenario routing while the gate API work is pending.

Why does my workflow testing scenario not reach the expected terminal node?▼

A workflow testing scenario might not reach the expected terminal node if the FakeProvider agent_outputs playback order is incorrect or the cli_scripts argv prefix matching fails. Running the Python driver with verbose output helps debug the routing failure.

Related Skills

Explore More Software Engineering Skills→
MihaiOnSoftwareMihaiOnSoftware
1

load-rules

Load user rules from ~/.ai/rules/ in defined order.

Community
Basic
matteingmatteing
56

ci-preflight

Run the full CI suite locally to verify Elixir and TypeScript checks before pushing.

Community
Intermediate
penghang1223penghang1223
1

test-master

Design and execute unit, integration, and end-to-end tests with CI/CD automation.

Community
Advanced
PolyphonyRequiemPolyphonyRequiem

polyphony-sdlc

Orchestrate Azure DevOps work item SDLC routing without hardcoded type names.

Community
Advanced