flow-fix-by-benchmarks

Skill failures are diagnosed by running benchmark scenarios against the target skill to collect pass/fail evidence. The benchmark-runner subagent executes tests from benchmarks/<skill>/scenarios/, reports results, and enforces a verification plan with re-runs to ensure stability before any fixes are applied.

Automating debugging for AssistFlow skills requires running benchmarks to isolate failures, drafting data-driven fixes with argumentation, and preparing a verification plan. This structured approach uses the benchmark-runner subagent to execute tests and enforces re-runs until stability is achieved before applying changes.

Reproducibility when fixing skill failures is ensured by enforcing a systematic critique, proposing a fix, and executing a verification plan. You must re-run the benchmarks using the benchmark-runner subagent until stability is confirmed before applying the actual changes to the target skill.

Yes, you can use benchmark scenarios to verify a proposed fix. After determining the root cause and drafting a fix, you prepare a verification plan to re-run the benchmarks. This ensures the target skill passes the scenarios and achieves stability before you apply the actual changes.

Benchmark scenarios for testing a target skill should be located in the benchmarks/<skill>/scenarios/ directory. The benchmark-runner subagent accesses these scenarios to execute tests, collect outputs, and report pass/fail results with concrete evidence for root-cause analysis.