skill-autoresearch

Automate skill optimization by running the target skill against a test set, scoring outputs with binary evals, and retaining mutations that improve performance. It iteratively mutates and benchmarks prompt quality within an isolated workspace.

A binary eval framework defines repeatable, yes/no checks for skill quality to score outputs. It establishes an untouched baseline and logs each mutation, enabling deterministic experiments to trace performance improvements.

Set up an autoresearch mutation loop by providing a target skill path, a defined test suite, and an evaluation framework. The system runs the skill, applies targeted mutations, and generates results.json and results.tsv artifacts for traceability.

Yes, the automated evaluation loop works for any SKILL.md-based workflow that benefits from iterative improvement. It leverages baseline management and mutation logging to build robust autoresearch loops across deterministic experiments.

Running automated skill evaluation generates results.json, results.tsv, and a self-contained status dashboard page. These artifacts monitor progress and record each experiment mutation alongside the baseline for complete traceability.

An isolated workspace is required to manage results, keep an untouched baseline, and record each experiment safely. This ensures deterministic experiments by preventing untested mutations from corrupting the original target skill.