autoresearch

Automate prompt engineering by running autonomous experimentation loops that execute a target skill, score outputs with binary evals, and mutate the prompt to discover better configurations. This process systematically improves skill performance through traceable, change-one-thing-at-a-time mutations.

Autonomous skill optimization via binary evals is a process where a skill is run repeatedly, its outputs are scored against binary pass/fail criteria, and the prompt is mutated based on those results to iteratively discover improved skill configurations. It ensures traceable improvements without overfitting.

Start an experimentation loop by pointing the autoresearch process at your target SKILL.md path. The system will run the skill multiple times with defined test prompts, apply binary evals, mutate the prompt, and generate artifacts like a changelog and a live monitoring dashboard.

Yes, autonomous experimentation works on any Claude Code skill by pointing the process at its SKILL.md path. The optimization loop applies generally by running executions with defined test prompts and scoring outputs with binary evals.

Automated skill optimization generates an improved SKILL.md file, a results log in JSON and TSV formats, a changelog of mutations, and a live dashboard for monitoring experiment progress. These artifacts track the iterative improvements and final output configurations.

The prompt mutation process changes one thing at a time to ensure traceable improvements and avoid overfitting to a single input. This approach isolates the effect of each modification, making it easier to identify which changes positively impact the binary eval scores.