Run repeated experiments with binary evals to optimize SKILL.md prompts.
npx skills add https://github.com/JBonfim/skill-developer --skill autoresearch-jbonfim
Please help me install this Agent Skill. Skill: autoresearch Source: https://github.com/JBonfim/skill-developer/tree/main/.agents/skills/autoresearch Command: npx skills add https://github.com/JBonfim/skill-developer --skill autoresearch-jbonfim
This Skill reduces the chance of “garbage” skill outputs by running an autonomous optimization loop that repeatedly tests, scores, and incrementally mutates a target SKILL.md until it improves measurable quality. It solves the problem of unreliable skill performance by converting subjective behavior into binary evaluations and using those signals to guide prompt changes.
Start autoresearch for the skill whose SKILL.md you want optimized, after confirming the target skill path, 3–5 test inputs, and 3–6 binary eval criteria with the user.
High-intent search queries and answers about installing and using this skill.
To improve Claude Code skill prompts automatically, you can use an autonomous experimentation loop that repeatedly tests fixed inputs, scores outputs with binary yes/no evals, and applies targeted mutations to the SKILL.md until quality measurably increases.
Binary scoring in prompt benchmarking is an evaluation method that defines 3–6 yes/no checks as pass/fail test questions for each output, converting subjective skill behavior into consistent signals to guide prompt mutations.
To set up autonomous experimentation for skill evaluation, you must confirm the target skill path, provide 3–5 fixed test inputs, and define 3–6 binary eval criteria to establish a baseline before the mutation loop begins.
Targeted prompt mutation prevents overfitting in skill optimization by applying one focused change at a time to the SKILL.md, isolating exactly which mutations improve binary eval scores before keeping or discarding the change.
During autonomous skill optimization, the process generates research logging artifacts including a baseline SKILL.md.baseline, results.tsv, results.json, changelog.md, and a self-contained auto-refreshing HTML dashboard visualizing score progression.
You should stop the autonomous prompt mutation loop when a predefined budget is reached or a convergence condition is met, ensuring the skill evaluation process halts once incremental binary score improvements diminish.
olelehmann1337Run autonomous experiments to iteratively optimize a SKILL.md against binary evaluation criteria.
compozyAutomate Claude Code skill optimization through autonomous experimentation loops with binary evals.
nikkogiblerRun autonomous experimentation loops to mutate prompts and improve Claude Code skills.
SimHackerCreate weighted rubrics and score outputs against defined criteria.
