skills.rest
engineai-nz
engineai-nzOFFICIAL
Explore all skills by this author →
WebsiteGitHub

autoresearch

Run autonomous mutation experiments to improve skill prompt pass rates.

Updated Mar 22, 2026
Search Tags:#quality assurance#benchmarking#skill optimization#checkpointing#prompt mutation#autonomous evaluation#binary evals
One-click install
npx skills add https://github.com/engineai-nz/engineai-skills --skill autoresearch-engineai-nz
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: autoresearch
Source: https://github.com/engineai-nz/engineai-skills/tree/main/meta/autoresearch
Command: npx skills add https://github.com/engineai-nz/engineai-skills --skill autoresearch-engineai-nz

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It reduces the chance that a Claude Code skill produces inconsistent or low-quality outputs by running autonomous optimization loops and selecting only prompt mutations that improve strict, evidence-based pass/fail evals.

Core Features & Use Cases

  • Binary eval-driven optimization: Turns user-defined yes/no criteria into an auditable scoring suite that requires quoted evidence for every decision.
  • Autonomous mutation loop: Repeatedly runs the target skill across multiple test inputs, mutates the prompt using a structured taxonomy, and keeps only improvements.
  • Resumable experimentation: Writes checkpoints and detailed logs (results, changelog, outputs, scores) so optimization can continue after interruptions.
  • Best-suited use cases: Optimize a specific skill’s prompt, benchmark changes across versions, fix recurring failure patterns, and improve reliability on representative scenarios.

Quick Start

Use the autoresearch skill to optimize a target skill by first providing the target SKILL.md path, 3–5 test inputs, 3–6 binary eval criteria, runs per experiment (default 5), and an optional budget cap (default 20) so it can begin with a baseline run and subsequent mutations.

Frequently Asked Questions about autoresearch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize a Claude skill prompt that only passes my tests 70% of the time?▼

Optimize inconsistent Claude skill prompts by running autonomous mutation loops against binary evaluation criteria. The system repeatedly tests your target skill, applies structured prompt mutations, and keeps only changes that improve evidence-quoted pass/fail rates.

How does binary evaluation work for autonomous prompt optimization?▼

Binary evaluation turns user-defined yes/no criteria into an auditable scoring suite. Every pass or fail decision requires quoted evidence from the output, ensuring strict, evidence-based scoring rather than subjective grading during the mutation loop.

What inputs do I need to provide for automated skill benchmarking?▼

Automated skill benchmarking requires a target SKILL.md path, 3 to 5 test inputs, 3 to 6 binary eval criteria, runs per experiment, and an optional budget cap. These parameters establish the baseline run and guide subsequent prompt mutations.

Can I resume a prompt mutation experiment after an interruption?▼

Resuming a prompt mutation experiment is fully supported through resumable checkpoints. The process writes detailed logs, results, and changelogs, allowing the autonomous optimization loop to continue exactly where it left off after any interruption.

What files are generated when autotuning a Claude Code skill?▼

Autotuning a Claude Code skill generates an improved SKILL.md file, a results.tsv containing experiment scores, and a changelog.md documenting per-mutation outcomes. These outputs provide a complete audit trail of the optimization process.

When should I use autonomous prompt mutation over manual prompt editing?▼

Autonomous prompt mutation is best suited for fixing recurring failure patterns, benchmarking changes across versions, and improving reliability on representative scenarios. It outperforms manual editing when a skill produces inconsistent outputs and needs structured, evidence-driven optimization.

Related Skills

Explore More Education & Research Skills→
pacphipacphi
6

mutation-testing

Mutate code and run tests to measure mutation score and kill rate.

Community
Advanced
cryborg1211cryborg1211

vc:autoresearch

Orchestrate iterative optimization loops over measurable software metrics with Git-backed rollback.

Community
Advanced
cognitedatacognitedata
5

skill-creator

Create, refine, and benchmark AI skills through iterative AI-guided workflows.

Official
Intermediate
mykcsmykcs
1

agents

Compare two skill outputs blindly and grade expectation evidence.

Community
Advanced