autoresearch

Runs a metric-driven evolve-score-verify loop that optimizes prompts, code, or creative artifacts against a numeric target.

4|1|Updated Jul 31, 2025
One-click install
npx skills add https://github.com/alvis/.agents --skill autoresearch-alvis
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: autoresearch
Source: https://github.com/alvis/.agents/tree/main/plugins/essential/skills/autoresearch
Command: npx skills add https://github.com/alvis/.agents --skill autoresearch-alvis

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Optimizing prompts, code, configs, or creative text by hand is slow and unmeasurable — you tweak, eyeball the result, and never know if you actually improved. This Skill turns a fuzzy optimization goal into an approved research brief with a concrete metric, baseline, and target, then runs an automated generate-score-verify-evolve loop until the target is met, the round budget is spent, or progress plateaus. ## Core Features & Use Cases - Metric-driven research loop: Define a metric, evaluator backend (programmatic command, LLM judge panel, or human scoring), baseline, and target, then evolve candidates across rounds with genetic mutation, recombination, and wildcard exploration. - Adversarial verification: Every round winner is attacked by a refuter checking constraint violations, metric gaming, harness bugs, and prompt injection before its score stands. - Auditable, resumable runs: Every round persists candidates, scores.yaml, verify.yaml, and round logs to disk, producing a final dossier, leaderboard, and best-artifact copy; runs resume via --resume. - Use Case: Optimize a prompt until accuracy reaches 0.90, get p95 latency under 200ms, or evolve taglines until a judge panel scores 8.5 — all with a full provenance trail. ## Quick Start Ask the agent to run autoresearch with the goal of optimizing your prompt until its accuracy on the eval set reaches 0.90.

Frequently Asked Questions about autoresearch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize a prompt against a numeric metric automatically?

Provide an optimization goal and an eval command that prints exactly one number on stdout. The skill interviews you to build a research brief with metric, baseline, target, and budget, then runs generate-score-verify-evolve rounds until the target, budget, or plateau stops the run.

What eval backends does the autoresearch loop support?

Three backends: programmatic (a command printing one parseable number), judges (a panel of at least 3 independent LLM judges using an anchored rubric with median consensus), and human (you score candidates each round through a structured input tool).

Can I resume an interrupted autoresearch run?

Yes. Pass --resume=<run-dir> and the skill reconstructs round, survivors, best candidate, fanout, and leaderboard from the persisted rounds/ directory. Every round writes candidates, scores.yaml, verify.yaml, and a round log, so any run is resumable from disk alone.

How does the loop prevent candidates from gaming the metric?

The eval command, script, rubric, and fixtures are auto-listed as immutable paths; any candidate touching them is disqualified. An adversarial refuter also attacks each round winner for hardcoded eval outputs, test-set overfitting, harness bugs, and prompt injection before its score stands.

When should I use deep-research instead of autoresearch?

Use deep-research for literature reviews, web research, and fact-finding reports. Autoresearch is only for goals with a nameable, computable metric where candidate solutions can be evolved and scored against a numeric target.

What happens when the optimization run stops?

The loop stops on target met, budget exhausted, or plateau, then asks you to accept, extend the budget, or pivot the brief. On acceptance it synthesizes a dossier, rewrites the leaderboard, copies the winning artifact to best/, and a reviewer verifies every score traces to disk.