autoresearch

Name: autoresearch
Availability: InStock
Author: Compozy

Automate Claude Code skill optimization through autonomous experimentation loops with binary evals.

2.5k|134|Updated Mar 28, 2026

Search Tags:#automation #experimentation #prompt-engineering #skill-improvement #skill-optimization #autoresearch #binary-evals

Ready to run

npx skills add https://github.com/compozy/compozy --skill autoresearch-compozy

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: autoresearch
Source: https://github.com/compozy/compozy/tree/main/.agents/skills/autoresearch
Command: npx skills add https://github.com/compozy/compozy --skill autoresearch-compozy

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Autonomously optimize a Claude Code skill by running autonomous experiments that score outputs against binary evals and mutate the prompt to discover better configurations.

Core Features & Use Cases

Autonomous experimentation loop: run the target skill multiple times, score outputs with binary evals, and mutate prompts to improve results.
Change-one-thing-at-a-time mutations: ensure traceable improvements and avoid overfitting to a single input.
Artifacts and visibility: generate an improved SKILL.md, a results log, a changelog, and a live monitoring dashboard.
General applicability: works on any skill by pointing at its SKILL.md path.

Quick Start

Start the autoresearch loop on a target SKILL.md to run experiments, score outputs with binary evals, and mutate the prompt.

Frequently Asked Questions about autoresearch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate prompt engineering to improve Claude Code skills?▼

Automate prompt engineering by running autonomous experimentation loops that execute a target skill, score outputs with binary evals, and mutate the prompt to discover better configurations. This process systematically improves skill performance through traceable, change-one-thing-at-a-time mutations.

What is autonomous skill optimization via binary evals?▼

Autonomous skill optimization via binary evals is a process where a skill is run repeatedly, its outputs are scored against binary pass/fail criteria, and the prompt is mutated based on those results to iteratively discover improved skill configurations. It ensures traceable improvements without overfitting.

How do I start an experimentation loop to optimize a SKILL.md file?▼

Start an experimentation loop by pointing the autoresearch process at your target SKILL.md path. The system will run the skill multiple times with defined test prompts, apply binary evals, mutate the prompt, and generate artifacts like a changelog and a live monitoring dashboard.

Can I use autonomous experimentation on any Claude Code skill?▼

Yes, autonomous experimentation works on any Claude Code skill by pointing the process at its SKILL.md path. The optimization loop applies generally by running executions with defined test prompts and scoring outputs with binary evals.

What artifacts are generated during automated skill optimization?▼

Automated skill optimization generates an improved SKILL.md file, a results log in JSON and TSV formats, a changelog of mutations, and a live dashboard for monitoring experiment progress. These artifacts track the iterative improvements and final output configurations.

Why does the prompt mutation process change one thing at a time?▼

The prompt mutation process changes one thing at a time to ensure traceable improvements and avoid overfitting to a single input. This approach isolates the effect of each modification, making it easier to identify which changes positively impact the binary eval scores.