skills.rest
OmarA1-Bakri
OmarA1-BakriCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

autoresearch

Automate code and prompt optimization experiments with Claude Code CLI.

Updated Mar 9, 2026
Search Tags:#prompt engineering#A/B testing#code optimization#experiment automation#autoresearch
One-click install
npx skills add https://github.com/OmarA1-Bakri/Claude_Code_Automations --skill autoresearch-omara1-bakri
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: autoresearch
Source: https://github.com/OmarA1-Bakri/Claude_Code_Automations/tree/main/agent_workflows/autoresearch
Command: npx skills add https://github.com/OmarA1-Bakri/Claude_Code_Automations --skill autoresearch-omara1-bakri

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires claude-code-cli, pytest, timeit, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill automates the process of optimizing code and prompts, allowing users to conduct experiments on a codebase and fine-tune prompts without manual intervention.

Core Features & Use Cases

  • Autonomous Experiment Loop: Automatically mutate files, run benchmark scripts, and check metrics, keeping or discarding changes based on the results.
  • Three Modes: Supports code optimization, prompt/template optimization, and outreach A/B testing.
  • Customizable Benchmarks: Run custom scripts to evaluate metrics based on specific criteria.
  • Integration with Claude Code CLI: Leverages Claude Code CLI for code mutation and prompt generation.
  • Use Case: Optimize code performance by running the autodev mode and watching the codebase evolve automatically. Similarly, fine-tune prompt templates by running the autoprompt mode.

Quick Start

Initialize the experiment by running: python autodev.py init --mode autodev --target train.py --metric "val_bpb" --direction lower. To run the experiment loop, execute: python autodev.py run --max-iterations 100.

Frequently Asked Questions about autoresearch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate code optimization experiments on my codebase?▼

To automate code optimization experiments, you can run the `autodev` mode to automatically mutate files, execute benchmark scripts, and keep or discard code changes based on the resulting metrics.

What is the best way to fine-tune prompt templates automatically?▼

Fine-tuning prompt templates automatically is handled by the `autoprompt` mode, which leverages Claude Code CLI to generate prompt variations and evaluates them against custom benchmark scripts.

How does automated A/B testing for software development work?▼

Automated A/B testing for software development works by running an autonomous experiment loop that mutates code or prompts, checks performance against custom metrics, and retains the winning variations.

Do I need Claude Code CLI to run code mutation and prompt generation experiments?▼

Yes, you need Claude Code CLI installed, as the experiment automation directly requires it to perform code mutation and generate prompts during the optimization loop.

How do I specify benchmark metrics when automating code performance tests?▼

You specify benchmark metrics when initializing the experiment using the `--metric` and `--direction` flags, such as `--metric "val_bpb" --direction lower`, to evaluate custom scripts.

Can I use custom benchmark scripts to evaluate code optimization results?▼

Yes, you can use custom benchmark scripts to evaluate code optimization results, allowing the automation loop to check specific metrics and automatically keep or discard file mutations.

Related Skills

Explore More Software Engineering Skills→
mi179mi179
1

benchmark-models

Run the same prompt across Claude, GPT, and Gemini to compare latency, tokens, cost, and quality.

Community
Advanced
sakamoto-family-smilesakamoto-family-smile

prompt-optimizer

Analyze draft prompts and generate ready-to-paste ECC-optimized prompts.

Community
Advanced
getspurgetspur
2

performance-investigation

Profile CLI applications and binary workloads to quantify on-CPU and off-CPU bottlenecks.

Official
Advanced
KsawyouxKsawyoux

prompt-engineer-toolkit

Evaluate, version, and A/B test AI prompts against defined test cases.

Community
Advanced