autoresearch

Automate iterative OpenCode experiments by scaffolding, running benchmarks, and logging results.

6|1|Updated Nov 23, 2023

Search Tags:#automation #metrics #benchmark #opencode #experiment #autoresearch

Ready to run

npx skills add https://github.com/gmcabrita/dotfiles --skill autoresearch-gmcabrita

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: autoresearch
Source: https://github.com/gmcabrita/dotfiles/tree/main/.config/opencode/skills/autoresearch
Command: npx skills add https://github.com/gmcabrita/dotfiles --skill autoresearch-gmcabrita

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Autoresearch automates the setup, execution, and evaluation of iterative experiments to discover optimal configurations and performance characteristics in OpenCode.

Core Features & Use Cases

Autonomous benchmark loop: scaffold canonical files, initialize experiments, run benchmarks, parse metrics, and cache results.
Safe, resumable workflow: supports resuming from the latest state and reverting losers to keep a clean history.
Monitoring and logging: records results to autoresearch/*.jsonl and provides status summaries via autoresearch_status.

Quick Start

Scaffold, initialize, and start the automated benchmarking loop.

Frequently Asked Questions about autoresearch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate iterative benchmarking to find optimal configurations?▼

Automated iterative benchmarking scaffolds experiment files, runs trials, parses metrics, and caches results to discover optimal performance configurations. The workflow safely resumes from the latest state and reverts losers to maintain a clean history.

Can I resume an experiment if my automated optimization loop stops halfway?▼

Yes, automated optimization loops support safe resumption from the latest recorded state. The system maintains state in autoresearch.jsonl and related artifacts, allowing you to revert losing configurations and continue iterative benchmarking without losing progress.

What metrics can I track using automated iterative experiments?▼

Automated iterative experiments track metric-driven optimization targets such as performance, size, and quality improvements. The system parses and generates METRIC lines for each trial, logging the results to monitor progress and discover the best settings.

Does this approach require external dependencies to run metric-driven optimization?▼

No external dependencies are required to run metric-driven optimization tasks. The workflow autonomously scaffolds its required files, initializes experiments, and maintains all state and logs locally within the autoresearch directory structure.

How does the automated benchmark loop keep experiment history clean?▼

The automated benchmark loop keeps history clean by reverting losing configurations after each trial. It records all results and status summaries in autoresearch.jsonl, ensuring only optimal configurations persist while maintaining a deterministic workflow.

What is the best way to monitor status during an iterative benchmark run?▼

The best way to monitor iterative benchmark status is by checking the generated autoresearch_status summaries. The system automatically records parsed metrics and results into autoresearch.jsonl, providing real-time visibility into ongoing optimization trials.