What problem does it solve? Optimization work often degrades into benchmark gambling: one-off timing runs, unrecorded failures, and changes kept on noisy evidence. This Skill enforces a disciplined experiment loop where every hypothesis is stated before testing, every trial is measured repeatedly, and every result is logged to a machine-readable ledger. ## Core Features & Use Cases - Structured experiment loop: Initialize a session with an explicit goal, metric, direction, and threshold, then record baselines and candidates as JSON experiment records. - Repeated measurement protocol: Runs warmup and measured trials, parses METRIC lines from benchmark output, and summarizes results with median, mean, min, and max. - Automatic disposition decisions: Classifies each experiment as keep, discard, checks_failed, or crash based on the declared improvement threshold and correctness gates. - Local reporting: Generates a CSV summary and a self-contained HTML report with SVG charts showing metric trends, best-so-far progression, and trial distributions. - Use Case: You want to reduce latency in a hot code path. The Skill scaffolds autoresearch.md and autoresearch.sh, records a baseline, then logs each candidate change with statistical evidence before deciding whether to keep it. ## Quick Start Ask the agent to run autoresearch to reduce latency in your hot path, specifying the benchmark command, the metric name, and that lower values are better.
