baseline

To capture an AI performance baseline, this Skill automates measuring accuracy, latency, cost, and error rate for a component. It dispatches a background agent to analyze the codebase and existing eval suites, then writes reproduction steps to a .plans/BASELINE-<name>.md file.

Benchmarking AI model components measures accuracy, latency, cost, and error rate. The Skill evaluates these dimensions by analyzing your codebase alongside any existing test fixtures or eval suites to establish a reliable reference point.

Yes, you can establish an AI evaluation baseline without blocking workflows. The Skill spawns a baseline-capturer agent on a separate background thread, allowing your development and deployment pipelines to continue running uninterrupted.

To reproduce AI evaluation results after modifications, the Skill generates a .plans/BASELINE-<name>.md file containing detailed reproduction steps. This ensures stakeholders can compare post-change outputs against the known reference.

Capturing an AI latency baseline does not strictly require existing eval suites. The baseline-capturer analyzes the codebase and any available test fixtures, coordinating with related agents like eval-designer and cost-estimator to measure performance comprehensively.

The best way to evaluate AI accuracy and cost before a release is running an automated baseline capture. It measures performance metrics against current code and eval suites, creating a reproducible reference document for post-change comparison.