harness:certify

What problem does it solve?

Ensure that an evolved LLM agent's reported evaluation score is consistent and statistically reliable by running repeated evaluations and computing summary statistics.

Core Features & Use Cases

• Repeated Evaluation Runs: Automatically run the project's evaluation script multiple times against the merged code to observe score variance.
• Statistical Certification: Aggregate combined_scores to compute mean, standard deviation, range, and a categorical verdict (STABLE, MARGINAL, UNSTABLE).
• Actionable Guidance: Provide next steps based on stability: deploy when stable, add rubrics or rerun heavy evolutions when unstable.
• Use Case: After an evolution run merges an improved agent, run certification to confirm the improvement holds across multiple independent evaluations before tagging and deploying.

Quick Start

Run the certify flow to evaluate the current merged agent with three independent evaluation runs and produce a mean ± std certification report.