eval-optimize

What problem does it solve?

Automated skill improvement loop that analyzes eval results, identifies judge failures, reads traces and rationale, edits the SKILL.md to fix issues, and re-runs validations to verify improvements.

Core Features & Use Cases

• Automated evaluation and iterative SKILL.md edits based on judge reasoning and transcripts.
• Re-run eval suites and verify without user prompts, iterating until all judges pass or max iterations are reached.
• Supports end-to-end evaluation workflows, handling per-case and batch modes depending on eval.yaml configuration, and ensures regressions are caught.

Quick Start

Trigger the optimization workflow to analyze current eval results and begin automatic skill improvements.