agent-eval-design

What problem does it solve?

This Skill helps you avoid false confidence in AI agents by turning ambiguous “seems good” behavior into measurable, repeatable behavioral contracts.

Core Features & Use Cases

• Eval coverage design: Define positives, near misses, and failure traces so your tests reflect real ambiguity.
• Routing and grading contracts: Create rubrics, graders, artifact checks, and hard negatives that verify decision boundaries.
• Regression readiness: Convert past failures into regression cases with explicit acceptance thresholds and health-state fields.

Quick Start

Create an agent evaluation plan with hard negatives, a rubric-based grader, and pass thresholds for whether your router selects the correct skill under near-miss prompts and previously observed failure traces.