eval-harness

Eval-driven development for Claude Code sessions is implemented by defining expected behavior before implementation and running evaluations continuously. This framework treats evals as unit tests to ensure task completion and agent reliability.

Capability evals test new features in your AI development workflow, while regression evals ensure existing functionality remains intact. Both eval types are supported to maintain agent reliability throughout development cycles.

To define an evaluation for a new feature in Claude Code, run the command `/eval define feature-name`. This initiates the eval-driven development framework to formalize expected behavior for the session.

Available grader types for evaluating AI agent reliability include code-based graders for deterministic checks, model-based graders for open-ended evaluation, and human graders. These options ensure flexible and accurate assessment.

Reliability measurement in eval-driven development uses pass@k and pass^k metrics. These metrics quantify the consistency and success rate of Claude Code sessions across multiple evaluation runs.

Yes, executing this evaluation framework requires Claude Code and its associated eval tools. You must have this environment configured to define, check, and report evaluations for your sessions.