eval-harness

Eval-driven development is a formal evaluation framework for Claude Code sessions that applies structured workflows from definition to reporting. It supports capability and regression evaluations using deterministic, model-based, and human graders to measure session performance.

You run regression evaluations by defining evals, implementing them, and executing the evaluation workflow. The framework supports grading with code-based, model-based, and human graders to assess Claude Code session capabilities and generate structured reports.

Yes, the evaluation framework supports pass@k and pass^k metrics with explicit baselines and reporting. These metrics allow you to measure and compare Claude Code session performance across multiple evaluation attempts and aggregation strategies.

Yes, the framework provides eval storage and an end-to-end workflow from definition to report. It captures evaluation outputs from capability and regression runs, applies the specified graders, and produces structured reporting for analysis.

The best way is to apply a formal evaluation workflow that defines evals, implements them with code-based or model-based graders, and generates a report. This framework supports pass@k and pass^k metrics with explicit baselines for accurate capability assessment.

You need formal evaluation workflows when measuring Claude Code session capabilities or running regression checks. The framework is essential when requiring deterministic grading, pass@k metrics, and structured reporting to validate session performance against explicit baselines.