eval-harness

You can run regression tests on Claude Code sessions by defining success criteria, implementing tests, and evaluating them against baselines to track regressions across model versions.

Eval-driven development for AI coding agents is a workflow that defines success criteria, runs capability and regression evals, and generates reports to drive reliable code generation.

You measure pass@k and pass^k metrics for AI testing benchmarks by running formal evaluations on Claude Code sessions, using deterministic and model-based graders to quantify reliability.

Yes, you can use model-based grading for AI evaluation instead of deterministic tests, as the framework supports multiple grader types including both deterministic and model-based options.

The best way to track regressions across Claude Code model versions is using a structured evaluation workflow that stores definitions and results in a standardized layout with regression baselines.

You structure an evaluation workflow for AI testing by following the define, implement, evaluate, and report lifecycle, storing eval definitions and results in a standardized layout.