eval-harness

Regression evals for Claude Code apply predefined success criteria to AI-assisted workflows, ensuring reliable QA across iterations. The harness automates this formal evaluation process to support eval-driven development and track performance benchmarks across model versions.

Eval-driven development requires formal evaluation of AI-assisted workflows against predefined success criteria. You need it when automating code evaluation, running regression tests, or comparing performance benchmarks across different Claude model versions.

Available grader types for code evaluation include code-based, model-based, and human graders. These support capability and regression eval definitions to measure standard metrics like pass@k and pass^k within a repeatable workflow.

Measure pass@k and pass^k metrics by running the eval workflow with configured grader types against capability eval definitions. The harness stores artifacts and provides traceability for each eval run to ensure repeatable benchmarking across model versions.

The eval harness supports artifact storage and traceability for eval runs. It automates formal evaluation of Claude Code sessions, storing results to maintain a repeatable workflow across regression and capability eval definitions.

You can benchmark performance across model versions using capability and regression eval definitions with predefined success criteria. The harness applies standard metrics like pass@k and pass^k to generate comparative eval reports for AI-assisted workflows.