eval-harness

Eval-driven development workflows for AI-assisted coding tasks require a formal evaluation harness to define, run, and track evals across pre-implementation, implementation, evaluation, and reporting stages.

Pass@k and pass^k metrics in AI capability and regression evaluations measure the consistency and reliability of code changes, prompts, and system behavior across multiple evaluation runs.

Automating quality assurance testing for Claude Code sessions involves configuring an evaluation framework with code-based, model-based, and human graders to assess system behavior and ensure robust regression evals.

Yes, you can use model-based and human graders together for AI prompt evaluation, combining automated scoring with manual reviews to ensure robust capability and regression assessments.

The best way to structure regression evals for AI system behavior changes is to follow an end-to-end workflow that includes pre-implementation, implementation, evaluation, and reporting steps with pass@k and pass^k metrics.