skills.rest
aaione
aaioneCOMMUNITY
Explore all skills by this author →
GitHub

eval-harness

Run capability and regression evals for Claude Code sessions.

25|13|Updated Jan 27, 2026
Search Tags:#workflow#testing#evaluation#regression#eval#edd
One-click install
npx skills add https://github.com/aaione/everything-claude-code-cn --skill eval-harness-aaione
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/aaione/everything-claude-code-cn/tree/main/skills/eval-harness
Command: npx skills add https://github.com/aaione/everything-claude-code-cn --skill eval-harness-aaione

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Formalizes eval-driven development to treat evals as unit tests for Claude Code sessions, enabling predictable validation and regression tracking.

Core Features & Use Cases

  • Capability Evals: define and run tests that validate new capabilities.
  • Regression Evals: ensure changes don't break existing features.
  • Grader Types: code-based, model-based, and human reviews to score changes.
  • Eval Workflow: Pre-implementation, implementation, evaluation, and reporting with .claude/evals storage.

Quick Start

Define the evals under .claude/evals and run /eval check feature-name to execute and verify the results.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up eval-driven development for Claude Code sessions?▼

Eval-driven development for Claude Code sessions is set up by defining test files under the .claude/evals directory. This framework formalizes validation by treating evals as unit tests, enabling predictable regression tracking and capability validation before deployment.

What is the workflow for running capability evals in Claude Code?▼

The capability eval workflow involves four stages: pre-implementation, implementation, evaluation, and reporting. You define evals under .claude/evals and execute them using the /eval check feature-name command to validate new features and verify results.

Does the eval-harness framework support pass@k metrics for model evaluation?▼

Yes, the eval-harness framework supports pass@k metrics for evaluating Claude Code sessions. It integrates these metrics within the reporting workflow alongside capability evals, regression evals, and various grader types to provide formal validation.

What grader types are available for scoring evals in Claude Code?▼

Grader types available for scoring evals include code-based, model-based, and human reviews. These graders evaluate changes during the evaluation workflow to provide formal, test-like validation for Claude Code sessions and regression tracking.

Related Skills

Explore More Software Engineering Skills→
D0NMEGAD0NMEGA
8

eval-harness

Automate agent reliability assessments for Claude Code sessions with eval-driven workflows.

Community
Advanced
kouisokouiso

eval-harness

Define capability and regression evals with code-based, model-based, or human graders.

Community
Advanced
xu-xiangxu-xiang
1.8k

eval-harness

Define, execute, and report capability and regression evals for Claude Code sessions.

Community
Intermediate
rivaldiekaptrrrrivaldiekaptrrr

eval-harness

Define and run eval suites to validate AI task performance.

Community
Advanced