eval-harness

Define, implement, and run capability and regression evaluations for Claude Code sessions.

2|Updated Mar 21, 2026

Search Tags:#automation #evaluation #framework #software-testing #eval-driven #graders

Ready to run

npx skills add https://github.com/Throokie/claude-code-skills --skill eval-harness-throokie

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/Throokie/claude-code-skills/tree/main/skills/eval-harness
Command: npx skills add https://github.com/Throokie/claude-code-skills --skill eval-harness-throokie

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Provides a formal evaluation framework to drive eval-driven development (EDD) across Claude Code sessions, enabling consistent quality checks, repeatable test plans, and traceable results.

Core Features & Use Cases

Capability Evals: predefine tests to verify new features meet acceptance criteria.
Regression Evals: guard existing behavior against changes and ensure stability.
Grader Diversity: supports code-based, model-based, and human graders to cover deterministic checks and qualitative assessments.
Storage & Traceability: organized eval definitions, run histories, and baselines for rapid audits and releases.

Quick Start

Run the Eval Harness to initialize evaluation workflows in your Claude Code projects.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

What is eval-driven development and how does an evaluation framework work?▼

Eval-driven development uses a formal evaluation framework to define, implement, and run capability and regression tests. It applies structured criteria across model versions and prompts to ensure consistent quality checks and traceable results.

How do I run regression and capability evaluations in Claude Code?▼

You can run capability and regression evaluations by initializing the eval harness in your Claude Code sessions. It applies predefined tests to verify new features meet acceptance criteria and guards existing behavior against changes.

Can I use model-based and human graders for qualitative software testing?▼

Yes, the evaluation framework supports code-based, model-based, and human graders. This grader diversity covers both deterministic checks for automated software testing and qualitative assessments for complex model outputs.

Does Claude Code support automated eval workflows without external dependencies?▼

Yes, Claude Code supports automated eval workflows directly. The eval harness initializes within your sessions to provide repeatable test plans, requiring no external dependencies to define and run structured capability evaluations.

What is the best way to store traceable evaluation results for model versions?▼

The best way to store traceable evaluation results is using a formal eval framework that organizes eval definitions, run histories, and baselines. This produces a traceable eval artifact for rapid audits and releases.

Why do I need an evaluation framework for prompt and agent workflow testing?▼

You need an evaluation framework to solve evaluation bottlenecks and enable consistent quality checks. It applies structured criteria across prompts and agent workflows to verify behavior and ensure stability during development.