eval-harness

Define and run capability and regression evaluations for Claude Code sessions.

Updated Feb 5, 2026

Search Tags:#workflow #evaluation #regression #quality-assurance #claude-code #pass@k #edd

Ready to run

npx skills add https://github.com/gugug168/claudecode-tutorial --skill eval-harness-gugug168

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/gugug168/claudecode-tutorial/tree/main/everything-claude-code-learning/02-Skills/eval-harness
Command: npx skills add https://github.com/gugug168/claudecode-tutorial --skill eval-harness-gugug168

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides a formal evaluation framework to govern Claude Code conversations using Evaluation-Driven Development (EDD). It helps teams define success criteria before building AI workflows, run ongoing assessments, and measure reliability with pass@k and pass^k metrics.

Core Features & Use Cases

Define capability and regression evaluations for AI tasks
Track evaluation results over time with baselines and reports
Choose scoring approaches (Code-Based, Model-Based, Human) to fit risk
Integrate with a simple command workflow and store results under .claude/evs (as example)

Quick Start

Create an evaluation definition and run it with /eval define feature-name, /eval check feature-name, and /eval report feature-name.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run regression evaluations for Claude Code workflows?▼

You can run regression evaluations for Claude Code workflows by defining success criteria and using the /eval check command. This framework applies pass@k and pass^k metrics to measure reliability across different model versions and prompts.

What is evaluation-driven development for AI coding tasks?▼

Evaluation-driven development (EDD) for AI coding tasks is a process of defining success criteria before building AI workflows. It helps teams run ongoing capability assessments and track evaluation results over time using baselines and automated reports.

How do I set up pass@k metrics for Claude Code capability assessments?▼

You set up pass@k metrics by creating an evaluation definition using the /eval define feature-name command. This framework stores results under the .claude/evals directory, enabling traceability and collaboration across projects.

Can I use code-based scoring for Claude Code prompt evaluations?▼

Yes, you can choose code-based scoring for Claude Code prompt evaluations. The framework supports code-based, model-based, and human scoring approaches so you can fit the evaluation method to your specific risk profile.

Does this evaluation framework work without external dependencies?▼

Yes, the evaluation framework operates without external dependencies. It uses a simple command workflow with /eval commands and stores all evaluation results locally under the .claude/evals directory for team collaboration.

What is the best way to track AI workflow quality over time?▼

The best way to track AI workflow quality over time is by generating automated reports with the /eval report feature-name command. This establishes baselines and tracks evaluation results across different model versions and prompts.