eval-harness

Automate eval-driven development workflows for Claude Code sessions.

Updated Mar 21, 2026

Search Tags:#automation #claude-code #grading #software-testing #evals #eval-driven-development

Ready to run

npx skills add https://github.com/UsernameTron/Claude-Code-Kickstart --skill eval-harness-usernametron

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/UsernameTron/Claude-Code-Kickstart/tree/main/plugins/claude-code-factory/downloads/eval-harness
Command: npx skills add https://github.com/UsernameTron/Claude-Code-Kickstart --skill eval-harness-usernametron

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Establishes a formal evaluation framework for Claude Code sessions, enabling eval-driven development to improve reliability and reduce regressions.

Core Features & Use Cases

Capability Evals: define and run targeted capability checks.
Regression Evals: ensure existing functionality remains intact across changes.
Graders: include code-based, model-based, and human evaluators for flexible validation.
Eval Storage & Tracking: store eval definitions, baselines, and results for audit and iteration.
Integration Patterns: predefined workflows to scaffold, execute, and report on evals.

Quick Start

Run the eval suite to define, implement, and evaluate feature changes for Claude Code sessions.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I set up eval-driven development for Claude Code sessions?▼

Eval-driven development for Claude Code sessions is set up by defining markdown-based capability and regression evals, configuring graders, and storing results in the .claude/evals directory to track reliability across revisions.

What types of graders can I use for software testing evals?▼

Software testing evals support three types of graders: code-based graders for automated validation, model-based graders for AI evaluation, and human evaluators for manual review, ensuring flexible validation across different testing contexts.

How do I prevent regressions when adding new features to Claude Code projects?▼

To prevent regressions in Claude Code projects, define and run regression evals that verify existing functionality remains intact, storing baselines and results in .claude/evals to audit changes across revisions.

Can I track eval results and baselines across multiple code revisions?▼

Yes, eval results and baselines can be tracked across revisions by storing eval definitions and outputs in the .claude/evals directory, enabling audit and iteration throughout the development lifecycle.

What is the best way to automate capability checks for Claude Code features?▼

Automating capability checks for Claude Code features is best done by defining targeted capability evals in markdown, applying code-based or model-based graders, and using predefined workflows to scaffold, execute, and report on evals.

Do I need external dependencies to run evals for Claude Code sessions?▼

No external dependencies are required to run evals for Claude Code sessions. The framework operates independently, using markdown-based eval definitions and built-in grader configurations stored locally in .claude/evals.