eval-harness

Create structured evaluation frameworks for Claude Code sessions with capability and regression checks.

1|Updated Mar 12, 2026

Search Tags:#workflow #claude-code #grading #evaluation-framework #regression-tests #eval-driven-development

Ready to run

npx skills add https://github.com/XiaoPuOuO/VFactory --skill eval-harness-xiaopuouo

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/XiaoPuOuO/VFactory/tree/main/paperclip-official/AgentSetting/skills/eval-harness
Command: npx skills add https://github.com/XiaoPuOuO/VFactory --skill eval-harness-xiaopuouo

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Streamlines and standardizes the evaluation of Claude Code sessions, converting subjective assessments into repeatable, metric-driven checks.

Core Features & Use Cases

Eval types for capability and regression checks to ensure model improvements do not break existing behavior.
Grader templates (code-based, model-based, and human) to provide flexible evaluation strategies.
End-to-end workflow from defining evals, implementing changes, to generating evaluation reports that inform releases.

Quick Start

Run a minimal eval suite to validate a Claude Code change end-to-end.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I set up regression tests for Claude Code sessions?▼

Regression tests for Claude Code sessions are set up using a formal evaluation framework that enforces explicit eval types. This enforces explicit eval types to ensure model improvements do not break existing behavior across versions.

What is eval-driven development for AI coding workflows?▼

Eval-driven development is a workflow that standardizes AI coding evaluation through capability and regression checks. It defines graders, metrics, and an end-to-end workflow from defining evals to generating evaluation reports that inform releases.

How do I standardize grading across different model versions and prompts?▼

Standardized grading across model versions and prompts is achieved by applying structured grader templates within an evaluation framework. These templates include code-based, model-based, and human graders to provide flexible evaluation strategies.

Can I run a minimal eval suite to validate a Claude Code change end-to-end?▼

Yes, you can run a minimal eval suite to validate a Claude Code change end-to-end. The framework provides a complete workflow from defining evals and implementing changes to generating evaluation reports.

Does the eval framework support human graders alongside model-based checks?▼

Yes, the eval framework supports human graders alongside model-based and code-based checks. These grader templates are integrated to provide flexible evaluation strategies for Claude Code sessions.

What's the best way to convert subjective coding assessments into metric-driven checks?▼

The best way to convert subjective coding assessments into metric-driven checks is by implementing a formal eval framework. It streamlines evaluation by defining explicit eval types, metrics, and graders for Claude Code sessions.