skills.rest
virgo777
virgo777COMMUNITY
Explore all skills by this author →
GitHub

eval-harness

Automate formal evaluation workflows for Claude Code sessions.

302|21|Updated May 10, 2026
Search Tags:#evaluation#reporting#regression#claude-code#grading#capability-evals#edd
One-click install
npx skills add https://github.com/virgo777/buddyme --skill eval-harness-virgo777
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/virgo777/buddyme/tree/main/buddyMe/skill_library/skills/eval-harness
Command: npx skills add https://github.com/virgo777/buddyme --skill eval-harness-virgo777

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

为 Claude Code 会话提供正式评测框架，支持评测驱动开发（EDD）原则，帮助在开发前定义评测、实时执行评测并追踪回归。

Core Features & Use Cases

  • Eval-Driven Development：把评测视为开发的一部分，在实现前定义评测，确保变更的可验证性。
  • 评测类型：能力评测（Capability Evals）、回归评测（Regression Evals）。
  • 评分器类型：Code-Based Grader、Model-Based Grader、Human Grader，便于多维评估。
  • 指标与存储：结果以标准化格式存储在 .claude/evals，支持基线与回归对比。
  • Use Case：在跨模型版本迭代的提示和实现中，快速回归验证与基准对比。

Quick Start

Use /eval to define, run, and generate an evaluation report in Claude Code.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up regression evaluation for Claude Code tasks?▼

You can set up regression evaluation for Claude Code tasks by applying an Eval-Driven Development framework to define evals before implementation, ensuring changes remain verifiable across iterations.

What is Eval-Driven Development and how does it work for capability evaluations?▼

Eval-Driven Development integrates evaluation into the development lifecycle, requiring you to define capability evaluations before implementation to track performance and verify task regressions.

Can I use multiple graders for Claude Code session evaluation?▼

Yes, you can orchestrate multiple graders including Code-Based, Model-Based, and Human Graders to perform multi-dimensional assessment of Claude Code sessions.

How do I store and report evaluation results for model version comparisons?▼

Evaluation results are stored in a standardized format within the .claude/evals directory, supporting baseline comparisons and regression tracking across different model versions and prompts.

Does the eval-harness framework require external dependencies to run?▼

No, the formal evaluation harness operates without external dependencies, running directly within the Claude Code environment to orchestrate graders and generate reports.

What's the best way to define and run an evaluation report in Claude Code?▼

The best way to define, run, and generate an evaluation report is by using the /eval command, which automates the formal evaluation workflow and outputs standardized results.

Related Skills

Explore More Software Engineering Skills→
yu13130122297yu13130122297

scout

Identify uncertainties in research framing, datasets, metrics, and baselines.

Community
Intermediate
xu-xiangxu-xiang
1.8k

eval-harness

Define, execute, and report capability and regression evals for Claude Code sessions.

Community
Intermediate
EotelEotel

eval-harness

Automate eval-driven validation for Claude Code changes with pass@k metrics.

Community
Advanced
riftzen-bitriftzen-bit

eval-harness

Define and run capability and regression evals for Gemini Code sessions.

Community
Advanced