skills.rest
aman-2709
aman-2709COMMUNITY
Explore all skills by this author →
GitHub

eval-harness

Define, run, and report Claude Code session evaluations with pass/fail criteria.

12|2|Updated Mar 22, 2026
Search Tags:#evaluation#claude-code#grading#regression-testing#testing-framework#eval#pass@k
One-click install
npx skills add https://github.com/aman-2709/superpowers-ecc --skill eval-harness-aman-2709
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/aman-2709/superpowers-ecc/tree/main/skills/eval-harness
Command: npx skills add https://github.com/aman-2709/superpowers-ecc --skill eval-harness-aman-2709

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Formal evaluation framework to guide eval-driven development for Claude Code sessions, enabling precise, repeatable assessments of agent behavior.

Core Features & Use Cases

  • Eval Types: capability and regression evaluations to validate new features and preserve existing functionality.
  • Grading Styles: supports code-based, rule-based, LLM-based, semantic similarity, and human review graders to suit different tasks.
  • Metrics & Storage: tracks pass@k, pass^k, and stores eval definitions and history for traceability.

Quick Start

Define an evaluation feature with /eval define feature-name, run checks with /eval check feature-name, and generate a report with /eval report feature-name.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up reproducible regression testing for Claude Code sessions?▼

Claude Code evaluation frameworks support code-based, rule-based, LLM-based, semantic similarity, and human review graders to suit different testing tasks. You can apply these diverse grading styles across activation, checks, and reporting.

What metrics are used for evaluating Claude Code agent behavior?▼

Eval-driven development for Claude Code requires defining pass/fail criteria and regression tests. You can establish capability and regression evaluation types to validate new features and preserve existing functionality during sessions.

How do I define pass/fail criteria for Claude Code evaluations?▼

Capability and regression evaluations are the two primary eval types available for Claude Code. Capability evaluations validate new features, while regression evaluations preserve existing functionality by running formal checks during sessions.

Can I use LLM-based graders to evaluate Claude Code testing frameworks?▼

No external dependencies are required to use the eval-harness framework for Claude Code sessions. The framework provides definable pass/fail criteria, regression tests, and diverse graders natively to implement eval-driven development.

Related Skills

Explore More Software Engineering Skills→
galeepgaleep

caveman-stats

Analyze Claude Code session logs for actual token usage and savings.

Community
Basic
oabdelmaksoudoabdelmaksoud
3

eval-harness

Implement evaluation-driven development with code and model-based evaluators.

Community
Advanced
sunnypatneedisunnypatneedi
11

ai-evaluation

Define and run structured evaluation suites for AI/LLM systems.

Community
Advanced
WUIX69WUIX69

eval-harness

Define evaluation frameworks for Claude Code tasks with pass@k metrics.

Community
Advanced