skills.rest
tan-yong-sheng
tan-yong-shengCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

eval-harness

Automate eval-driven development workflows for Claude Code sessions.

1|1|Updated Mar 5, 2026
Search Tags:#testing#claude#regression#code#eval#pass@k#eval-harness
One-click install
npx skills add https://github.com/tan-yong-sheng/GrowChat --skill eval-harness-tan-yong-sheng
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/tan-yong-sheng/GrowChat/tree/main/.claude/skills/eval-harness
Command: npx skills add https://github.com/tan-yong-sheng/GrowChat --skill eval-harness-tan-yong-sheng

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.

Core Features & Use Cases

  • Capability Evals: define and run capability-based checks that verify new features.
  • Regression Evals: track regressions across model changes with clear baselines.
  • Grader Types and Metrics: deterministic code-based checks, model-based graders, and pass/fail reporting with pass@k and pass^k metrics.

Quick Start

Set up an eval-driven workflow to define pass/fail criteria, run capability and regression evals, and generate an evaluation report.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What is eval-driven development for Claude Code and why use it?▼

Eval-driven development is a formal evaluation framework for Claude Code sessions that defines pass/fail criteria, runs capability and regression evals, and tracks reliability with pass@k metrics across model versions.

How do I track regressions across model versions with Claude Code?▼

Run regression evals by defining clear baselines and using deterministic code-based checks or model-based graders. This tracks regressions across model changes to ensure new updates do not break existing capabilities.

Can I use both deterministic and model-based graders for pass/fail reporting?▼

Yes, you can use deterministic code-based graders and model-based graders. The framework generates pass/fail reporting with pass@k and pass^k metrics to measure capability and regression eval reliability.

How do I set up capability evals to verify new features?▼

Set up an eval-driven workflow to define pass/fail criteria and run capability-based checks. This automates capability evals to formally verify new features within your Claude Code sessions.

Does the eval-harness workflow require external dependencies?▼

No external dependencies are required. The framework operates natively within Claude Code sessions to automate evaluation workflows, utilizing built-in deterministic and model-based graders.

Related Skills

Explore More Software Engineering Skills→
xu-xiangxu-xiang
1.8k

eval-harness

Define, execute, and report capability and regression evals for Claude Code sessions.

Community
Intermediate
fredericvahrenhorstfredericvahrenhorst

eval-harness

Run capability and regression evals for Claude Code sessions with pass@k metrics.

Community
Advanced
alangebalangeb

test-suite-monitor

Run and monitor test suites in background tmux sessions with comparative analysis.

Community
Intermediate
bugrabilgebugrabilge
10

agent-evaluation

Design benchmark test suites for evaluating LLM agent reliability.

Community
Advanced