skills.rest
aayushsoam
aayushsoamCOMMUNITY
Explore all skills by this author →
GitHub

eval-harness

Evaluate Claude Code sessions with pass/fail criteria and regression metrics.

1|Updated Mar 31, 2026
Search Tags:#testing#framework#claude-code#model-evaluation#eval-driven#regression-tests
One-click install
npx skills add https://github.com/aayushsoam/clawbot-plus --skill eval-harness-aayushsoam
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/aayushsoam/clawbot-plus/tree/main/skills/eval-harness
Command: npx skills add https://github.com/aayushsoam/clawbot-plus --skill eval-harness-aayushsoam

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Formalizes evaluation for Claude Code sessions by providing an eval-driven framework that defines success criteria, runs tests, and records results to guide development workflows.

Core Features & Use Cases

  • End-to-end eval workflow (define, implement, evaluate, report) for AI-driven coding tasks.
  • Supports capability and regression evals with pass@k and pass^k style metrics.
  • Provides deterministic code-based graders and model-based rubrics to ensure reliability across model versions.

Quick Start

Run the evaluation workflow to define your first eval and generate a report.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up regression tests for Claude Code model versions?▼

Regression tests for Claude Code are formalized using an eval-driven framework that defines success criteria, runs tests, and records results to track performance across model versions.

What is eval-driven testing for AI coding tasks?▼

Eval-driven testing for AI coding tasks defines success criteria, runs tests, and records results to guide development workflows using pass@k and pass^k metrics.

How do I evaluate Claude Code agent interactions deterministically?▼

Claude Code agent interactions are evaluated deterministically by applying code-based graders and model-based rubrics to ensure reliable, reproducible pass/fail assessments.

Can I use pass@k metrics for Claude Code prompt evaluation?▼

Yes, pass@k and pass^k style metrics are supported for Claude Code prompt evaluation, measuring capability and regression performance across model versions.

Does the eval-driven framework support end-to-end reporting for coding sessions?▼

The eval-driven framework supports end-to-end reporting for coding sessions by guiding a full workflow from define to report, implementing tests, evaluating results, and generating reports.

What's the best way to standardize Claude Code session assessments?▼

Standardizing Claude Code session assessments is best achieved using an eval-driven framework that defines eval types, graders, and metrics to produce deterministic, reproducible results.

Related Skills

Explore More Software Engineering Skills→
tan-yong-shengtan-yong-sheng
1

eval-harness

Automate eval-driven development workflows for Claude Code sessions.

Community
Advanced
rivaldiekaptrrrrivaldiekaptrrr

eval-harness

Define and run eval suites to validate AI task performance.

Community
Advanced
xu-xiangxu-xiang
1.8k

eval-harness

Define, execute, and report capability and regression evals for Claude Code sessions.

Community
Intermediate
officechbusinessservices-creatorofficechbusinessservices-creator

agentic-engineering

Coordinate AI-driven engineering tasks with eval-first evaluation and tiered model routing.

Community
Advanced