skills.rest
luongldptit
luongldptitCOMMUNITY
Explore all skills by this author →
GitHub

eval-harness

Define pass/fail criteria and track pass@k reliability metrics for Claude Code evals.

Updated Mar 26, 2026
Search Tags:#claude code#regression testing#eval-driven development#agent reliability#pass@k metrics#ai workflow benchmarking#capability evals
One-click install
npx skills add https://github.com/luongldptit/move-ticket --skill eval-harness-luongldptit
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/luongldptit/move-ticket/tree/main/.agent/.agents/skills/eval-harness
Command: npx skills add https://github.com/luongldptit/move-ticket --skill eval-harness-luongldptit

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill eliminates the guesswork around Claude Code agent performance and inconsistent task completion by providing a structured framework to measure and improve AI workflow reliability.

Core Features & Use Cases

  • Eval Definition & Tracking: Create formal pass/fail criteria for AI tasks before implementation, with support for both capability and regression evals.
  • Reliability Measurement: Track pass@k and pass^k metrics to quantify agent performance across multiple attempts and model versions.
  • Regression Prevention: Build automated test suites to catch breaking changes to prompts or agent configurations before they impact production. Use case: For engineering teams building AI-powered coding tools, use this Skill to define evals for new features, run them continuously during development, and ensure updates don't break existing functionality.

Quick Start

Use the eval-harness skill to define pass/fail criteria for your new Claude Code feature and track its reliability across test runs.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What is eval-driven development for AI coding agents?▼

To define pass/fail criteria for Claude Code features, establish formal pass/fail conditions for AI tasks before implementation. This Skill supports both capability and regression evals to measure agent reliability and track performance across multiple attempts and model versions.

How do I prevent regressions when updating AI agent configurations?▼

Pass@k and pass^k metrics quantify AI agent performance by tracking reliability across multiple attempts and model versions. These metrics provide standardized eval reports that measure consistency and capability for AI-assisted development workflows.

How do I benchmark Claude Code agent performance across model versions?▼

Yes, you can use this eval-harness framework for capability testing by defining evals for new features and running them continuously. It supports tracking pass@k metrics to measure agent performance and ensure updates do not break existing functionality.

Do I need external dependencies to set up AI workflow benchmarking?▼

If AI workflow benchmarking is not capturing consistent task completion, verify that formal pass/fail criteria are explicitly defined for AI tasks before implementation. The framework relies on structured eval definitions to accurately track pass@k and pass^k reliability metrics.

Related Skills

Explore More Software Engineering Skills→
rongarederongarede
4

eval-harness

Evaluate Claude Code sessions with capability and regression evals.

Community
Advanced
sayasaya8039sayasaya8039
2

eval-harness

Define pass/fail criteria and run regression evals for Claude Code agents.

Community
Advanced
riftzen-bitriftzen-bit

eval-harness

Define and run capability and regression evals for Gemini Code sessions.

Community
Advanced
kk20300113-pngkk20300113-png

eval-harness

Define pass/fail criteria and calculate pass@k metrics for Claude Code agent evals.

Community
Advanced