skills.rest
linnefromice
linnefromiceCOMMUNITY
Explore all skills by this author →
Website𝕏@linnefromiceGitHub

eval-harness

Evaluate Claude Code sessions with code-based, model-based, and human graders.

Updated Jan 22, 2026
Search Tags:#Claude Code#CI#evaluate#eval-driven development#capability eval#regression eval#grader
One-click install
npx skills add https://github.com/linnefromice/claude-code-workspace --skill eval-harness-linnefromice
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/linnefromice/claude-code-workspace/tree/main/template-.claude/skills/eval-harness
Command: npx skills add https://github.com/linnefromice/claude-code-workspace --skill eval-harness-linnefromice

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides a formal framework to evaluate Claude Code sessions, aligning development with eval-driven principles.

Core Features & Use Cases

  • Capability evaluations to verify new features against defined success criteria.
  • Regression evaluations to ensure changes do not break existing behavior.
  • Flexible grading: supports code-based, model-based, and human graders for comprehensive assessments.

Quick Start

Run the evaluator against your project baseline: ./eval-harness --baseline main --type capability. Then review the generated EVAL REPORT to guide iterative improvements.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run regression evaluations for Claude Code sessions?▼

Run regression evaluations for Claude Code sessions by executing the evaluator against a project baseline with the regression type flag. This verifies that changes do not break existing behavior by generating an EVAL REPORT to guide iterative improvements.

What is eval-driven development for Claude Code?▼

Eval-driven development for Claude Code is a formal evaluation framework that aligns development with defined success criteria. It supports capability and regression evaluations across iterative workflows to verify new features and prevent behavioral regressions.

Can I integrate Claude Code capability evaluations into a CI pipeline?▼

You can integrate Claude Code capability evaluations into a CI pipeline. The evaluation framework is designed for CI integration, enforcing metrics such as pass@k to automatically verify new features against defined success criteria.

What grading methods are supported for Claude Code session evaluations?▼

Claude Code session evaluations support three grading methods: code-based, model-based, and human graders. This flexible grading approach allows for comprehensive assessments of both capability and regression test results.

Does the evaluation framework enforce specific metrics like pass@k?▼

The evaluation framework enforces metrics such as pass@k. It provides a formal structure to evaluate Claude Code sessions, ensuring iterative workflows meet defined success criteria through code-based, model-based, or human grading.

How do I evaluate new features against defined success criteria using Claude Code?▼

Evaluate new features against defined success criteria by running a capability evaluation. Execute the evaluator against your project baseline and review the generated EVAL REPORT to guide iterative improvements for your Claude Code sessions.

Related Skills

Explore More Software Engineering Skills→
jinyuanlujinyuanlu
3

eval-harness

Define and run eval-driven development tests for Claude Code sessions.

Community
Advanced
ldk-hubldk-hub

eval-harness

Automate eval-driven development workflows for Claude Code sessions.

Community
Advanced
riftzen-bitriftzen-bit

eval-harness

Define and run capability and regression evals for Gemini Code sessions.

Community
Advanced
jakezpjakezp
17

eval-harness

Run formal evaluation workflows for Claude Code sessions with pass@k metrics.

Community
Advanced