skills.rest
ValenBorras
ValenBorrasCOMMUNITY
Explore all skills by this author →
𝕏@ValenBorras_GitHub

eval-harness

Create repeatable evaluation frameworks for Claude Code sessions with pass/fail criteria.

1|Updated Mar 18, 2026
Search Tags:#ai#evaluation#reliability#code#grading#eval
One-click install
npx skills add https://github.com/ValenBorras/upscribe --skill eval-harness-valenborras
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/ValenBorras/upscribe/tree/main/.github/skills/eval-harness
Command: npx skills add https://github.com/ValenBorras/upscribe --skill eval-harness-valenborras

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles.

Core Features & Use Cases

  • Establish capability Evals to test new features and regressions to ensure stability.
  • Define multiple Grader Types: Code-Based, Model-Based, Human Grader to cover deterministic and open-ended checks.
  • Use metrics like pass@k and pass^k to quantify reliability across runs and changes.
  • Provide an auditable workflow and storage pattern for eval definitions and results.

Quick Start

Define an evaluation using /eval define feature-name and run checks with /eval check feature-name to begin.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up an evaluation framework for AI coding sessions?▼

Measure AI reliability by using pass@k and pass^k metrics to quantify task success across multiple runs. These metrics track regression and ensure stability when evaluating changes or comparing different model versions.

How do I grade open-ended AI code generation tasks?▼

Grade open-ended AI code tasks using model-based or human graders alongside deterministic code-based graders. This multi-grader approach covers both rigid output matching and subjective code quality assessment.

What is the best way to track AI regressions across model versions?▼

Track AI regressions across model versions by implementing an auditable storage pattern for evaluation definitions and results. This workflow maintains historical eval data to ensure stability as features evolve.

Does eval-driven development work without external dependencies?▼

Start evaluating AI reliability by defining an evaluation with /eval define feature-name, then run checks with /eval check feature-name. This establishes a repeatable framework to measure task success and track regressions.

Related Skills

Explore More Software Engineering Skills→
istinataFTSistinataFTS
2

caveman-stats

Analyze Claude Code session JSONL logs for token usage and savings.

Community
Basic
LincyawLincyaw

Eval Harness Skill

Implement eval-driven development with code, model, and human graders.

Community
Advanced
hieuckhieuck

eval-harness

Coordinate eval-driven development workflows for Claude Code sessions.

Community
Advanced
KuaaMUKuaaMU
4

eval-harness

Implement Eval-Driven Development frameworks for AI code generation sessions.

Community
Advanced