skills.rest
krishnakanthb13
krishnakanthb13COMMUNITY
Explore all skills by this author →
Website𝕏@KrishnaKanthB13GitHub

eval-harness

Automate eval-driven evaluation for Claude Code sessions with formal definitions and pass@k metrics.

89|19|Updated Feb 9, 2026
Search Tags:#workflow#ai#evaluation#claude-code#grading#tests#eval
One-click install
npx skills add https://github.com/krishnakanthb13/everything-antigravity --skill eval-harness-krishnakanthb13
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/krishnakanthb13/everything-antigravity/tree/main/skills/eval-harness
Command: npx skills add https://github.com/krishnakanthb13/everything-antigravity --skill eval-harness-krishnakanthb13

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides a formal evaluation framework to guide and validate Claude Code sessions using eval-driven development principles, ensuring tests and criteria are defined before implementation.

Core Features & Use Cases

  • Capability Evals: define and verify new features with concrete success criteria.
  • Regression Evals: protect existing behavior across changes and refactors.
  • Grader Types: include code-based, model-based, and human graders for robust evaluation.
  • Metrics & Storage: track pass@k, pass^k, and store eval definitions and results for traceability.

Quick Start

Run the eval harness to set up, execute, and report on your feature's capability and regression evals.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up eval-driven testing for Claude Code sessions?▼

To set up eval-driven testing for Claude Code, you apply formal eval definitions before implementation. This framework automates setup, execution, and reporting for capability and regression workflows.

What is the difference between capability and regression evals in AI testing?▼

Capability evals define and verify new features with concrete success criteria, while regression evals protect existing behavior across model updates, code changes, and feature rollouts.

How do I calculate pass@k metrics for AI model evaluation?▼

You calculate pass@k and pass^k metrics using the standardized storage and reporting workflow. This tracks evaluation results and ensures traceability across model updates.

Can I use model-based graders alongside code-based graders for AI evaluation?▼

Yes, you can use model-based, code-based, and human graders together. This framework supports multiple grader types to provide robust evaluation for Claude Code sessions.

Does eval-driven development work for tracking regressions across model updates?▼

Eval-driven development works for tracking regressions by applying formal eval definitions across model updates, code changes, and feature rollouts, storing results for traceability.

Related Skills

Explore More Software Engineering Skills→
thmspithmspi

eval-harness

Automate formal evaluation workflows for Claude Code sessions with pass@k metrics.

Community
Advanced
LuohaothuLuohaothu
24

eval

Define eval criteria and track pass@k metrics for AI development.

Community
Advanced
nazrulsoftwaredevnazrulsoftwaredev

eval-harness

Define and run evaluation suites for AI-assisted development sessions.

Community
Advanced
AlbertLin821AlbertLin821
2

caveman-stats

Analyze Claude Code JSONL session logs to calculate token usage and cost savings.

Community
Basic