skills.rest
Ced3-han
Ced3-hanCOMMUNITY
Explore all skills by this author →
Website𝕏@JiahanCedGitHub

eval-harness

Evaluate Claude Code sessions with capability and regression evals.

Updated Apr 30, 2026
Search Tags:#AI development#Claude Code#eval-driven development#evaluation framework#metrics tracking
One-click install
npx skills add https://github.com/Ced3-han/Harness-Settings --skill eval-harness-ced3-han
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/Ced3-han/Harness-Settings/tree/main/skills/eval-harness
Command: npx skills add https://github.com/Ced3-han/Harness-Settings --skill eval-harness-ced3-han

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides a formal evaluation framework for Claude Code sessions, ensuring eval-driven development (EDD) principles are followed.

Core Features & Use Cases

  • Capability Evals: Test Claude's new capabilities.
  • Regression Evals: Ensure changes don't break existing functionality.
  • Grader Types: Utilize code-based, model-based, or human graders.
  • Metrics: Track pass@k metrics for reliability.
  • Integration Patterns: Pre-implementation, during implementation, and post-implementation support.
  • Eval Storage: Organize evals in project directories.
  • Best Practices: Follow best practices for eval-driven development.

Quick Start

Use the eval-harness skill to define and run capability and regression evals for your Claude Code session.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I implement eval-driven development for Claude Code sessions?▼

Capability and regression evals are structured testing frameworks used to validate AI capabilities and prevent regressions. Capability evals test new Claude functions, while regression evals ensure changes do not break existing functionality in your codebase.

What metrics should I track for AI evaluation frameworks?▼

Pass@k metrics are tracked to measure reliability during AI evaluation. You can gather these metrics by applying code-based, model-based, or human graders to your capability and regression evals within your project directories.

When do I need to run regression evals in an AI testing workflow?▼

Regression evals are needed during and after implementation to ensure code changes do not break existing functionality. They provide post-implementation verification to maintain system stability throughout the AI development lifecycle.

Can I use code-based and model-based graders together for Claude Code evaluation?▼

Yes, you can use code-based, model-based, and human graders together for Claude Code evaluation. The framework supports utilizing multiple grader types simultaneously to comprehensively assess your AI testing sessions.

How do I organize evaluation files for an AI development project?▼

You organize evaluation files by storing evals in structured project directories. This storage pattern supports best practices for eval-driven development by keeping capability and regression tests accessible throughout the implementation lifecycle.

Related Skills

Explore More Software Engineering Skills→
ysyecustysyecust
40

eval-harness

Codify and automate Claude Code session evaluations with pass@k metrics.

Community
Intermediate
SeanChenRSeanChenR

eval-harness

Automate evaluation and regression tracking for AI-assisted coding sessions.

Community
Advanced
zardusai-cyberzardusai-cyber
1

eval-harness

Define and execute eval-driven tests for Claude Code sessions.

Community
Advanced
ryasrkryasrk

eval-harness

Formalize evaluation frameworks for AI code sessions with pass@k and pass^k metrics.

Community
Advanced