skills.rest
infantesromeroadrian
infantesromeroadrianCOMMUNITY
Explore all skills by this author →
Website𝕏@infantes_adrianGitHub

ecc-eval-harness

Evaluate Claude Code sessions with capability and regression evals using pass@k metrics.

Updated Jun 21, 2026
Search Tags:#AI development#evaluation#Claude Code#reliability#eval-driven development
One-click install
npx skills add https://github.com/infantesromeroadrian/arca-agent --skill ecc-eval-harness-infantesromeroadrian
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ecc-eval-harness
Source: https://github.com/infantesromeroadrian/arca-agent/tree/main/template/skills/ecc-eval-harness
Command: npx skills add https://github.com/infantesromeroadrian/arca-agent --skill ecc-eval-harness-infantesromeroadrian

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill offers a formal evaluation framework to ensure AI-assisted workflows with Claude Code are reliable and meet defined criteria.

Core Features & Use Cases

  • Formal Evaluation Framework: Implements eval-driven development principles for Claude Code sessions.
  • Evaluation Types: Handles capability and regression evaluations.
  • Grader Types: Offers code-based, model-based, and human graders for assessment.
  • Metrics: Uses pass@k for reliability measurement.
  • Workflow Integration: Integrates with Claude Code for seamless evaluation processes.

Quick Start

Use the /eval define command to create an evaluation for your Claude Code task.

Frequently Asked Questions about ecc-eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What is eval-driven development for AI coding sessions?▼

Eval-driven development integrates formal evaluation frameworks directly into AI coding sessions to ensure reliability. It uses capability and regression evaluations alongside pass@k metrics to verify AI-assisted workflows meet defined criteria.

How do I create an evaluation for a Claude Code task?▼

To create an evaluation for a Claude Code task, use the /eval define command. This initiates the formal evaluation framework to assess your specific AI-driven development workflow.

What grading methods are available for AI code regression testing?▼

AI code regression testing supports code-based, model-based, and human graders. These grading methods assess capability and regression evaluations to measure workflow reliability using pass@k metrics.

Can I measure AI coding reliability using pass@k metrics in Claude Code?▼

Yes, you can measure AI coding reliability using pass@k metrics in Claude Code. The evaluation framework calculates these metrics to formally assess whether your AI-assisted workflows meet defined reliability criteria.

Does Claude Code evaluation support both capability and regression testing?▼

Yes, Claude Code evaluation supports both capability and regression testing. The framework handles both evaluation types to ensure your AI-assisted workflows maintain reliability and meet defined criteria across sessions.

Related Skills

Explore More Software Engineering Skills→
githubnextgithubnext
4.9k

skills

Generate structured task specifications for GitHub Copilot agents in gh-aw workflows.

Official
Advanced
KuaaMUKuaaMU
4

eval-harness

Implement Eval-Driven Development frameworks for AI code generation sessions.

Community
Advanced
rongarederongarede
4

eval-harness

Evaluate Claude Code sessions with capability and regression evals.

Community
Advanced
bradtaylorsfbradtaylorsf
8

evaluation-frameworks

Evaluate AI models, code quality, and agent performance with structured frameworks.

Community
Advanced