skills.rest
unju-ai
unju-aiOFFICIAL
Explore all skills by this author →
WebsiteGitHub

eval-harness

Implement Eval-Driven Development with capability and regression evals.

Updated Mar 2, 2026
Search Tags:#testing#ai development#metrics#evaluation#regression testing#edd
One-click install
npx skills add https://github.com/unju-ai/ecc --skill eval-harness-unju-ai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/unju-ai/ecc/tree/main/.agents/skills/eval-harness
Command: npx skills add https://github.com/unju-ai/ecc --skill eval-harness-unju-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill provides a structured approach to evaluating AI-generated outputs, ensuring reliability and consistency through Eval-Driven Development (EDD) principles.

Core Features & Use Cases

  • Eval-Driven Development (EDD): Implement unit testing for AI development by defining expected behavior before implementation.
  • Automated Testing: Run capability and regression evals to measure agent reliability and catch regressions.
  • Metrics: Track performance using pass@k and pass^k metrics.
  • Use Case: Before implementing a new feature for your AI agent, define its expected behavior using capability evals. Then, after coding, run these evals to ensure the feature works as intended and hasn't broken existing functionality.

Quick Start

Use the eval-harness skill to define a new capability evaluation for the 'user-authentication' feature.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What is eval-driven development for AI agents and when do I need it?▼

AI regression testing catches performance degradation by running regression evals against modified agent codebases. It ensures new feature additions do not break existing functionality or degrade output quality.

How do I measure AI agent reliability using pass@k metrics?▼

AI regression testing catches performance degradation by running regression evals against modified agent codebases. It ensures new feature additions do not break existing functionality or degrade output quality.

How do I set up capability evals for a new AI feature?▼

AI regression testing catches performance degradation by running regression evals against modified agent codebases. It ensures new feature additions do not break existing functionality or degrade output quality.

What grader types can I use for AI evaluation testing?▼

AI regression testing catches performance degradation by running regression evals against modified agent codebases. It ensures new feature additions do not break existing functionality or degrade output quality.

Does this formal evaluation framework support automated regression testing?▼

AI regression testing catches performance degradation by running regression evals against modified agent codebases. It ensures new feature additions do not break existing functionality or degrade output quality.

Related Skills

Explore More Software Engineering Skills→
yu13130122297yu13130122297

scout

Identify uncertainties in research framing, datasets, metrics, and baselines.

Community
Intermediate
LincyawLincyaw

Eval Harness Skill

Implement eval-driven development with code, model, and human graders.

Community
Advanced
kouisokouiso

eval-harness

Define capability and regression evals with code-based, model-based, or human graders.

Community
Advanced
xu-xiangxu-xiang
1.8k

eval-harness

Define, execute, and report capability and regression evals for Claude Code sessions.

Community
Intermediate