skills.rest
excatt
excattCOMMUNITY
Explore all skills by this author →
GitHub

eval-harness

Implement Eval-Driven Development workflows for AI feature evaluation and reporting.

14|2|Updated Jan 28, 2026
Search Tags:#quality assurance#prompt engineering#benchmarking#AI development#evaluation#LLM
One-click install
npx skills add https://github.com/excatt/superclaude-plusplus --skill eval-harness-excatt
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/excatt/superclaude-plusplus/tree/main/skills/eval-harness
Command: npx skills add https://github.com/excatt/superclaude-plusplus --skill eval-harness-excatt

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill addresses the challenge of reliably developing and evaluating AI features by establishing a rigorous Eval-Driven Development (EDD) process, ensuring AI outputs meet defined success criteria before deployment.

Core Features & Use Cases

  • EDD Workflow: Guides users through defining, implementing, evaluating, and reporting on AI features.
  • Evaluation Types: Supports Capability, Regression, and Safety evaluations.
  • Grading Methods: Integrates code-based, model-based, and human grading.
  • Use Case: When developing a new AI summarization feature, use this Skill to first define what constitutes a "good" summary (e.g., length, keyword inclusion, accuracy), then implement the feature, run the defined evaluations, and finally generate a report on its performance against the criteria.

Quick Start

Define a new evaluation for the document summarizer feature by running the command /eval define document-summarizer.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What is eval-driven development for AI features?▼

AI feature evaluation combines code-based assertions, LLM judges, and human review to grade model outputs. This hybrid grading approach validates whether AI performance meets predefined success criteria across capability, regression, and safety tests.

How do I benchmark LLM performance for regression and safety?▼

To start an AI evaluation, define the target feature and its success criteria, such as length or accuracy. Then implement the feature, run the defined capability and regression tests, and generate a performance report.

Can I use LLM judges alongside code-based assertions for AI quality assurance?▼

Yes, you can evaluate AI summarization by first defining what constitutes a good summary, such as length and keyword inclusion. Then implement the feature and run evaluations to generate a performance report.

When do I need to run safety evaluations during AI development?▼

You need safety, capability, and regression evaluations when developing AI features to ensure outputs meet defined benchmarks. This prevents deploying LLMs that fail quality assurance criteria or exhibit unsafe behavior.

Related Skills

Explore More Software Engineering Skills→
selcukyucelselcukyucel
1

eval-suite

Generate structured evaluation datasets with binary rubric scoring for AI outputs.

Community
Advanced
JaiminVadadoriyaJaiminVadadoriya
1

ai-system-testing

Test AI features including LLM prompts, tool calls, and nondeterministic outputs.

Community
Advanced
dtsongdtsong
1

AI Evaluation

Design evaluation frameworks for AI features with golden datasets and scoring rubrics.

Community
Advanced
deepan-alvedeepan-alve

eval-harness

Define and run structured capability and regression evals for Claude Code sessions.

Community
Advanced