skills.rest
Rekan-Maqsoud
Rekan-MaqsoudCOMMUNITY
Explore all skills by this author →
GitHub

eval-harness

Run eval-driven capability and regression evaluations for Claude Code sessions.

Updated Aug 23, 2026
Search Tags:#testing#metrics#framework#regression#grading#ai-assistance#eval-driven
One-click install
npx skills add https://github.com/Rekan-Maqsoud/college-community-app --skill eval-harness-rekan-maqsoud
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/Rekan-Maqsoud/college-community-app/tree/main/.agent/.agents/skills/eval-harness
Command: npx skills add https://github.com/Rekan-Maqsoud/college-community-app --skill eval-harness-rekan-maqsoud

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automates and standardizes evaluation for AI-assisted tasks, enabling objective pass/fail criteria and regression checks.

Core Features & Use Cases

  • Eval types including capability and regression evaluations
  • Graders: code-based, model-based, and human review
  • Metrics tracking with pass@k and pass^k
  • End-to-end eval workflow from definition to report

Quick Start

Define and run an eval cycle using the official eval commands to validate agent performance.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run regression testing for AI-assisted development tasks?▼

Regression testing for AI-assisted development tasks is handled through an eval-driven framework that defines repeatable pass/fail criteria and tracks performance metrics across sessions.

What is eval-driven testing and how does it grade AI agent performance?▼

Eval-driven testing is an evaluation framework that grades AI agent performance using code-based, model-based, and human review workflows to apply objective pass/fail criteria.

How do I measure AI capability metrics like pass@k and pass^k?▼

AI capability metrics like pass@k and pass^k are measured by defining capability evaluations and running an end-to-end grading workflow that tracks these performance indicators from definition to report.

Can I use code-based, model-based, and human graders in one evaluation workflow?▼

Multiple grader types including code-based, model-based, and human review can be used within a single evaluation workflow to assess AI session capability and regression results.

What's the best way to standardize AI session evaluations and prevent regressions?▼

Standardizing AI session evaluations requires an eval-driven testing harness that automates capability checks, applies multiple grader types, and generates repeatable performance metrics reports.

Related Skills

Explore More Software Engineering Skills→
selcukyucelselcukyucel
1

eval-suite

Generate structured evaluation datasets with binary rubric scoring for AI outputs.

Community
Advanced
TakMczkTakMczk
1

eval-harness

Implement Eval-Driven Development evaluations with code, model, and human graders.

Community
Advanced
ryasrkryasrk

eval-harness

Formalize evaluation frameworks for AI code sessions with pass@k and pass^k metrics.

Community
Advanced
vrcmsvrcms
1

eval-harness

Defines and runs pass/fail evaluations for AI-assisted coding tasks.

Community
Advanced