skills.rest
The-Skyy-Rose-Collection-LLC
The-Skyy-Rose-Collection-LLCOFFICIAL
Explore all skills by this author →
GitHub

eval-harness

Define, run, and report evaluations with /eval commands.

3|1|Updated Oct 28, 2025
Search Tags:#workflow#testing#tdd#quality-assurance#eval#feature-validation
One-click install
npx skills add https://github.com/The-Skyy-Rose-Collection-LLC/DevSkyy --skill eval-harness-the-skyy-rose-collection-llc
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-harness
Source: https://github.com/The-Skyy-Rose-Collection-LLC/DevSkyy/tree/main/.claude/skills/eval-harness
Command: npx skills add https://github.com/The-Skyy-Rose-Collection-LLC/DevSkyy --skill eval-harness-the-skyy-rose-collection-llc

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Clarifies expected behavior before coding to prevent scope creep and misalignment, enabling faster, more reliable delivery.

Core Features & Use Cases

  • Eval Types: Capability (test new functionality), Regression (ensure existing behavior unchanged)
  • Metrics: pass@k and pass^k definitions for success criteria
  • Workflow: /eval define feature-name, /eval check feature-name, /eval report feature-name
  • Eval Definition: EVAL: feature-name with Capability and Regression sections
  • Graders: Code-based, Model-based, Human review
  • Related Tools: /eval command, tdd-guide, tdd-workflow

Quick Start

Define an evaluation with /eval define feature-name and validate its results with /eval check feature-name.

Frequently Asked Questions about eval-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I define feature evaluations before coding to prevent scope creep?▼

Feature evaluations codify expected behavior before coding using the /eval define feature-name command, specifying Capability and Regression sections to set measurable success criteria and prevent scope creep.

What metrics are used for testing feature validation success criteria?▼

Feature validation uses pass@k and pass^k metrics to define success criteria, measuring whether the evaluated functionality meets the expected behavior specified in the eval definition.

Can I use model-based graders for regression testing workflows?▼

Yes, regression testing workflows support code-based, model-based, and human review graders to verify existing behavior remains unchanged when running /eval check feature-name evaluations.

What's the best way to generate reports after running feature evaluations?▼

Generating evaluation reports uses the /eval report feature-name command after running /eval check, producing validation outputs that summarize capability and regression test results for feature assessments.

Does TDD workflow integrate with eval definitions for quality assurance?▼

TDD workflow integrates with eval definitions through the tdd-guide and tdd-workflow related tools, connecting test-driven development processes with feature validation and quality assurance evaluations.

Related Skills

Explore More Software Engineering Skills→
pingwupingwu
12

test-first-development

Generate BDD/Gherkin feature files and TDD test scaffolding before coding.

Community
Intermediate
rongarederongarede
4

eval-harness

Evaluate Claude Code sessions with capability and regression evals.

Community
Advanced
LiGoldragonLiGoldragon
1

test-driven-development

Automate TDD cycles for test generation, execution, and code refactoring.

Community
Advanced
johannesfritzjohannesfritz
1

run-test-suite

Run pytest or Playwright/vitest suites and report pass/fail results.

Community
Advanced