skills.rest
brolag
brolagCOMMUNITY
Explore all skills by this author →
Website𝕏@brolagGitHub

evaluator

Run golden-tasks tests to validate AI skills and output pass/fail metrics.

11|Updated Dec 18, 2025
Search Tags:#performance#validation#metrics#quality-assurance#tests#evaluate#golden-tasks
One-click install
npx skills add https://github.com/brolag/neural-claude-code --skill evaluator-brolag
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluator
Source: https://github.com/brolag/neural-claude-code/tree/main/skills/evaluator
Command: npx skills add https://github.com/brolag/neural-claude-code --skill evaluator-brolag

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automate validation of AI skills by running predefined golden-tasks tests. This reduces manual QA effort and provides reproducible performance signals.

Core Features & Use Cases

  • Automated evaluation: run golden-tasks, compare outputs, and produce a concise scorecard.
  • Regression detection: surface performance regressions across skill changes.
  • CI/CD integration: plug into pipelines to validate releases and optimizations.

Quick Start

Run a full evaluation pass against the golden-tasks collection to quantify current performance and detect regressions.

Frequently Asked Questions about evaluator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate AI skill validation with golden tasks?▼

Automate AI skill validation by running predefined golden-tasks tests from a JSON file to compute pass/fail metrics and output a structured scorecard. This reduces manual QA effort and provides reproducible performance signals.

Can I integrate performance metrics testing into my CI pipelines?▼

Yes, performance metrics testing can plug directly into CI pipelines to validate releases and optimizations. It runs automated evaluations against golden tasks to detect regressions and measure improvements during scheduled reviews.

What is the best way to detect AI performance regressions across skill changes?▼

Detect AI performance regressions by running automated golden-tasks tests that compare outputs and produce a concise scorecard. This surfaces performance drops across skill changes during periodic reviews or CI validation.

Do I need a specific file format to load golden tasks for evaluation?▼

You need to load golden tasks from a .claude/eval/golden-tasks.json file. The evaluator reads this predefined collection to run tests, compute metrics, and output structured results for dashboards.

How does automated validation produce results for performance dashboards?▼

Automated validation computes pass/fail metrics from golden-tasks tests and outputs structured results formatted for dashboards. This provides concise scorecards and reproducible performance signals for review.

Related Skills

Explore More Data & Analytics Skills→
ImaginerLabsImaginerLabs

skill-creator

Create and refine AI skill definitions with SKILL.md and evaluation workflows.

Official
Advanced
ValenBorrasValenBorras
1

eval-harness

Create repeatable evaluation frameworks for Claude Code sessions with pass/fail criteria.

Community
Advanced
alexmaruccialexmarucci

cicd

Monitor CI/CD pipeline metrics, tests, and deployment gates via Datadog.

Community
Advanced
0xtaoist0xtaoist

ln-636-manual-test-auditor

Audit manual test scripts for harness usage, golden-file coverage, and idempotency.

Community
Advanced