skills.rest
eugene-belkovich
eugene-belkovichCOMMUNITY
Explore all skills by this author →
GitHub

evaluation

Build multi-dimensional evaluation pipelines for agent systems using fixed rubrics.

Updated Aug 23, 2026
Search Tags:#testing#metrics#evaluation#agents#rubrics#production-monitoring
One-click install
npx skills add https://github.com/eugene-belkovich/ai-setup --skill evaluation-eugene-belkovich
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluation
Source: https://github.com/eugene-belkovich/ai-setup/tree/main/claude/profiles/work/skills/evals/evaluation
Command: npx skills add https://github.com/eugene-belkovich/ai-setup --skill evaluation-eugene-belkovich

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Evaluation of agent systems is challenging due to non-determinism, multi-agent interactions, and the need for robust, repeatable measurement across development and production.

Core Features & Use Cases

  • Multi-dimensional rubrics for factual accuracy, completeness, citation quality, source quality, and tool efficiency.
  • Test-set driven evaluation that supports simple, medium, and complex scenarios and produces actionable insights.
  • Production monitoring hooks to track performance and alert on degradation over time.

Quick Start

Run a basic evaluation pipeline against a pre-defined test set to generate a report on agent performance and stability.

Frequently Asked Questions about evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate agent performance using test sets and rubrics?▼

To evaluate agent performance, you build scalable pipelines using fixed rubrics and test sets across factual accuracy, completeness, and tool efficiency. The system computes weighted scores to validate outcomes and generate actionable insights.

What is the best way to monitor agent systems for performance degradation in production?▼

Production monitoring tracks agent performance over time using hooks that alert on degradation. It applies multi-dimensional rubrics to measure stability and validate outcomes within live production environments.

Can I use evaluation rubrics across unit, integration, and production contexts?▼

Yes, evaluation rubrics apply across unit, integration, and production contexts. They measure multi-dimensional metrics like citation quality and tool efficiency, supporting simple, medium, and complex test scenarios.

Do I need a reporting layer to measure agent performance?▼

Yes, a reporting layer is required to compute weighted scores and alerts. Alongside a rubric, test sets, and an agent interface, it generates reports on agent performance and stability.

Why does non-determinism make evaluating agent systems challenging?▼

Non-determinism and multi-agent interactions make repeatable measurement difficult. Evaluation pipelines solve this by applying fixed rubrics across test sets to consistently quantify factual accuracy, completeness, and tool efficiency.

What metrics should I track when testing agent systems?▼

Track factual accuracy, completeness, citation quality, source quality, and tool efficiency. These multi-dimensional metrics form the rubrics used to compute weighted scores and validate agent outcomes.

Related Skills

Explore More Data & Analytics Skills→
NAKKA-KNAKKA-K

eval-driven-development

Define rubrics, run baseline evaluations, and compare before-and-after outcomes.

Community
Advanced
hopeoverturehopeoverture
6

performance-budget-enforcer

Enforce performance budgets on bundle sizes, Lighthouse scores, and Web Vitals in CI/CD pipelines.

Community
Advanced
phonowellphonowell
1

evaluation

Evaluate context-engineered agents using configurable weighted rubrics and test sets.

Community
Advanced
selcukyucelselcukyucel
1

prompt-test

Run a prompt against test inputs and score outputs with a binary rubric.

Community
Advanced