skills.rest
FVossebeld
FVossebeldCOMMUNITY
Explore all skills by this author →
GitHub

evaluation

Evaluate agent outputs with weighted rubric criteria and per-dimension scores.

Updated May 24, 2026
Search Tags:#testing#monitoring#agent#evaluation#benchmark#rubric
One-click install
npx skills add https://github.com/FVossebeld/agent-skills-for-context-engineering --skill evaluation-fvossebeld
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluation
Source: https://github.com/FVossebeld/agent-skills-for-context-engineering/tree/main/skills/evaluation
Command: npx skills add https://github.com/FVossebeld/agent-skills-for-context-engineering --skill evaluation-fvossebeld

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Agents and autonomous systems produce varied outputs across tasks, making objective evaluation difficult without a structured rubric and reproducible test sets.

Core Features & Use Cases

  • Multidimensional rubrics: score factual accuracy, completeness, citations, sources, and tool efficiency to reveal strengths and gaps.
  • Test-set management: build, filter, and stratify test cases to analyze performance across complexity levels and over time.
  • Production monitoring: sample real interactions, compute pass rates and average scores, and trigger alerts for quality drift.

Quick Start

Run the AgentEvaluator on a prepared TestSet to obtain per-dimension scores and an overall pass/fail verdict.

Frequently Asked Questions about evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate agent performance against a custom rubric?▼

You can evaluate agent performance by scoring outputs against multi-dimensional rubrics with weighted criteria, generating per-dimension scores for factual accuracy, completeness, and tool efficiency.

What is a multi-dimensional rubric for testing autonomous agents?▼

A multi-dimensional rubric for testing autonomous agents objectively scores factual accuracy, completeness, citations, sources, and tool efficiency to reveal specific strengths and gaps.

How do I monitor agent quality drift in production?▼

To monitor agent quality drift in production, sample real interactions, compute pass rates and average scores, and trigger alerts when quality drifts.

Can I run batch evaluation tests across development and production?▼

Yes, you can run batch evaluation tests across test sets in development and production for continuous quality monitoring using the evaluation runner and test-set manager.

How do I stratify test cases to analyze agent performance over time?▼

You stratify test cases by building and filtering them within the test-set manager, allowing you to analyze agent performance across complexity levels and over time.

Related Skills

Explore More Software Engineering Skills→
Ikram-AlamIkram-Alam

evaluation

Evaluate agent outputs against rubrics and ground truth with multi-dimensional scoring.

Community
Basic
nahisahonahisaho
3

scientific-model-monitoring

Detect data and concept drift in production ML models.

Community
Advanced
phonowellphonowell
1

evaluation

Evaluate context-engineered agents using configurable weighted rubrics and test sets.

Community
Advanced
SyntaxAsSpiralSyntaxAsSpiral
5

evaluation

Evaluate agent systems using rubric-based scoring across test sets.

Community
Advanced