evaluation

Evaluate agent outputs against multi-dimensional rubrics to produce scores and pass/fail judgments.

1|Updated Jan 4, 2026

Search Tags:#monitoring #metrics #agent #rubric #evaluate

Ready to run

npx skills add https://github.com/ChakshuGautam/games --skill evaluation-chakshugautam

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: evaluation
Source: https://github.com/ChakshuGautam/games/tree/main/.claude/skills/evaluation
Command: npx skills add https://github.com/ChakshuGautam/games --skill evaluation-chakshugautam

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

The Evaluation skill provides a structured framework to measure, compare, and monitor agent performance across diverse tasks, ensuring consistent quality and governance.

Core Features & Use Cases

Build multi-dimensional rubrics (factual accuracy, completeness, citation accuracy, source quality, tool efficiency) to evaluate outputs.
Run automated evaluations against ground truth, tool logs, and context histories to generate per-dimension scores and an overall pass/fail verdict.
Use in development and production to track quality over time, compare configurations, and enforce quality gates for pipelines.

Quick Start

Produce an evaluation report by feeding a task, its expected ground truth, and agent output into the evaluation pipeline to obtain dimension scores and an overall score.

Frequently Asked Questions about evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I evaluate agent outputs against custom rubrics?▼

To evaluate agent outputs against custom rubrics, define a multi-dimensional rubric with weights, provide the expected ground truth and tool-use logs, and the pipeline will compute per-dimension scores and an overall pass/fail judgment.

What is a multi-dimensional rubric for agent monitoring?▼

A multi-dimensional rubric for agent monitoring is a defined set of weighted criteria, such as factual accuracy, citation accuracy, and tool efficiency, used to systematically score agent performance against ground truth.

Can I use rubric evaluation for multi-agent configurations?▼

Yes, you can use rubric evaluation for multi-agent configurations by running automated evaluations against context histories and tool logs to generate scores and pass/fail verdicts across single-agent and multi-agent setups.

What do I need to generate pass/fail judgments for agent quality?▼

You need a defined rubric with weights, ground truth data, and access to agent tool-use logs and context histories to generate structured dimension scores and overall pass/fail judgments.

Does automated agent evaluation work for continuous monitoring in production?▼

Automated agent evaluation works for continuous monitoring in production by tracking quality over time, comparing configurations, and enforcing quality gates within development and production pipelines.