evaluation

Evaluate AI agent performance using systematic testing and multi-dimensional rubrics.

3|Updated Dec 23, 2025

Search Tags:#testing #quality assurance #metrics #evaluation #rubrics #agent performance

Ready to run

npx skills add https://github.com/boazcstrike/opencode --skill evaluation-boazcstrike

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: evaluation
Source: https://github.com/boazcstrike/opencode/tree/main/skills/evaluation
Command: npx skills add https://github.com/boazcstrike/opencode --skill evaluation-boazcstrike

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of systematically testing and validating the performance of AI agents, ensuring their reliability and effectiveness.

Core Features & Use Cases

Systematic Testing: Build robust evaluation frameworks for agent systems.
Quality Measurement: Define and apply multi-dimensional rubrics to assess agent outputs.
Use Case: You've updated your agent's prompt. Use this Skill to run it against a suite of test cases and verify that its performance metrics (accuracy, completeness, efficiency) have improved or at least not degraded.

Quick Start

Use the evaluation skill to build a test framework for your agent.

Frequently Asked Questions about evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I evaluate AI agent performance systematically?▼

Build a test framework using multi-dimensional rubrics and detailed metrics to measure accuracy, completeness, and efficiency across a suite of test cases for your AI agents.

What is the best way to test non-deterministic AI agents with multiple valid paths?▼

Testing non-deterministic agents requires a comprehensive evaluation framework that accommodates multiple valid execution paths, utilizing both LLM-as-judge and human evaluation methodologies to ensure thorough quality assurance.

How do I measure quality assurance metrics after updating an agent's prompt?▼

Run your updated agent against a suite of test cases to verify that quality assurance metrics such as accuracy, completeness, and efficiency have improved or at least not degraded.

Can I use LLM-as-judge for scalable evaluation of agent outputs?▼

Yes, LLM-as-judge is supported as an evaluation methodology alongside human evaluation, providing a scalable approach to thoroughly assess agent outputs against defined multi-dimensional rubrics.

How do I build a test framework to measure agent quality using rubrics?▼

Build a robust evaluation framework by defining and applying multi-dimensional rubrics to systematically test agent systems, ensuring their reliability and effectiveness through detailed metrics.