run-evals

Name: run-evals
Availability: InStock
Author: Flutter

Orchestrate automated evaluation of agent skills using isolated subagents and rubrics.

2.8k|166|Updated Feb 25, 2026

Search Tags:#automation #testing #quality assurance #benchmarking #evaluations #regression

Ready to run

npx skills add https://github.com/flutter/agent-plugins --skill run-evals-flutter

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: run-evals
Source: https://github.com/flutter/agent-plugins/tree/main/tool/dart_skills_lint/.agents/skills/run-evals
Command: npx skills add https://github.com/flutter/agent-plugins --skill run-evals-flutter

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill solves the challenge of manually verifying and benchmarking agent skill performance, ensuring that updates to skills do not introduce regressions or unexpected behaviors.

Core Features & Use Cases

Automated Evaluation Orchestration: Automatically triggers subagents to run integration tests and benchmarks based on defined rubrics.
Comparative Analysis: Supports A/B testing by comparing baseline agent performance against versions with specific skills enabled.
Use Case: Use this skill when you need to validate that a new skill update passes all quality criteria or when you want to benchmark the performance improvement of a specific skill against a baseline agent.

Quick Start

Use the run-evals skill to execute the evaluation suite for the target skill and generate a summary report.

Frequently Asked Questions about run-evals

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate regression testing for agent skills?▼

Automated regression testing for agent skills is orchestrated by spawning isolated subagents to execute test cases against defined rubrics, generating git diffs and summary reports to verify updates introduce no unexpected behaviors.

What is the best way to benchmark agentic behavior performance?▼

Benchmarking agentic behavior performance is achieved through comparative A/B testing, measuring baseline agent performance against versions with specific skills enabled to quantify performance improvements using rubric-based grading.

Can I use subagents to run isolated integration tests for skill evaluations?▼

Yes, you can use subagents to run isolated integration tests for skill evaluations. The orchestration framework manages workspace isolation, ensuring test cases are executed independently against defined rubrics for accurate grading.

Do I need an agent orchestration framework to execute skill evaluations?▼

Yes, an agent orchestration framework is required to execute skill evaluations. It provides the necessary infrastructure for workspace isolation, git diff generation, and rubric-based grading of the execution outputs.

When should I use automated benchmarking for quality assurance?▼

Automated benchmarking for quality assurance should be used when validating that a new skill update passes all quality criteria, or when you need to compare specific skill performance against a baseline agent to prevent regressions.

Why does my skill evaluation require workspace isolation?▼

Skill evaluation requires workspace isolation to ensure that subagents execute test cases independently without interference. This separation guarantees accurate rubric-based grading and reliable git diff generation for comparative analysis.