mad-eval

What problem does it solve?

Building and maintaining composite AI agents carries risk of undetected definition errors, unmeasured context overhead, and unquantified quality regressions over time. This Skill eliminates those gaps by providing a standardized evaluation framework to validate agent correctness, measure context efficiency, and track quality trends.

Core Features & Use Cases

• Structural Validation (Zero LLM Cost): Automatically check agent definitions for required frontmatter fields, mandatory tools, output formats, and turn limits to catch configuration errors before deployment.
• Functional & Comparison Evals: Run composite agents against smoke and enterprise test scenarios, then compare their performance against direct agent chains to quantify context savings and orchestrator efficiency gains.
• LLM-as-Judge Grading: Optional automated qualitative scoring of agent output against rubric templates to validate code quality, process adherence, and technical correctness.
• Use Case: A team building composite coding agents can use this Skill to run pre-deployment structural checks, execute regression tests after agent updates, and generate dashboard reports to share quality metrics with stakeholders.

Quick Start

Invoke the mad-eval skill to run a full suite of local agent evaluations, including structural checks, functional tests, and comparison metrics, then open the built-in dashboard to review the results.