evaluator

Agent performance evaluation systematically tests your AI agents against task datasets to measure accuracy, quality, and improvement over time. This Skill automates the testing process, executes test tasks, computes metrics, and generates performance reports without manual grading overhead.

Rule compliance auditing scans your codebase against loaded compliance rules to ensure standards are met. This Skill loads rules, validates code, and generates compliance reports to identify violations and track conformance across development teams.

Workflow quality assessment validates execution outputs and artifact completeness by applying code-based, model-based, and human grading methods. This Skill evaluates whether workflows produce correct results and outputs comprehensive quality reports.

This Skill applies across development teams and CI/CD pipelines to test agents on task datasets automatically. It executes evaluations as part of your pipeline, generates performance and compliance reports, and enables continuous validation of agent quality.

Performance and compliance validation uses multi-method evaluation: code-based analysis for rule compliance, model-based assessment for quality metrics, and human grading for nuanced evaluation. Reports aggregate results across all methods for comprehensive performance visibility.