model-evaluation-benchmark

Model evaluation benchmarks automate end-to-end testing of AI models by executing predefined tasks, measuring efficiency and quality metrics, then generating comprehensive reports. This Skill orchestrates the full process: reads your benchmark task definitions, runs evaluations against selected models, analyzes results, and produces documented reports and GitHub artifacts for reproducible comparison.

Yes. This Skill applies to agentic workflows comparing multiple AI models. It executes the same benchmark tasks against each model, aggregates performance scores across efficiency, quality, and workflow adherence dimensions, then generates side-by-side reports so you can validate which model performs best for your specific use cases.

Benchmark automation produces BENCHMARK_REPORT_V3.md with aggregated scores and analysis, creates GitHub issues and pull requests for tracking results, generates artifact archives for reproducibility, and performs cleanup. All outputs document test coverage, score breakdowns, and workflow compliance so results remain reproducible across runs.

You define benchmark tasks in BENCHMARK_TASKS.md; the Skill reads this file and automates the execution and analysis. This approach lets you customize what gets tested while the automation handles orchestration, parallel evaluation, and reporting—reducing manual work while maintaining control over test scope.

Benchmark evaluation measures efficiency, quality, and workflow adherence through parallel subagent reviewer calls that analyze results across multiple dimensions. This multi-faceted scoring approach captures not just speed but behavioral correctness and whether models follow your workflow requirements.

Yes. The Skill generates GitHub issues and pull requests as outputs, allowing benchmark results to be tracked and reviewed within your repository. This integration supports regression testing and makes model performance changes visible to your team through native GitHub tooling.