skill-benchmark

What problem does it solve?

Evaluates the real-world impact of a skill by comparing agent outputs with and without activation, enabling evidence-based validation for skill improvements, PRs, and regression testing.

Core Features & Use Cases

• Benchmark design: defines reproducible tasks and standardized prompts for consistent evaluation.
• Baseline vs. skill comparison: captures parallel outputs to measure delta in quality and coverage.
• Reporting and scoring: produces a scored, interpretable report showing where the skill adds value.

Quick Start

Select a representative benchmark task, run baseline output without the skill, run with the skill activated, and compare results using the predefined scoring framework.