What problem does it solve?
Creating effective AI agent skills requires more than just writing instructions — it demands systematic testing, objective measurement, and iterative refinement to ensure skills trigger correctly and deliver consistent results. Without a structured workflow, developers waste time on skills that underperform or fail to activate when needed.
Core Features & Use Cases
- End-to-End Skill Creation: Guides you through capturing intent, drafting SKILL.md, writing test cases, and iterating based on quantitative and qualitative feedback.
- Automated Evaluation & Benchmarking: Spawns parallel with-skill and without-skill runs, grades outputs against assertions, and aggregates pass rates, timing, and token usage with variance analysis.
- Description Optimization: Uses a train/test loop to iteratively improve the skill's description field for better triggering accuracy, preventing overfitting with held-out test sets.
- Blind Comparison & Analysis: Provides rigorous A/B testing between skill versions and post-hoc analysis to understand why one version outperforms another.
Quick Start
Use the skill-creator skill to build a new skill for processing customer support tickets, run benchmark evals to measure its performance, and optimize its description for better triggering accuracy.
