skill-creator

You can automate skill evaluation and benchmarking by coordinating drafting, running test cases, and iterating improvements based on results. This structured workflow establishes baselines and refines triggering accuracy through batch experiments.

The best way to refine SKILL.md descriptions is through an iteration loop that runs evaluations, benchmarks performance, and optimizes descriptions. This process improves triggering accuracy based on structured test results.

You set up evaluation scaffolding by capturing skill intent, generating drafts, and defining eval prompts and metrics. This facilitates running baseline comparisons and tracking provenance for skill versions.

Yes, you need pyyaml installed to run skill evaluations and batch experiments. The dependency supports the scripts required to automate the end-to-end workflow of creating and refining skills.

Yes, you can track provenance and compare baselines when testing workflows. The system supports batch experiments and baseline comparisons to validate improvements during the iteration loop.

Triggering accuracy fails when descriptions lack optimization based on benchmark evaluations. Iterating improvements by analyzing structured test case results refines the description and corrects performance gaps.