skill-creator

To evaluate AI skills, you run with-skill and baseline comparison runs using generated test prompts, then grade outputs and aggregate results into benchmark JSON artifacts. This tracks timing, tokens, and performance metrics for iterative improvement.

The best way to create a SKILL.md file is using guided templates to define the name, description, and operational instructions. This structured drafting ensures the skill reliably triggers and performs during evaluation runs.

Iterative description optimization produces trigger test sets and runs an optimization loop that refines the skill description to improve triggering accuracy. It optionally calls the claude CLI to iteratively enhance the description based on test results.

Yes, you need Python to run the provided benchmarking scripts, which require the pyyaml dependency to process configurations. These scripts generate eval_metadata, grading, and benchmark JSON artifacts during the evaluation loop.

Yes, you can visualize benchmarking results by generating benchmark.json and benchmark.md files, then launching a self-contained HTML viewer. This enables qualitative and quantitative review of aggregated run statistics and skill performance.

AI skills fail to trigger correctly when the description lacks clarity or fails to match user intents. Running a description optimization loop with trigger test sets iteratively refines the description to improve triggering accuracy across various prompts.