skill-creator

Skill authoring with benchmarking starts by capturing your intent, drafting a structured skill plan, generating test prompts, and running with-skill versus baseline comparisons to measure performance. You then aggregate quantitative and qualitative results to iteratively revise the skill.

Trigger optimization improves by updating the skill's YAML frontmatter description using a held-out evaluation set with variance-safe iteration. This process tests whether the skill activates correctly when needed, reducing false triggers across different prompt variations.

Skill evaluation uses structured test prompts to run with-skill and baseline comparisons, aggregating results through variance-aware benchmarking. This combines quantitative metrics with qualitative review to identify weaknesses and guide iterative improvement of the skill.

Python and Claude are required dependencies for running the skill evaluation and benchmarking workflow. Python scripts handle generating structured test prompts, executing with-skill and baseline comparisons, and aggregating results for variance-aware analysis.

Inaccurate triggering often stems from an unoptimized YAML frontmatter description that fails to match user intent. Applying trigger optimization with a held-out eval set and variance-safe iteration calibrates the description to activate reliably on relevant prompts.

Skill-creator applies to both creating new skills from scratch and refining existing drafts. For existing skills, it focuses on iterative improvement through measurable evaluation prompts, qualitative review, and variance-aware benchmarking to enhance performance.