skill-creator

Iteratively improve Claude skill descriptions through trigger evaluations and structured edit loops.

Updated Aug 27, 2026
One-click install
npx skills add https://github.com/stellarone/evaluate --skill skill-creator-stellarone
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/stellarone/evaluate/tree/main/.agents/skills/skill-creator
Command: npx skills add https://github.com/stellarone/evaluate --skill skill-creator-stellarone

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires claude, python, yaml, lsof, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

Skill-creator helps you design new AI skills or refine existing ones by turning vague “try it and see” iteration into a structured loop: draft the skill, run trigger-evaluation tests against realistic prompts, review qualitative outputs, then iteratively improve until the skill triggers reliably and performs well.

Core Features & Use Cases

  • Skill drafting and improvement loop: Guides you through intent capture, drafting the skill, then iterating based on evaluation results.
  • Quantitative trigger evaluation: Runs Claude “trigger/not trigger” tests using a held-out set to reduce overfitting to your prompt set.
  • Qualitative review workflow: Launches an eval viewer so you can inspect outputs and leave human feedback.
  • Benchmarking and variance analysis: Aggregates pass rates and timing/token usage across runs to spot inconsistent or flaky behavior.
  • Description optimization: Improves the SKILL.md description so Claude invokes the skill at the right times.
  • Test-case growth strategy: Encourages expanding the test set and retrying at larger scale for robustness.

Quick Start

Use this skill to create a new skill by telling it what you want Claude to do, when it should trigger, what output format you expect, and a few example prompts you consider success.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate LLM prompt triggering against realistic test cases?

To evaluate LLM prompt triggering, you run structured trigger-evaluation tests using a held-out set of prompts to measure pass rates, reducing overfitting to your initial prompt set. This quantitative benchmarking measures whether Claude invokes the skill at the right times.

What is the best way to iteratively improve AI skill descriptions and instructions?

Iteratively improving AI skill descriptions involves a structured skill-edit loop: draft the skill, run trigger evaluations, review qualitative outputs in an eval viewer, then refine the YAML frontmatter and instructions until the skill triggers reliably.

How do I benchmark LLM behavior and analyze pass rate variance across runs?

Benchmarking LLM behavior and analyzing pass rate variance requires aggregating timing and token usage across multiple test runs. This structured benchmarking workflow spots inconsistent or flaky behavior, helping you expand your test set for robustness.

Do I need YAML frontmatter to create and optimize Claude skills?

Yes, you need YAML frontmatter metadata to create and optimize Claude skills. It provides the required structure for skill instructions and optional compatibility fields, enabling the description optimization needed for reliable skill triggering.

Why does my Claude skill trigger inconsistently on similar prompts?

Inconsistent Claude skill triggering often occurs due to overfitting to a small prompt set. Expanding your test cases and running quantitative trigger evaluations with a held-out test set improves the SKILL.md description so Claude invokes the skill reliably.

Can I use Python to automate the skill evaluation and iteration loop?

Yes, you can use Python to automate the skill evaluation and iteration loop. The workflow uses Python scripts alongside Claude to run trigger tests, launch the eval viewer for qualitative review, and aggregate benchmarking data for analysis.