skill-creator

Create, evaluate, and iteratively improve Claude skills with benchmarked test runs.

Updated Aug 25, 2017
One-click install
npx skills add https://github.com/loki495/dotfiles --skill skill-creator-loki495
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/loki495/dotfiles/tree/main/ai/skills/synced/4f2cd280-c105-47f5-b498-fb931ff17906_01354434-3473-483d-87d9-0aa269eff282/skill-creator
Command: npx skills add https://github.com/loki495/dotfiles --skill skill-creator-loki495

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Writing a skill that triggers reliably and produces good outputs is hard to do by intuition alone. This Skill guides the full lifecycle of skill development: drafting SKILL.md files, running test prompts with and without the skill, grading results against assertions, and optimizing the description field for accurate triggering. ## Core Features & Use Cases - Guided Skill Authoring: Interviews the user about intent, trigger contexts, and output formats, then writes a structured SKILL.md following progressive disclosure and skill-writing best practices. - Evaluation & Benchmarking: Spawns parallel with-skill and baseline runs, grades outputs against quantitative assertions, and aggregates pass rates, timing, and token usage into benchmark.json with mean and standard deviation. - Description Optimization: Generates realistic should-trigger and should-not-trigger queries, runs an automated train/test optimization loop, and selects the best description by held-out test score. - Use Case: You want a skill that converts meeting notes into Jira tickets. This Skill helps you draft it, creates test prompts, runs comparative evaluations in an HTML review viewer, collects your feedback, and iterates until the outputs are consistently correct. ## Quick Start Help me create a new skill that turns my weekly meeting notes into structured action items, then test and improve it.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Claude skill from scratch?▼

Start by defining what the skill should do, when it should trigger, and its output format. Then write a SKILL.md with name and description frontmatter, add optional scripts, references, or assets directories, and validate it with realistic test prompts.

How do I test whether a skill actually improves outputs?▼

Run each test prompt twice in parallel: once with the skill and once without as a baseline. Grade both against objective assertions, then aggregate pass rates, timing, and token usage into a benchmark to compare configurations.

How do I improve a skill description for better triggering?▼

Generate 20 realistic queries split between should-trigger and should-not-trigger cases, then run an optimization loop that evaluates descriptions on a train set and selects the winner by held-out test score to avoid overfitting.

What makes a good skill evaluation assertion?▼

Good assertions are objectively verifiable checks with descriptive names, such as confirming an output file exists or contains required fields. Subjective qualities like writing style are better evaluated through human review than forced assertions.

Can I run skill evaluations without subagents or a browser?▼

Yes. On Claude.ai, run test prompts sequentially yourself and present outputs inline for feedback, skipping baselines and quantitative benchmarking. In headless environments, generate a static HTML review file instead of starting a server.

When should a skill bundle a script instead of instructions?▼

Bundle a script when test runs show agents repeatedly writing the same helper code across cases, or when a task is deterministic and repetitive. Scripts in the scripts directory execute without loading into context, saving tokens and improving reliability.