skill-creator

Create, evaluate, and iteratively improve Claude skills with benchmarks and description optimization.

1|Updated Mar 2, 2025
One-click install
npx skills add https://github.com/marjorg/setup --skill skill-creator-marjorg
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/marjorg/setup/tree/main/home/.agents/skills/skill-creator
Command: npx skills add https://github.com/marjorg/setup --skill skill-creator-marjorg

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Writing an effective skill is hard to get right on the first try: descriptions may fail to trigger, instructions may be ambiguous, and there is no built-in way to know whether a skill actually improves outcomes. This Skill provides a structured draft-test-review-iterate workflow with quantitative benchmarking so skills are validated against real test prompts rather than guesswork. ## Core Features & Use Cases - Guided Skill Authoring: Interviews the user about intent, trigger contexts, and output formats, then drafts a SKILL.md following progressive disclosure and writing-style best practices. - Evaluation & Benchmarking: Runs test prompts with and without the skill via parallel subagents, grades assertions, and aggregates pass rates, timing, and token usage into benchmark reports with an interactive HTML review viewer. - Description Optimization: Generates realistic trigger/no-trigger eval queries, runs an automated optimization loop with train/test splits, and applies the best-performing description to improve triggering accuracy. - Use Case: A user wants a skill that formats weekly status reports. This Skill helps draft it, creates three realistic test prompts, runs with-skill and baseline comparisons, presents results in a browser viewer, and iterates on the instructions until the outputs satisfy the user. ## Quick Start Help me create a new skill that converts my meeting notes into structured action items, then test it with a few realistic prompts.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Claude skill from scratch?

Start by defining what the skill should do, when it should trigger, and its expected output format. Then write a SKILL.md with name and description frontmatter, draft 2-3 realistic test prompts, run them with and without the skill, and iterate based on feedback.

How do I test whether my skill actually improves results?

Run each test prompt twice in parallel: once with the skill and once without as a baseline. Grade both against objective assertions, then aggregate pass rates, timing, and token usage into a benchmark comparing the two configurations.

How do I improve a skill description so it triggers correctly?

Generate about 20 realistic should-trigger and should-not-trigger queries, then run the automated optimization loop. It evaluates description variants on a train split, selects the best by held-out test score, and updates the SKILL.md frontmatter.

What is the difference between with-skill and baseline benchmark runs?

With-skill runs execute the test prompt with the skill loaded, while baseline runs use no skill for new skills or the previous version for improvements. Comparing them isolates the skill's actual contribution to output quality.

When should a skill not include quantitative test assertions?

Skip assertions for skills with subjective outputs like writing style or design quality, where human judgment matters more than programmatic checks. Assertions work best for objectively verifiable outputs such as file transforms, data extraction, or fixed workflow steps.