skill-creator

Create, evaluate, and iteratively improve GitHub Copilot CLI skills with benchmarked test runs.

1|Updated May 19, 2026
One-click install
npx skills add https://github.com/thomasmartinsen/talks --skill skill-creator-thomasmartinsen
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/thomasmartinsen/talks/tree/main/20260519%20-%20Agentic%20engineering/primitives/skills/creator/skills/skill-creator
Command: npx skills add https://github.com/thomasmartinsen/talks --skill skill-creator-thomasmartinsen

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Writing a skill that triggers reliably and produces good outputs is hard to do blind. This Skill guides the full lifecycle of building GitHub Copilot CLI skills: drafting SKILL.md files, running test prompts with and without the skill, grading outputs against assertions, and iterating based on quantitative benchmarks and user feedback. ## Core Features & Use Cases - Guided Skill Authoring: Interviews the user to capture intent, then writes a SKILL.md with a triggering-optimized description, progressive disclosure structure, and bundled scripts/references/assets. - Evaluation Harness: Spawns paired with-skill and baseline runs via sub-agents, grades outputs against assertions, and aggregates results into benchmark.json with pass-rate, timing, and token statistics. - Interactive Review Viewer: Serves a local HTML viewer (eval-viewer/generate_review.py) where users inspect outputs side-by-side, leave feedback, and compare iterations. - Description Optimization: Generates should-trigger and should-not-trigger eval queries, reviews them in an editable HTML page, and iteratively rewrites the skill description to improve triggering accuracy. - Use Case: You want a skill that formats weekly sales spreadsheets. This Skill helps you draft it, runs three test prompts with and without the skill, shows you a benchmark comparing pass rates, and loops until the outputs satisfy you. ## Quick Start Help me create a new Copilot CLI skill that converts meeting notes into action items, then test it with a few prompts.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a GitHub Copilot CLI skill?

Define a directory containing a SKILL.md file with YAML frontmatter specifying a name and description, followed by Markdown instructions. Optionally add scripts/, references/, and assets/ directories for executable code, documentation, and templates.

How do I test whether my skill actually improves outputs?

Run paired evaluations: execute each test prompt once with the skill and once without, then grade both against objective assertions. The aggregate_benchmark.py script computes pass rates, timing, and token deltas between configurations.

What makes a skill description trigger reliably?

Include both what the skill does and specific contexts for when to use it, since the description is the primary triggering mechanism. Test against 20 realistic should-trigger and should-not-trigger queries, then rewrite to fix false negatives and false positives.

Can I compare two versions of a skill objectively?

Yes, use blind comparison: give both outputs to an independent comparator agent without revealing which skill produced which, then run the analyzer agent to identify why the winner won and generate improvement suggestions.

Why does my skill fail to trigger on some prompts?

Copilot only consults skills for tasks it cannot easily handle alone, so simple one-step queries may not trigger even with a matching description. Ensure eval queries are substantive and the description covers varied phrasings of the intent.