skill-creator

Create, evaluate, and iteratively improve Agent Skills with benchmark-driven testing loops.

Updated Mar 25, 2026
One-click install
npx skills add https://github.com/Tyler-R-Kendrick/copilot-auto-training --skill skill-creator-tyler-r-kendrick
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/Tyler-R-Kendrick/copilot-auto-training/tree/main/.agents/skills/skill-creator
Command: npx skills add https://github.com/Tyler-R-Kendrick/copilot-auto-training --skill skill-creator-tyler-r-kendrick

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Writing an effective Agent Skill is hard to get right on the first try: descriptions may fail to trigger, instructions may be ambiguous, and there is no built-in way to know whether a skill actually improves outcomes. This Skill provides a structured create-test-review-iterate workflow with quantitative benchmarks and human feedback loops. ## Core Features & Use Cases - Guided Skill Authoring: Interviews the user about intent, trigger contexts, and output formats, then drafts a SKILL.md following progressive-disclosure and writing-style best practices. - Evaluation Harness: Runs test prompts with and without the skill in parallel, grades outputs against assertions, aggregates pass rates, timing, and token usage into benchmark.json, and renders results in an HTML review viewer. - Description Optimization: Generates realistic should-trigger and should-not-trigger queries, then runs an automated train/test optimization loop to improve the skill's triggering accuracy. - Use Case: You want to build a skill that formats weekly sales reports. This Skill helps you draft it, creates test prompts, runs side-by-side comparisons against a baseline, shows you the results in a browser viewer, and iterates until the outputs meet your standards. ## Quick Start Ask the agent to help you create a new skill for your workflow, or point it at an existing skill directory and ask it to run evaluations and improve it.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Agent Skill from scratch?

Start by describing what the skill should do and when it should trigger. The skill-creator interviews you about intent and edge cases, drafts a SKILL.md with proper frontmatter, then helps you build test cases and iterate based on evaluation results.

How do I test whether my skill actually improves outputs?

Run test prompts in parallel with and without the skill, grade outputs against assertions, and aggregate results into a benchmark comparing pass rates, time, and token usage. An HTML viewer lets you review outputs qualitatively alongside the quantitative metrics.

How do I improve a skill description so it triggers correctly?

Generate 20 realistic should-trigger and should-not-trigger queries, review them in the provided HTML template, then run the automated optimization loop. It evaluates description variants on train and held-out test queries and returns the best-performing description.

Does skill evaluation work without subagents, like on Claude.ai?

Yes, but with adaptations. Without subagents you run test cases sequentially yourself, skip baseline comparisons and quantitative benchmarking, and present results inline in the conversation for qualitative feedback instead of using the browser viewer.

What is the difference between with_skill and without_skill benchmark runs?

With_skill runs execute the test prompt with the skill loaded, while without_skill runs use no skill as a baseline. Comparing the two configurations shows whether the skill measurably improves pass rates and reveals its time and token overhead.