skill-creator

Create, evaluate, and iteratively improve AI agent skills with benchmarked testing workflows.

6|1|Updated Sep 3, 2026
One-click install
npx skills add https://github.com/jasonzhu0922-sketch/agentloop --skill skill-creator-jasonzhu0922-sketch
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/jasonzhu0922-sketch/agentloop/tree/main/packages/agentloop-skills/skills/skill-creator
Command: npx skills add https://github.com/jasonzhu0922-sketch/agentloop --skill skill-creator-jasonzhu0922-sketch

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and assets (resource) components.

What problem does it solve? Building effective AI agent skills requires more than writing instructions — descriptions must trigger reliably, outputs must be verified, and improvements must be validated against baselines. This Skill provides a complete workflow for drafting skills, running controlled evaluations, benchmarking performance, and optimizing trigger descriptions. ## Core Features & Use Cases - Skill Authoring: Guides you through intent capture, interviewing, and writing SKILL.md files with proper frontmatter, progressive disclosure, and bundled scripts/references/assets. - Evaluation & Benchmarking: Runs test prompts with and without the skill in parallel, grades outputs against assertions, and aggregates pass rates, timing, and token usage into benchmark reports with an interactive HTML viewer. - Description Optimization: Generates realistic trigger/no-trigger eval queries and runs an automated train/test optimization loop to improve skill triggering accuracy. - Use Case: You want to build a skill that converts meeting notes into structured action items. This Skill helps you draft it, create test cases, compare outputs against a no-skill baseline, review results in a browser viewer, and iterate until quality is satisfactory. ## Quick Start Help me create a new skill that extracts action items from meeting transcripts, then test it with a few realistic prompts.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new AI agent skill from scratch?

Start by capturing intent: what the skill should do, when it should trigger, and the expected output format. Then write a SKILL.md with name and description frontmatter, draft 2-3 realistic test prompts, and iterate based on evaluation results.

How do I test whether a skill actually improves agent output?

Run each test prompt twice in parallel — once with the skill and once without as a baseline. Grade both outputs against objective assertions, then aggregate pass rates, timing, and token usage into a benchmark for comparison.

How can I improve a skill description for better triggering?

Generate about 20 realistic should-trigger and should-not-trigger queries, then run the automated optimization loop. It evaluates description variants on a train split and selects the best by held-out test score to avoid overfitting.

What is the structure of a valid skill directory?

A skill requires a SKILL.md file with YAML frontmatter containing name and description, plus markdown instructions. Optional directories include scripts/ for executable code, references/ for on-demand docs, and assets/ for templates or files.

When should I skip quantitative benchmarking for a skill?

Skip benchmarking when the skill produces subjective outputs like writing style or design quality that need human judgment. Also skip it in environments without subagents, since baseline comparisons are not meaningful without parallel independent runs.