What problem does it solve?
The Skill Creator removes the friction of designing, testing, and iterating AI skills by providing a structured workflow for drafting SKILL.md, generating test prompts, running paired with-skill and baseline evaluations, grading outputs, and producing benchmark reports. It centralizes the evaluation lifecycle so creators can move from a draft to a validated, packaged skill with measurable improvements and reproducible runs.
Core Features & Use Cases
- Guided Skill Authoring: Walks the user through intent capture, required inputs/outputs, trigger phrasing, and writing the SKILL.md frontmatter and body.
- Test Generation & Execution: Produces realistic test prompts, orchestrates paired with-skill and baseline runs, captures timing and token usage, and saves iteration workspaces.
- Evaluation & Benchmarking: Drafts assertions, runs graders, aggregates metrics into benchmarks (pass rates, time, tokens), surfaces high-variance or nondiscriminating evals, and generates a viewer for human review.
- Improvement Loop & Packaging: Guides iterative revisions, suggests bundling reusable scripts, and packages skills into distributable .skill files.
- Use Case Example: Build a new CSV-cleaning skill, create 3 test prompts, run iteration-1 with both with-skill and baseline runs, grade outputs, and produce a benchmark report for the user to review.
Quick Start
Create a new skill draft, add two realistic test prompts, run paired evaluations (with_skill and without_skill), then open the generated review to inspect outputs and benchmark metrics.
