skill-creator

Create, test, and optimize AI skills with benchmarking workflows.

1|Updated Feb 19, 2025
One-click install
npx skills add https://github.com/trystan2k/favoritable-old --skill skill-creator-trystan2k
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/trystan2k/favoritable-old/tree/main/.agents/skills/skill-creator
Command: npx skills add https://github.com/trystan2k/favoritable-old --skill skill-creator-trystan2k

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires yaml, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

Building effective AI skills is hit-or-miss without systematic testing. Skills may fail to trigger, produce inconsistent outputs, or have descriptions that don't match real user queries. This skill provides the complete workflow for creating, evaluating, and refining skills with quantitative rigor.

Core Features & Use Cases

  • End-to-End Skill Creation: Guide users from intent capture through draft, testing, and iteration for both new and existing skills.
  • Quantitative Benchmarking: Run controlled experiments comparing with-skill vs without-skill performance, tracking pass rates, timing, and token usage with statistical variance analysis.
  • Description Optimization: Automatically tune skill descriptions using train/test eval loops to improve triggering accuracy across diverse user queries.
  • Use Case: A developer building a code review skill can capture their review workflow, draft the skill instructions, generate test cases from real PRs, run blind comparisons against baseline behavior, and optimize the description so Claude reliably invokes it when users ask for reviews.

Quick Start

Use the skill-creator skill to build a new skill for automating code review workflows by capturing your review process, drafting the SKILL.md instructions, and running benchmark tests to measure its triggering accuracy and output quality.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test and benchmark AI skills for consistent triggering?

You can test and benchmark AI skills by running controlled experiments that compare with-skill and without-skill performance, tracking pass rates, timing, and token usage with statistical variance analysis.

What is the best way to optimize skill descriptions for better prompt engineering?

The best way to optimize skill descriptions is using automated train and test evaluation loops that tune triggering accuracy across diverse user queries, ensuring reliable invocation based on real search intent.

How do I create an end-to-end AI skill from a workflow intent?

Creating an AI skill end-to-end involves capturing workflow intent, drafting instructions, generating test cases from real examples, executing baseline comparisons, and refining descriptions to improve output quality.

Do I need Python tooling to generate skill evaluation reports?

Yes, you need Python tooling to aggregate evaluation data, analyze benchmark results, and generate HTML reports that visualize pass rates and statistical variance for skill testing.

Can I run blind comparisons against baseline behavior for code review skills?

Yes, you can run blind comparisons against baseline behavior by generating test cases from real pull requests, executing controlled experiments, and measuring the skill's impact on output quality.

Why does my AI skill fail to trigger on matching user queries?

AI skills fail to trigger when descriptions do not match real user queries, which you can fix by running description optimization loops that automatically tune triggering accuracy using train and test evaluations.