skill-creator

Create and evaluate AI skills with SKILL.md files and baseline tests.

Updated Jun 26, 2022
One-click install
npx skills add https://github.com/xlsama/use --skill skill-creator-xlsama
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/xlsama/use/tree/main/config/agents/skills/skill-creator
Command: npx skills add https://github.com/xlsama/use --skill skill-creator-xlsama

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill streamlines the entire lifecycle of creating and refining AI capabilities, from initial concept to performance measurement, making it easier to build powerful, reliable AI tools.

Core Features & Use Cases

  • Skill Creation: Guides users through defining intent, interviewing for edge cases, and writing SKILL.md files.
  • Iterative Improvement: Facilitates testing, evaluation, and rewriting of skills based on performance metrics.
  • Performance Benchmarking: Runs skills against test cases, comparing them to baselines and providing detailed analysis.
  • Description Optimization: Automatically refines skill descriptions for better triggering accuracy using an iterative loop.
  • Use Case: A developer wants to create a new skill to summarize meeting transcripts. They use this Skill to draft the SKILL.md, write test cases, run evaluations, analyze the results, and optimize the skill's description for optimal triggering.

Quick Start

Use the skill-creator to help me build a new skill that can summarize meeting transcripts.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create an AI skill from scratch?

To create an AI skill, you define the skill intent, address edge cases, and write a `SKILL.md` file containing YAML frontmatter and markdown instructions, which establishes the core logic and bundles necessary resources.

How can I evaluate and optimize LLM skills for better performance?

You can evaluate LLM skills by running automated tests against defined test cases and comparing results to baselines. This benchmarking provides detailed analysis to facilitate iterative rewriting and performance optimization.

What is the best way to improve AI skill triggering accuracy?

The best way to improve AI skill triggering accuracy is through description optimization. This feature uses an iterative loop to automatically refine skill descriptions, ensuring the skill activates precisely when intended.

Do I need to write test cases manually to benchmark prompt engineering results?

Yes, you need to define test cases to benchmark prompt engineering results effectively. The skill runs these cases against your implementation, comparing outputs to baselines to provide detailed performance analysis and evaluation metrics.

Can I bundle external scripts and references when building AI tools?

Yes, you can bundle external scripts and references when building AI tools. The skill creation process supports packaging these resources alongside the `SKILL.md` file to ensure the AI has all necessary context and capabilities.