skills.rest
Paradicat
ParadicatCOMMUNITY
Explore all skills by this author →
GitHub

skill-creator

Create, modify, and evaluate AI skills with SKILL.md documentation and test cases.

1|1|Updated Jun 15, 2023
Search Tags:#skill development#prompt engineering#code generation#AI agent#evaluation#performance testing
One-click install
npx skills add https://github.com/Paradicat/demo --skill skill-creator-paradicat
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/Paradicat/demo/tree/main/agent_pack/third_party_skills/skills/skills/skill-creator
Command: npx skills add https://github.com/Paradicat/demo --skill skill-creator-paradicat

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill empowers users to create new AI skills from scratch, improve existing ones, and rigorously test their performance, ensuring they are effective and accurate.

Core Features & Use Cases

  • Skill Creation: Guide users through defining a skill's purpose, triggers, and output.
  • Iterative Improvement: Refine skills based on user feedback and performance metrics.
  • Performance Measurement: Run evaluations to benchmark skill accuracy and identify areas for optimization.
  • Use Case: You have an idea for a new AI capability, like summarizing meeting notes. Use this Skill to draft the initial skill, write test cases, evaluate its performance, and iterate until it reliably summarizes notes as intended.

Quick Start

Use the skill creator to help me draft a new skill that can generate commit messages from code changes.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build and test AI agents for prompt engineering?▼

To build and test AI agents, define the skill intent, triggers, and output formats, then draft documentation and establish test cases for deterministic and subjective outputs. This enables iterative refinement through quantitative benchmarks and qualitative feedback.

What is the best way to evaluate AI skill performance and trigger accuracy?▼

Evaluating AI skill performance involves running quantitative benchmarks and qualitative user feedback to measure output accuracy. You can optimize trigger accuracy by iteratively refining the skill description based on these test case results.

Can I use this to generate a SKILL.md file for new code generation skills?▼

Yes, you can use this to generate a SKILL.md file for new code generation skills by guiding you through defining the skill purpose, drafting the documentation, and establishing test cases for reliable outputs.

How do I refine AI skills that are not triggering correctly?▼

Refine AI skills not triggering correctly by iteratively optimizing the skill description based on performance measurement results. Run evaluations to benchmark trigger accuracy and apply qualitative user feedback to improve intent matching.

Do I need to write test cases for subjective AI skill outputs?▼

Yes, you need to write test cases for subjective AI skill outputs to establish evaluation criteria. Setting up these test cases allows you to measure performance, run quantitative benchmarks, and iteratively refine the skill.

Related Skills

Explore More Software Engineering Skills→
slowman2084slowman2084
6

meta-ideal-state

Generate structured ideal_state.md documents defining Agent goals, intents, safety rules, and output templates.

Community
Advanced
amhuppertamhuppert
2

skill-creator

Create and refine Claude Skill units with YAML frontmatter and Markdown instructions.

Community
Intermediate
thechrickenrun-cyberthechrickenrun-cyber

skill-creator

Create, evaluate, and package Claude skills from SKILL.md workflows.

Community
Advanced
vuralserhat86vuralserhat86
43

llm_evaluation

Evaluate LLM applications with automated metrics, human feedback, and benchmarking.

Community
Advanced