Draft, evaluate, and benchmark SKILL.md-driven workflows with bundled scripts.
npx skills add https://github.com/dy9759/SkillCollection --skill skill-creator-dy9759
Please help me install this Agent Skill. Skill: skill-creator Source: https://github.com/dy9759/SkillCollection/tree/main/skills/skill-creator Command: npx skills add https://github.com/dy9759/SkillCollection --skill skill-creator-dy9759
This Skill removes friction from building, testing, and improving Claude-compatible skills by providing a repeatable workflow for drafting SKILL.md, generating test prompts, running baseline vs. with-skill evaluations, and producing human-readable benchmark reports.
Use the skill-creator to draft or update a SKILL.md, create a small eval set, run the eval loop, and open the generated review in your browser.
High-intent search queries and answers about installing and using this skill.
To create and benchmark AI skills, use the skill-creator workflow to draft a SKILL.md, generate test prompts, and run baseline comparisons. It captures timing and token metrics, grades outputs, and aggregates pass rates into human-readable benchmark reports for document transforms and data extraction tasks.
Evaluating AI skills through baseline comparisons works by running parallel eval runs with and without the skill active. The bundled run_eval.py script captures token usage and timing metrics, then grades the outputs to calculate performance deltas for your test prompts.
The best way to automate iterative improvement of a SKILL.md file is using the run_loop.py and improve_description utilities. These scripts orchestrate repeated eval runs, apply description optimizations, and measure benchmark deltas to refine skill behavior automatically.
Yes, you need pyyaml installed to run skill evaluation and benchmarking scripts. It is the only required dependency for the skill-creator, ensuring the bundled Python scripts can parse SKILL.md frontmatter and process benchmark configurations correctly.
Yes, you can generate a review HTML and benchmark.json for stakeholder presentations. The aggregate_benchmark.py script aggregates pass rates and timing metrics, producing human-readable review artifacts and structured JSON outputs directly from your eval runs.
Limitations of blind baseline comparisons for tool-driven automations include the need for realistic test prompts to yield meaningful deltas. The evaluation mechanism grades outputs based on predefined criteria, so poorly defined test sets may not accurately reflect skill performance.
VjayRamDefine DSPy signatures and modules with deterministic evaluation across splits.
xg-gh-25Generate single-file HTML artifacts with inline CSS from AI outputs.
gsd-buildCreate standardized GSD Skill units with YAML frontmatter and XML body structure.
CyapstayeBenchmark large language models across academic and reasoning tasks.
