skills.rest
dotnet
dotnetOFFICIAL
Explore all skills by this author →
Website𝕏@dotnetfdnGitHub

improve-skill-quality

Diagnoses failing skill evaluations and applies targeted fixes to skills, fixtures, or eval specs.

5.3k|403|Updated Feb 3, 2026
Search Tags:#activation#diagnose#skill evaluation#statistical power#baseline comparison#DotNet#eval triage
One-click install
npx skills add https://github.com/dotnet/skills --skill improve-skill-quality
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: improve-skill-quality
Source: https://github.com/dotnet/skills/tree/main/.agents/skills/improve-skill-quality
Command: npx skills add https://github.com/dotnet/skills --skill improve-skill-quality

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

When a skill evaluation in the dotnet/skills repository returns a regression, an underpowered verdict, a timeout, or "no credible improvement", the most common mistake is rewriting skill prose when the real cause is the eval spec, the fixtures, or the harness. This Skill enforces a classify-first workflow so you fix the actual root cause instead of editing content blindly.

Core Features & Use Cases

  • Failure classification: A symptom-to-cause table routes verdicts to the right fix path — fixture defects, harness/reliability failures, statistical power limits, eval design flaws, skill content, or activation/routing problems.
  • Statistical power analysis: Explains the two-bar gate (≥5 distinct stimuli and p ≤ 0.05 sign test over discordant votes) so you know when to add discriminating stimuli instead of padding runs.
  • Content and activation fixes: Reference guides cover writing skill content that beats the unskilled baseline and repairing frontmatter descriptions that fail to route.
  • Use Case: A PR's /evaluate comment reports 16W/8T/1L but the verdict fails. Use this Skill to determine the ties made a pass statistically unreachable, then add discriminating stimuli rather than rewriting the skill.

Quick Start

Ask the agent to diagnose why the latest /evaluate run for your skill reported a regression and recommend the correct fix.

Frequently Asked Questions about improve-skill-quality

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fix a skill that lost to its baseline in evaluation?▼

First classify the failure using the symptom table before editing content. Quote a losing trial and the judge's stated reason, then apply content patterns like decision tables, concrete output contracts, and stop-conditions from the writing-for-baseline-delta reference.

Why did my evaluation fail with a positive win-loss record?▼

A positive record with a failing verdict is a statistical power problem, not a content problem. The gate requires at least 5 distinct stimuli and p ≤ 0.05 on a sign test over discordant votes, so ties can make a pass unreachable. Add discriminating stimuli rather than raising runs.

What does an underpowered verdict mean in skill evaluation?▼

Underpowered means the eval has fewer than 5 distinct stimuli, so it can never report a pass or regression regardless of results. Fix it by adding stimuli that exercise new behavior, not by increasing repeated runs, which only measure reliability.

Why is my skill not activated during evaluation?▼

Activation failures are frontmatter description problems, not body problems. Put the user's own words in the description — symptoms, error codes, artifact names — claim ambiguous routing words against sibling skills, and check both isolated and plugin arm activation.

When should I not use this skill?▼

Do not use it to scaffold a brand-new skill (use create-skill) or a new eval.yaml (use create-skill-test), and do not use it to change the harness itself such as eng/skill-validator or the evaluation workflows.

Related Skills

Explore More Software Engineering Skills→
T-ravT-rav
5

skill-stocktake

Analyze Claude SKILL.md definitions and generate per-skill verdicts with results.json.

Community
Advanced
mmartinsolivmmartinsoliv

pr-review-pending

Create pending GitHub pull request review comments from code review findings.

Community
Advanced
IPADS-SAIIPADS-SAI
44

skill-creator

Create and improve Claude skills with tests and evaluation feedback.

Official
Advanced
constellizeconstellize
1

retire-memory-bank

Archive outdated memory-bank entries and document their rationale with cross-references.

Official
Advanced