evaluation-methodology

Explains PluginEval scoring dimensions, rubrics, composite formulas, and badge thresholds for plugin quality evaluation.

Updated Apr 23, 2026
One-click install
npx skills add https://github.com/SanketAdlak/PDMProjectDesign --skill evaluation-methodology-sanketadlak
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: evaluation-methodology
Source: https://github.com/SanketAdlak/PDMProjectDesign/tree/main/.agents/skills/evaluation-methodology
Command: npx skills add https://github.com/SanketAdlak/PDMProjectDesign --skill evaluation-methodology-sanketadlak

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Plugin and skill authors often receive quality scores without understanding how those numbers were computed, which dimensions matter most, or how to improve a low grade. This Skill provides the authoritative reference for the PluginEval methodology so you can interpret reports, prioritize fixes, and calibrate thresholds. ## Core Features & Use Cases - Three-Layer Evaluation Model: Documents static analysis, LLM judge, and Monte Carlo simulation layers with per-dimension blend weights. - Composite Scoring & Badges: Explains the ten weighted dimensions, anti-pattern penalties, letter grades, and Bronze-to-Platinum badge thresholds including Elo ranking. - Improvement Guidance: Provides weight-ordered fix priorities, anti-pattern remediation, and troubleshooting for common scoring surprises. - Use Case: A skill receives a D in triggering_accuracy. Use this Skill to learn that the description needs a "Use this skill when..." phrase with three or more concrete trigger contexts, then verify the fix with a quick-depth rescore. ## Quick Start Ask the assistant to explain why a skill scored low on orchestration fitness and what changes would raise that dimension's grade.

Frequently Asked Questions about evaluation-methodology

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How is the PluginEval composite score calculated?

The composite score sums each dimension score multiplied by its weight, scales by 100, then applies an anti-pattern penalty of max(0.5, 1.0 minus 0.05 per flag). Dimension scores blend static, judge, and Monte Carlo layers using per-dimension weights.

How do I improve a low triggering accuracy score?

Rewrite the frontmatter description to include the exact phrase "Use this skill when" followed by three or more specific, comma-separated trigger contexts. Test with five prompts that should trigger and five that should not to check discrimination.

What is the difference between quick, standard, and deep evaluation depth?

Quick depth runs only static analysis in under two seconds. Standard adds the LLM judge, taking 30 to 90 seconds. Deep depth adds Monte Carlo simulation with 50 invocations and Elo ranking, taking 15 to 20 minutes via the certify command.

Why did my score drop after adding more content to SKILL.md?

Additional content likely triggered anti-pattern flags such as OVER_CONSTRAINED or BLOATED_SKILL, and the penalty multiplies the whole score. Run with JSON output and inspect layers[0].anti_patterns, then move detail into a references directory.

What badge thresholds does PluginEval use?

Platinum requires composite 90 or higher and Elo 1600 or higher; Gold needs 80 and 1500; Silver needs 70 and 1400; Bronze needs 60 and 1300. Below 60 earns no badge, and the Elo check is skipped when Elo has not been computed.

When should I not rely on quick-depth scoring?

Quick depth only runs static analysis, so it cannot judge output quality or detect orchestration problems, and rewrites that move content to references may score lower statically. Use standard depth for fair comparisons that include the LLM judge.