skills.rest
viktorbezdek
viktorbezdekCOMMUNITY
Explore all skills by this author →
𝕏@viktorbezdekGitHub

plugin-evaluation

Run trigger-evals and output-evals to report plugin activation pass rates.

10|2|Updated Mar 12, 2026
Search Tags:#offline#plugin#activation#harness#evaluate#evals
One-click install
npx skills add https://github.com/viktorbezdek/skillstack --skill plugin-evaluation
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: plugin-evaluation
Source: https://github.com/viktorbezdek/skillstack/tree/main/plugin-dev/skills/plugin-evaluation
Command: npx skills add https://github.com/viktorbezdek/skillstack --skill plugin-evaluation

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Measures whether a Claude Code plugin activates reliably and produces correct outputs by running trigger-evals and output-evals, enabling teams to validate activation, iterate on frontmatter, and catch coverage gaps before shipping.

Core Features & Use Cases

Two eval types are supported: Trigger evals (does the model invoke the skill) and Output evals (does the produced result meet expectations), all orchestrated by a lightweight harness that supports offline smoke tests, CI workflows, and incremental iteration.

Quick Start

Run the evaluation harness against the plugin-evaluation skill using the plugin-dev/scripts/run_eval.py script with the proper --plugin-dir and --skill options.

Frequently Asked Questions about plugin-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate whether a Claude Code plugin activates reliably?▼

To evaluate plugin activation, run trigger-evals and output-evals through a lightweight harness that checks whether the model invokes the skill and whether produced results meet expectations, reporting pass rates.

What is the difference between trigger evals and output evals?▼

Trigger evals measure whether the model successfully invokes the skill, while output evals measure whether the produced result meets expectations, both orchestrated by the harness to catch coverage gaps.

How do I set up offline smoke tests for plugin evaluation?▼

Offline smoke tests are supported by the harness workflow; provide a SKILL.md with name and description, create an evals directory containing trigger-evals.json and evals.json, and run the harness script.

Can I use the evaluation harness in CI workflows?▼

Yes, the harness workflow supports CI workflows, enabling teams to validate activation, iterate on frontmatter, and catch coverage gaps before shipping the plugin.

What files do I need to run plugin evals?▼

You need a SKILL.md frontmatter with name and description, an evals directory containing trigger-evals.json and evals.json, plus optional references or scripts resources consumed by the harness.

Why is my Claude Code plugin not activating on expected trigger phrases?▼

Plugin activation failures can be diagnosed by running trigger-evals through the harness, which measures whether the model invokes the skill and reports pass rates to identify frontmatter or trigger phrase gaps.

Related Skills

Explore More Software Engineering Skills→
richfremrichfrem
5

create-skill

Scaffold Agent Skill directories with SKILL.md frontmatter and evals.json.

Community
Advanced
anymouschinaanymouschina
523

skill-creator

Draft, refine, and evaluate Claude Skills with SKILL.md and evals.

Community
Advanced
Teng-bioTeng-bio
2

minimal-run-and-audit

Automate non-training command execution and generate standardized output files.

Community
Advanced
stopitdanstopitdan

run-evals

Run the recommendation engine evaluation suite and summarize pass rates, NDCG, and failures.

Community
Advanced