plugin-evaluation

Run trigger-evals and output-evals to report plugin activation pass rates.

10|2|Updated Mar 12, 2026

Search Tags:#offline #plugin #activation #harness #evaluate #evals

Ready to run

npx skills add https://github.com/viktorbezdek/skillstack --skill plugin-evaluation

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: plugin-evaluation
Source: https://github.com/viktorbezdek/skillstack/tree/main/plugin-dev/skills/plugin-evaluation
Command: npx skills add https://github.com/viktorbezdek/skillstack --skill plugin-evaluation

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Measures whether a Claude Code plugin activates reliably and produces correct outputs by running trigger-evals and output-evals, enabling teams to validate activation, iterate on frontmatter, and catch coverage gaps before shipping.

Core Features & Use Cases

Two eval types are supported: Trigger evals (does the model invoke the skill) and Output evals (does the produced result meet expectations), all orchestrated by a lightweight harness that supports offline smoke tests, CI workflows, and incremental iteration.

Quick Start

Run the evaluation harness against the plugin-evaluation skill using the plugin-dev/scripts/run_eval.py script with the proper --plugin-dir and --skill options.

Frequently Asked Questions about plugin-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I evaluate whether a Claude Code plugin activates reliably?▼

To evaluate plugin activation, run trigger-evals and output-evals through a lightweight harness that checks whether the model invokes the skill and whether produced results meet expectations, reporting pass rates.

What is the difference between trigger evals and output evals?▼

Trigger evals measure whether the model successfully invokes the skill, while output evals measure whether the produced result meets expectations, both orchestrated by the harness to catch coverage gaps.

How do I set up offline smoke tests for plugin evaluation?▼

Offline smoke tests are supported by the harness workflow; provide a SKILL.md with name and description, create an evals directory containing trigger-evals.json and evals.json, and run the harness script.

Can I use the evaluation harness in CI workflows?▼

Yes, the harness workflow supports CI workflows, enabling teams to validate activation, iterate on frontmatter, and catch coverage gaps before shipping the plugin.

What files do I need to run plugin evals?▼

You need a SKILL.md frontmatter with name and description, an evals directory containing trigger-evals.json and evals.json, plus optional references or scripts resources consumed by the harness.

Why is my Claude Code plugin not activating on expected trigger phrases?▼

Plugin activation failures can be diagnosed by running trigger-evals through the harness, which measures whether the model invokes the skill and reports pass rates to identify frontmatter or trigger phrase gaps.