zz-plugin-define

Derives plugin-specific evaluation rubrics and gates scoring on human approval.

Updated Sep 11, 2026
One-click install
npx skills add https://github.com/zhixuan312/zz-stack --skill zz-plugin-define-zhixuan312
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: zz-plugin-define
Source: https://github.com/zhixuan312/zz-stack/tree/main/catalog/zz/zz-plugin-eval/skills/zz-plugin-define
Command: npx skills add https://github.com/zhixuan312/zz-stack --skill zz-plugin-define-zhixuan312

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Evaluating plugins with a shared generic rubric produces meaningless scores, and deciding pass/fail thresholds after seeing results invites bias. This Skill guides you through deriving what 'good' means for a specific plugin from its own profile, writing it into a gated rulers.md document, and getting a person to approve it before any scoring begins. ## Core Features & Use Cases - Custom rubric derivation: Builds per-plugin rulers with qualitative dimensions (every level named, low end first) and quantitative dimensions (tool-computed facts with human-set thresholds and stated reasons). - Registry validation: Uses ruler_record and ruler_affirm to reject dimensions with missing levels, empty thresholds, or figures that do not exist on the plugin's facts sheet. - Approval gating: Writes rulers.md via document_write, presents it with document_present, and records human agreement with document_approve before judging can start. - Use Case: After profiling a delivery-method plugin, you define a 'document depth' qualitative dimension with five named levels and a 'tool fit' quantitative threshold over never_called counts, then get stakeholder sign-off before the judge scores any artifact. ## Quick Start Run the zz-plugin-define stage to derive evaluation rulers for this plugin from its profile and write them into rulers.md for approval.

Frequently Asked Questions about zz-plugin-define

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I define evaluation criteria for a plugin before scoring it?▼

Read the plugin's profile with ruler_read, then write a rulers.md containing qualitative dimensions with every level named and quantitative dimensions with explicit thresholds and reasons. Record it with ruler_record and get human approval via document_approve before judging starts.

What is the difference between qualitative and quantitative rubric dimensions?▼

Qualitative dimensions name every level of a scale and are marked by a judge reading artifact text. Quantitative dimensions set a threshold over a tool-computed fact, where the tool supplies the number and a person decides where the line is.

Why does ruler_record refuse a quantitative dimension?▼

ruler_record refuses a quantitative dimension with no threshold, no stated reason, or a reads path naming a figure missing from the plugin's facts sheet. An unanswerable line scores as a failure the plugin never earned, so the gate rejects it upfront.

Can I reuse a rubric from a similar plugin?▼

No. Each plugin gets its own ruler derived from its own profile, because a delivery method and a credential surface measure different things. Figures also differ per plugin, so a line valid on one sheet may be unanswerable on another.

When should a rule be enforced by the tool instead of measured?▼

If a property should be impossible to violate rather than merely checked, enforce it in the tool instead of writing a threshold. A rule enforced at the door is true by construction and needs no line, figure, or judge.