What problem does it solve?
Teams using plugin-eval often need evaluation criteria beyond the built-in rubric, but writing schema-compatible custom checks and metrics from scratch is error-prone. This Skill guides the creation of local metric packs that integrate cleanly with the plugin-eval analysis pipeline.
Core Features & Use Cases
- Custom Rubric Design: Clarify rubric categories and target kinds, then define the smallest useful checks[] and metrics[] payload.
- Manifest and Script Generation: Create a metric-pack manifest plus a script that prints JSON to stdout for deterministic evaluation.
- Stable, Comparable Output: Keep IDs stable across runs and emit only checks[], metrics[], and optional artifacts[] without overwriting the core score.
- Use Case: A team wants to enforce internal code-quality rules during plugin evaluation. Use this Skill to build a local metric pack, then run it via plugin-eval analyze with the --metric-pack flag.
Quick Start
Design a custom metric pack for plugin-eval that checks my plugin for our team's naming conventions and emits schema-compatible metrics.
