skills.rest
openai
openaiOFFICIAL
Explore all skills by this author →
WebsiteGitHub

plugin-eval

Evaluates local Codex skills and plugins with scoring, token budgets, and benchmark setup.

5.3k|765|Updated Mar 4, 2026
Search Tags:#CLI#benchmark#skill evaluation#token budget#CODEX#plugin analysis#Scoring
One-click install
npx skills add https://github.com/openai/plugins --skill plugin-eval
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: plugin-eval
Source: https://github.com/openai/plugins/tree/main/plugins/plugin-eval/skills/plugin-eval
Command: npx skills add https://github.com/openai/plugins --skill plugin-eval

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Engineers building Codex skills and plugins lack a clear way to measure quality, understand scores, and know what to fix first. This Skill routes natural-language requests to the right evaluation workflow and produces actionable reports.

Core Features & Use Cases

  • Skill and Plugin Analysis: Runs plugin-eval analyze to score a local skill or plugin and explain why it scored that way.
  • Token Budget Measurement: Explains static token budget estimates and plans real measured token usage via benchmarks.
  • Benchmark Setup: Initializes .plugin-eval/benchmark.json with starter scenarios and dry-run commands.
  • Use Case: You ask "Give me an analysis of the game dev skill" and the Skill resolves the path, runs the analysis report, initializes a benchmark, and shows the setup questions to tailor scenarios.

Quick Start

Ask the assistant to evaluate this skill and explain what to fix first, pointing it at your local skill or plugin directory.

Frequently Asked Questions about plugin-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate a local Codex skill or plugin?▼

Run `plugin-eval analyze <path> --format markdown` against the skill or plugin directory to get a scored report. You can also use `plugin-eval start <path> --request "Evaluate this skill"` to route a natural-language request automatically.

How do I measure the real token usage of a skill?▼

Initialize a benchmark with `plugin-eval init-benchmark <path>`, then run the benchmark flow and use `plugin-eval measurement-plan` to plan measured token usage. Reports distinguish static budget estimates from measured harness results.

Can I evaluate a skill by name instead of a path?▼

Yes. The workflow checks `~/.codex/skills/<skill-name>` first, then any repo-local `skills/<skill-name>` directory. If the name is still ambiguous, it asks one short clarifying question before continuing.

What should I do after getting an evaluation score?▼

The report leads with Fix First and Recommended Next Step sections so you know the highest-impact change. For rewrite help it routes to the improve-skill skill, and for custom rubrics it routes to metric-pack-designer.

What is the difference between evaluating a skill and a plugin?▼

Skill-specific questions hand off to the evaluate-skill workflow, while plugin bundle questions hand off to evaluate-plugin. The plugin-eval entrypoint resolves the target type first and routes accordingly.

Related Skills

Explore More Software Engineering Skills→
haryelramalhoharyelramalho

plugin-creator

Scaffold and validate Codex plugin directories, manifests, and marketplace entries.

Community
Advanced
AlejfanjulAlejfanjul

prompt-optimization-analyzer

Diagnose skill prompts and generate rewrite recommendations for token efficiency.

Community
Advanced
atholaathola
326

evaluation-framework

Evaluate artifacts using weighted criteria and threshold-based decision rules.

Community
Basic
adriannoesadriannoes
46

autoresearch

Run iterative mutations and binary scoring to refine Claude Code skill prompts.

Community
Advanced