skills.rest
openai
openaiOFFICIAL
Explore all skills by this author →
WebsiteGitHub

evaluate-plugin

Evaluates local Codex plugins and generates prioritized fix reports.

5.3k|765|Updated Mar 4, 2026
Search Tags:#audit#CLI#benchmark#Report#plugin evaluation#Code quality#codex plugin
One-click install
npx skills add https://github.com/openai/plugins --skill evaluate-plugin
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluate-plugin
Source: https://github.com/openai/plugins/tree/main/plugins/plugin-eval/skills/evaluate-plugin
Command: npx skills add https://github.com/openai/plugins --skill evaluate-plugin

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Plugin authors lack a structured way to audit a local Codex plugin's manifest, nested skills, and code coverage, making it hard to know what to fix first or how versions compare over time.

Core Features & Use Cases

  • Plugin Analysis: Runs plugin-eval analyze against a plugin root containing .codex-plugin/plugin.json and surfaces a prioritized "Fix First" section before detailed findings.
  • Benchmarking and Comparison: Initializes starter benchmarks, runs dry-run benchmarks, and compares two JSON outputs to track score trends across versions.
  • Report Generation: Exports evaluation results as Markdown or HTML reports for sharing with a team.
  • Use Case: Before publishing a plugin, run an evaluation to see the strongest and weakest skills, fix the top issues, then compare the new JSON output against the previous run to confirm improvement.

Quick Start

Ask the assistant to evaluate this plugin by pointing it at the plugin root directory containing the .codex-plugin/plugin.json manifest.

Frequently Asked Questions about evaluate-plugin

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate a local Codex plugin?▼

Run plugin-eval analyze <plugin-root> --format markdown against the plugin root containing .codex-plugin/plugin.json. Read the Fix First section before drilling into manifest, skill, and coverage findings.

How do I compare two plugin evaluation results?▼

Use plugin-eval compare before.json after.json with two saved JSON outputs. This shows score trends between versions so you can verify whether fixes improved the evaluation.

What directory structure does plugin evaluation require?▼

The target must be a plugin root containing a .codex-plugin/plugin.json manifest. Plugins with multiple skills are supported, and the evaluation summarizes the strongest and weakest skills explicitly.

Can I generate an HTML report from a plugin evaluation?▼

Yes, run plugin-eval report result.json --format html --output ./plugin-eval-report.html to convert a saved JSON result into a shareable HTML report.

How do I benchmark a Codex plugin before running it fully?▼

Run plugin-eval init-benchmark <plugin-root> to scaffold a starter benchmark, then plugin-eval benchmark <plugin-root> --dry-run to preview measured usage without executing the full benchmark.

Related Skills

Explore More Software Engineering Skills→
erichugyerichugy

plugin-creator

Create and scaffold Codex plugin directories and plugin.json manifests.

Community
Intermediate
albertms22albertms22

plugin-forge

Automate Claude Code plugin scaffolding, versioning, and marketplace publishing.

Community
Intermediate
manifoldlogicmanifoldlogic

eleven-category-test-structure

Organize iTerm plugin shell script tests into 11 categories.

Official
Advanced
netalertxnetalertx
6.9k

netalertx-plugin-run-development

Create, run, and test NetAlertX plugins with script.py and config.json.

Official
Intermediate