skills.rest
Adithiya-S
Adithiya-SCOMMUNITY
Explore all skills by this author →
GitHub

vercel-plugin-eval

Runs live eval sessions against the vercel-plugin to verify hook firing, skill injection, and dedup correctness.

Updated Aug 21, 2025
Search Tags:#claude code#plugin testing#wezterm#eval#dedup#coverage report#hook verification
One-click install
npx skills add https://github.com/Adithiya-S/AI-Study-Companion --skill vercel-plugin-eval-adithiya-s
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: vercel-plugin-eval
Source: https://github.com/Adithiya-S/AI-Study-Companion/tree/main/.agents/skills/vercel-plugin-eval
Command: npx skills add https://github.com/Adithiya-S/AI-Study-Companion --skill vercel-plugin-eval-adithiya-s

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Verifying that a Claude Code plugin's hooks actually fire, inject the right skills, and deduplicate correctly requires launching real interactive sessions, which is tedious and error-prone to do manually. This Skill standardizes that eval workflow with exact commands, monitoring queries, and a structured coverage report. ## Core Features & Use Cases - Live Session Launching: Spawns real Claude Code sessions via WezTerm with the plugin installed through npx add-plugin, avoiding non-interactive modes where hooks never fire. - Hook & Dedup Monitoring: Provides grep-based checks for all 8 registered hooks (SessionStart, PreToolUse, UserPromptSubmit, PostToolUse, SessionEnd) and compares skill injections against atomic claim files to validate dedup. - Coverage Reporting: Maps scenario types (AI chat, monorepo, edge auth, payments) to expected skill triggers and produces a COVERAGE.md report with hook matrices, injection tables, and issues found. - Use Case: After modifying the vercel-plugin's hook logic, run a kitchen-sink eval session, confirm all 44 skills trigger with correct dedup, and document regressions before release. ## Quick Start Ask the agent to run a vercel-plugin eval session for an AI chat app scenario and produce a coverage report of hook firing and skill dedup.

Frequently Asked Questions about vercel-plugin-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test that Claude Code plugin hooks actually fire?▼

Launch a real interactive session via WezTerm with the plugin installed using npx add-plugin, then grep the debug log in ~/.claude/debug for SessionStart, PreToolUse, UserPromptSubmit, PostToolUse, and SessionEnd success entries. Hooks do not fire in claude --print or -p mode.

How do I verify skill injection dedup in a Claude Code plugin?▼

Compare the count of skillInjection entries in the debug log (divided by 3, since each injection logs three times) against the number of atomic claim files in the session's seen-skills directory under the OS temp dir. Equal counts confirm correct dedup.

Why don't hooks fire when I run claude with --print?▼

Non-interactive modes like claude --print or -p skip hook execution and create no session files, so plugin behavior cannot be evaluated there. Use a real terminal session spawned through WezTerm with /bin/zsh -ic instead.

How do I trigger hard-to-reach plugin skills during eval?▼

Some skills require explicit technology references in the prompt because agents do not naturally reach for them. For example, say "use the AI Elements component registry" for ai-elements or "install the geist font package" for geist.

What are the limitations of this plugin eval approach?▼

The workflow is tied to a specific local setup: WezTerm, the x alias for claude, zsh, and a macOS-style home directory. It also forbids writing standalone eval scripts, requiring all steps to run as Bash tool calls in the conversation.

Related Skills

Explore More Software Engineering Skills→
sharad07072007sharad07072007

vercel-plugin-eval

Runs live eval sessions against the vercel-plugin to verify hook firing, skill injection, and dedup correctness.

Community
Advanced
sharad07072007sharad07072007

plugin-audit

Audits vercel-plugin skill injection by testing hook matching against Claude Code conversation logs.

Community
Intermediate
ubie-ossubie-oss
16

verify-and-fix

Format, lint, test, and build Go and Terraform provider codebases locally.

Official
Intermediate
vercelvercel
246

vercel-plugin-eval

Run Claude Code sessions to evaluate Vercel plugin hook logs and coverage.

Official
Advanced