benchmark-agents

Runs interactive Claude Code eval sessions via WezTerm to verify Vercel plugin skill injection.

Updated Sep 5, 2026
One-click install
npx skills add https://github.com/dsgalkar/dnyaneshwar_portfolio --skill benchmark-agents-dsgalkar
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: benchmark-agents
Source: https://github.com/dsgalkar/dnyaneshwar_portfolio/tree/main/.agents/plugins/vercel/.claude/skills/benchmark-agents
Command: npx skills add https://github.com/dsgalkar/dnyaneshwar_portfolio --skill benchmark-agents-dsgalkar

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Testing whether a Claude Code plugin correctly injects skills and fires hooks requires real interactive sessions, since print-mode runs never trigger PreToolUse, PostToolUse, or UserPromptSubmit hooks. This Skill provides the exact commands, monitoring queries, and verification checks to run reliable end-to-end evals of the Vercel plugin. ## Core Features & Use Cases - Interactive Eval Launch: Spawns WezTerm panes running Claude Code sessions with the plugin installed via npx add-plugin, using timestamped directories and debug logging. - Injection Monitoring: Checks skill claim directories and debug logs to count injected skills, hook firings, and PostToolUse validation catches. - Code Verification: Greps generated projects for correct patterns such as withWorkflow config, AI Gateway usage, gemini image models, and AI Elements components. - Use Case: Run three parallel eval sessions with natural-language prompts (e.g., a durable content pipeline), then produce a coverage report showing which of the 43 plugin skills triggered and which code patterns need fixing before release. ## Quick Start Launch three benchmark eval sessions for the Vercel plugin using the tarot-deck, interior-designer, and superhero-origin scenarios and report which skills were injected.

Frequently Asked Questions about benchmark-agents

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test whether a Claude Code plugin injects skills correctly?

Launch an interactive Claude Code session in a WezTerm pane with the plugin installed via npx add-plugin, then inspect the session's claim directory in the temp folder. Each injected skill appears as a file in the vercel-plugin-<session-id>-seen-skills.d directory.

Why doesn't claude --print work for testing plugin hooks?

Print mode generates text without executing tools, so PreToolUse, PostToolUse, and UserPromptSubmit hooks never fire. No files are created, no session_id exists, and dedup, profiler, and claim files do not work.

How do I run multiple Claude Code eval sessions in parallel?

Create timestamped directories, install the plugin in each with npx add-plugin, then spawn a separate WezTerm pane per session using wezterm cli spawn with /bin/zsh -ic. Each pane runs an independent interactive session with its own debug log.

How do I check which hooks fired during a Claude Code session?

Grep the session's debug log in ~/.claude/debug for patterns like SessionStart success, executePreToolHooks, provided additionalContext, and VALIDATION. Setting VERCEL_PLUGIN_LOG_LEVEL=debug at launch enables this output.

What prompt style triggers Vercel plugin skills during evals?

Describe products in natural language rather than naming technologies, since the plugin matches prompt signals like multi-step pipeline or streams progress. Explicitly naming SDKs or config options bypasses the pattern matching being tested.