skills.rest
sharad07072007
sharad07072007COMMUNITY
Explore all skills by this author →
GitHub

benchmark-agents

Runs interactive Claude Code eval sessions via WezTerm to verify Vercel plugin skill injection.

Updated Sep 10, 2026
Search Tags:#claude code#benchmark#wezterm#eval#skill injection#hook testing#vercel plugin
One-click install
npx skills add https://github.com/sharad07072007/paras --skill benchmark-agents-sharad07072007
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: benchmark-agents
Source: https://github.com/sharad07072007/paras/tree/main/.agents/plugins/vercel/.claude/skills/benchmark-agents
Command: npx skills add https://github.com/sharad07072007/paras --skill benchmark-agents-sharad07072007

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Testing whether a Claude Code plugin correctly injects skills, fires hooks, and produces valid generated code requires real interactive sessions — claude --print and scripted runners do not trigger plugin hooks. This Skill provides the exact, proven procedure for launching, monitoring, and verifying plugin eval sessions end to end. ## Core Features & Use Cases - Interactive eval launch: Spawns WezTerm panes running real Claude Code sessions with the Vercel plugin installed, using exact copy-paste Bash commands and timestamped directory naming. - Injection monitoring: Checks skill claim directories and debug logs to count injected skills, hook firings, and PostToolUse validation catches. - Code verification: Greps generated projects for correct patterns — gateway vs direct provider usage, image model selection, withWorkflow config, and AI Elements installation. - Use Case: Run 12 benchmark scenarios (doc-qa-agent, multi-agent-research, compliance-auditor, etc.) across three complexity tiers, then write a coverage report and feed findings back into skill fixes before release. ## Quick Start Launch three benchmark eval sessions for the Vercel plugin using the tarot-deck, interior-designer, and superhero-origin prompts, then report which skills were injected in each session.

Frequently Asked Questions about benchmark-agents

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test Claude Code plugin skill injection?▼

Install the plugin with npx add-plugin in a timestamped directory, then spawn a WezTerm pane running an interactive Claude Code session with VERCEL_PLUGIN_LOG_LEVEL=debug. Check the session's claim directory in the temp folder to see which skills were injected.

Why doesn't claude --print work for plugin evals?▼

Print mode generates text without executing tools, so PreToolUse, PostToolUse, and UserPromptSubmit hooks never fire. No files are created, no session_id exists, and dedup, profiler, and claim files do not work. Interactive sessions are the only valid method.

How do I check which skills were injected in a session?▼

List the contents of the claim directory at $TMPDIR/vercel-plugin-<session-id>-seen-skills.d. Each file represents one injected skill. You can also grep the debug log in ~/.claude/debug/ for 'provided additionalContext' to count actual injections.

What prompts trigger the workflow skill in evals?▼

Use natural language phrases like 'multi-step pipeline', 'streams progress', 'streams each phase', or 'durable pipeline'. Describe products rather than naming technologies, so the plugin's promptSignals pattern matching is what triggers injection.

Why do eval directory names need lowercase and timestamps?▼

npm rejects uppercase letters in directory names, which breaks create-next-app scaffolding. Appending a timestamp in yyyymmdd-hhmm format prevents reruns from overwriting previous eval projects.

What code patterns should generated Vercel projects be checked for?▼

Verify gateway() or provider/model strings instead of direct openai() calls, gemini image models instead of dall-e-3, withWorkflow in next.config, no src/ directory for Workflow SDK projects, and installed AI Elements components.

Related Skills

Explore More Software Engineering Skills→
Extremez-SuryaExtremez-Surya

benchmark-testing

Run benchmark tests for vercel-plugin skill injection in isolated directories.

Community
Advanced
TbK0ngTbK0ng

improve-ut

Analyze changed files and unit-test specs to update and validate tests.

Community
Intermediate
l395624064l395624064

cloud-functions

Plan, develop, and manage CloudBase cloud functions with correct runtimes and deployment.

Community
Advanced
Prompt-or-Die-LabsPrompt-or-Die-Labs
7

testing-helper

Generate unit, integration, and end-to-end tests for elizaOS plugins, characters, and actions.

Official
Intermediate