skills.rest
Adithiya-S
Adithiya-SCOMMUNITY
Explore all skills by this author →
GitHub

benchmark-testing

Creates isolated test projects and launches Claude Code sessions to benchmark vercel-plugin skill injection.

Updated Aug 21, 2025
Search Tags:#automation#testing#claude code#plugin#benchmark#wezterm#skill injection
One-click install
npx skills add https://github.com/Adithiya-S/AI-Study-Companion --skill benchmark-testing-adithiya-s
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: benchmark-testing
Source: https://github.com/Adithiya-S/AI-Study-Companion/tree/main/.agents/skills/benchmark-testing
Command: npx skills add https://github.com/Adithiya-S/AI-Study-Companion --skill benchmark-testing-adithiya-s

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Manually verifying that a plugin's skill injection triggers correctly across diverse scenarios is slow and inconsistent. This Skill automates the creation of realistic benchmark projects and launches parallel Claude Code sessions with crafted prompts to exercise skill injection at scale. ## Core Features & Use Cases - Isolated Test Project Setup: Creates numbered test directories and installs the vercel-plugin into each via npx add-plugin, generating per-project .claude/settings.json configurations. - Parallel Session Launching: Spawns WezTerm panes running Claude Code with precise flags (--cwd, --settings, unset CLAUDECODE) to avoid nested-session errors and system overload. - Prompt Engineering Guidelines: Provides technology-agnostic prompt templates mapped to expected skill triggers (auth, payments, cron-jobs, ai-sdk) so injection behavior can be validated against expectations. - Use Case: After modifying the vercel-plugin's skill routing logic, run this Skill to spin up nine diverse product scenarios (recipe platform, trivia game, status page) and visually confirm the correct skills are injected in each session. ## Quick Start Ask the agent to set up benchmark test projects for the vercel-plugin and launch Claude Code sessions in WezTerm panes using the provided prompt table.

Frequently Asked Questions about benchmark-testing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test vercel-plugin skill injection across multiple scenarios?▼

Create isolated test directories, install the plugin in each with npx add-plugin, then launch Claude Code sessions with crafted prompts describing products without naming technologies. Compare injected skills against the expected skills listed in the prompt table.

How to launch Claude Code in WezTerm panes without nested-session errors?▼

Run unset CLAUDECODE before invoking Claude Code, and use wezterm cli spawn with --cwd set to the absolute project path. Pass --settings .claude/settings.json so the plugin configuration loads correctly.

Why should benchmark prompts avoid naming specific technologies?▼

Naming technologies like Next.js or Stripe bypasses the plugin's inference logic, defeating the test's purpose. Describing only the product and features forces the plugin to infer which skills to inject, validating its actual routing behavior.

Why does Claude Code fail to load the plugin in test directories?▼

The plugin only loads when Claude Code is started with --settings pointing to the .claude/settings.json file created by npx add-plugin. Using --settings project or omitting the flag prevents the enabledPlugins configuration from being read.

What are the limitations of launching many WezTerm panes?▼

The split-pane approach runs out of terminal space after roughly four panes, so spawn (new tabs) must be used instead. Launches should also be staggered by about 10 seconds to avoid overwhelming the system.

Related Skills

Explore More Software Engineering Skills→
sharad07072007sharad07072007

benchmark-testing

Creates isolated test projects and launches WezTerm panes running Claude Code to benchmark vercel-plugin skill injection.

Community
Intermediate
mauromeddamauromedda
13

improve-skill

Analyze Claude Code session transcripts to generate structured skill improvement prompts.

Community
Intermediate
vercel-labsvercel-labs
246

vercel-plugin-eval

Run live Claude Code eval sessions for the vercel-plugin and generate coverage reports.

Official
Advanced
dotnetdotnet
6.2k

cli-e2e-testing

Automate Aspire CLI end-to-end tests using Hex1b terminal scripting.

Official
Advanced