benchmark-sandbox

Name: benchmark-sandbox
Availability: InStock
Author: Vercel

Automate end-to-end sandbox-based AI benchmarking for Vercel plugins.

246|42|Updated Mar 4, 2026

Search Tags:#automation #benchmark #claude-code #sandbox #haiku #vercel-plugin

Ready to run

npx skills add https://github.com/vercel/vercel-plugin --skill benchmark-sandbox-vercel

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: benchmark-sandbox
Source: https://github.com/vercel/vercel-plugin/tree/main/.claude/skills/benchmark-sandbox
Command: npx skills add https://github.com/vercel/vercel-plugin --skill benchmark-sandbox-vercel

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires @vercel/sandbox.

What problem does it solve?

Run vercel-plugin eval scenarios in Vercel Sandboxes instead of local WezTerm panels. Provisions ephemeral microVMs with Claude Code + plugin pre-installed, runs benchmark prompts, extracts hook artifacts, and produces coverage reports.

Core Features & Use Cases

Ephemeral sandboxes: spin up Firecracker microVMs to run evaluation prompts without local setup.
3-phase eval: BUILD, VERIFY, and DEPLOY phases with artifact extraction and haiku scoring.
Automated scoring: per-phase haiku JSON scoring and per-scenario coverage reports for quick insight.

Quick Start

Run bun run .claude/skills/benchmark-sandbox/run-eval.ts to start benchmarking.

Frequently Asked Questions about benchmark-sandbox

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run sandboxed AI benchmarks for Vercel plugins?▼

Run sandboxed AI benchmarks by provisioning ephemeral Firecracker microVMs with Claude Code, executing evaluation prompts, and extracting hook artifacts. The benchmark-sandbox skill automates this end-to-end pipeline to produce structured scoring and coverage reports.

What is automated 3-phase eval pipeline scoring in Claude Code sandboxes?▼

Automated 3-phase eval scoring runs plugin scenarios through BUILD, VERIFY, and DEPLOY phases inside microVMs. It extracts artifacts from each phase and applies haiku JSON scoring to generate per-scenario coverage reports for analytics.

Does benchmarking Vercel plugins require a local WezTerm setup?▼

Benchmarking Vercel plugins does not require local WezTerm setup. The skill provisions ephemeral Vercel Sandboxes to run evaluation scenarios remotely, eliminating local environment configuration and directly extracting artifacts for scoring.

How do I start an end-to-end sandbox benchmark evaluation?▼

Start an end-to-end sandbox benchmark evaluation by running the bun command on the run-eval TypeScript script. This orchestrates plugin upload, microVM provisioning, sandbox execution, and structured scoring across all eval phases.

What is the best way to score AI plugin artifacts across multiple deployment phases?▼

The best way to score AI plugin artifacts is using a multi-phase eval pipeline that extracts outputs during BUILD, VERIFY, and DEPLOY stages. The skill applies per-phase haiku scoring to generate structured analytics for iteration.

Are there limitations to running ephemeral microVM benchmarks for Claude Code?▼

Ephemeral microVM benchmarks are limited to the Vercel Sandbox environment and require the @vercel/sandbox dependency. Scoring is constrained to the structured BUILD, VERIFY, and DEPLOY phases with haiku-generated JSON outputs.