benchmark-sandbox

Automate sandboxed deployment benchmarks by orchestrating a three-phase BUILD, VERIFY, and DEPLOY evaluation pipeline inside ephemeral Vercel Sandboxes using Claude Code. The pipeline collects per-scenario haiku scores and extracts artifacts to measure skill coverage.

A Vercel Sandbox is an ephemeral, isolated environment used for running safe and repeatable AI-enhanced deployment benchmarks. You need it when provisioning fresh environments to install Claude Code, execute dynamic scenarios, and collect structured results without affecting production.

Run the Benchmark Sandbox Runner to spin up ephemeral Vercel Sandboxes, provision lightweight plugins, and execute the three-phase BUILD, VERIFY, and DEPLOY workflow. It manages deterministic sandbox lifecycles to ensure reproducible benchmarks across dynamic scenarios.

Yes, the benchmark sandbox supports dynamic scenarios with haiku-based scoring to measure skill coverage. It operates on these scenarios during the evaluation pipeline and emits structured scores and artifacts for each run.

Yes, you need the @vercel/sandbox dependency to provision fresh sandboxes and manage deterministic sandbox lifecycles. It provides the ephemeral environments required to securely install Claude Code and orchestrate the evaluation phases.

The primary limitation of ephemeral sandbox evaluations is managing deterministic sandbox lifetimes, plugin provisioning, and secure auths. You must ensure safe, repeatable runs by carefully managing these lifecycles to avoid premature termination before artifact extraction completes.