skills.rest
vercel-labs
vercel-labsOFFICIAL
Explore all skills by this author →
Website𝕏@vercelGitHub

benchmark-e2e

Inject AI skills into Vercel projects and analyze JSONL session logs for improvement reports.

246|42|Updated Mar 4, 2026
Search Tags:#automation#testing#e2e#skills#vercel#benchmark
One-click install
npx skills add https://github.com/vercel-labs/vercel-plugin --skill benchmark-e2e
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: benchmark-e2e
Source: https://github.com/vercel-labs/vercel-plugin/tree/main/.claude/skills/benchmark-e2e
Command: npx skills add https://github.com/vercel-labs/vercel-plugin --skill benchmark-e2e

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates the rigorous testing and validation of AI skills within the Vercel ecosystem, ensuring reliable performance and identifying areas for improvement.

Core Features & Use Cases

  • End-to-End Testing: Runs realistic projects through skill injection and verifies functionality.
  • Performance Analysis: Analyzes conversation logs to generate improvement reports.
  • Self-Improvement Loop: Facilitates a closed feedback cycle for continuous skill enhancement.
  • Use Case: Developers can automatically run a comprehensive suite of tests against new or updated Vercel plugin skills, receiving detailed reports on performance and actionable recommendations for optimization.

Quick Start

Execute the full benchmark suite by running bun run scripts/benchmark-e2e.ts.

Frequently Asked Questions about benchmark-e2e

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate end-to-end testing for Vercel plugin skills?▼

End-to-end testing for Vercel plugin skills is automated by injecting skills into realistic projects, verifying dev server functionality, and analyzing JSONL session logs to produce improvement reports.

What is the best way to benchmark AI skills in realistic projects?▼

Benchmarking AI skills is best achieved by running `bun run scripts/benchmark-e2e.ts`, which automatically injects skills via `claude --print` and validates their functionality in project environments.

How does conversation log analysis work for AI skill improvement loops?▼

Conversation log analysis for skill improvement works by parsing JSONL session outputs to evaluate performance, identify functional gaps, and generate actionable reports for continuous self-improvement loops.

Do I need Bun to run automated e2e benchmarking scripts?▼

Bun is required to execute the automated e2e benchmarking scripts, specifically through the `bun run scripts/benchmark-e2e.ts` command to initiate the full project validation suite.

Can I verify dev server functionality automatically after skill injection?▼

Dev server functionality is verified automatically after skill injection, ensuring the injected AI skills operate correctly within the realistic project environment without manual intervention.

What are the limitations of automated skill benchmarking?▼

Automated skill benchmarking limitations include its strict dependency on creating realistic projects, requiring successful `claude --print` injection, and needing valid JSONL session logs to generate accurate improvement reports.

Related Skills

Explore More Software Engineering Skills→
ccstudentccccstudentcc

skill-creator

Create, modify, and benchmark Claude skills with SKILL.md workflows.

Community
Advanced
karaabd23-cryptokaraabd23-crypto
1

session-logs

Parse JSONL session logs with jq to extract and summarize historical user messages.

Community
Intermediate
NodeOps-appNodeOps-app
2

vercel-to-createos

Migrate Vercel projects to CreateOS via MCP server deployments.

Official
Advanced
qcmuuqcmuu

evolving-ai-agents

Evolve AI agents across prompts, skills, and memory using LLM-guided mutation cycles.

Community
Advanced