skills.rest
sanjanb
sanjanbCOMMUNITY
Explore all skills by this author →
GitHub

benchmark-models

Runs identical prompts through Claude, GPT, and Gemini to compare latency, cost, and quality.

1|Updated Jul 23, 2026
Search Tags:#llm evaluation#claude#model comparison#latency#benchmark#gemini#gpt
One-click install
npx skills add https://github.com/sanjanb/my-agent-harness --skill benchmark-models-sanjanb
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: benchmark-models
Source: https://github.com/sanjanb/my-agent-harness/tree/main/skills/gstack-benchmark-models
Command: npx skills add https://github.com/sanjanb/my-agent-harness --skill benchmark-models-sanjanb

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Choosing between AI models for a given task is usually guesswork. This Skill replaces vibes with data by running the same prompt through Claude, GPT (via Codex CLI), and Gemini side-by-side, then reporting latency, token usage, cost, and optionally a quality score from an LLM judge. ## Core Features & Use Cases - Cross-model comparison: Executes one prompt across multiple providers and produces a comparison table of speed, cost, and tokens. - Optional LLM judge: Uses Anthropic's Claude to score each model's output on a 0-10 quality scale for roughly $0.05 per run. - Skill benchmarking: Can benchmark an existing gstack skill's SKILL.md as the prompt, exposing tool-use differences between models rather than just raw generation. - Use Case: You want to know which model handles your /review skill best. Run the benchmark against that skill's SKILL.md, enable the judge, and get a table showing Claude vs GPT vs Gemini on latency, cost, and quality. ## Quick Start Ask the agent to run a cross-model benchmark comparing Claude, GPT, and Gemini on a prompt of your choice with the quality judge enabled.

Frequently Asked Questions about benchmark-models

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compare Claude, GPT, and Gemini on the same prompt?▼

Run the benchmark-models skill, which sends one prompt to all three providers via their CLIs and prints a table comparing latency, tokens, and cost. Enable the judge option to also get a 0-10 quality score per model.

How do I benchmark an AI model against an existing skill file?▼

Choose the option to benchmark a gstack skill when prompted, then pick the skill's SKILL.md file. The benchmark uses that file as the prompt, which exposes tool-use and instruction-following differences between models.

What happens if a model provider is not authenticated?▼

The dry-run step shows which providers are OK versus NOT READY with remediation hints. Unauthenticated providers are skipped cleanly without aborting the batch, but at least one authed provider is required to run.

How much does the LLM quality judge cost?▼

The judge uses Anthropic's Claude to score each output on a 0-10 scale and adds roughly $0.05 per run. It requires an ANTHROPIC_API_KEY or Claude credentials and can be skipped to measure only speed, cost, and tokens.

What is the difference between benchmark-models and the /benchmark skill?▼

benchmark-models measures AI model performance on prompts, comparing latency, cost, and quality across providers. The /benchmark skill measures web page performance such as Core Web Vitals and load times.

Related Skills

Explore More Software Engineering Skills→
vocalbridgeaivocalbridgeai
7

login

Authenticate with Vocal Bridge using a user-provided API key via the vb CLI.

Official
Basic
NVIDIANVIDIA
3.4k

eagle3-new-model

Configure EAGLE3 offline pipeline YAML files for new model checkpoints.

Official
Intermediate
theodrosyimertheodrosyimer

skill-evaluator

Execute eval suites combining structural checks and LLM rubric scoring.

Community
Advanced
microsoftmicrosoft
19

scaffolding-yarp-proxy-project

Scaffold an ASP.NET Core project with YARP reverse proxy for incremental migration.

Official
Intermediate