benchmark-models

Benchmarks Claude, GPT, and Gemini on the same prompt to compare latency, cost, and quality.

Updated Sep 13, 2026
One-click install
npx skills add https://github.com/abdulazeezoj/monovella-poc --skill benchmark-models-abdulazeezoj
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: benchmark-models
Source: https://github.com/abdulazeezoj/monovella-poc/tree/main/.agents/skills/gstack/benchmark-models
Command: npx skills add https://github.com/abdulazeezoj/monovella-poc --skill benchmark-models-abdulazeezoj

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Choosing between AI models for a task usually relies on guesswork. This Skill runs the same prompt through Claude, GPT (via Codex CLI), and Gemini side-by-side and reports latency, token usage, cost, and optional quality scores so model selection is based on measured data. ## Core Features & Use Cases - Cross-model execution: Runs one prompt through multiple providers and compares speed, cost, and tokens in a single table. - LLM quality judge: Optionally scores each model's output 0-10 using Anthropic's Claude as a tiebreaker, adding roughly $0.05 per run. - Auth preflight: A dry-run shows which providers are authenticated before any paid API calls are made. - Use Case: You want to know whether GPT or Gemini handles your /ship review prompt better. Run the benchmark against that skill's SKILL.md, enable the judge, and save the JSON result as a baseline to detect quality drift after provider model updates. ## Quick Start Ask the assistant to run a cross-model benchmark comparing Claude, GPT, and Gemini on one of your gstack skills. ## Quick Start Say: run a cross-model benchmark on my ship skill with the quality judge enabled.

Frequently Asked Questions about benchmark-models

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compare Claude, GPT, and Gemini on the same prompt?

Run the benchmark with a prompt file or inline text and a comma-separated model list such as claude,gpt,gemini. The tool executes the prompt through each provider and prints a table comparing latency, tokens, and cost.

What is the difference between benchmark-models and the benchmark skill?

The benchmark skill measures web page performance like Core Web Vitals and load times. benchmark-models measures AI model performance, comparing providers on speed, cost, and output quality for a given prompt.

How does the LLM quality judge work in model benchmarking?

The judge uses Anthropic's Claude to score each model's output on a 0-10 scale, adding roughly $0.05 per run. It requires an Anthropic API key or credentials and is only enabled when the user explicitly opts in.

What happens if a provider is not authenticated before benchmarking?

The dry-run step shows each provider's auth status with remediation hints. Unauthenticated providers are skipped cleanly without aborting the batch, but if zero providers are authed the benchmark stops and suggests running claude login, codex login, or setting GOOGLE_API_KEY.

Can I save benchmark results to track model quality over time?

Yes. Results can be saved as dated JSON files under ~/.gstack/benchmarks/. Saving a baseline lets you diff future runs against it to catch quality regressions when providers update their models.