Verified:GitHub

benchmark-models

Benchmark multiple LLM providers on the same gstack prompt and return a cost/latency table.

Updated May 7, 2026

Search Tags:#latency #api authentication #prompt evaluation #llm comparison #token cost #model benchmarking #quality judging

Ready to run

npx skills add https://github.com/TumeloRamaphosa/StudEx-Valley-OS --skill benchmark-models-tumeloramaphosa

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: benchmark-models
Source: https://github.com/TumeloRamaphosa/StudEx-Valley-OS/tree/main/.claude/skills/gstack/benchmark-models
Command: npx skills add https://github.com/TumeloRamaphosa/StudEx-Valley-OS --skill benchmark-models-tumeloramaphosa

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Cross-model benchmarking decides which LLM is actually best for a specific gstack skill prompt by measuring latency, tokens, and cost, with an optional quality judge.

Core Features & Use Cases

Side-by-side model runs: executes the same prompt across Claude, GPT (Codex CLI), and Gemini to compare outcomes on the same input.
Tradeoff visibility: reports latency, tokens, and cost so you can choose based on real performance, not vibes.
Optional quality judging: can add an LLM judge pass to score output quality and break ties when you care about correctness and usefulness.
Skill-focused benchmarking: supports benchmarking an existing skill’s prompt (or any prompt you provide) to answer “which model should I use for this task?” consistently.
Voice triggers: supports common “compare models / model shootout / which model is best” voice commands.

Quick Start

Run benchmark-models to benchmark one of your existing gstack skills by choosing option A when asked for the prompt source.

Frequently Asked Questions about benchmark-models

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I compare LLM models to find the best one for my prompts?▼

To compare LLM models for your prompts, you can benchmark the same prompt across multiple providers like Claude, GPT, and Gemini. This process measures latency, token usage, and cost side-by-side to determine which model performs best in practice.

What metrics are included in a model benchmarking comparison?▼

A model benchmarking comparison includes latency, token usage, and total cost metrics. It can also be augmented with an optional LLM judge score to evaluate output quality and break ties based on correctness and usefulness.

Do I need to verify API authentication before running a model shootout?▼

Yes, you need to verify API authentication before running a model shootout. The benchmarking process requires a dry-run to confirm provider authentication readiness across your selected LLM providers before executing the actual prompt comparison.

Can I evaluate prompt quality across different LLM providers?▼

Yes, you can evaluate prompt quality across different LLM providers by applying an optional quality judging pass. This uses an LLM judge to score the output quality of side-by-side runs to help you choose the best model.

What is the best way to decide which model to use for a specific task?▼

The best way to decide which model to use for a specific task is to run a skill-focused benchmark. This executes your existing prompt across multiple models and returns a cost and latency table to reveal the optimal tradeoff.