Benchmark LLM API latency and throughput across streaming endpoints.
npx skills add https://github.com/OpenDCAI/leonai --skill bench
Please help me install this Agent Skill. Skill: bench Source: https://github.com/OpenDCAI/leonai/tree/main/.claude/skills/bench Command: npx skills add https://github.com/OpenDCAI/leonai --skill bench
Quick benchmark of third-party API models for latency and throughput to help developers compare production-ready performance across endpoints.
Use a concise instruction to start the benchmark by providing the API key and base URL for the endpoint.
High-intent search queries and answers about installing and using this skill.
To benchmark LLM API latency and throughput, use streaming-based Python scripts to measure TTFB and tokens-per-second across OpenAI-like /v1/chat/completions and Anthropic /v1/messages endpoints, aggregating timing data for multiple models.
TTFB (Time to First Byte) measures the initial latency before a model generates its first token. It is measured by parsing streaming deltas from chat completions endpoints to calculate the exact time elapsed before the first response byte arrives.
Yes, you can compare token generation speed across both interfaces using dedicated scripts. The benchmark applies to OpenAI-like /v1/chat/completions and Anthropic /v1/messages formats to test multiple models and aggregate timing data cross-compatibly.
Automatically discover available models by querying the /models endpoint, then filter to the latest mainstream text models like Claude 4.x, GPT-5.x, and Gemini-3.x, excluding outdated or irrelevant variants for performance comparison.
Yes, API benchmarking requires a streaming-capable endpoint to measure tokens per second accurately. Client-side scripts parse streaming deltas to compute TTFB, total time, and generated token counts for valid performance metrics.
To start testing model latency with aiohttp, you need to provide a streaming-capable endpoint, an API key, a base URL, and a test prompt. The scripts handle parsing streaming deltas and computing timing data automatically.
RobbyMo81Summarize text and transcripts from URLs, podcasts, and local files.
felvieiraAggregate tokens, API usage, and execution time to generate per-skill and session cost reports.
manikostoProvide reusable async HTTP client patterns for httpx and aiohttp.
SstoboStream real-time agent response deltas to multiple clients via Convex APIs.
