gem-llm-load-test

What problem does it solve?

This Skill eliminates uncertainty in GEM-LLM performance by running repeatable load tests that expose real throughput, latency (including p99), and quota/rate-limit behaviors under concurrent users.

Core Features & Use Cases

• Concurrent load generation (locust + asyncio bench): Runs both a single-key Locust workload and a multi-key multi-user asyncio benchmark to measure realistic behavior.
• Quota and rate-limit scenario validation: Confirms 60RPM and daily token limits via controlled concurrency and user-key setup.
• Production-oriented metrics capture: Produces latency percentiles (p50/p95/p99/max), success/failure classifications (HTTP 401/429/500), and token throughput (tok/s), saving results into reports for analysis.

Use Case

Validate whether a new deployment can sustain a target concurrency (e.g., ~50 concurrent requests), keeps p99 latency under a threshold (e.g., <5s), and meets throughput requirements (e.g., >30 req/s), while ensuring quota/rate-limit responses behave correctly.

Quick Start

Run the skill with install.sh for gem-llm-load-test, then execute the Locust single-key test or the multi-user-bench script to generate load and save metrics reports.