What problem does it solve?
Running large benchmark grids or adaptive searches against LLM inference endpoints on Kubernetes requires orchestrating many sequential benchmark jobs, enforcing cardinality caps, handling credentials safely, and aggregating results across variations — all of which is error-prone to do by hand.
Core Features & Use Cases
- Sweep orchestration: Submit one
AIPerfSweep CR with aiperf kube sweep and get one child AIPerfJob per (variation, trial), executed sequentially by a dedicated sweep-controller pod.
- Grid, zip, scenario, Sobol, Latin Hypercube, and adaptive/Bayesian search: Author parameter axes with dotted paths rooted in the
benchmark: block, or define a search space with objectives for Bayesian optimization.
- Safety guardrails: Credential-bearing axes are rejected before creation, the 200-variation and 10-trial caps are enforced, and nested in-process sweeps are blocked under operator management.
- Use Case: Benchmark a vLLM deployment by sweeping
phases.profiling.concurrency over [4, 8, 16, 32], then run aiperf kube results <sweep-name> to download the aggregate and per-child exports.
Quick Start
Ask the AI to author a sweep.yaml sweeping concurrency over a few values and submit it with aiperf kube sweep against your benchmark namespace.