skills.rest
spacecat2002
spacecat2002COMMUNITY
Explore all skills by this author →
GitHub

sglang-auto-benchmark

Search SGLang serving configurations for maximum throughput under latency SLAs.

Updated Jun 16, 2026
Search Tags:#benchmarking#performance tuning#speculative decoding#sglang#kv cache#latency sla#qps optimization
One-click install
npx skills add https://github.com/spacecat2002/oscar --skill sglang-auto-benchmark-spacecat2002
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sglang-auto-benchmark
Source: https://github.com/spacecat2002/oscar/tree/main/sglang-research/.claude/skills/sglang-auto-benchmark
Command: npx skills add https://github.com/spacecat2002/oscar --skill sglang-auto-benchmark-spacecat2002

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This skill replaces one-off SGLang benchmark commands and manual tuning with a repeatable workflow for finding serving configurations that maximize throughput while meeting latency targets.

Core Features & Use Cases

  • Tiered Configuration Search: Sweep server flags such as attention backends, batching limits, prefill settings, parallelism, and cache controls with bounded candidate budgets.
  • Dataset Preparation and Validation: Convert ShareGPT, custom conversation data, random scenarios, and shared-prefix workloads into canonical benchmark JSONL and validate the resulting prompts and generation lengths.
  • SLA and QPS Benchmarking: Search for the highest QPS that satisfies TTFT and TPOT targets, export CSV and JSONL results, preserve failed candidates, and support interruption and resume.
  • Remote Run Observability: Mirror cleaned progress logs and final result artifacts from remote benchmark environments for local monitoring and review.
  • Use Case: Tune a multi-GPU SGLang deployment for a real production request distribution, compare candidate launch configurations under latency SLAs, and optionally run a second-stage EAGLE or speculative decoding search.

Quick Start

Use the SGLang auto-benchmark skill to validate my dataset, run a tier-2 search against the specified model and latency SLA, and report the best configuration with paths to all result artifacts.

Frequently Asked Questions about sglang-auto-benchmark

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I tune SGLang serving configurations for maximum QPS under latency SLAs?▼

Benchmark custom conversation datasets by converting ShareGPT, random scenarios, or shared-prefix workloads into canonical benchmark JSONL. The process validates resulting prompts and generation lengths before executing the latency SLA search.

How do I benchmark custom conversation datasets in SGLang?▼

Benchmark custom conversation datasets by converting ShareGPT, random scenarios, or shared-prefix workloads into canonical benchmark JSONL. The process validates resulting prompts and generation lengths before executing the latency SLA search.

Can I evaluate speculative decoding performance with SGLang benchmarks?▼

Evaluate speculative decoding performance by running an optional second-stage EAGLE search. This tests speculative decoding configurations alongside standard server flags to measure throughput improvements under latency constraints.

What do I need to set up before running an SGLang latency benchmark?▼

Running an SGLang latency benchmark requires a launchable SGLang environment, an accessible model and tokenizer, and properly prepared canonical datasets. These components validate prompts and generation lengths before bounded searches begin.

How do I export and resume SGLang benchmark results if a run is interrupted?▼

Resume interrupted SGLang benchmark runs using resumable result logging that preserves failed candidates. Export final benchmark artifacts as CSV and JSONL files to maintain complete records of QPS and latency performance.

Related Skills

Explore More Software Engineering Skills→
FutureMLS-LabFutureMLS-Lab
556

sglang-auto-benchmark

Search SGLang server configurations against throughput and latency objectives.

Official
Advanced
flagos-aiflagos-ai
18

perf-test-flagos

Benchmark vLLM-served AI models across five workload profiles for throughput, latency, TTFT, and TPOT.

Official
Advanced
cxnaivecxnaive

tensorrt-llm

Optimize large language model inference with NVIDIA TensorRT-LLM.

Community
Advanced
Nabilhassan12345Nabilhassan12345

sglang-auto-benchmark

Automate SGLang benchmark runs with tiered server-flag sweeps and YAML configs.

Community
Advanced