skills.rest
FutureMLS-Lab
FutureMLS-LabOFFICIAL
Explore all skills by this author →
GitHub

sglang-auto-benchmark

Search SGLang server configurations against throughput and latency objectives.

556|81|Updated May 19, 2026
Search Tags:#benchmarking#performance tuning#speculative decoding#sglang#kv cache#throughput optimization#latency analysis
One-click install
npx skills add https://github.com/FutureMLS-Lab/OSCAR --skill sglang-auto-benchmark-futuremls-lab
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sglang-auto-benchmark
Source: https://github.com/FutureMLS-Lab/OSCAR/tree/main/sglang-research/.claude/skills/sglang-auto-benchmark
Command: npx skills add https://github.com/FutureMLS-Lab/OSCAR --skill sglang-auto-benchmark-futuremls-lab

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill eliminates the manual effort of testing SGLang server configurations one flag at a time, helping you identify higher-throughput and lower-latency deployments while preserving target service-level objectives.

Core Features & Use Cases

  • Tiered Configuration Search: Sweep attention backends, scheduling, batching, memory, parallelism, and other server flags using bounded or exhaustive search strategies.
  • Dataset Preparation and Validation: Convert and validate ShareGPT, custom, random, and shared-prefix workloads into canonical benchmark JSONL.
  • SLA and Throughput Evaluation: Search fixed QPS or maximum SLA-compliant QPS while exporting results, summaries, and per-candidate server logs.
  • Optional Speculative Tuning: Run a second-stage EAGLE or EAGLE3 search after establishing the best non-speculative configuration.
  • Use Case: For a Qwen3 deployment, compare FlashInfer and FlashAttention backends, chunked-prefill sizes, and concurrency limits against TTFT and TPOT targets, then resume an interrupted search from its saved results.

Quick Start

Ask the SGLang auto-benchmark skill to prepare your workload, run a tier-2 search against the target model and SLA, and report the best configuration with paths to all result artifacts.

Frequently Asked Questions about sglang-auto-benchmark

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automatically find the best SGLang server configuration for throughput and latency?▼

You can automate SGLang serving performance optimization by searching server configurations against throughput and latency objectives. The tool sweeps attention backends, scheduling, batching, memory, and parallelism flags using bounded or exhaustive search strategies.

Can I benchmark SGLang using ShareGPT workloads and SLA constraints?▼

Yes, SGLang benchmarking supports ShareGPT, synthetic, shared-prefix, or custom request distributions. You can search for fixed QPS or maximum SLA-compliant QPS while exporting results, summaries, and per-candidate server logs.

Does SGLang performance tuning support speculative decoding with EAGLE?▼

Yes, SGLang performance tuning offers optional speculative decoding. You can run a second-stage EAGLE or EAGLE3 search after establishing the best non-speculative configuration to further optimize serving latency.

What do I need to run an SGLang benchmarking search?▼

You need a launchable SGLang environment, an available model and tokenizer, validated canonical benchmark data, and suitable GPU resources to run the configuration search and evaluation.

How do I prepare custom request distributions for SGLang latency analysis?▼

You can prepare custom request distributions by converting and validating ShareGPT, custom, random, and shared-prefix workloads into canonical benchmark JSONL format before running the evaluation.

Is it possible to resume an interrupted SGLang configuration search?▼

Yes, you can resume an interrupted SGLang configuration search from its saved results. The tool exports results and summaries, allowing you to continue evaluating flags like chunked-prefill sizes and concurrency limits.

Related Skills

Explore More Software Engineering Skills→
spacecat2002spacecat2002

sglang-auto-benchmark

Search SGLang serving configurations for maximum throughput under latency SLAs.

Community
Advanced
sqjiansqjian

sglang-auto-benchmark

Automates SGLang benchmarking by coordinating datasets, flag sweeps, and result CSV export.

Community
Advanced
Supporter09Supporter09

tensorrt-llm

Configure TensorRT-LLM inference with quantization, batching, and multi-GPU serving.

Community
Advanced
jpoindexterjpoindexter
1

inference-performance

Diagnose and optimize LLM inference performance metrics like TTFT and TPOT.

Community
Advanced