skills.rest
BBuf
BBufCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

sglang-sota-performance

Benchmark models across SGLang, vLLM, and TensorRT-LLM to identify performance gaps.

721|65|Updated Apr 1, 2026
Search Tags:#performance#benchmarking#profiling#sglang#sota#torch-profiler
One-click install
npx skills add https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS --skill sglang-sota-performance
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sglang-sota-performance
Source: https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/tree/main/skills/sglang-sota-performance
Command: npx skills add https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS --skill sglang-sota-performance

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automates end-to-end performance optimization for a single model by coordinating cross-framework benchmarking, profiling, and patching to reach SOTA results on a target hardware setup.

Core Features & Use Cases

  • Benchmark across SGLang, vLLM, and TensorRT-LLM to identify best deployment commands for a given model.
  • Run torch-profiler analysis to locate kernel bottlenecks, overlap opportunities, and fusion patterns.
  • Patch SGLang conservatively based on evidence and revalidate with the same workload to close performance gaps.

Quick Start

Provide a model identifier, target hardware, workload description, and artifact root, then run the SGLang SOTA loop to start the optimization cycle.

Frequently Asked Questions about sglang-sota-performance

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark and compare performance across SGLang, vLLM, and TensorRT-LLM?▼

To benchmark SGLang, vLLM, and TensorRT-LLM, you run a coordinated cross-framework workload to identify performance gaps and determine the best deployment commands for your specific model and target hardware.

How does torch-profiler help diagnose SGLang performance bottlenecks?▼

Torch-profiler analysis diagnoses SGLang performance bottlenecks by identifying kernel issues, overlap opportunities, and fusion patterns, which guides targeted patching to close performance gaps.

How do I patch SGLang to reach SOTA performance on target hardware?▼

To patch SGLang for SOTA performance, apply conservative, evidence-based fixes derived from profiler analysis and revalidate the workload to confirm the performance gaps are closed.

What do I need to provide to start the SGLang optimization loop?▼

To start the SGLang optimization loop, you need to provide a model identifier, target hardware, workload description, and artifact root to run the automated benchmarking and patching cycle.

What artifacts are generated when profiling and patching LLM inference frameworks?▼

Profiling and patching LLM inference frameworks generates a reproducible artifact set including benchmarks, profiles, applied patches, and a final report to guide deployment decisions.

Related Skills

Explore More Software Engineering Skills→
hey-konghey-kong

sglang-sota-performance

Coordinate benchmarking, profiling, patching, and revalidation across SGLang, vLLM, and TensorRT-LLM.

Community
Advanced
QingHuan-6QingHuan-6

sglang-sota-performance

Compare SGLang, vLLM, and TensorRT-LLM deployment commands and generate benchmark reports.

Community
Advanced
dyyounggdyyoungg

llm-torch-profiler-analysis

Analyze LLM torch-profiler traces to identify kernel, overlap, and fusion patterns.

Community
Advanced
QingHuan-6QingHuan-6

llm-serving-auto-benchmark

Benchmark SGLang, vLLM, and TensorRT-LLM serving to produce Markdown/CSV summaries.

Community
Advanced