skills.rest
BBuf
BBufCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

sglang-sota-humanize-loop

Runs a Humanize-governed RLCR loop that benchmarks, profiles, and patches SGLang until it matches competing serving frameworks.

783|67|Updated Apr 1, 2026
Search Tags:#benchmark#SGLang#kernel tuning#LLM serving#Performance optimization#rlcr loop#profiler analysis
One-click install
npx skills add https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS --skill sglang-sota-humanize-loop
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sglang-sota-humanize-loop
Source: https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/tree/main/skills/sglang-sota-humanize-loop
Command: npx skills add https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS --skill sglang-sota-humanize-loop

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Closing the performance gap between SGLang and competing LLM serving frameworks (vLLM, TensorRT-LLM, TokenSpeed) normally requires manual benchmarking, profiler triage, and iterative code patching. This Skill automates that entire campaign as a single recoverable, auditable loop.

Core Features & Use Cases

  • Fixed Fair Benchmark Gate: Runs one bounded cross-framework benchmark with identical model weights, precision, workload, and SLA before any patching, producing winner commands and fairness diagnostics.
  • Humanize RLCR Loop: Each round decides the remaining gap, runs torch-profiler and layer-pipeline analysis, patches SGLang source code, optionally gathers Nsight Compute kernel evidence, and revalidates on the real model.
  • Model PR History Gate: Consults model-family optimization PR history before choosing patch targets, recording PR state, source heads, and validation evidence.
  • Use Case: Given a model like a new MoE checkpoint, the Skill benchmarks SGLang against vLLM and TensorRT-LLM, then iteratively patches SGLang kernels and runtime paths until SGLang matches or beats the best competitor under the same SLA.

Quick Start

Ask the agent to run the SGLang SOTA loop for your model against vLLM and TensorRT-LLM on your target GPU environment, and it will benchmark, profile, patch, and revalidate until SGLang reaches parity.

Frequently Asked Questions about sglang-sota-humanize-loop

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I make SGLang match vLLM or TensorRT-LLM serving performance?▼

Run this Skill with a target model and GPU environment. It first runs a fixed fair benchmark across SGLang and the selected comparison frameworks, then starts a Humanize RLCR loop that profiles bottlenecks, patches SGLang code, and revalidates until SGLang matches or beats the best competitor.

What frameworks does the SGLang SOTA loop compare against?▼

SGLang is always the target framework being patched. The comparison set defaults to vLLM, TensorRT-LLM, and TokenSpeed when available, but honors explicit user selections or exclusions, which are recorded in the run manifest.

Can I limit the comparison to only vLLM?▼

Yes. If you specify only vLLM as the comparison framework, the loop runs SGLang plus vLLM only and does not silently add TensorRT-LLM or TokenSpeed. The selection is recorded in manifest.md and the fairness diagnostics.

Does the loop require Nsight Compute for kernel optimization?▼

Nsight Compute via ncu-report-skill is optional and used only when a kernel edit needs deeper evidence than torch-profiler provides. If NCU cannot run on the host, the blocker is recorded and the loop continues with profiler and source evidence.

When does the SGLang optimization loop stop?▼

The loop stops when SGLang beats the best SLA-passing competitor, ties within a stable 1% threshold after repeat runs, the gap is proven external to SGLang, or profiling shows the hot path is near hardware limits with no low-risk patch remaining.

What happens if the Humanize runtime is not installed?▼

The Skill resolves the Humanize runtime from installed Claude Code or Codex skill directories, or a plugin containing scripts/setup-rlcr-loop.sh. If setup fails, it stops and reports the error rather than patching SGLang outside the governed loop.

Related Skills

Explore More Software Engineering Skills→
NVIDIANVIDIA
3.7k

deployment

Serve quantized or unquantized LLM checkpoints as OpenAI-compatible endpoints using vLLM, SGLang, or TRT-LLM.

Official
Advanced
QingHuan-6QingHuan-6

sglang-sota-performance

Compare SGLang, vLLM, and TensorRT-LLM deployment commands and generate benchmark reports.

Community
Advanced
spacecat2002spacecat2002

sglang-torch-profiler-analysis

Analyze SGLang torch-profiler traces to rank GPU kernels and identify optimization opportunities.

Community
Advanced
NVIDIANVIDIA
2.8k

perf-workload-profiling

Instrument GPU workloads with CUDA event timing and NVTX labeling.

Official
Intermediate