sglang-skill

What problem does it solve?

This Skill eliminates the hassle of manually navigating scattered SGLang source code and documentation to develop, debug, or optimize LLM serving systems, reducing time spent searching for implementation details of core components like attention backends, KV cache, and MoE routing.

Core Features & Use Cases

• Structured Source Navigation: Provides clear paths to SGLang's core runtime (SRT), JIT Triton kernels, custom CUDA kernels (sgl-kernel), and frontend DSL for fast code lookup.
• Development Workflow Guidance: Includes step-by-step instructions for common tasks like adding new attention backends, implementing custom model support, adding quantization methods, and configuring distributed serving (TP/PP/EP).
• Use Case Example: If you need to add FlashInfer MLA support for DeepSeek V3 models, this Skill directs you to the correct backend implementation files and registration points to complete the task efficiently.

Quick Start

Use the sglang-skill to add a custom FP8 quantization module to the SGLang serving engine and register it for use with LLaMA 3 models.