sgl-project
Official@sgl-project
High-performance inference acceleration framework providing JIT CUDA and Triton kernel integration for diffusion and large-scale model deployment.
Agent Skills by sgl-project
Showing 31 vetted skills indexed across 2 GitHub repositories.
sgl-projectsglang-diffusion-modelopt-quant
Quantize diffusion DiT models with NVIDIA ModelOpt and validate FP8 or NVFP4 checkpoints in SGLang Diffusion.
sgl-projectsglang-diffusion-benchmark-profile
Benchmarks denoise latency and profiles diffusion hotspots in SGLang multimodal generation.
sgl-projectsglang-diffusion-add-model
Implements new diffusion models and Diffusers pipelines in SGLang multimodal_gen.
sgl-projectsglang-diffusion-performance
Selects optimal SGLang Diffusion performance flags for a given model, GPU, and VRAM budget.
sgl-projectclean-startup-log
Detect and fix noisy warnings and duplicate prints in SGLang server startup logs.
sgl-projectsglang-cherrypick
Triggers and monitors bot-cherry-pick GitHub workflow runs for merged PRs onto SGLang release branches.
sgl-projectci-workflow-guide
Explains SGLang CI workflow orchestration including stage gating, fast-fail, partitioning, and failure debugging.
sgl-projectspeculative-naming
Enforces naming conventions for identifiers in SGLang speculative decoding code.
sgl-projectdebug-cuda-crash
Diagnose CUDA crashes in SGLang using kernel API logging and crash-safe tensor dumps.
sgl-projectscripted-runtime-notes
Guides API design decisions for the SGLang scripted runtime test harness.
sgl-projectlarge-class-style
Enforces frozen-code and init-helper conventions for SGLang's Scheduler, TokenizerManager, and ModelRunner classes.
sgl-projectkl-consistency-test
Write, calibrate, and debug prefill-vs-decode logprob KL consistency tests in sglang.
sgl-projectcompute-mamba-ratio
Computes the optimal memory split ratio between state and KV pools for hybrid Mamba model serving.
sgl-projectsglang-runtime-context
Documents SGLang's RuntimeContext architecture for config bags, overrides, and per-forward state.
sgl-projectbabysit-pr-to-pass-ci
Monitor and drive selected GitHub Actions workflows to green on an SGLang pull request.
sgl-projectllm-torch-profiler-analysis
Analyzes torch.profiler traces from LLM serving frameworks into kernel, overlap, and fusion tables.
sgl-projectcookbook-migrate-model
Migrate legacy SGLang cookbook pages to the config-driven template format.
sgl-projectdebug-distributed-hang
Diagnose hangs in SGLang distributed inference using py-spy, NCCL logs, and per-rank diffing.
sgl-projectcookbook-add-model
Adds a new model page to the SGLang Cookbook docs using config-driven templates.
sgl-projectenv-var-conventions
Enforces conventions for defining, naming, and deprecating SGLang environment variables.
sgl-projectcookbook-review-pr
Reviews SGLang Cookbook pull requests against a config-driven contribution checklist.
sgl-projectmechanical-refactor-verify
Verifies mechanical refactoring commits by reproducing them byte-for-byte from faithful relocation primitives.
sgl-projecttune-ci-thresholds
Calibrate ASR, TTS, and Qwen3-Omni CI thresholds with statistical validation.
sgl-projectrunning-eval-suite
Automate performance benchmark execution and result table synchronization in sglang-omni.
Frequently Asked Questions About sgl-project
FAQPage SchemaWhat specific tasks are enabled by the SGLang framework?▼
The framework enables the integration of custom JIT CUDA and Triton kernels to accelerate diffusion model inference. It provides structured guidance for adding new diffusion models, calibrating CI thresholds for multimodal benchmarks, and executing performance evaluation suites to ensure low-latency model serving.
Which engineers should utilize these performance optimization capabilities?▼
These capabilities are designed for machine learning infrastructure engineers and kernel developers focused on high-performance model serving. It is ideal for practitioners tasked with optimizing inference speed, managing complex CI regression cycles, and deploying custom C++ or CUDA kernels within production-grade model environments.
What are the prerequisites for implementing custom kernels?▼
Implementation requires proficiency in CUDA and Triton programming to interface with the underlying framework architecture. Users must be familiar with the SGLang repository structure, specifically the kernel library, and possess the ability to configure environment variables and runtime flags for hardware-specific performance tuning.