Navigate SGLang source code and documentation for development and optimization.
npx skills add https://github.com/slowlyC/agent-gpu-skills --skill sglang-skill
Please help me install this Agent Skill. Skill: sglang-skill Source: https://github.com/slowlyC/agent-gpu-skills/tree/main/sglang_skill Command: npx skills add https://github.com/slowlyC/agent-gpu-skills --skill sglang-skill
This Skill eliminates the hassle of manually navigating scattered SGLang source code and documentation to develop, debug, or optimize LLM serving systems, reducing time spent searching for implementation details of core components like attention backends, KV cache, and MoE routing.
Use the sglang-skill to add a custom FP8 quantization module to the SGLang serving engine and register it for use with LLaMA 3 models.
High-intent search queries and answers about installing and using this skill.
To add a custom attention backend to SGLang, navigate to the core runtime (SRT) implementation files and follow the structured registration points to integrate your specific backend. This workflow provides step-by-step guidance for efficiently completing backend implementation and registration.
SGLang source code for optimizing KV cache and MoE routing is accessible via structured paths to the core runtime, JIT Triton kernels, and custom CUDA kernels (sgl-kernel). This structured navigation eliminates manual searching across scattered documentation to locate performance optimization interfaces.
The best way to implement custom model support in an SGLang serving system is by following its standard development workflows for adding new model architectures. This Skill directs you to the correct implementation files and registration points to efficiently integrate and deploy custom models.
Yes, SGLang supports configuring distributed inference including tensor parallelism, pipeline parallelism, and expert parallelism. Development workflow guidance provides structured steps for setting up these distributed serving configurations to optimize production deployment.
To add a new quantization method like FP8 to the SGLang serving engine, use the provided development workflow guidance to locate the correct module implementation files. You can then register the custom quantization module for use with target models like LLaMA 3.
Yes, you can use SGLang to troubleshoot LLM serving latency and throughput issues by leveraging structured access to its source code and implementation patterns. This allows developers to efficiently debug performance bottlenecks within the serving engine's core components.
agentic-inSelect inference paths, serving stacks, and benchmarking plans for ML models.
handsomelong922Deploy scalable LLM serving with vLLM for OpenAI-compatible endpoints.
IsNoobgrammerOptimize Triton and CUDA kernels using roofline-based trial-loop benchmarking.
ancolemanDeploy LLM and ML models for production inference with vLLM, TensorRT-LLM, Ollama, and BentoML.
