Add custom CUDA kernels to the SGLang JIT kernel module.
npx skills add https://github.com/guqiong96/Lsglang --skill add-jit-kernel-guqiong96
Please help me install this Agent Skill. Skill: add-jit-kernel Source: https://github.com/guqiong96/Lsglang/tree/main/.claude/skills/add-jit-kernel Command: npx skills add https://github.com/guqiong96/Lsglang --skill add-jit-kernel-guqiong96
This Skill provides a structured, step-by-step guide to integrate custom CUDA kernels into the SGLang framework, enhancing its computational capabilities for specific AI model inference tasks.
Follow the tutorial to implement a new JIT kernel in the elementwise/scale.cuh file.
High-intent search queries and answers about installing and using this skill.
Adding custom CUDA kernels to SGLang involves integrating them into the JIT kernel module using provided C++ and CUDA abstractions for tensor validation, memory access, and kernel launching.
Wrapping a custom CUDA kernel in Python within SGLang is part of a complete workflow that follows kernel implementation, utilizing provided abstractions to ensure safety and consistency before testing.
You should use JIT compiled kernels in SGLang when you need to enhance inference speed by adding a custom kernel for a novel activation function or specialized tensor operation not supported by default.
Yes, SGLang provides specific C++ and CUDA abstractions for tensor validation, memory access, and kernel launching to ensure safety, readability, and consistency when adding custom kernels.
Testing and benchmarking custom CUDA kernels in SGLang are the final steps in the workflow, following the implementation and Python wrapping phases, to verify and measure the kernel's performance.
FutureMLS-LabAdd validated JIT CUDA kernels to SGLang's jit_kernel module.
guqiong96Develop and optimize Triton and CUDA JIT kernels for diffusion models.
spacecat2002Diagnose CUDA crashes in SGLang by logging tensors at kernel API boundaries.
pedestrianloveManage tensor parallelism, inference optimizations, and mixed-precision support for Megatron-LM.
