add-jit-kernel

Add a lightweight JIT CUDA kernel to SGLang's jit_kernel module. This workflow provides a complete end-to-end pipeline including the kernel source, Python wrapper, unit tests, and benchmark scaffold to ensure safe and reproducible custom GPU operations.

SGLang ensures JIT kernel safety through TensorMatcher-based input validation and LaunchKernel-based safe execution. These mechanisms validate tensor inputs and manage secure kernel launches, maintaining correctness and preventing runtime errors during custom GPU operations.

Yes, SGLang JIT kernels support cross-type workflows across FP16, BF16, and FP32. The skill provides guidance for precision support, vector width selection, and vectorized memory access via AlignedVector to optimize operations across these formats.

The best way is using the integrated benchmark scaffold provided when adding a JIT kernel. This scaffold accompanies the host wrapper and unit tests, ensuring reproducible performance measurements for your custom GPU operations within the SGLang framework.

You need a SGLang environment configured for GPU operations and familiarity with the sgl_kernel library. The setup requires utilizing TensorMatcher, AlignedVector, and LaunchKernel abstractions to prepare host wrappers and ensure vectorized memory access for the new kernel.

The jit_kernel module enforces consistent abstractions and automates the end-to-end integration of custom GPU operations. It reduces iteration time by providing built-in TensorMatcher validation, LaunchKernel safety, cross-type precision support, and testing scaffolds that standalone code lacks.