add-jit-kernel

You can add a CUDA kernel without ahead-of-time compilation by implementing it in SGLang's jit_kernel module. This workflow uses TensorMatcher, LaunchKernel, and device utilities to build lightweight GPU operations with runtime compilation.

The process for wrapping a JIT kernel in Python involves creating a thin, cached Python wrapper that handles dtype validation and runtime argument management. This integrates the CUDA kernel with SGLang's Python execution environment.

To register correctness tests and performance benchmarks for a new GPU kernel, follow SGLang's CI suite conventions. This ensures your CUDA operation is validated for both accuracy and performance within the project's test suite.

Yes, validating a lightweight CUDA operation in SGLang requires CUDA-capable hardware. The JIT kernel abstractions and benchmarking suite depend on actual GPU execution to test correctness and measure performance.

Operations suitable for SGLang's JIT kernel module include element-wise, normalization, and quantization tasks. These lightweight CUDA operations benefit from vectorized memory access and type traits without requiring full ahead-of-time builds.