add-jit-kernel

To add a JIT CUDA kernel to SGLang, create a CUDA file under jit_kernel/csrc, wrap it with a Python launcher, and use LaunchKernel for safe GPU launches. This streamlines integration and iterative testing for custom tensor operations.

The best way to validate custom CUDA kernels is by enforcing tensor validation via TensorMatcher and integrating unit tests. This ensures safe GPU launches and confirms computational correctness before deploying kernels.

SGLang handles GPU kernel launches safely using the LaunchKernel wrapper and supports Programmatic Dependent Launch (PDL) for advanced scheduling. This provides structured runtime launching patterns for custom CUDA operations.

You can benchmark a new CUDA kernel by using provided scaffolding to integrate performance checks alongside unit tests for CI. This automates performance validation and ensures the custom kernel meets required efficiency standards.

Yes, you can extend SGLang with vectorized tensor operations by following guidance on CUDA kernel organization and vectorization. This allows rapid validation of custom tensor operations through Python wrappers and tests.