add-jit-kernel

To add a JIT CUDA kernel to SGLang, implement the kernel using TensorMatcher and LaunchKernel, create a cached Python wrapper with dtype specialization, and register correctness tests and benchmarks in CI. This workflow ensures tensor validation, vectorized memory access, and CUDA error handling conventions are met.

Tensor validation and error handling in SGLang CUDA kernels use TensorMatcher for shape and dtype checks, and LaunchKernel for safe execution. Cached JIT compilation manages device abstractions, while supported dtypes and vectorized memory access optimize performance without large AOT dependencies.

Yes, you can benchmark SGLang JIT CUDA kernels against PyTorch. The integration workflow includes adding performance comparisons and edge-case coverage to the test suite. CI registration ensures benchmarks run automatically, validating vectorized memory access and dtype specialization.

Yes, the SGLang jit_kernel module supports lightweight element-wise operations without large AOT dependencies. It uses cached JIT compilation, TensorMatcher validation, and vectorized memory access to implement and execute kernels efficiently across supported dtypes.

SGLang CUDA kernel tests may fail during CI discovery if project conventions for test-suite registration are not followed. Ensure your implementation includes TensorMatcher validation, LaunchKernel execution, supported dtypes, vectorized memory access, and CUDA error handling to pass correctness and edge-case tests.

The best way to structure a cached JIT compilation wrapper for SGLang CUDA kernels is to build a thin Python integration layer with dtype specialization, CUDA validation, and optional output handling. This aligns with SGLang conventions for TensorMatcher and LaunchKernel execution.