add-jit-kernel

To add a custom CUDA kernel to SGLang, you implement a simple element-wise operation, validate it with TensorMatcher, ensure vectorized memory access via AlignedVector, and wrap it in a Python module that compiles and caches the JIT kernel.

Testing element-wise GPU operations across data types is done by using a Python wrapper that compiles and executes JIT kernels, applying runtime safety checks and deterministic launchers to validate FP16, BF16, and FP32 tensor operations.

Yes, you can use JIT compilation for simple CUDA ops by leveraging a Python wrapper that handles compilation, caching, and execution, allowing you to rapidly prototype element-wise operations like scale without complex C++ wrappers.

To ensure vectorized memory access and runtime safety when developing a CUDA kernel, you apply TensorMatcher for validation and use AlignedVector for memory access, wrapping the execution through a LaunchKernel wrapper.

Yes, the SGLang jit_kernel module supports BF16 and FP16 data types for GPU acceleration, allowing developers to implement and test simple element-wise CUDA operations across FP16, BF16, and FP32 formats.