add-jit-kernel

To add a custom CUDA kernel to SGLang, you implement a lightweight JIT kernel within the jit_kernel module, utilizing TensorMatcher for validation and vectorized memory access via AlignedVector, then expose it through a Python wrapper for PyTorch workflows.

The best way to validate tensor inputs for a JIT CUDA kernel is using TensorMatcher-based validation, which ensures safe runtime checks and enforces correct tensor dimensions and types before the LaunchKernel-based launcher executes the GPU operation.

Yes, you can use PyTorch workflows with custom SGLang JIT kernels by implementing a Python wrapper that exposes the underlying CUDA JIT kernel, enabling rapid iteration and quick experiments directly within your PyTorch environment.

Vectorized memory access in SGLang JIT kernels works by utilizing AlignedVector for memory loads, which optimizes data reading patterns during fast element-wise operations and improves overall GPU kernel throughput.

SGLang supports JIT compilation to enable rapid iteration and deployment of custom GPU kernels, allowing developers to quickly build and test minimal element-wise scale operations through its dedicated jit_kernel module.

Building element-wise operations as JIT kernels in SGLang requires strict adherence to TensorMatcher validation and AlignedVector constraints, meaning non-aligned memory access patterns may not benefit from the vectorized loads and safe runtime checks provided.