add-sgl-kernel

To add a CUDA kernel to SGLang, you implement the C++ kernel, wire the Python extension, configure the build, and validate it with tests and benchmarks. This provides end-to-end guidance for integrating AOT kernels into sgl-kernel.

Integrating a CUTLASS kernel into sgl-kernel involves implementing the C++ kernel, registering the Torch extension, and configuring the build system. This covers full integration from C++ implementation to Python binding and validation.

The workflow for adding an AOT CUDA C++ kernel to a Torch extension includes writing the C++ code, setting up Python bindings, and building the extension. It ensures kernel development and Torch extension registration are handled correctly.

Yes, sgl-kernel supports adding CUDA kernels that depend on large C++ projects like CUTLASS. The workflow specifically applies to adding new kernels with heavyweight dependencies and covers the required build wiring.

You validate a new CUDA kernel in SGLang by running unit tests and a benchmark suite. This ensures the correctness and performance of the integrated kernel are comprehensively verified before deployment.