add-cuda-kernel

To add a custom CUDA kernel to FlashInfer, follow the complete workflow: define the kernel, write a host launcher, bind via TVM-FFI, generate a JIT module, and expose a Python API for production-grade integration.

TVM-FFI binding is used to bridge the C++ host launcher with Python, enabling JIT module generation for dynamic workloads and allowing the custom CUDA kernel to be called directly from the FlashInfer Python API.

Yes, FlashInfer supports production-ready patterns for custom kernels including dtype dispatch, CUDA stream handling, error reporting, and optional pre-allocated outputs to ensure robust operation.

You need a JIT module to handle dynamic workloads efficiently. It compiles the CUDA kernel on the fly via TVM-FFI, allowing the FlashInfer Python API to adapt to varying input shapes and types without static recompilation.

Yes, the integration workflow supports optional pre-allocated outputs. This production-ready pattern allows you to manage memory externally and pass existing buffers to the CUDA kernel via the host launcher.