Add custom CUDA kernels to FlashInfer with C++ launchers and Python APIs.
npx skills add https://github.com/fw-ai/flashinfer --skill add-cuda-kernel-fw-ai
Please help me install this Agent Skill. Skill: add-cuda-kernel Source: https://github.com/fw-ai/flashinfer/tree/main/.claude/skills/add-cuda-kernel Command: npx skills add https://github.com/fw-ai/flashinfer --skill add-cuda-kernel-fw-ai
This Skill provides a comprehensive, step-by-step guide to integrating new custom CUDA kernels into the FlashInfer library, enabling users to extend its functionality with their own high-performance GPU operations.
Follow the tutorial to add a new element-wise scale CUDA kernel to FlashInfer.
High-intent search queries and answers about installing and using this skill.
Adding custom CUDA kernels to FlashInfer requires defining the kernel in C++, creating a launcher, binding with TVM-FFI, generating JIT modules, and exposing Python APIs for seamless integration with novel attention mechanisms.
Binding a C++ CUDA launcher to Python using TVM-FFI involves integrating the launcher with the TVM-FFI framework, generating Just-In-Time (JIT) modules, and exposing the resulting Python APIs to enable custom GPU operations within FlashInfer.
Yes, FlashInfer supports dtype dispatch and CUDA architecture-specific compilation for custom GPU programming. These best practices ensure your custom kernels handle various data types and target the correct GPU architectures effectively.
The best way to handle errors when developing CUDA kernels for FlashInfer is to follow the library's demonstrated best practices, which include proper error handling mechanisms within the C++ launchers and Python API integrations to ensure robust custom GPU operations.
Yes, you can generate JIT modules for custom kernel development by following the step-by-step integration guide, which covers TVM-FFI binding and module generation, enabling Python API exposure without requiring extensive underlying TVM framework knowledge.
LeightonWangModify, pack, and test FlashInfer contest kernels with Modal validation.
spacecat2002Add and validate lightweight CUDA JIT kernels in SGLang's jit_kernel module.
jwu1980-testAdd CUDA kernels to FlashInfer with JIT and TVM-FFI bindings.
hung-phanDevelop Python GPU kernels with explicit shared memory and warp-level control.
