add-cuda-kernel

Add custom CUDA kernels to FlashInfer with C++ launchers and Python APIs.

2|1|Updated Oct 21, 2025

Search Tags:#performance optimization #kernel development #cuda #flashinfer #tvm #gpu programming

Ready to run

npx skills add https://github.com/fw-ai/flashinfer --skill add-cuda-kernel-fw-ai

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: add-cuda-kernel
Source: https://github.com/fw-ai/flashinfer/tree/main/.claude/skills/add-cuda-kernel
Command: npx skills add https://github.com/fw-ai/flashinfer --skill add-cuda-kernel-fw-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides a comprehensive, step-by-step guide to integrating new custom CUDA kernels into the FlashInfer library, enabling users to extend its functionality with their own high-performance GPU operations.

Core Features & Use Cases

End-to-End Workflow: Covers defining CUDA kernels, creating C++ launchers, binding with TVM-FFI, generating JIT modules, and creating Python APIs.
Best Practices: Demonstrates proper error handling, dtype dispatch, and architecture-specific compilation.
Use Case: A researcher needs to implement a novel attention mechanism not supported by FlashInfer. They can follow this guide to add their custom kernel, complete with Python bindings and tests, ensuring seamless integration and performance optimization.

Quick Start

Follow the tutorial to add a new element-wise scale CUDA kernel to FlashInfer.

Frequently Asked Questions about add-cuda-kernel

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I add custom CUDA kernels to FlashInfer for novel attention mechanisms?▼

Adding custom CUDA kernels to FlashInfer requires defining the kernel in C++, creating a launcher, binding with TVM-FFI, generating JIT modules, and exposing Python APIs for seamless integration with novel attention mechanisms.

What is the process for binding a C++ CUDA launcher to Python using TVM-FFI?▼

Binding a C++ CUDA launcher to Python using TVM-FFI involves integrating the launcher with the TVM-FFI framework, generating Just-In-Time (JIT) modules, and exposing the resulting Python APIs to enable custom GPU operations within FlashInfer.

Does FlashInfer support dtype dispatch and architecture-specific compilation for custom GPU programming?▼

Yes, FlashInfer supports dtype dispatch and CUDA architecture-specific compilation for custom GPU programming. These best practices ensure your custom kernels handle various data types and target the correct GPU architectures effectively.

What's the best way to handle error handling when developing CUDA kernels for FlashInfer?▼

The best way to handle errors when developing CUDA kernels for FlashInfer is to follow the library's demonstrated best practices, which include proper error handling mechanisms within the C++ launchers and Python API integrations to ensure robust custom GPU operations.

Can I generate JIT modules for custom kernel development without deep TVM knowledge?▼

Yes, you can generate JIT modules for custom kernel development by following the step-by-step integration guide, which covers TVM-FFI binding and module generation, enabling Python API exposure without requiring extensive underlying TVM framework knowledge.