Guides adding new CUDA kernels to the FlashInfer library.
npx skills add https://github.com/ariusewy/flashinfer_dev --skill add-cuda-kernel-ariusewy
Please help me install this Agent Skill. Skill: add-cuda-kernel Source: https://github.com/ariusewy/flashinfer_dev/tree/main/.claude/skills/add-cuda-kernel Command: npx skills add https://github.com/ariusewy/flashinfer_dev --skill add-cuda-kernel-ariusewy
This Skill provides a comprehensive, step-by-step guide to integrating new custom CUDA kernels into the FlashInfer library, enabling users to extend its functionality with their own optimized operations.
Follow the detailed steps in the tutorial to add a new CUDA kernel to FlashInfer.
High-intent search queries and answers about installing and using this skill.
To add custom CUDA kernels to FlashInfer, you define the kernel in C++, implement launchers, create TVM-FFI bindings, generate JIT modules, and expose Python APIs. This end-to-end workflow ensures seamless integration of novel operations into your inference pipeline.
Creating JIT modules for FlashInfer CUDA kernels involves compiling architecture-specific code and binding it via TVM-FFI. You then wrap these bindings in Python APIs to enable dtype dispatch and proper error handling for the custom operations.
Yes, FlashInfer supports compiling custom CUDA kernels for various data types and CUDA architectures. The integration workflow includes dtype dispatch and architecture-specific compilation to ensure optimized performance across different hardware configurations.
The best way to test and benchmark custom GPU kernels in FlashInfer is to follow the provided integration guide, which includes rigorous testing and benchmarking steps. This ensures your custom operations maintain correctness and achieve optimized performance.
Yes, you can implement TVM-FFI bindings for FlashInfer kernel development without external dependencies. The skill provides a self-contained workflow covering C++ launcher implementation and binding generation to seamlessly connect your C++ code with Python.
dezverevGenerate AnimalAI plugin scaffolds with classes, tests, and kernel registrations.
flashinfer-aiExtract GPU kernel schemas from SGLang models into deduplicated Definition JSON files.
fw-aiAdd custom CUDA kernels to FlashInfer with C++ launchers and Python APIs.
guqiong96Develop and optimize Triton and CUDA JIT kernels for diffusion models.
