flashinfer
High-performance GPU kernels for fast LLM inference serving
All Skills in This Repository (4)
Pure Emerald Level Indicators
flashinfer-aibenchmark-kernel
Benchmark FlashInfer kernels across backends using CUPTI timing and export CSV reports.
flashinfer-aidebug-cuda-crash
Diagnose CUDA crashes by capturing pre-crash API inputs with FlashInfer logging.
flashinfer-aiadd-cuda-kernel
Add CUDA kernels to FlashInfer with launchers, TVM-FFI bindings, and JIT modules.
flashinfer-aicutedsl_megamoe
Updates and audits the vendored CuTeDSL MegaMoE kernel drop for FlashInfer expert-parallel MoE backends.
Frequently Asked Questions
FAQPage SchemaHow to install FlashInfer?▼
Run `npx skills add flashinfer-ai/flashinfer --all -g -y` in your terminal to install all skills in this suite globally.
What does FlashInfer do?▼
FlashInfer provides high-performance GPU kernels for LLM inference, covering attention, matrix multiplication, and mixture-of-experts operations. It powers serving frameworks like vLLM, SGLang, and TensorRT-LLM.
Which GPUs does FlashInfer support?▼
It supports NVIDIA GPUs from Turing (SM75) through Blackwell (SM120), including A100, H100, B200, and RTX series cards.
How to debug CUDA crashes in FlashInfer?▼
Enable API logging with the FLASHINFER_LOGLEVEL environment variable to capture tensor shapes and values before a crash. The debug skill guides your agent through diagnosing memory errors and NaN issues.
Can I add custom CUDA kernels with AI help?▼
Yes. The included skills walk your AI agent through writing, binding, and testing new CUDA kernels using FlashInfer's JIT compilation system.
Related Repositories in Software Engineering
View All in Software Engineering→
openclawopenclaw
Run a personal AI assistant across your devices and chat apps
obrasuperpowers
Gives coding agents a disciplined workflow from idea to merged code
facebookreact
AI agent skills for building, testing, and porting React core