FlashInfer
Official@flashinfer-ai
Offers high-performance GPU kernel optimization, CUDA debugging, and benchmarking infrastructure for large-scale model inference acceleration.
Agent Skills by FlashInfer
Showing 7 vetted skills indexed across 2 GitHub repositories.
flashinfer-aicutedsl_megamoe
Updates and audits the vendored CuTeDSL MegaMoE kernel drop for FlashInfer expert-parallel MoE backends.
flashinfer-aibenchmark-kernel
Benchmark FlashInfer kernels across backends using CUPTI timing and export CSV reports.
flashinfer-aidebug-cuda-crash
Diagnose CUDA crashes by capturing pre-crash API inputs with FlashInfer logging.
flashinfer-aiadd-cuda-kernel
Add CUDA kernels to FlashInfer with launchers, TVM-FFI bindings, and JIT modules.
flashinfer-aiclone-repos
Clone and update SGLang and FlashInfer repositories with submodules from GitHub.
flashinfer-aiextract-kernel-definitions
Extract GPU kernel schemas from SGLang models into deduplicated Definition JSON files.
flashinfer-aiadd-reference-tests
Generate and validate pytest tests for flashinfer_trace reference implementations against FlashInfer or SGLang ground truth.
Frequently Asked Questions About FlashInfer
FAQPage SchemaWhat specific tasks does FlashInfer enable for GPU developers?▼
FlashInfer enables developers to benchmark custom kernels, debug CUDA crashes through input logging, and integrate new GPU kernels using TVM-FFI bindings. It facilitates the extraction of kernel schemas from model definitions and provides a framework for validating reference implementations against ground truth data.
Which technical personas benefit most from these capabilities?▼
These capabilities are designed for GPU kernel engineers, performance optimization specialists, and machine learning infrastructure researchers. Professionals working on low-level inference acceleration, CUDA memory management, and model deployment optimization will find these utilities essential for maintaining high-throughput production environments.
What are the primary dependencies for running FlashInfer kernel benchmarks?▼
Running benchmarks requires a CUDA-capable GPU environment, the CUPTI profiling library for timing, and the TVM-FFI interface for kernel binding. Users must also have the SGLang repository configured as a submodule to ensure compatibility with the extracted kernel schemas and reference test suites.