Benchmark FlashInfer GPU kernels with CUPTI timing and CSV metrics.
npx skills add https://github.com/fw-ai/flashinfer --skill benchmark-kernel-fw-ai
Please help me install this Agent Skill. Skill: benchmark-kernel Source: https://github.com/fw-ai/flashinfer/tree/main/.claude/skills/benchmark-kernel Command: npx skills add https://github.com/fw-ai/flashinfer --skill benchmark-kernel-fw-ai
This Skill provides a robust framework for accurately measuring the performance of FlashInfer's GPU kernels, enabling detailed performance analysis and comparison across different backends.
Run a benchmark for the BatchDecodeWithPagedKVCacheWrapper routine using FlashAttention-2 and cuDNN backends with specified batch and sequence lengths.
High-intent search queries and answers about installing and using this skill.
You can benchmark FlashInfer GPU kernels accurately using this framework, which utilizes CUPTI for precise timing or falls back to CUDA events to measure performance metrics like median time and TFLOPS.
Yes, you can compare performance across multiple backends including FlashAttention-2/3, cuDNN, CUTLASS, and TensorRT-LLM for attention, GEMM, and MoE operations to identify the optimal kernel.
Benchmarking GPU kernels generates reproducible CSV reports containing detailed performance metrics such as median time, TFLOPS, and memory bandwidth, alongside the exact commands to reproduce the results.
Yes, cupti-python is a required dependency to enable precise CUPTI timing for measuring GPU kernel execution times, with the system falling back to standard CUDA events if unavailable.
You can benchmark attention routines with different batch and sequence lengths by running specified configurations across various backends, such as testing BatchDecodeWithPagedKVCacheWrapper with FlashAttention-2 and cuDNN.
flashinfer-aiExtract GPU kernel schemas from SGLang models into deduplicated Definition JSON files.
alibabaAnalyzes GPU kernel performance using Roofline models and profiling data.
ariusewyBenchmark FlashInfer GPU kernels with CUPTI or CUDA events.
drunkcodingBenchmark FlashInfer kernel performance across backends using CUPTI timing.
