Benchmark FlashInfer GPU kernel performance with CUPTI timing across backends.
npx skills add https://github.com/sunxxuns/flashinfer --skill benchmark-kernel
Please help me install this Agent Skill. Skill: benchmark-kernel Source: https://github.com/sunxxuns/flashinfer/tree/main/.claude/skills/benchmark-kernel Command: npx skills add https://github.com/sunxxuns/flashinfer --skill benchmark-kernel
This Skill provides a guided workflow to measure GPU kernel performance for FlashInfer using CUPTI timing, delivering accurate runtime metrics and comparisons across backends.
Install CUPTI-python and CUDA drivers, then run one of the sample benchmarks such as:
python benchmarks/flashinfer_benchmark.py
--routine BatchDecodeWithPagedKVCacheWrapper
--backends fa2 fa2_tc cudnn
--page_size 16
--batch_size 32
--s_kv 2048
--num_qo_heads 32
--num_kv_heads 8
--head_dim_qk 128
--head_dim_vo 128
--refcheck
-vv
Or run a smaller test:
python benchmarks/flashinfer_benchmark.py
--routine bmm_fp8
--backends cudnn cutlass
--batch_size 64
--m 1 --n 1024 --k 2048
--refcheck
High-intent search queries and answers about installing and using this skill.
To benchmark FlashInfer GPU kernels using CUPTI timing, run the flashinfer_benchmark.py script with specified routines, backends, and tensor dimensions to collect precise execution times and export CSV results.
CUPTI-based kernel timing is used for GPU performance profiling to measure precise kernel execution times, enabling accurate comparisons across multiple backends like fa2, fa2_tc, cudnn, and trtllm during FlashInfer workloads.
Yes, you need CUPTI-python, CUDA drivers, and the flashinfer benchmark scripts to measure GPU kernel performance and collect reproducible runtime metrics in CUDA-enabled environments.
Yes, you can compare fa2, fa2_tc, cudnn, and trtllm backends in a single GPU benchmarking run by passing multiple backend names to the --backends argument of the benchmark script.
You export reproducible benchmark results for FlashInfer kernel comparisons by running the benchmark script with --refcheck and verbose flags, which generates standardized reports and CSV exports for later analysis.
CUPTI timing for GPU kernel benchmarking is limited to CUDA-enabled environments and requires specific dependencies like CUPTI-python and CUDA drivers to function correctly for collecting precise execution metrics.
jwu1980-testBenchmark FlashInfer GPU kernels with CUPTI timing and output median throughput metrics.
flashinfer-aiBenchmark FlashInfer kernels across backends using CUPTI timing and export CSV reports.
ariusewyBenchmark FlashInfer GPU kernels with CUPTI or CUDA events.
drunkcodingBenchmark FlashInfer kernel performance across backends using CUPTI timing.
