skills.rest
flashinfer-ai
flashinfer-aiOFFICIAL
Explore all skills by this author →
WebsiteGitHub

benchmark-kernel

Benchmark FlashInfer kernels across backends using CUPTI timing and export CSV reports.

6.1k|1.3k|Updated Jul 22, 2023
Search Tags:#timing#benchmark#gpu#backends#flashinfer#cupti#kernel-performance
One-click install
npx skills add https://github.com/flashinfer-ai/flashinfer --skill benchmark-kernel-flashinfer-ai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: benchmark-kernel
Source: https://github.com/flashinfer-ai/flashinfer/tree/main/.claude/skills/benchmark-kernel
Command: npx skills add https://github.com/flashinfer-ai/flashinfer --skill benchmark-kernel-flashinfer-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Benchmark kernel performance across FlashInfer backends using precise timing to enable apples-to-apples comparisons and reproducible results.

Core Features & Use Cases

  • CUPTI timing as the primary method for accurate GPU timing, with CUDA events as a safe fallback.
  • Cross-backend benchmarking across fa2, fa2_tc, cudnn, cutlass, and trtllm-gen.
  • A ready-to-use benchmark script (benchmarks/flashinfer_benchmark.py) to run common routines and generate a CSV report with reproducer commands.
  • Quick-start guidance and troubleshooting to ensure reliable results across hardware.

Quick Start

Run a CUPTI-based benchmark with the included script to measure kernel times across backends and save results to a CSV file.

Frequently Asked Questions about benchmark-kernel

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark FlashInfer kernel performance across different backends?▼

To benchmark FlashInfer kernel performance, you can use the benchmarks/flashinfer_benchmark.py script to measure execution times across backends like fa2, fa2_tc, cudnn, cutlass, and trtllm-gen, then export the results to a CSV file.

Does GPU kernel benchmarking with CUPTI require a specific CUDA version?▼

Yes, GPU kernel benchmarking with CUPTI requires CUDA 13 or higher to utilize the primary CUPTI-based timing method, though the tool automatically falls back to standard CUDA events if the version requirement is not met.

What GPU workloads can I measure timing data for using this benchmarking approach?▼

You can measure GPU timing data for multiple workload types including decode, prefill, GEMM, and MOE operations, allowing direct performance comparisons across the supported FlashInfer backends on CUDA-enabled GPUs.

Why use CUPTI timing instead of CUDA events for FlashInfer backend comparisons?▼

CUPTI timing is preferred for FlashInfer backend comparisons because it provides more accurate GPU timing data, enabling precise apples-to-apples comparisons, while CUDA events are only used as a safe fallback mechanism.

Can I generate reproducible commands when benchmarking FlashInfer kernels?▼

Yes, when benchmarking FlashInfer kernels, the benchmark script generates a CSV report that includes reproducer commands, ensuring that your performance comparisons and timing results remain fully reproducible across different runs.

Related Skills

Explore More Software Engineering Skills→
flashinfer-aiflashinfer-ai
264

extract-kernel-definitions

Extract GPU kernel schemas from SGLang models into deduplicated Definition JSON files.

Official
Advanced
fw-aifw-ai
2

benchmark-kernel

Benchmark FlashInfer GPU kernels with CUPTI timing and CSV metrics.

Official
Intermediate
jwu1980-testjwu1980-test

benchmark-kernel

Benchmark FlashInfer GPU kernels with CUPTI timing and output median throughput metrics.

Official
Intermediate
drunkcodingdrunkcoding
4

benchmark-kernel

Benchmark FlashInfer kernel performance across backends using CUPTI timing.

Community
Intermediate