benchmark-kernel

Name: benchmark-kernel
Availability: InStock
Author: FlashInfer

Benchmark FlashInfer kernels across backends using CUPTI timing and export CSV reports.

6.1k|1.3k|Updated Jul 22, 2023

Search Tags:#timing #benchmark #gpu #backends #flashinfer #cupti #kernel-performance

Ready to run

npx skills add https://github.com/flashinfer-ai/flashinfer --skill benchmark-kernel-flashinfer-ai

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: benchmark-kernel
Source: https://github.com/flashinfer-ai/flashinfer/tree/main/.claude/skills/benchmark-kernel
Command: npx skills add https://github.com/flashinfer-ai/flashinfer --skill benchmark-kernel-flashinfer-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Benchmark kernel performance across FlashInfer backends using precise timing to enable apples-to-apples comparisons and reproducible results.

Core Features & Use Cases

CUPTI timing as the primary method for accurate GPU timing, with CUDA events as a safe fallback.
Cross-backend benchmarking across fa2, fa2_tc, cudnn, cutlass, and trtllm-gen.
A ready-to-use benchmark script (benchmarks/flashinfer_benchmark.py) to run common routines and generate a CSV report with reproducer commands.
Quick-start guidance and troubleshooting to ensure reliable results across hardware.

Quick Start

Run a CUPTI-based benchmark with the included script to measure kernel times across backends and save results to a CSV file.

Frequently Asked Questions about benchmark-kernel

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I benchmark FlashInfer kernel performance across different backends?▼

To benchmark FlashInfer kernel performance, you can use the benchmarks/flashinfer_benchmark.py script to measure execution times across backends like fa2, fa2_tc, cudnn, cutlass, and trtllm-gen, then export the results to a CSV file.

Does GPU kernel benchmarking with CUPTI require a specific CUDA version?▼

Yes, GPU kernel benchmarking with CUPTI requires CUDA 13 or higher to utilize the primary CUPTI-based timing method, though the tool automatically falls back to standard CUDA events if the version requirement is not met.

What GPU workloads can I measure timing data for using this benchmarking approach?▼

You can measure GPU timing data for multiple workload types including decode, prefill, GEMM, and MOE operations, allowing direct performance comparisons across the supported FlashInfer backends on CUDA-enabled GPUs.

Why use CUPTI timing instead of CUDA events for FlashInfer backend comparisons?▼

CUPTI timing is preferred for FlashInfer backend comparisons because it provides more accurate GPU timing data, enabling precise apples-to-apples comparisons, while CUDA events are only used as a safe fallback mechanism.

Can I generate reproducible commands when benchmarking FlashInfer kernels?▼

Yes, when benchmarking FlashInfer kernels, the benchmark script generates a CSV report that includes reproducer commands, ensuring that your performance comparisons and timing results remain fully reproducible across different runs.