benchmark-kernel

Benchmark FlashInfer GPU kernels accurately by utilizing CUPTI for precise kernel execution timing, which falls back to CUDA events when CUPTI is unavailable. This enables direct performance comparison across different backends and configurations.

Yes, you can compare FlashAttention performance against cuDNN, CUTLASS, and TensorRT-LLM backends. The benchmarking outputs median execution time and achieved TFLOPS to facilitate direct performance comparison across these multiple backends.

The best way to measure GPU kernel execution time for attention and GEMM operations is using CUPTI for precise timing. It supports detailed performance analysis by outputting results to CSV for further evaluation across different batch sizes and sequence lengths.

You need the cupti-python dependency to enable precise GPU kernel execution timing via CUPTI. If CUPTI is unavailable, the benchmarking process automatically falls back to using standard CUDA events for timing your FlashInfer kernels.

Generate reproducible benchmark commands for MoE operations by using the built-in command generation feature. This supports saving your benchmarking results to CSV format, ensuring detailed analysis and reproducibility across different hardware and workload configurations.

CUDA events timing differs because it serves as a fallback when CUPTI is unavailable. CUPTI provides precise GPU kernel execution measurements, while CUDA events offer standard timing, which may vary slightly during FlashInfer performance analysis.