Benchmark custom GPU kernels against PyTorch baselines with CUDA event timing.
npx skills add https://github.com/fmh66/kernel-opt-agent --skill kernel-benchmark
Please help me install this Agent Skill. Skill: kernel-benchmark Source: https://github.com/fmh66/kernel-opt-agent/tree/main/skills/kernel-benchmark Command: npx skills add https://github.com/fmh66/kernel-opt-agent --skill kernel-benchmark
You need an objective way to compare a custom GPU kernel implementation (CUDA-C++, CUTLASS, CuTe DSL, or Triton) against known PyTorch baselines, while ensuring correctness before you trust any timing results.
Run the benchmark by providing your solution file, a reference module that defines reference(**kwargs), a directory to write benchmark.md, and the kernel dimension/scalar arguments your reference expects.
High-intent search queries and answers about installing and using this skill.
To benchmark custom GPU kernels against PyTorch baselines, you provide a solution file and a reference module defining reference(**kwargs). The tool uses configurable warmup, trial timing, and KernelBench-style CUDA events to generate latency and speedup results.
Latency measurements can be inaccurate due to measurement bias, but this benchmark applies L2 cache thrashing and uses CUDA event timing by default with configurable warmup and trial control to reduce bias and ensure trustworthy results.
Validating kernel correctness before benchmarking is required. The tool checks solution outputs against a required PyTorch eager reference function prior to collecting any timing numbers to ensure you only benchmark correct implementations.
You can use torch.compile and FlashInfer as optional baselines when benchmarking. The tool compares your custom CUDA-C++, CUTLASS, CuTe DSL, or Triton implementations against these alternatives alongside the standard PyTorch eager mode.
Benchmarking a CUDA-C++ shared library kernel requires mapping the solution ABI. The tool solves for .cu/.so files, validates correctness against a PyTorch reference, times execution with CUDA events, and outputs a benchmark.md summary report.
fmh66Profile CUDA-C++ kernels with Nsight Compute metrics and classify bottlenecks.
nickm538Benchmark and optimize claude-flow v3 for Flash Attention, HNSW search, and memory usage.
flashinfer-aiExtract GPU kernel schemas from SGLang models into deduplicated Definition JSON files.
yo-stevenClassify SOL% bottlenecks and diagnose CUDA kernel underperformance from Nsight Compute output.
