cuda-roofline-strategy

To determine the next CUDA kernel optimization, you classify the roofline bottleneck from NCU metrics like SM throughput, achieved occupancy, and top warp stall reasons. This identifies whether the kernel is compute-bound, bandwidth-bound, occupancy-limited, or latency-limited to guide your next experiment.

Choosing a CUDA optimization strategy based on iteration phase involves mapping your current stage—early, mid, late, or plateau—to a technique tier. This scales recommendations by how desperate the run is, ensuring heavy rewrites are only suggested when warranted during the iterative kernel optimization loop.

Classifying roofline bottlenecks with NVIDIA NCU involves analyzing SM throughput, DRAM/SOL throughput, and achieved occupancy. You use the top warp stall reason breakdown to refine the classification into compute, bandwidth, occupancy, latency, or balanced-limited behavior for explicit next experiment justification.

CUDA kernel optimization can plateau with high achieved occupancy if the kernel is actually compute-bound or latency-limited. Analyzing the roofline position and warp stall breakdown reveals the true limiting factor, preventing wasted experiments on occupancy tuning when other bottlenecks dominate.

You should re-profile slow outlier workloads separately during CUDA kernel optimization to avoid misleading averages. Per-workload refinement adapts dispatch when regimes differ, ensuring the selected technique tier matches the specific roofline position and stall behavior of the outlier.