debug-cuda-crash

To debug a CUDA crash in SGLang, enable Kernel API logging to capture the last successful API boundary and its tensor inputs. This workflow isolates illegal memory accesses and NaN/Inf issues by providing actionable context for root cause analysis.

Illegal memory accesses during CUDA kernel execution often stem from shape or dtype mismatches. Enabling Kernel API logging captures tensor metadata at the crash boundary, revealing the exact inputs that triggered the memory failure.

You can capture tensor inputs before a CUDA crash by enabling Kernel API logging in your environment. This records the tensor metadata of the last successful API boundary, allowing you to observe the exact inputs that led to the failure.

To diagnose CUDA device-side asserts, use level-based logging progression such as Level 3 to balance detail and performance. You can adjust levels up or down to capture the necessary API boundaries without overwhelming the system.

Yes, this CUDA crash debugging workflow applies to diffusion workloads. The Kernel API logging mechanism captures tensor metadata and crash boundaries across all CUDA-enabled components, including both LLM and diffusion executions.

The best way to reproduce a CUDA crash for root cause analysis is to run a crash repro script with Kernel API logging enabled. This captures the last successful boundary and generates crash-safe dumps to guide your diagnosis.