debug-cuda-crash

To debug a CUDA crash like an illegal memory access, you can use API logging to capture tensors and trace function calls before the failure. This enables post-mortem analysis of faulty inputs to identify the root cause.

NaN or Inf values during CUDA execution often stem from invalid input tensors or mathematical operations. API logging helps by recording these tensors and metadata before the crash, allowing you to isolate the specific inputs that triggered the error.

You can trace CUDA API calls to find out-of-bounds errors by enabling API logging in your Python scripts. This records the sequence of function calls and system information, providing a guided workflow to locate faulty inputs.

Yes, capturing API logs for CUDA crash analysis requires setting FLASHINFER_LOGLEVEL and FLASHINFER_LOGDEST environment variables. You must also use the @flashinfer_api decorator to log inputs, metadata, and system information during execution.

Yes, you can reproduce CUDA crashes using logged system information. The API logging feature records GPU, CUDA, and PyTorch versions alongside execution metadata, which provides the environment visibility needed to replicate issues.