debug-distributed-hang

To debug distributed SGLang hangs, apply per-rank logging and traces to pinpoint divergence, use py-spy watchdogs for stack snapshots, enable NCCL debug steps, and perform binary-search methodology to locate the first diverge point.

Multi-GPU inference hangs in tensor-parallel and pipeline-parallel setups occur due to diverging state across ranks. Identifying the first diverge point through per-rank traces and cross-rank divergence analysis stabilizes the distributed run.

Yes, you can use py-spy as a watchdog to trace SGLang NCCL hangs. Attach py-spy to the running processes to capture stack traces, which helps locate the exact diverge point during AllGather or AllReduce collectives.

The best way to identify cross-rank divergence in distributed inference is using a binary-search methodology to locate the first diverge point, combined with per-rank logging and cross-rank divergence analysis to stabilize multi-GPU runs.

You need CUDA core-dumps guidance as part of deterministic debugging workflows to diagnose distributed SGLang hangs. Collecting core-dumps alongside per-rank traces helps identify the root cause of multi-GPU run failures.