ml-debug

Debug NaN loss and CUDA OOM during PyTorch Lightning training by categorizing symptoms, mapping them to root causes like gradient overflow, and prescribing fixes such as gradient clipping or mixed precision toggles to restore stability.

PyTorch Geometric GNN training failures from graph OOM and over-smoothing are resolved through framework-aware checks that diagnose node memory limits and prescribe configuration adjustments to prevent layer degradation.

Troubleshoot slow training throughput and convergence problems by inspecting GPU utilization and loss metrics from logs, then applying checkpoint and profiling workflows to identify bottlenecks and adjust learning rates.

You can generate a structured debug report from existing training logs and checkpoints by analyzing metrics in your logs directory, mapping performance and data symptoms to root causes, and outputting actionable remediation steps.

Fix training failures beyond learning rate adjustments by implementing numerical stability improvements, gradient accumulation, and mixed precision toggles to address underlying data issues and prevent loss divergence.

The debugging workflow supports experiment troubleshooting with Hydra configs by mapping categorized training symptoms to likely root causes and prescribing configuration or code-level fixes to resolve failures.