Precision + Numerics Stabilizer (bf16/fp16 Done Right)

What problem does it solve?

Add a precision layer and numerics instrumentation that makes mixed-precision training safe to run unattended by detecting NaNs/Inf, monitoring gradient norms, and providing reproducible failure snapshots.

Core Features & Use Cases

• Automatic detection of NaNs, Inf, and gradient overflow with auto-abort or debug snapshots.
• GradScaler integration for fp16 with automatic scale management and safe unscale/clip ordering.
• Flexible precision modes: bf16, fp16, or fp32 with autocast constraints and master-weights-in-fp32 invariant.
• Numerics sentinel: gradient norms, logit monitoring, activation checks, and weight non-finite checks.
• Snapshotting: 7-file failure snapshots with atomic writes for reproducibility, diagnostic data, and repro scripts.

Quick Start

Integrate this stabilizer into your PyTorch training workflow and run your loop to enable automatic NaN detection, mixed-precision safety, and reproducible failure snapshots.