training-check

To monitor WandB metrics for anomaly detection during ML training, this Skill automates periodic evaluation of active sessions to detect NaN values, loss divergence, or plateauing metrics early. It reads WandB history including loss trends, eval metrics, and gradient norms to catch issues before long-running runs finish.

You can automate ML training health checks that periodically read WandB metrics to catch problems like NaN or Inf values, stagnation, and divergence early. This ongoing quality check helps you avoid wasting GPU hours by discovering issues before long-running experiments finish.

When WandB metrics are ambiguous during training health checks, the Skill triggers an escalation workflow via a secondary Codex reviewer. It also adapts the check interval dynamically, ensuring uncertain anomaly detection results receive further scrutiny without constant manual monitoring.

Yes, monitoring training health in hyperparameter sweeps and long-running experiments is a core use case. The Skill applies periodic quality checks to active training sessions, reading signals like eval metrics, gradient norms, and learning rate to detect early anomalies.

No specific dependencies are required to run periodic WandB evaluations for anomaly detection. The Skill operates standalone to read WandB history and signals such as loss trends, eval metrics, NaN or Inf values, gradient norms, and learning rate during your ML training.