What problem does it solve? Training neural networks involves dozens of interdependent decisions — optimizer choice, learning rate schedules, mixed precision setup, memory management, and debugging loss spikes or OOM errors — and getting any of them wrong wastes GPU hours or produces broken models. This Skill consolidates production-tested PyTorch training patterns into actionable recipes so you can configure, train, and debug models correctly the first time. ## Core Features & Use Cases - Optimizer & LR Configuration: Per-parameter-group setups combining Muon for 2D weight matrices with AdamW for embeddings, plus cosine, WSD, and time-based LR schedules with dimension-based scaling rules. - Domain-Specific Guidance: Architecture selection tables and training patterns for LLMs, vision, diffusion, medical imaging, protein/drug discovery, genomics, single-cell omics, and clinical NLP. - Debugging & Performance: Systematic checklists for loss explosions, NaN, low MFU, OOM resolution steps, and silent failures like data leakage or tokenizer mismatch. - Use Case: You are pretraining a small LLM and hit a loss spike at step 2,000. Use this Skill to apply the debugging checklist — reduce LR, add gradient clipping, verify logit soft capping — then reconfigure the optimizer with the Muon/AdamW hybrid recipe and resume training with proper MFU monitoring. ## Quick Start Ask the AI to set up a PyTorch training loop for your model with the recommended optimizer configuration, mixed precision, and learning rate schedule from the ml-training-recipes skill.
