Self-Supervised Training Loop (AMP + EMA + DDP)

What problem does it solve?

This Skill bundles a production-grade workflow for self-supervised learning that unifies AMP, EMA target networks, and distributed data-parallel training to accelerate and stabilize representation learning at scale.

Core Features & Use Cases

• End-to-end SSL training loop with automatic mixed precision (AMP) using autocast and GradScaler
• Exponential moving average (EMA) target network with cosine-annealed tau for stable representations
• DDP-ready setup: online encoder wrapped for gradients; target encoder updated only via EMA
• Robust checkpointing with six-state save/load, auto-resume, and pruning for fault tolerance
• Integrated monitoring utilities and templates (scheduler, wandb logger, diagnostics) for production-like workflows
• Ready-to-use templates for trainer, scheduler, EMA, and utilities to accelerate SSL experimentation

Quick Start

Initialize the Self-Supervised Trainer with a minimal config and run train on a tiny synthetic dataset to verify the end-to-end AMP, EMA, and DDP workflow.