What problem does it solve? Scaling PyTorch training from a single GPU to multi-GPU or multi-node clusters normally requires rewriting code for DDP, DeepSpeed, FSDP, or Megatron, each with different APIs and launcher setups. This Skill provides a unified approach that adds distributed support with minimal code changes. ## Core Features & Use Cases - Unified Distributed API: Convert any PyTorch script to distributed training with four lines of code using Accelerator and accelerator.prepare(). - Backend Flexibility: Switch between DDP, DeepSpeed ZeRO, FSDP, and Megatron-LM through configuration without changing training code. - Mixed Precision & Memory Optimization: Enable FP16, BF16, or FP8 training, gradient accumulation, gradient checkpointing, and CPU/NVMe offloading. - Use Case: You have a single-GPU training script that runs out of memory on a 70B model. Use this Skill to configure FSDP with full sharding and BF16 mixed precision, then launch across 8 GPUs with one accelerate launch command. ## Quick Start Convert my PyTorch training script to run on multiple GPUs using HuggingFace Accelerate and show me the launch command.
