What problem does it solve? Scaling PyTorch training from a single GPU to multi-GPU or multi-node clusters normally requires rewriting code for DDP, DeepSpeed, or FSDP, which is error-prone and time-consuming. This Skill provides the patterns to make that transition with minimal code changes. ## Core Features & Use Cases - Unified Distributed API: Convert any PyTorch training loop to run on DDP, DeepSpeed ZeRO, FSDP, or Megatron-LM by adding roughly four lines of code. - Mixed Precision Training: Enable FP16, BF16, or FP8 training with automatic gradient scaling and device placement. - Configuration & Launch: Generate accelerate configs interactively and launch jobs across GPUs, nodes, or TPUs with a single command. - Use Case: You have a single-GPU training script that now needs to run on an 8-GPU node with BF16 and gradient accumulation. Apply the Accelerator pattern, run accelerate config, and launch with accelerate launch --multi_gpu --num_processes 8 train.py. ## Quick Start Convert my PyTorch training script to run on multiple GPUs using HuggingFace Accelerate and show me the launch command.
