Verified:GitHub

huggingface-accelerate

Adds distributed training support to PyTorch scripts with a unified API for DDP, DeepSpeed, FSDP, and Megatron.

Updated Jun 7, 2026

Search Tags:#PyTorch #HuggingFace #DeepSpeed #distributed training #fsdp #mixed precision #multi-GPU

One-click install

npx skills add https://github.com/Chensihakniroth/ANAKOT-AGENT --skill huggingface-accelerate-chensihakniroth

Or copy as Structured Prompt for Agent▼

Please help me install this Agent Skill.
Skill: huggingface-accelerate
Source: https://github.com/Chensihakniroth/ANAKOT-AGENT/tree/main/optional-skills/mlops/accelerate
Command: npx skills add https://github.com/Chensihakniroth/ANAKOT-AGENT --skill huggingface-accelerate-chensihakniroth

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires accelerate, torch, transformers, and includes references (resource) components.

What problem does it solve? Scaling PyTorch training from a single GPU to multi-GPU or multi-node clusters normally requires rewriting code for each distributed backend. This Skill shows how to add distributed support with four lines of code, handling device placement, mixed precision, and sharding automatically. ## Core Features & Use Cases - Unified Distributed API: One codebase runs on DDP, DeepSpeed ZeRO, FSDP, or Megatron-LM without modification. - Automatic Mixed Precision: Enable FP16, BF16, or FP8 training with a single parameter, including gradient scaling. - Interactive Configuration: Generate launch configs via `accelerate config` and run any setup with one `accelerate launch` command. - Use Case: You have a single-GPU training script that runs out of memory on a 70B model. Convert it with Accelerate, configure FSDP full sharding with BF16, and launch across 8 GPUs without changing your training loop. ## Quick Start Convert my PyTorch training script to run on multiple GPUs using HuggingFace Accelerate and show me the launch command.

Frequently Asked Questions about huggingface-accelerate

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

Can Accelerate train large models with Megatron tensor parallelism?▼

Yes, Accelerate integrates Megatron-LM through MegatronLMPlugin, supporting tensor, pipeline, data, and sequence parallelism. It requires installing Megatron-LM and NVIDIA Apex, and tensor parallelism works best with NVLink-connected GPUs.