skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

huggingface-accelerate

Automate distributed training for PyTorch models with HuggingFace Accelerate.

2|Updated Apr 12, 2026
Search Tags:#pytorch#distributed-training#fsdp#ddp#deepspeed#accelerate#megatron-lm
One-click install
npx skills add https://github.com/Clay-HHK/claude-config --skill huggingface-accelerate-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: huggingface-accelerate
Source: https://github.com/Clay-HHK/claude-config/tree/main/skills/AI-research-SKILLs/08-distributed-training/accelerate
Command: npx skills add https://github.com/Clay-HHK/claude-config --skill huggingface-accelerate-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

HuggingFace Accelerate simplifies distributed training by enabling DDP/DeepSpeed/FSDP/Megatron with a single script and automatic device placement, plus unified mixed‑precision support.

Core Features & Use Cases

  • Unified API: One script works across CPU, single-GPU, and multi-GPU environments with minimal changes.
  • Backend Flexibility: Switch between DDP, DeepSpeed, FSDP, and Megatron with interchangeable configurations.
  • Automatic Optimization: Automatic device placement, mixed precision, and interactive configuration to reduce boilerplate.

Quick Start

Install accelerate and run your script with accelerate launch

Frequently Asked Questions about huggingface-accelerate

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run distributed training across DDP, DeepSpeed, and FSDP with a single PyTorch script?▼

Distributed training across DDP, DeepSpeed, and FSDP uses a unified API to run a single script across multiple backends. It automates device placement and mixed precision, requiring only an interactive configuration before launching the workflow.

What is the best way to switch between DeepSpeed and FSDP for multi-GPU training?▼

Switching between DeepSpeed and FSDP for multi-GPU training is done through interchangeable backend configurations. This unified approach applies automatic device placement and mixed precision, allowing you to change distributed strategies with minimal script modifications.

Does HuggingFace Accelerate work with CPU and single-GPU PyTorch workflows?▼

HuggingFace Accelerate works with CPU, single-GPU, and multi-GPU PyTorch workflows. The unified API applies automatic device placement across all these environments, meaning the same script runs with minimal changes regardless of hardware scale.

Do I need to manually manage device placement when using Accelerate for mixed precision training?▼

Manual device placement is not needed when using Accelerate for mixed precision training. The library applies automatic device placement and unified mixed precision support, reducing boilerplate so you can focus on the training logic.

Can I use Megatron-LM configurations alongside DDP in my PyTorch distributed training setup?▼

Megatron-LM configurations can be used alongside DDP in your PyTorch distributed training setup. The unified API supports switching between DDP, DeepSpeed, FSDP, and Megatron through interchangeable configurations within a single-launch workflow.

Related Skills

Explore More Software Engineering Skills→
ovachieverovachiever
52

pytorch-fsdp

Apply PyTorch FSDP for sharded, mixed-precision distributed training of large models.

Community
Advanced
musingfoxmusingfox
2

mermaid-display

Convert Mermaid diagram syntax into PNG or SVG images.

Community
Advanced
markwang2658markwang2658
31

huggingface-tokenizers

Provide Rust-based tokenization with BPE, WordPiece, and Unigram models.

Community
Intermediate
JKhyroJKhyro

huggingface-accelerate

Scale PyTorch training with a unified distributed API for multi-GPU and mixed-precision workflows.

Community
Advanced