skills.rest
Plaidmustache
PlaidmustacheCOMMUNITY
Explore all skills by this author →
GitHub

huggingface-accelerate

Automate distributed training setup for PyTorch with HuggingFace Accelerate.

Updated May 4, 2026
Search Tags:#pytorch#distributed-training#fsdp#ddp#deepspeed#accelerate#megatron
One-click install
npx skills add https://github.com/Plaidmustache/hermes-nulab --skill huggingface-accelerate-plaidmustache
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: huggingface-accelerate
Source: https://github.com/Plaidmustache/hermes-nulab/tree/main/optional-skills/mlops/accelerate
Command: npx skills add https://github.com/Plaidmustache/hermes-nulab --skill huggingface-accelerate-plaidmustache

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Simplest distributed training API that eliminates boilerplate by enabling a unified workflow for PyTorch scripts across DeepSpeed, FSDP, Megatron, and DDP with automatic device placement and mixed precision.

Core Features & Use Cases

  • Unified API: Seamlessly switch backends (DDP, DeepSpeed, FSDP, Megatron) without changing training code.
  • Automatic configuration: Interactive accelerate config and single-launch workflow for multi-GPU or multi-node clusters.
  • Real-world use: Scale a single-script training from CPU to multi-GPU cluster with minimal changes.

Quick Start

Install accelerate and run your training script with accelerate launch to enable unified distributed training.

Frequently Asked Questions about huggingface-accelerate

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run PyTorch distributed training across multiple GPUs without changing my code?▼

PyTorch distributed training across multiple GPUs can run without code changes by using a unified API that automates backend switching and device placement. This eliminates boilerplate, letting you scale single-script training seamlessly.

What is the best way to switch between DeepSpeed and FSDP for PyTorch training?▼

Switching between DeepSpeed and FSDP is best handled by a unified training API that supports both backends natively. It enables seamless switching without modifying your underlying PyTorch training code.

Does HuggingFace Accelerate work with Megatron-LM pipelines?▼

Yes, HuggingFace Accelerate works with Megatron-LM pipelines. It supports configurable launch options and plugins to enable Megatron alongside DDP, DeepSpeed, and FSDP within a unified workflow.

How do I configure multi-node GPU clusters for PyTorch distributed training?▼

Multi-node GPU clusters for PyTorch distributed training are configured using an interactive setup tool. This generates launch configurations, enabling a single-launch workflow for multi-GPU or multi-node cluster environments.

Can I use automatic mixed precision and device placement for single-node PyTorch training?▼

Yes, automatic mixed precision and device placement are supported for single-node PyTorch training. The unified API applies these configurations automatically, allowing scripts to scale from CPU to multi-GPU with minimal changes.

Related Skills

Explore More Software Engineering Skills→
SriRamkunamsettySriRamkunamsetty

huggingface-accelerate

Launch distributed PyTorch training with HuggingFace Accelerate across CPU, GPU, and TPU environments.

Community
Advanced
ovachieverovachiever
52

pytorch-fsdp

Apply PyTorch FSDP for sharded, mixed-precision distributed training of large models.

Community
Advanced
PeterHddPeterHdd
10

engineering-ml-engineer

Develop machine learning models with PyTorch, HuggingFace Transformers, scikit-learn, and XGBoost.

Community
Advanced
richardnguyen0715richardnguyen0715

pytorch-lightning

Automate distributed PyTorch training with a high-level API and minimal boilerplate.

Community
Advanced