perf-megatron-fsdp
OfficialOptimize Megatron training with Fully Sharded Data Parallelism
Content & Communication#scalability#pytorch#deeplearning#megatronsdpfp#distributedtraining#fSDP#machinelearning
AuthorNVIDIA-NeMo
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill provides guidance and configuration instructions for enabling Megatron's Fully Sharded Data Parallel (FSDP) training mode, improving scalability and memory efficiency in large model training.
Core Features & Use Cases
- Configuration Guidance: Instructions for setting up Megatron FSDP in Megatron-Bridge, including code snippets and validation steps.
- Compatibility and Pitfalls: Highlights common issues, constraints, and pitfalls when deploying FSDP, ensuring reliable training.
- Verification Procedures: Outlines testing and validation steps to confirm correct deployment and performance benefits.
Quick Start
Configure Megatron-Bridge to enable FSDP by setting the appropriate parameters in the configuration file as instructed, then run the provided validation tests to verify setup.
Dependency Matrix
Required Modules
None requiredComponents
referencesscripts
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: perf-megatron-fsdp Download link: https://github.com/NVIDIA-NeMo/Megatron-Bridge/archive/main.zip#perf-megatron-fsdp Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.