parallelism-strategies
OfficialOptimizes 3D parallelism for Megatron Bridge.
AuthorNVIDIA
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Guides engineers in selecting and combining parallelism strategies (TP, PP, DP, SP, CP, EP) for Megatron Bridge to achieve scalable training performance and efficient resource usage.
Core Features & Use Cases
- Provides decision guidance for dense and MoE models across single-node to multi-node deployments.
- Offers sizing rules, hardware-topology mappings, and combined parallelism configurations to optimize throughput and memory.
- Use case: when training a 236B MoE model on a 256-GPU cluster, apply recommended EP, PP, TP settings to maximize throughput while controlling interconnect overhead.
Quick Start
Configure and optimize TP, PP, DP, SP, and CP settings for Megatron Bridge across given hardware topology.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: parallelism-strategies Download link: https://github.com/NVIDIA/skills/archive/main.zip#parallelism-strategies Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.