moe-comm-overlap
OfficialBoost MoE throughput with overlap.
AuthorNVIDIA
Version1.0.0
Installs0
System Documentation
What problem does it solve?
MoE expert-parallel communication overlap in Megatron Bridge to improve training throughput by overlapping communication with computation when MoE models and multiple experts are used.
Core Features & Use Cases
- Enable overlap of MoE expert-parallel communication to reduce step time in MoE training.
- Provide prerequisites and activation notes for correct operation, including dispatcher choices, precision requirements, and VPP usage when pipeline parallelism is involved.
- Suitable for Megatron Bridge deployments using DeepEP-like backends, flex or alltoall token dispatchers, and complex MoE routing scenarios.
Quick Start
Enable the overlap by setting cfg.comm_overlap.overlap_moe_expert_parallel_comm to True and validate the change with a small, representative workload.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: moe-comm-overlap Download link: https://github.com/NVIDIA/skills/archive/main.zip#moe-comm-overlap Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.