expert-parallel-overlap
OfficialOverlap MoE expert-parallel compute for speed.
Software Engineering#dispatch#overlap#distributed-training#gpu-training#megatron#moe#compute-optimization
AuthorNVIDIA
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Overlap expert-parallel communication with compute in MoE workloads to reduce step time and improve throughput, especially when dispatching tokens across experts.
Core Features & Use Cases
- Overlap MoE token dispatch and gather with expert compute using both alltoall and flex-based backends.
- Support for optional delayed wgrad scheduling to further hide communication latency.
- Works with high-EP configurations and large-scale MoE models to unlock throughput gains in training.
Quick Start
Run a MoE training setup with EP overlap enabled using the alltoall dispatcher to validate performance gains.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: expert-parallel-overlap Download link: https://github.com/NVIDIA/skills/archive/main.zip#expert-parallel-overlap Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.