nemo-mbridge-perf-moe-comm-overlap

What problem does it solve?

This skill enables performance-tuning of MoE training on Megatron Bridge by enabling overlapping of expert-parallel communication, reducing idle compute time.

Core Features & Use Cases

• MoE communication overlap knob: enables overlap between token dispatch/combine and expert computation for MoE layers.
• Optional delayed wgrad scheduling: allows additional overlap opportunities when compatible with CUDA graph scopes.
• Flex vs all-to-all dispatcher awareness: guides valid configurations for different dispatcher backends and dependencies between flags.
• Use Case: tune throughput of large MoE training jobs when EP>1 and profile shows communication time as a bottleneck.

Quick Start

Execute a basic activation of overlap on a representative run and verify throughput improvements.