nemo-mbridge-perf-moe-comm-overlap

Enable expert-parallel communication overlap for Megatron Bridge MoE training.

Updated Apr 16, 2026

Search Tags:#throughput #moe #megatron-bridge #communication-overlap #model-parallel #token-dispatch

Ready to run

npx skills add https://github.com/sayalinvidia/sayali-skills-test --skill nemo-mbridge-perf-moe-comm-overlap

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: nemo-mbridge-perf-moe-comm-overlap
Source: https://github.com/sayalinvidia/sayali-skills-test/tree/main/skills/nemo-mbridge-perf-moe-comm-overlap
Command: npx skills add https://github.com/sayalinvidia/sayali-skills-test --skill nemo-mbridge-perf-moe-comm-overlap

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill enables performance-tuning of MoE training on Megatron Bridge by enabling overlapping of expert-parallel communication, reducing idle compute time.

Core Features & Use Cases

MoE communication overlap knob: enables overlap between token dispatch/combine and expert computation for MoE layers.
Optional delayed wgrad scheduling: allows additional overlap opportunities when compatible with CUDA graph scopes.
Flex vs all-to-all dispatcher awareness: guides valid configurations for different dispatcher backends and dependencies between flags.
Use Case: tune throughput of large MoE training jobs when EP>1 and profile shows communication time as a bottleneck.

Quick Start

Execute a basic activation of overlap on a representative run and verify throughput improvements.

Frequently Asked Questions about nemo-mbridge-perf-moe-comm-overlap

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I improve MoE training throughput when expert-parallel communication is a bottleneck?▼

To improve MoE training throughput, enable expert-parallel communication overlap to parallelize token dispatch and combine operations with expert computation, reducing idle compute time when expert model parallel size is greater than one.

What are the prerequisites for enabling communication overlap in Megatron Bridge MoE training?▼

Enabling communication overlap requires expert_model_parallel_size greater than 1, num_moe_experts greater than 1, and a token dispatcher type of either alltoall or flex. If pipeline parallelism is used, virtual pipeline parallelism is also required.

Does communication overlap work with both alltoall and flex token dispatchers?▼

Yes, communication overlap supports both alltoall and flex token dispatchers. Configuring flex dispatcher specifically requires setting moe_token_dispatcher_type to flex to ensure valid backend operations.

When should I enable delayed wgrad scheduling for MoE layers?▼

You should enable delayed wgrad scheduling to create additional overlap opportunities when the configuration is compatible with CUDA graph scopes, further optimizing MoE training throughput.

How do I know if communication time is causing idle compute in my MoE training profile?▼

If your MoE training profile shows visible token dispatch or combine times when expert parallelism is greater than one, communication is a bottleneck and overlapping expert-parallel communication can tune throughput.