perf-moe-comm-overlap

Overlap expert-parallel dispatch and combine communication with expert compute in Megatron-Bridge.

Updated May 23, 2026

Search Tags:#dispatch #moe #expert-parallelism #megatron-bridge #communication overlap #alltoall #flex dispatcher

Ready to run

npx skills add https://github.com/yo-steven/skills-exploration-20260522 --skill perf-moe-comm-overlap-yo-steven

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: perf-moe-comm-overlap
Source: https://github.com/yo-steven/skills-exploration-20260522/tree/main/skills/Megatron-Bridge/perf-moe-comm-overlap
Command: npx skills add https://github.com/yo-steven/skills-exploration-20260522 --skill perf-moe-comm-overlap-yo-steven

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It reduces MoE throughput regressions caused by expert-parallel token dispatch/combining communication time by enabling communication overlap with expert computation in Megatron-Bridge.

Core Features & Use Cases

Expert-parallel comm overlap enablement: Activates overlap of MoE expert-parallel dispatch/combining with expert compute via comm_overlap.overlap_moe_expert_parallel_comm.
Flex dispatcher routing support: Enables compatible MoE dispatcher behavior by coordinating moe_token_dispatcher_type with flex/DeepEP-style backends.
Optional delayed expert wgrad scheduling: Adds delay_wgrad_compute to increase overlap potential when constraints allow it.

Use case: you observe that a profiling trace shows visible time spent in MoE dispatch/combine under EP>1, and you want to tune throughput after keeping dispatcher/routing/recompute decisions stable.

Quick Start

Enable comm overlap by setting cfg.comm_overlap.overlap_moe_expert_parallel_comm = True and also set cfg.model.moe_shared_expert_overlap = False when using the dispatch-overlap path.

Frequently Asked Questions about perf-moe-comm-overlap

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I overlap MoE expert-parallel communication with expert compute in Megatron-Bridge?▼

To overlap MoE expert-parallel communication with expert compute, set cfg.comm_overlap.overlap_moe_expert_parallel_comm to True. You must also set cfg.model.moe_shared_expert_overlap to False when using the dispatch-overlap path to ensure correct routing behavior.

When should I enable communication overlap for Mixture of Experts models?▼

Enable MoE communication overlap when profiling traces show visible time spent in expert-parallel dispatch or combine steps under EP>1. This tuning step reduces throughput regressions after keeping dispatcher, routing, and recompute decisions stable.

What are the requirements for using MoE communication overlap with a flex dispatcher?▼

Using MoE communication overlap requires expert_model_parallel_size and num_moe_experts greater than one. The moe_token_dispatcher_type must be set to alltoall or flex, and the model must use supported BF16 or FP16 precision.

Can I delay weight gradient computation to increase MoE communication overlap?▼

Yes, you can add delay_wgrad_compute to increase overlap potential when constraints allow it. Using this optional delayed expert wgrad scheduling requires careful pairing with CUDA-graph and recompute configurations to avoid destabilizing the training process.

Does MoE communication overlap work with DeepEP-style backends in Megatron-Bridge?▼

Yes, the communication overlap path supports DeepEP-style backends by coordinating the flex dispatcher. It activates compatible MoE dispatcher behavior, enabling overlapping of expert-parallel dispatch and combining operations with expert compute.