skills.rest
NVIDIA
NVIDIAOFFICIAL
Explore all skills by this author →
WebsiteGitHub

moe-comm-overlap

Enable MoE expert-parallel communication overlap with compute in Megatron Bridge.

2.8k|332|Updated Feb 25, 2026
Search Tags:#throughput#overlap#distributed-training#moe#megatron-bridge#expert-parallel
One-click install
npx skills add https://github.com/NVIDIA/skills --skill moe-comm-overlap
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: moe-comm-overlap
Source: https://github.com/NVIDIA/skills/tree/main/skills/Megatron-Bridge/perf-techniques/moe-comm-overlap
Command: npx skills add https://github.com/NVIDIA/skills --skill moe-comm-overlap

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

MoE expert-parallel communication overlap in Megatron Bridge to improve training throughput by overlapping communication with computation when MoE models and multiple experts are used.

Core Features & Use Cases

  • Enable overlap of MoE expert-parallel communication to reduce step time in MoE training.
  • Provide prerequisites and activation notes for correct operation, including dispatcher choices, precision requirements, and VPP usage when pipeline parallelism is involved.
  • Suitable for Megatron Bridge deployments using DeepEP-like backends, flex or alltoall token dispatchers, and complex MoE routing scenarios.

Quick Start

Enable the overlap by setting cfg.comm_overlap.overlap_moe_expert_parallel_comm to True and validate the change with a small, representative workload.

Frequently Asked Questions about moe-comm-overlap

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What are the prerequisites for overlapping MoE expert-parallel communication with compute?▼

Prerequisites for MoE expert-parallel communication overlap include expert_model_parallel_size > 1, num_moe_experts > 1, moe_token_dispatcher_type set to alltoall or flex, and BF16 or FP16 precision.

Does overlapping MoE expert-parallel communication work with pipeline parallelism?▼

When using pipeline parallelism with MoE expert-parallel communication overlap, you must set virtual_pipeline_model_parallel_size to satisfy the prerequisites for correct operation.

Which token dispatchers support MoE expert-parallel communication overlap?▼

The alltoall and flex token dispatcher backends support MoE expert-parallel communication overlap, accommodating complex MoE routing scenarios within Megatron Bridge deployments.

Why should I enable MoE expert-parallel communication overlap during distributed training?▼

Enabling MoE expert-parallel communication overlap improves distributed training throughput by executing communication and computation concurrently when using MoE models with multiple experts.

Related Skills

Explore More Software Engineering Skills→
yo-stevenyo-steven

perf-moe-comm-overlap

Overlap expert-parallel dispatch and combine communication with expert compute in Megatron-Bridge.

Community
Advanced
NVIDIA-NeMoNVIDIA-NeMo
852

perf-moe-optimization-workflow

Optimize MoE model training throughput by addressing memory, communication, and compute bottlenecks.

Official
Advanced
yo-stevenyo-steven

perf-expert-parallel-overlap

Overlap expert-parallel token dispatch and combine communication with expert feed-forward compute in Megatron-Bridge MoE models.

Community
Advanced
NVIDIANVIDIA
2.8k

moe-optimization-workflow

Diagnose and optimize MoE training bottlenecks across memory, compute, and communication.

Official
Advanced