skills.rest
sayalinvidia
sayalinvidiaCOMMUNITY
Explore all skills by this author →
GitHub

nemo-mbridge-perf-moe-comm-overlap

Enable expert-parallel communication overlap for Megatron Bridge MoE training.

Updated Apr 16, 2026
Search Tags:#throughput#moe#megatron-bridge#communication-overlap#model-parallel#token-dispatch
One-click install
npx skills add https://github.com/sayalinvidia/sayali-skills-test --skill nemo-mbridge-perf-moe-comm-overlap
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: nemo-mbridge-perf-moe-comm-overlap
Source: https://github.com/sayalinvidia/sayali-skills-test/tree/main/skills/nemo-mbridge-perf-moe-comm-overlap
Command: npx skills add https://github.com/sayalinvidia/sayali-skills-test --skill nemo-mbridge-perf-moe-comm-overlap

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill enables performance-tuning of MoE training on Megatron Bridge by enabling overlapping of expert-parallel communication, reducing idle compute time.

Core Features & Use Cases

  • MoE communication overlap knob: enables overlap between token dispatch/combine and expert computation for MoE layers.
  • Optional delayed wgrad scheduling: allows additional overlap opportunities when compatible with CUDA graph scopes.
  • Flex vs all-to-all dispatcher awareness: guides valid configurations for different dispatcher backends and dependencies between flags.
  • Use Case: tune throughput of large MoE training jobs when EP>1 and profile shows communication time as a bottleneck.

Quick Start

Execute a basic activation of overlap on a representative run and verify throughput improvements.

Frequently Asked Questions about nemo-mbridge-perf-moe-comm-overlap

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I improve MoE training throughput when expert-parallel communication is a bottleneck?▼

To improve MoE training throughput, enable expert-parallel communication overlap to parallelize token dispatch and combine operations with expert computation, reducing idle compute time when expert model parallel size is greater than one.

What are the prerequisites for enabling communication overlap in Megatron Bridge MoE training?▼

Enabling communication overlap requires expert_model_parallel_size greater than 1, num_moe_experts greater than 1, and a token dispatcher type of either alltoall or flex. If pipeline parallelism is used, virtual pipeline parallelism is also required.

Does communication overlap work with both alltoall and flex token dispatchers?▼

Yes, communication overlap supports both alltoall and flex token dispatchers. Configuring flex dispatcher specifically requires setting moe_token_dispatcher_type to flex to ensure valid backend operations.

When should I enable delayed wgrad scheduling for MoE layers?▼

You should enable delayed wgrad scheduling to create additional overlap opportunities when the configuration is compatible with CUDA graph scopes, further optimizing MoE training throughput.

How do I know if communication time is causing idle compute in my MoE training profile?▼

If your MoE training profile shows visible token dispatch or combine times when expert parallelism is greater than one, communication is a bottleneck and overlapping expert-parallel communication can tune throughput.

Related Skills

Explore More Software Engineering Skills→
yo-stevenyo-steven

perf-expert-parallel-overlap

Overlap expert-parallel token dispatch and combine communication with expert feed-forward compute in Megatron-Bridge MoE models.

Community
Advanced
NVIDIANVIDIA
2.8k

expert-parallel-overlap

Overlap expert-parallel communication with compute in MoE training.

Official
Advanced
yo-stevenyo-steven

perf-moe-comm-overlap

Overlap expert-parallel dispatch and combine communication with expert compute in Megatron-Bridge.

Community
Advanced
NVIDIANVIDIA
2.8k

moe-comm-overlap

Enable MoE expert-parallel communication overlap with compute in Megatron Bridge.

Official
Advanced