skills.rest
yo-steven
yo-stevenCOMMUNITY
Explore all skills by this author →
GitHub

perf-expert-parallel-overlap

Overlap expert-parallel token dispatch and combine communication with expert feed-forward compute in Megatron-Bridge MoE models.

Updated May 23, 2026
Search Tags:#moe#megatron-bridge#communication overlap#throughput tuning#cuda graphs#expert parallel#flex dispatcher
One-click install
npx skills add https://github.com/yo-steven/skills-exploration-20260522 --skill perf-expert-parallel-overlap-yo-steven
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: perf-expert-parallel-overlap
Source: https://github.com/yo-steven/skills-exploration-20260522/tree/main/skills/Megatron-Bridge/perf-expert-parallel-overlap
Command: npx skills add https://github.com/yo-steven/skills-exploration-20260522 --skill perf-expert-parallel-overlap-yo-steven

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

MoE training can spend a significant fraction of step time on expert token dispatch/combine communication, leaving expert compute underutilized.

Core Features & Use Cases

  • Enables expert-parallel communication overlap: sets comm_overlap.overlap_moe_expert_parallel_comm to overlap MoE all-to-all dispatch/combine with expert FFN compute.
  • Optionally delays expert weight-gradient compute: sets comm_overlap.delay_wgrad_compute to defer wgrad so it overlaps with the next layer’s forward, reducing idle time when supported.
  • Selects dispatcher backends for deeper overlap: supports model.moe_token_dispatcher_type = "alltoall" and flex (via apply_flex_dispatcher_backend) to target DeepEP/HybridEP-style overlap.
  • Validation-first guardrails: enforces critical constraints (EP degree, precision, recompute settings, VPP with PP, PyTorch/TE versions, and mutual exclusivity with moe_shared_expert_overlap).

What problem does it solve?

  • Real-world use case: When profiling a MoE run and communication shows up as a bottleneck, turn on EP overlap to reduce step time without changing convergence behavior.

Quick Start

Enable the skill by setting cfg.comm_overlap.overlap_moe_expert_parallel_comm = True, keeping cfg.comm_overlap.delay_wgrad_compute = False initially, and setting cfg.model.moe_token_dispatcher_type = "alltoall" with cfg.model.moe_shared_expert_overlap = False for a correctness-first start.

Frequently Asked Questions about perf-expert-parallel-overlap

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I overlap expert-parallel token dispatch communication with MoE compute?▼

To overlap expert-parallel token dispatch communication with MoE compute, enable `cfg.comm_overlap.overlap_moe_expert_parallel_comm` to overlap all-to-all dispatch/combine traffic with expert feed-forward compute in Megatron-Bridge models.

Why does Mixture-of-Experts training leave expert compute underutilized during token dispatch?▼

MoE training leaves expert compute underutilized because step time is consumed by expert-parallel token dispatch/combine communication. Enabling communication overlap hides this latency by running expert feed-forward compute concurrently with the traffic.

Can I delay weight-gradient compute to reduce idle time during MoE communication overlap?▼

Yes, you can delay weight-gradient compute to reduce idle time by setting `cfg.comm_overlap.delay_wgrad_compute = True`. This defers wgrad to overlap with the next layer's forward, requiring TE-gated support and enabling EP overlap.

Does expert-parallel communication overlap work with the flex dispatcher backend?▼

Yes, expert-parallel communication overlap works with the flex dispatcher backend. You can set `cfg.model.moe_token_dispatcher_type = "flex"` via `apply_flex_dispatcher_backend` to target DeepEP/HybridEP-style overlap.

What are the configuration constraints for enabling EP overlap in MoE models?▼

Configuration constraints for enabling EP overlap require expert-parallelism degree > 1, expert count > 1, PyTorch >= 2.6.0, BF16/FP16 precision, recompute/VPP exclusions, and mutual exclusivity with `moe_shared_expert_overlap`.

When should I not use expert-parallel communication overlap?▼

You should not use expert-parallel communication overlap when dispatcher latency is immaterial, when using `moe_shared_expert_overlap`, or when violating recompute/VPP exclusions and PyTorch >= 2.6.0 constraints, as correctness guardrails will block execution.

Related Skills

Explore More Software Engineering Skills→
sayalinvidiasayalinvidia

nemo-mbridge-perf-expert-parallel-overlap

Enable MoE expert-parallel communication overlap in Megatron-Bridge.

Community
Advanced
sayalinvidiasayalinvidia

nemo-mbridge-perf-moe-comm-overlap

Enable expert-parallel communication overlap for Megatron Bridge MoE training.

Community
Advanced
NVIDIANVIDIA
2.8k

moe-optimization-workflow

Diagnose and optimize MoE training bottlenecks across memory, compute, and communication.

Official
Advanced
NVIDIANVIDIA
2.8k

moe-comm-overlap

Enable MoE expert-parallel communication overlap with compute in Megatron Bridge.

Official
Advanced