skills.rest
NVIDIA-NeMo
NVIDIA-NeMoOFFICIAL
Explore all skills by this author →
WebsiteGitHub

perf-moe-comm-overlap

Configure and validate expert-parallel communication overlap in MoE models.

852|445|Updated May 21, 2025
Search Tags:#performance#communication#dispatch#training optimization#parallelism#moe
One-click install
npx skills add https://github.com/NVIDIA-NeMo/Megatron-Bridge --skill perf-moe-comm-overlap
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: perf-moe-comm-overlap
Source: https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/skills/perf-moe-comm-overlap
Command: npx skills add https://github.com/NVIDIA-NeMo/Megatron-Bridge --skill perf-moe-comm-overlap

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the need to improve the efficiency of expert-parallel communication in MoE models, reducing communication bottlenecks.

Core Features & Use Cases

  • Communication Overlap Tuning: Provides configurations and guidance for overlapping token dispatch and combine communication with computation in MoE training.
  • Flex Dispatcher Activation: Supports flexible dispatcher backends for advanced expert routing strategies.
  • Use Case: A researcher tuning large-scale MoE models can use this Skill to experiment with overlap configurations, aiming to improve training throughput and scalability.

Quick Start

Use the perf-moe-comm-overlap skill to optimize expert-parallel communication by enabling overlap configurations and validating improvements in training performance.

Frequently Asked Questions about perf-moe-comm-overlap

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize MoE communication overlap for higher training throughput?▼

To optimize MoE communication overlap, configure expert-parallel settings to overlap token dispatch and combine operations with computation, reducing bottlenecks and improving large-scale training efficiency.

What is the flex dispatcher used for in expert-parallel MoE models?▼

The flex dispatcher in expert-parallel MoE models activates flexible dispatcher backends to enable advanced token routing strategies, allowing communication and computation to overlap for better training scalability.

Does MoE communication overlap work with BF16 or FP16 precision and pipeline parallelism?▼

Yes, MoE communication overlap works with BF16 or FP16 precision and pipeline parallelism, but requires virtual pipeline support to ensure compatibility during large-scale model training.

How do I validate improvements after tuning expert-parallel communication configurations?▼

Validate expert-parallel communication improvements by measuring training throughput and efficiency before and after applying overlap configurations, ensuring the tuning effectively reduces communication bottlenecks.

When should I tune MoE token dispatch and combine communication?▼

Tune MoE token dispatch and combine communication when large-scale model training faces communication bottlenecks involving multiple experts and specialized dispatchers, requiring performance optimization and system debugging.

Related Skills

Explore More Software Engineering Skills→
NVIDIANVIDIA
2.8k

moe-comm-overlap

Enable MoE expert-parallel communication overlap with compute in Megatron Bridge.

Official
Advanced
NVIDIA-NeMoNVIDIA-NeMo
852

perf-tp-dp-comm-overlap

Configure TP, DP, and PP communication overlap in Megatron-Bridge training.

Official
Intermediate
yo-stevenyo-steven

perf-expert-parallel-overlap

Overlap expert-parallel token dispatch and combine communication with expert feed-forward compute in Megatron-Bridge MoE models.

Community
Advanced
yo-stevenyo-steven

perf-moe-optimization-workflow

Profile MoE training bottlenecks and apply optimization phases in Megatron Bridge.

Community
Advanced