perf-moe-comm-overlap

Name: perf-moe-comm-overlap
Availability: InStock
Author: NVIDIA-NeMo

Configure and validate expert-parallel communication overlap in MoE models.

852|445|Updated May 21, 2025

Search Tags:#performance #communication #dispatch #training optimization #parallelism #moe

Ready to run

npx skills add https://github.com/NVIDIA-NeMo/Megatron-Bridge --skill perf-moe-comm-overlap

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: perf-moe-comm-overlap
Source: https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/skills/perf-moe-comm-overlap
Command: npx skills add https://github.com/NVIDIA-NeMo/Megatron-Bridge --skill perf-moe-comm-overlap

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the need to improve the efficiency of expert-parallel communication in MoE models, reducing communication bottlenecks.

Core Features & Use Cases

Communication Overlap Tuning: Provides configurations and guidance for overlapping token dispatch and combine communication with computation in MoE training.
Flex Dispatcher Activation: Supports flexible dispatcher backends for advanced expert routing strategies.
Use Case: A researcher tuning large-scale MoE models can use this Skill to experiment with overlap configurations, aiming to improve training throughput and scalability.

Quick Start

Use the perf-moe-comm-overlap skill to optimize expert-parallel communication by enabling overlap configurations and validating improvements in training performance.

Frequently Asked Questions about perf-moe-comm-overlap

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I optimize MoE communication overlap for higher training throughput?▼

To optimize MoE communication overlap, configure expert-parallel settings to overlap token dispatch and combine operations with computation, reducing bottlenecks and improving large-scale training efficiency.

What is the flex dispatcher used for in expert-parallel MoE models?▼

The flex dispatcher in expert-parallel MoE models activates flexible dispatcher backends to enable advanced token routing strategies, allowing communication and computation to overlap for better training scalability.

Does MoE communication overlap work with BF16 or FP16 precision and pipeline parallelism?▼

Yes, MoE communication overlap works with BF16 or FP16 precision and pipeline parallelism, but requires virtual pipeline support to ensure compatibility during large-scale model training.

How do I validate improvements after tuning expert-parallel communication configurations?▼

Validate expert-parallel communication improvements by measuring training throughput and efficiency before and after applying overlap configurations, ensuring the tuning effectively reduces communication bottlenecks.

When should I tune MoE token dispatch and combine communication?▼

Tune MoE token dispatch and combine communication when large-scale model training faces communication bottlenecks involving multiple experts and specialized dispatchers, requiring performance optimization and system debugging.