skills.rest
NVIDIA-NeMo
NVIDIA-NeMoOFFICIAL
Explore all skills by this author →
WebsiteGitHub

perf-moe-dispatcher-selection

Select optimal MoE token dispatchers based on hardware and model size.

852|445|Updated May 21, 2025
Search Tags:#performance optimization#model scaling#distributed training#dispatchers#moe#hardware tuning
One-click install
npx skills add https://github.com/NVIDIA-NeMo/Megatron-Bridge --skill perf-moe-dispatcher-selection
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: perf-moe-dispatcher-selection
Source: https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/skills/perf-moe-dispatcher-selection
Command: npx skills add https://github.com/NVIDIA-NeMo/Megatron-Bridge --skill perf-moe-dispatcher-selection

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps developers choose the optimal MoE token dispatcher (such as alltoall, DeepEP, or HybridEP) based on hardware platform, model size, and EP degree, ensuring maximum performance and efficiency.

Core Features & Use Cases

  • Dispatcher Recommendation: Guides users in selecting the best dispatcher configuration for H100, B200, GB200, or GB300 systems.
  • Performance Tuning Advice: Provides insights on tuning SM counts and routing modes for specific models and hardware.
  • Use Case: A researcher working on large-scale MoE models on GB200 systems can determine whether to use DeepEP or HybridEP for optimal throughput and memory utilization.

Quick Start

Ask the AI which MoE dispatcher setting is best for a 685B model running on a 256×GB200 system to improve performance.

Frequently Asked Questions about perf-moe-dispatcher-selection

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I choose the best MoE dispatcher for large-scale distributed training?▼

Choosing the best MoE dispatcher depends on your hardware platform, model size, and expert parallelism degree. This Skill guides you in selecting the optimal token dispatcher, ensuring maximum performance, memory efficiency, and scalability for your specific distributed training scenario.

What is the difference between alltoall, DeepEP, and hybridEP dispatchers?▼

Alltoall, DeepEP, and hybridEP are MoE token dispatchers that handle routing and communication differently. This Skill provides recommendations on selecting among them based on specific hardware capabilities and model scaling requirements to optimize throughput and memory utilization.

Does the DeepEP dispatcher work well on GB200 systems for 685B models?▼

DeepEP can be highly effective on GB200 systems for large models like 685B parameters. This Skill helps you determine whether DeepEP or HybridEP offers the optimal throughput and memory utilization for your specific hardware and model scale.

How to tune SM counts and routing modes for MoE dispatchers?▼

Tuning SM counts and routing modes requires matching parameters to your specific hardware and model size. This Skill provides targeted performance tuning advice to configure these settings safely for maximum efficiency in distributed training.

When should I switch from alltoall to HybridEP for model scaling?▼

You should switch from alltoall to HybridEP when model scaling demands higher throughput and memory optimization across large expert parallelism degrees. This Skill evaluates your scaling factors to recommend the safest and most efficient dispatcher transition.

Related Skills

Explore More Software Engineering Skills→
KappTech88KappTech88

moe-training

Configure DeepSpeed MoE training pipelines with expert parallelism and load balancing.

Community
Advanced
blakeoxblakeox

performance-engineer

Identify system bottlenecks and performance inefficiencies in software systems.

Community
Advanced
MesferAliMesferAli

ray-train

Orchestrate distributed machine learning training across clusters with Ray Train.

Community
Advanced
NVIDIANVIDIA
2.8k

moe-dispatcher-selection

Select optimal MoE token dispatchers based on hardware, EP degree, and workload.

Official
Advanced