moe-optimization-workflow

Diagnose and optimize MoE training bottlenecks across memory, compute, and communication.

2.8k|332|Updated Feb 25, 2026

Search Tags:#workflow #dispatcher #moe #cuda-graphs #megatron-bridge #training-optimization

Ready to run

npx skills add https://github.com/NVIDIA/skills --skill moe-optimization-workflow

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: moe-optimization-workflow
Source: https://github.com/NVIDIA/skills/tree/main/skills/Megatron-Bridge/perf-techniques/moe-optimization-workflow
Command: npx skills add https://github.com/NVIDIA/skills --skill moe-optimization-workflow

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Streamlines MoE training optimization by providing a repeatable workflow aligned with Megatron Bridge and Megatron Core principles to diagnose and fix bottlenecks.

Core Features & Use Cases

Three Walls framework guidance for memory, communication, and compute bottlenecks.
Parallel Folding and recompute strategy to decouple MoE and attention parallelism.
Dispatcher selection and CUDA Graphs guidance to maximize throughput on scalable GPU setups.

Quick Start

Run the moe-optimization-workflow with Megatron-Bridge to guide end-to-end MoE training tuning.

Frequently Asked Questions about moe-optimization-workflow

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I optimize MoE training bottlenecks across memory, compute, and communication?▼

To optimize MoE training bottlenecks, apply the Three Walls framework to diagnose memory, compute, and communication limits, then adjust parallel folding and FP8 strategies within Megatron-Bridge runs to resolve the identified constraints.

What is the Three Walls framework for MoE training optimization?▼

The Three Walls framework for MoE training optimization is a diagnostic methodology that categorizes performance bottlenecks into memory, communication, and compute limits, guiding targeted tuning strategies like parallel folding and dispatcher selection to maximize throughput.

How do I decouple MoE and attention parallelism in Megatron-Bridge?▼

To decouple MoE and attention parallelism in Megatron-Bridge, apply the parallel folding and recompute strategy provided by the optimization workflow, which separates the parallelism dimensions to prevent resource contention during training.

Do I need to know Megatron and hardware topology to tune MoE dispatcher settings?▼

Yes, tuning MoE dispatcher settings requires existing knowledge of Megatron, MoE architectures, and hardware topology, as the workflow relies on this understanding to guide configuration changes and profiling for scalable GPU setups.

When should I use CUDA Graphs for MoE training throughput?▼

You should use CUDA Graphs for MoE training throughput when seeking to maximize performance on scalable GPU setups, applying dispatcher tuning alongside it as needed to reduce overhead during Megatron-Bridge execution.

What is the best way to apply FP8 strategies to Megatron Core MoE runs?▼

The best way to apply FP8 strategies to Megatron Core MoE runs is to follow the structured optimization workflow, which integrates FP8 configuration with parallel folding and dispatcher tuning to systematically address compute and memory walls.