nemo-mbridge-perf-moe-optimization-workflow

What problem does it solve?

This workflow provides a structured approach to diagnose and improve throughput for MoE training on Megatron Bridge, addressing memory, communication, and compute bottlenecks.

Core Features & Use Cases

• Phase-driven optimization using the Three Walls framework to guide memory, communication, and compute improvements.
• Parallel Folding decouples attention and MoE parallelism, enabling scalable multi-GPU configurations.
• Dispatcher selection, FP8 mapping guidance, and CUDA graph considerations to accelerate MoE workloads across hardware (e.g., Hopper and Blackwell).
• Use cases include diagnosing throughput regressions after commits and performing end-to-end MoE throughput tuning sweeps.

Quick Start

Initiate a three-phase MoE optimization by guiding the agent through fit, scale, profile, and retune steps using the Parallel Folding meshes and recommended dispatcher and FP8 mappings.