nemo-mbridge-perf-moe-optimization-workflow

MoE training throughput in Megatron Bridge is optimized using a three-phase workflow: fit, scale, profile, and retune. This process targets memory, communication, and compute bottlenecks using the Three Walls framework and Parallel Folding meshes.

The Three Walls framework is a diagnostic model that categorizes MoE training performance bottlenecks into memory, communication, and compute limits. It guides phase-specific optimization recommendations to systematically resolve throughput constraints.

Yes, the workflow provides explicit FP8 mapping guidance and CUDA graph considerations for MoE workloads. These features are designed to accelerate training across supported hardware platforms, including Hopper and Blackwell architectures.

Parallel Folding decouples attention and MoE parallelism, enabling scalable multi-GPU configurations. By separating these parallelism dimensions, it allows more flexible and efficient resource utilization during training.

Throughput regressions in MoE training can be diagnosed by running the workflow's profile and retune phases. These steps identify whether the bottleneck stems from memory, communication, or compute changes introduced by the new code.

Yes, this workflow provides specific optimization guidance for both Hopper and Blackwell platforms. It includes hardware-aware recommendations for dispatcher selection, FP8 mappings, and CUDA graphs to maximize throughput on either architecture.