perf-moe-optimization-workflow

To optimize MoE training throughput, follow a structured workflow that ensures memory feasibility, selects scale-oriented parallelism, and profiles to target the dominant bottleneck with dispatcher, overlap, FP8, and CUDA-graph adjustments.

Diagnose a MoE throughput regression by profiling to pinpoint the dominant bottleneck, checking if memory pressure, all-to-all/collective traffic, or host/launch overhead is the real limiter rather than applying random tweaks.

Parallel Folding helps MoE performance tuning by providing scale-oriented parallelism selection during the second phase of the optimization workflow, ensuring the run configuration supports efficient throughput before profiling bottlenecks.

Apply FP8 recipe selection and CUDA-graphs bring-up during the final optimization phase after confirming memory feasibility and parallelism, specifically targeting the dominant compute or host overhead bottleneck with recompute compatibility constraints.

Yes, the MoE optimization workflow is designed for Megatron Bridge environments, applying the Three Walls framework to systematically target memory, communication, and compute bottlenecks for full throughput tuning sweeps.

MoE training throughput is often limited because the dominant bottleneck—memory pressure, all-to-all/collective traffic, or host/launch overhead—has not been correctly identified and addressed through systematic profiling rather than random tweaks.