nemo-mbridge-perf-moe-vlm-training

What problem does it solve?

Practical guidance for training MoE VLMs in Megatron Bridge. Compares FSDP and 3D-parallel approaches, using rounded lessons from Qwen3-VL, Qwen3-Next, and other multimodal experiments.

Core Features & Use Cases

• Guidance on choosing between FSDP and 3D-parallel for MoE VLM training
• Tuning tips on memory, recompute, and CUDA graph considerations
• Real-world use case: selecting best parallel strategy for GB200/B200 scale VLMs

Quick Start

Execute a two-path MoE VLM training plan on Megatron Bridge by evaluating FSDP-first and 3D-parallel configurations and documenting their trade-offs.