moe-vlm-training

What problem does it solve?

Practical, field-tested guidance for training MoE-based VLMs in Megatron Bridge, helping engineers choose between FSDP and 3D-parallel approaches and stabilize multi-GPU workflows across diverse hardware.

Core Features & Use Cases

• Decision framework for selecting FSDP vs 3D-parallel paths in MoE-VLM training.
• Hardware and memory guidance tailored to GB200/B200-class clusters with real-data considerations.
• Real-world usage scenarios spanning Qwen3-VL-style models and related multimodal experiments.

Quick Start

Run the MoE-VLM training guide to select a training path and initialize a Megatron Bridge experiment.