nemo-mbridge-mlm-bridge-training

What problem does it solve?

Megatron-LM MLM training and Megatron Bridge workflows are often difficult to compare and debug in isolation, hindering reproducibility and governance of model-training experiments.

Core Features & Use Cases

• Correlation testing: compare MLM and Bridge losses across runs to verify consistency.
• Arg mapping and translation: convert Megatron-LM CLI args to Bridge overrides and vice versa.
• Multi-GPU training guidance: provide end-to-end instructions for scaling experiments and reproducing results across configurations.

Quick Start

Invoke vanilla_gpt_pretrain_config to perform a correlation test between MLM and Bridge using the Bridge entry point run_recipe.py and the MLM entry point pretrain_gpt.py.