mlm-bridge-training

Match Megatron-LM and Megatron Bridge loss curves by running the same GPT pretraining setup in both frameworks, then comparing iteration-level lm loss logs to verify agreement within BF16 rounding.

Loss divergence between Megatron-LM and Megatron Bridge training runs is typically caused by mismatched distributed settings, incorrect CLI parameter mapping, or inconsistent scheduler settings and checkpoint cleanup.

Map Megatron-LM CLI arguments into Megatron Bridge overrides by using documented translation tables and YAML recipe workflows to translate common MLM intent into the correct Bridge config fields.

You can reproduce multi-GPU tensor parallel and sequence parallel behavior by running the provided TP and SP examples in both MLM and Bridge to validate distributed scaling artifacts and loss correlation.

You need a YAML-described recipe workflow to launch Bridge training via run_recipe.py and ensure correct parameter mapping against the Megatron-LM pretrain_gpt.py command for accurate loss correlation.

Guardrails to prevent loss mismatch include enforcing fresh checkpoint cleanup, maintaining consistent scheduler settings, and applying correct parameter mapping across both training configurations.