skills.rest
sayalinvidia
sayalinvidiaCOMMUNITY
Explore all skills by this author →
GitHub

nemo-mbridge-mlm-bridge-training

Compare Megatron-LM MLM training and Bridge workflows across multi-GPU configurations.

Updated Apr 16, 2026
Search Tags:#training#correlation#megatron-lm#multi-gpu#config-translation#mlm#megatron-bridge
One-click install
npx skills add https://github.com/sayalinvidia/sayali-skills-test --skill nemo-mbridge-mlm-bridge-training
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: nemo-mbridge-mlm-bridge-training
Source: https://github.com/sayalinvidia/sayali-skills-test/tree/main/skills/nemo-mbridge-mlm-bridge-training
Command: npx skills add https://github.com/sayalinvidia/sayali-skills-test --skill nemo-mbridge-mlm-bridge-training

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Megatron-LM MLM training and Megatron Bridge workflows are often difficult to compare and debug in isolation, hindering reproducibility and governance of model-training experiments.

Core Features & Use Cases

  • Correlation testing: compare MLM and Bridge losses across runs to verify consistency.
  • Arg mapping and translation: convert Megatron-LM CLI args to Bridge overrides and vice versa.
  • Multi-GPU training guidance: provide end-to-end instructions for scaling experiments and reproducing results across configurations.

Quick Start

Invoke vanilla_gpt_pretrain_config to perform a correlation test between MLM and Bridge using the Bridge entry point run_recipe.py and the MLM entry point pretrain_gpt.py.

Frequently Asked Questions about nemo-mbridge-mlm-bridge-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compare Megatron-LM MLM training and Megatron Bridge workflows?▼

You compare Megatron-LM MLM training and Bridge workflows by invoking vanilla_gpt_pretrain_config to run correlation tests, verifying BF16 loss agreement across runs using the pretrain_gpt.py and run_recipe.py entry points.

How do I translate Megatron-LM CLI args to Bridge overrides?▼

Arg mapping and translation converts Megatron-LM CLI args to Bridge overrides and vice versa, enabling configuration parity across multi-GPU training experiments using different model sizes and datasets.

What is the best way to launch multi-GPU training for MLM and Bridge correlation testing?▼

Launch multi-GPU training via uv run python -m torch.distributed.run, ensuring proper MLM PYTHONPATH setup and a fresh nemo_experiments cleanup before executing the correlation test across configurations.

Does Megatron Bridge support BF16 loss agreement checks against vanilla Megatron-LM?▼

Yes, Megatron Bridge supports BF16 loss agreement checks against vanilla Megatron-LM by coordinating end-to-end comparisons across different model sizes and datasets to ensure reproducibility.

Why are my MLM and Bridge training losses mismatching across multi-GPU configurations?▼

Loss mismatches between MLM and Bridge training often occur when arg mapping translation is incorrect or when the nemo_experiments directory is not cleaned up, causing stale configuration conflicts across multi-GPU runs.

Related Skills

Explore More Software Engineering Skills→
OmniNode-aiOmniNode-ai
3

using-git-worktrees

Create and manage isolated Git worktrees with safety checks and cleanup.

Official
Advanced
ovachieverovachiever
52

simpo-training

Optimize LLM preference alignment without a reference model using comparison data.

Community
Advanced
ovachieverovachiever
52

pytorch-fsdp

Apply PyTorch FSDP for sharded, mixed-precision distributed training of large models.

Community
Advanced
NVIDIANVIDIA
2.8k

mlm-bridge-training

Coordinate Megatron-LM MLM pretraining and Megatron Bridge training experiments on multi-GPU setups.

Official
Advanced