Megatron-Bridge
Train and convert large AI models across GPU clusters
All Skills in This Repository (53)
Pure Emerald Level Indicators
NVIDIA-NeMoperf-sequence-packing
Configure PackedSequenceSpecs for sequence packing in model training.
NVIDIA-NeMoperf-moe-vlm-training
Guide MoE vision-language model training with parallelism strategy comparisons.
NVIDIA-NeMoperf-activation-recompute
Recompute transformer activations to reduce GPU memory usage during training.
NVIDIA-NeMoperf-moe-dispatcher-selection
Select optimal MoE token dispatchers based on hardware and model size.
NVIDIA-NeMoperf-moe-optimization-workflow
Optimize MoE model training throughput by addressing memory, communication, and compute bottlenecks.
NVIDIA-NeMoadding-model-support
Add language or multimodal models to Megatron-Bridge via config analysis and custom weight mapping.
NVIDIA-NeMoperf-parallelism-strategies
Select and configure parallelism strategies for large-scale model training in Megatron Bridge.
NVIDIA-NeMoperf-tp-dp-comm-overlap
Configure TP, DP, and PP communication overlap in Megatron-Bridge training.
NVIDIA-NeMolinting-and-formatting
Enforce Python and shell script style rules with ruff, isort, and pre-commit.
NVIDIA-NeMocicd
Diagnose CI/CD failures by locating logs and detecting failure patterns.
NVIDIA-NeMotesting
Organize, execute, and manage unit and functional tests for Megatron Bridge.
NVIDIA-NeMoperf-expert-parallel-overlap
Validates and tunes expert-parallel communication overlap for Megatron-Bridge MoE training on GPU clusters.
Frequently Asked Questions
FAQPage SchemaHow to install Megatron-Bridge?▼
Run `npx skills add NVIDIA-NeMo/Megatron-Bridge --all -g -y` in your terminal to install all skills in this suite globally.
What does Megatron-Bridge do?▼
It converts Hugging Face model checkpoints into NVIDIA Megatron-Core format and back, then runs pretraining, SFT, and LoRA fine-tuning at large GPU scale. The skills guide agents through conversion, training, and performance tuning.
How to fix GPU out-of-memory errors in Megatron training?▼
The memory tuning skill recommends setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True first, since most OOMs come from fragmentation, not capacity. Further skills cover activation recompute, CPU offloading, and parallelism resizing.
How to scale training across multiple nodes with Slurm?▼
The multi-node Slurm skill converts single-node scripts into sbatch jobs using the srun-native launch pattern, with shared filesystem paths and NCCL timeout debugging steps. Megatron-Bridge derives rank and world size from Slurm variables automatically.
Does Megatron-Bridge work with Claude Code and other AI agents?▼
Yes. The AGENTS.md orchestrator and SKILL.md files follow the open agent-skills standard, so Claude Code, Codex, and compatible agents can load the right guide for each training or conversion task.
Related Repositories in Software Engineering
View All in Software Engineering→
openclawopenclaw
Run a personal AI assistant across your devices and chat apps
obrasuperpowers
Gives coding agents a disciplined workflow from idea to merged code
facebookreact
AI agent skills for building, testing, and porting React core