multi-node-slurm
OfficialScale single-node scripts to multi-node Slurm.
AuthorNVIDIA
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Convert single-node Megatron Bridge scripts into scalable multi-node Slurm sbatch jobs and provide structured debugging guidance for common multi-node failures.
Core Features & Use Cases
- Two-phase launch pattern: srun-native (preferred) and uv run torch.distributed (legacy), with automatic distribution initialization (RANK, WORLD_SIZE, LOCAL_RANK, MASTER_ADDR, MASTER_PORT) derived from SLURM vars.
- Containerized environments and resource setup, including container mounts, environment variable management, and log organization for multi-node runs.
- Debugging assistance for NCCL timeouts, OOM sizing for MoE models, and interactive allocation workflows; suitable for scaling Megatron-Bridge scripts to multi-node deployments.
Quick Start
Use when scaling a Megatron-Bridge script from a single node to a multi-node Slurm job, following the two-phase srun workflow.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: multi-node-slurm Download link: https://github.com/NVIDIA/skills/archive/main.zip#multi-node-slurm Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.