Configure GPU cluster parallelism and manage Megatron-LM training workflows.
npx skills add https://github.com/tylertitsworth/skills --skill megatron-lm
Please help me install this Agent Skill. Skill: megatron-lm Source: https://github.com/tylertitsworth/skills/tree/main/megatron-lm Command: npx skills add https://github.com/tylertitsworth/skills --skill megatron-lm
Megatron-LM enables scalable training of transformer models at massive parameter counts across GPU clusters.
Configure a multi-node GPU cluster with tensor, pipeline, and context parallelism, then run the Megatron-LM training script to start a scalable model training job.
High-intent search queries and answers about installing and using this skill.
To scale transformer training, you configure tensor, pipeline, sequence, context, and expert parallelism across a multi-node GPU cluster. This distributes massive parameter counts efficiently during the training process.
Yes, you can convert checkpoints between HuggingFace and Megatron formats using Megatron Bridge. This enables seamless integration for large-scale training workflows and model deployment.
Yes, NVIDIA GPUs with CUDA are required for Megatron training. The environment also needs PyTorch and Megatron Core, with optional Megatron Bridge support for checkpoint conversion.
Training MoE models across GPU clusters requires configuring expert parallelism alongside tensor, pipeline, and context parallelism. This scales massive transformer models while managing mixed precision and data loading.
Mixed precision in large transformer training optimizes GPU memory usage and computational speed across distributed clusters. It is managed natively during the scalable training process for models with massive parameter counts.
Megatron training is limited to NVIDIA GPU clusters with CUDA and requires PyTorch, Megatron Core, and optional Megatron Bridge. It is designed for massive parameter scales and requires multi-node deployment infrastructure.
Clay-HHKProvision dedicated GPU cloud instances and Slurm clusters for ML training.
BirgerMoellConvert HuggingFace and Megatron-LM checkpoints with numerical equivalence validation.
overviewlabsIntegrates seamlessly with neural network frameworksGitHub-like APIs for converting and encoding raw text streams and Hugging Face Hub datasets using the BRAIN, LIP, and/or LIPREADING models.
tadod12Configure DeepSpeed distributed training for large transformer models with ZeRO and mixed precision.
