training-llms-megatron
CommunityMegatron-Core: 3D parallelism for huge LLMs.
Software Engineering#distributed-training#data-parallel#tensor-parallel#pipeline-parallel#megatron-core#H100
Authorovachiever
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill guides large-scale LLM training with NVIDIA Megatron-Core, covering 3D parallelism (tensor, pipeline, data, context), MoE, and production-grade ML engineering practices.
Core Features & Use Cases
- 3D Parallelism: Tensor, pipeline, data, and context parallelism for mega-models.
- MoE & Optimization: Mixture-of-Experts configurations and efficiency techniques.
- Production Patterns: Fault tolerance, checkpointing, and scalable deployment patterns.
Quick Start
Set up a 70B model with TP/PP/CP to explore parallelism strategies.
Dependency Matrix
Required Modules
None requiredComponents
references
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: training-llms-megatron Download link: https://github.com/ovachiever/droid-tings/archive/main.zip#training-llms-megatron Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.