Megatron-LM

Manage tensor parallelism, inference optimizations, and mixed-precision support for Megatron-LM.

Updated Feb 16, 2026

Search Tags:#llm #inference #cuda #distributed training #triton #megatron-lm #tensor parallelism

Ready to run

npx skills add https://github.com/pedestrianlove/skills --skill megatron-lm-pedestrianlove

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: Megatron-LM
Source: https://github.com/pedestrianlove/skills/tree/main/skills/Megatron-LM
Command: npx skills add https://github.com/pedestrianlove/skills --skill megatron-lm-pedestrianlove

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill provides a collection of utilities and functions specifically designed to support agents interacting with or leveraging the Megatron-LM framework. It aims to streamline complex operations related to large language model development and deployment.

Core Features & Use Cases

Tensor Parallelism Utilities: Functions for splitting, gathering, and managing tensors across parallel processes.
Inference Optimizations: Kernels and utilities for efficient text generation and inference, including memory management and communication.
FP8 and Quantization Support: Tools for handling mixed-precision training and inference with FP8 tensors.
Use Case: An agent can use this skill to manage the distribution of model weights across multiple GPUs for training a large transformer model, or to optimize the inference speed of a deployed Megatron-LM model.

Quick Start

Use the Megatron-LM skill to split a tensor along its last dimension into 8 partitions.

Frequently Asked Questions about Megatron-LM

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I split tensors across multiple GPUs for distributed training?▼

Tensor parallelism utilities split, gather, and manage tensors across parallel processes to distribute model weights efficiently during large transformer model training.

What's the best way to optimize inference speed for a deployed Megatron-LM model?▼

Inference optimizations provide specialized kernels and utilities for efficient text generation, memory management, and communication to accelerate deployed large language model inference.

Can I use FP8 mixed-precision training with tensor parallelism?▼

FP8 and quantization support tools handle mixed-precision training and inference with FP8 tensors, working alongside tensor parallelism utilities for distributed large language model operations.

Does Megatron-LM support Triton kernel implementations for collective communication?▼

Triton kernel implementations are included for collective communication and tensor operations, providing optimized performance for distributed training and inference workflows.

How do I manage unified memory and CUDA graph integration for large language models?▼

Utilities for CUDA graph integration and unified memory management facilitate efficient memory handling during high-performance inference and distributed training of large language models.

Do I need CUDA to use the tensor parallelism and inference optimization tools?▼

CUDA integration is required as the tools provide CUDA graph integration, Triton kernel implementations, and GPU-based tensor operations for distributed training and inference.