skills.rest
pedestrianlove
pedestrianloveCOMMUNITY
Explore all skills by this author →
GitHub

Megatron-LM

Manage tensor parallelism, inference optimizations, and mixed-precision support for Megatron-LM.

Updated Feb 16, 2026
Search Tags:#llm#inference#cuda#distributed training#triton#megatron-lm#tensor parallelism
One-click install
npx skills add https://github.com/pedestrianlove/skills --skill megatron-lm-pedestrianlove
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: Megatron-LM
Source: https://github.com/pedestrianlove/skills/tree/main/skills/Megatron-LM
Command: npx skills add https://github.com/pedestrianlove/skills --skill megatron-lm-pedestrianlove

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill provides a collection of utilities and functions specifically designed to support agents interacting with or leveraging the Megatron-LM framework. It aims to streamline complex operations related to large language model development and deployment.

Core Features & Use Cases

  • Tensor Parallelism Utilities: Functions for splitting, gathering, and managing tensors across parallel processes.
  • Inference Optimizations: Kernels and utilities for efficient text generation and inference, including memory management and communication.
  • FP8 and Quantization Support: Tools for handling mixed-precision training and inference with FP8 tensors.
  • Use Case: An agent can use this skill to manage the distribution of model weights across multiple GPUs for training a large transformer model, or to optimize the inference speed of a deployed Megatron-LM model.

Quick Start

Use the Megatron-LM skill to split a tensor along its last dimension into 8 partitions.

Frequently Asked Questions about Megatron-LM

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I split tensors across multiple GPUs for distributed training?▼

Tensor parallelism utilities split, gather, and manage tensors across parallel processes to distribute model weights efficiently during large transformer model training.

What's the best way to optimize inference speed for a deployed Megatron-LM model?▼

Inference optimizations provide specialized kernels and utilities for efficient text generation, memory management, and communication to accelerate deployed large language model inference.

Can I use FP8 mixed-precision training with tensor parallelism?▼

FP8 and quantization support tools handle mixed-precision training and inference with FP8 tensors, working alongside tensor parallelism utilities for distributed large language model operations.

Does Megatron-LM support Triton kernel implementations for collective communication?▼

Triton kernel implementations are included for collective communication and tensor operations, providing optimized performance for distributed training and inference workflows.

How do I manage unified memory and CUDA graph integration for large language models?▼

Utilities for CUDA graph integration and unified memory management facilitate efficient memory handling during high-performance inference and distributed training of large language models.

Do I need CUDA to use the tensor parallelism and inference optimization tools?▼

CUDA integration is required as the tools provide CUDA graph integration, Triton kernel implementations, and GPU-based tensor operations for distributed training and inference.

Related Skills

Explore More Software Engineering Skills→
SriRamkunamsettySriRamkunamsetty

distributed-llm-pretraining-torchtitan

Enable PyTorch-native distributed LLM pretraining with 4D parallelism on GPU clusters.

Community
Advanced
kngender5kngender5

colab-llm-inference

Deploy LLMs on Google Colab with a public Gradio URL for text generation.

Community
Advanced
xchang1121xchang1121
6

triton-cuda-patterns

Implement Triton CUDA kernel templates for elementwise, reduction, and matmul workloads.

Community
Intermediate
ihatesea69ihatesea69
3

pytorch-fsdp2

Integrate PyTorch FSDP2 fully_shard into training scripts for large models.

Community
Advanced