perf-hybrid-context-parallel
OfficialOptimize hierarchical context parallelism in Megatron-Bridge.
AuthorNVIDIA-NeMo
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill provides guidance and configuration instructions for enabling hierarchical context parallelism in Megatron-Bridge, helping users scale model training efficiently.
Core Features & Use Cases
- Configuration Guidance: Explains how to set
context_parallel_sizeandhierarchical_context_parallel_sizes. - Validation & Constraints: Details the validation rules for hierarchical parallelism parameters and available code anchors.
- Use Case: Use this Skill when scaling Megatron models beyond KV heads or troubleshooting OOM issues related to context parallelism; for example, deploying a large model with layered parallel splits.
Quick Start
Refer to the provided documentation and code snippets to activate hierarchical context parallelism in your Megatron-Bridge setup.
Dependency Matrix
Required Modules
None requiredComponents
scriptsreferences
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: perf-hybrid-context-parallel Download link: https://github.com/NVIDIA-NeMo/Megatron-Bridge/archive/main.zip#perf-hybrid-context-parallel Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.