activation-recompute
OfficialReduce GPU memory with selective recompute.
AuthorNVIDIA
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Activation recompute trades GPU memory for compute by discarding intermediate activations during the forward pass and recomputing them during backward in Megatron Bridge.
Core Features & Use Cases
- Supports selective recompute (per-submodule) and full-layer recompute (per transformer layer) to tune memory-compute tradeoffs.
- Provides clear enablement guidelines and compatibility notes for CUDA graphs, CPU offloading, and model configurations.
- Applies to large transformer training workflows where memory is a bottleneck, enabling deeper or larger models without hardware upgrades.
Quick Start
Configure recompute_granularity to selective and set recompute_modules to your target submodules to begin trading memory for compute in Megatron Bridge.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: activation-recompute Download link: https://github.com/NVIDIA/skills/archive/main.zip#activation-recompute Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.