nemo-mbridge-perf-activation-recompute

Reduce GPU memory usage by configuring selective and full activation recompute in Megatron Bridge. Discard intermediate activations during the forward pass and recompute them during backward to enable training under tighter memory budgets.

Configure selective activation recompute first by targeting specific submodules like core_attn and optionally adding layernorm. This saves memory with moderate compute cost, keeping TE-scoped CUDA graphs in mind for Llama3-70b compatibility.

Escalate to full recompute when memory pressure persists after applying selective recompute. Use recompute_num_layers and recompute_method to maximize memory savings when memory pressure is extreme.

TE-scoped CUDA graphs have specific compatibility constraints when toggling to full recompute. Ensure selective recompute has a non-empty recompute_modules list and apply these compatibility limitations during memory tuning for large models.

Activation recompute trades GPU memory for compute by discarding intermediate activations during the forward pass and recomputing them during backward, enabling training under tighter memory budgets for large Transformer models.

Selective recompute requires a non-empty recompute_modules list to specify which submodules to recompute. Without specifying target modules like core_attn or layernorm, the selective recompute mechanism cannot apply memory savings.