perf-activation-recompute

What problem does it solve?

It reduces GPU memory usage during training by selectively recomputing activations, allowing larger models or batch sizes without crashing.

Core Features & Use Cases

• Memory Optimization: Recomputes specific modules like attention and layernorm to save memory.
• Training Efficiency: Helps investigate and fix out-of-memory issues caused by memory fragmentation or large model sizes.
• Use Case: When training a large transformer model that exceeds available GPU memory, apply recompute strategies to fit the model into hardware constraints.

Quick Start

Configure your training setup to enable selective recompute on the attention modules and run your training script to reduce peak memory usage.