perf-cpu-offloading

What problem does it solve?

This Skill helps users reduce GPU memory usage and improve training stability by enabling CPU offloading of activations and optimizer states.

Core Features & Use Cases

• Memory Management: Offload transformer layer activations and optimizer states to CPU to prevent OOM errors during large model training.
• Performance Tuning: Adjust offloading fractions and configurations for balanced memory savings and training throughput.
• Use Case: An engineer training a 30B model on limited GPU memory can offload 50% of optimizer states to enable larger batch sizes without upgrading hardware.

Quick Start

Use the perf-cpu-offloading skill to configure optimizer offloading with 50% fraction and enable activation offloading for small models that fit with PP=1.