perf-cuda-graphs

What problem does it solve?

This Skill enables efficient GPU computation by capturing CUDA graphs to reduce host-driver overhead, significantly speeding up training iterations.

Core Features & Use Cases

• CUDA Graph Capture: Supports local full-iteration and TE-scoped graphs for attention, MLP, and MoE modules.
• Performance Optimization: Applicable for training large models with static shapes, improving throughput and reducing iteration time.
• Use Case: Use this Skill to accelerate training of large-scale language models on GPU clusters by minimizing kernel launch overhead.

Quick Start

Use the cuda-graphs skill to enable CUDA graph capture during training by setting the implementation to 'transformer_engine' and configuring the scope for attention and MoE modules.