perf-moe-long-context

What problem does it solve?

This Skill provides guidance for efficient training of MoE models with extensive context lengths, addressing memory constraints and throughput challenges.

Core Features & Use Cases

• Long-Sequence Training Optimization: Offers strategies for scaling MoE models to sequences up to 256K tokens, including CP sizing and recompute methods.
• Configuration Guidance: Details specific dispatcher choices, precision settings, and CUDA graph policies for big models.
• Use Case: Developers training long-context MoE models can implement these best practices to improve performance and resource utilization, such as handling 128K token sequences in NLP tasks.

Quick Start

Refer to the recommended configuration patterns and scaling rules in the documentation to set up your long-context MoE training environment effectively.