perf-moe-optimization-workflow

What problem does it solve?

This Skill provides a systematic workflow for optimizing the training performance of Mixture-of-Experts models, helping users enhance throughput and reduce bottlenecks.

Core Features & Use Cases

• Performance Tuning: Guides users through memory, communication, and compute optimizations for MoE models.
• Workflow Phases: Offers a clear, phased approach from fitting models into memory to profiling and scaling.
• Use Case: Ideal for deep learning engineers tuning large-scale MoE training jobs on GPU clusters, such as Megatron-LM, to improve efficiency and scalability.

Quick Start

Describe the workflow steps for diagnosing and enhancing MoE model training performance.