miles-rl-training

What problem does it solve?

This Skill provides an enterprise-grade Reinforcement Learning (RL) framework optimized for training large-scale Mixture-of-Experts (MoE) models, addressing challenges in stability, low-precision training, and train-inference alignment.

Core Features & Use Cases

• Low-Precision Training: Supports FP8 and INT4 quantization-aware training for massive models.
• Train-Inference Alignment: Ensures bit-wise identical alignment between training and inference.
• Speculative RL: Achieves maximum throughput via speculative decoding for faster rollouts.
• Use Case: Train a 1TB+ MoE model like DeepSeek V3 or Qwen3-MoE using FP8 quantization for reduced memory footprint and faster training, while ensuring the final model behaves identically during inference.

Quick Start

Use the miles skill to train a Qwen3-30B-a3b model using the GRPO advantage estimator and the specified HuggingFace checkpoint.