miles-rl-training

Train large-scale Mixture-of-Experts models with low-precision RL and Rollout Routing Replay.

1|Updated Feb 21, 2026

Search Tags:#enterprise #reinforcement learning #fp8 #sglang #moe #int4 #speculative rl

Ready to run

npx skills add https://github.com/tianhao909/AI-Research-SKILLs-cn --skill miles-rl-training-tianhao909

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: miles-rl-training
Source: https://github.com/tianhao909/AI-Research-SKILLs-cn/tree/main/06-post-training/miles
Command: npx skills add https://github.com/tianhao909/AI-Research-SKILLs-cn --skill miles-rl-training-tianhao909

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve?

This Skill provides a robust and optimized framework for training large-scale Mixture-of-Experts (MoE) models using Reinforcement Learning (RL), addressing challenges like low-precision training and train-inference alignment in enterprise settings.

Core Features & Use Cases

Low-Precision Training: Supports FP8 and INT4 quantization-aware training for massive models.
Train-Inference Alignment: Ensures bit-wise identical expert selection for MoE models using Rollout Routing Replay (R3).
Speculative RL: Achieves maximum throughput with speculative decoding for faster rollouts.
Use Case: Train a 1TB+ MoE model like DeepSeek V3 or Qwen3-MoE efficiently on enterprise hardware, ensuring consistent performance between training and inference phases.

Quick Start

Use the miles skill to train a Qwen3-MoE model with FP8 precision and speculative RL.

Frequently Asked Questions about miles-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I train large MoE models with reinforcement learning using FP8 or INT4 precision?▼

This Skill enables enterprise-grade reinforcement learning training for large-scale Mixture-of-Experts models by supporting low-precision FP8 and INT4 quantization-aware training to optimize performance on production hardware.

What is Rollout Routing Replay (R3) for train-inference alignment in MoE models?▼

Rollout Routing Replay (R3) is a mechanism that ensures bit-wise identical expert selection during training and inference, providing essential train-inference alignment for large-scale Mixture-of-Experts models.

Does this Skill support training 1TB+ models like DeepSeek V3 or Qwen3-MoE?▼

Yes, this Skill is designed for enterprise environments to train massive 1TB+ Mixture-of-Experts models like DeepSeek V3 or Qwen3-MoE efficiently, ensuring consistent performance between training and inference phases.

Can I use speculative decoding to increase reinforcement learning rollout throughput?▼

Yes, you can achieve maximum throughput with speculative RL, which utilizes speculative decoding techniques to perform faster rollouts during the reinforcement learning training process.

What frameworks do I need for enterprise-scale reinforcement learning with sglang and Ray?▼

To run enterprise-scale reinforcement learning, you need dependencies including sglang-router, Ray, PyTorch (torch), and Hugging Face transformers to support the distributed training and inference infrastructure.