skills.rest
qcmuu
qcmuuCOMMUNITY
Explore all skills by this author →
GitHub

miles-rl-training

Run stable FP8/INT4 reinforcement learning post-training for large MoE models.

Updated May 2, 2026
Search Tags:#fp8#sglang#int4#miles#moe rl training#routing replay
One-click install
npx skills add https://github.com/qcmuu/AI-Research-Skills --skill miles-rl-training-qcmuu
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: miles-rl-training
Source: https://github.com/qcmuu/AI-Research-Skills/tree/main/06-post-training/miles
Command: npx skills add https://github.com/qcmuu/AI-Research-Skills --skill miles-rl-training-qcmuu

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve?

miles-rl-training helps you perform stable, production-grade reinforcement learning post-training for large Mixture-of-Experts (MoE) models while maintaining train-inference consistency under low precision (FP8/INT4).

Core Features & Use Cases

  • Unified FP8 + INT4 Quantization-Aware RL: Enables end-to-end low-precision training that reduces numerical mismatch and supports very large MoE models.
  • Rollout Routing Replay (R3) for MoE alignment: Records expert routing decisions during SGLang inference and replays them during training to keep expert selection identical.
  • Speculative RL for throughput: Uses speculative RL (via SGLang/EAGLE) to increase rollout throughput for large-scale training runs.
  • Use Case: You are fine-tuning a Qwen3-MoE or DeepSeek-V3-class MoE model with FP8/INT4 and need bit-wise train-inference expert alignment plus faster rollout generation without RL collapse.

Quick Start

Run miles RL training for a large MoE model with GRPO advantage estimation using your HF checkpoint and rollout sampling configuration.

Frequently Asked Questions about miles-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I maintain train-inference consistency for MoE models during FP8 or INT4 reinforcement learning?▼

To maintain train-inference consistency for MoE models during FP8 or INT4 reinforcement learning, you can use Rollout Routing Replay (R3) to record SGLang expert routing decisions and replay them during training for deterministic expert behavior.

Can I use speculative decoding to accelerate RL rollout throughput for large MoE models?▼

Yes, you can accelerate RL rollout throughput for large MoE models by using speculative RL with SGLang-based EAGLE speculative decoding, which increases generation speed for enterprise-scale training runs.

What is the best way to perform stable GRPO post-training on DeepSeek-V3 or Qwen3-MoE architectures?▼

The best way to perform stable GRPO post-training on DeepSeek-V3 or Qwen3-MoE architectures is applying quantization-aware RL with off-policy correction and routing replay to prevent RL collapse under low precision.

Does miles-rl-training require SGLang and Ray for enterprise-scale MoE fine-tuning?▼

Yes, miles-rl-training requires SGLang for rollout and speculative decoding, Ray for distributed execution, plus Torch and Transformers to support enterprise-scale MoE fine-tuning with low-precision quantization.

Why does my FP8 MoE reinforcement learning collapse without routing replay?▼

FP8 MoE reinforcement learning collapses without routing replay because numerical mismatch alters expert selection during inference, causing train-inference inconsistency that destabilizes the RL post-training process.

How to configure miles and slime for TIS and MIS routing replay in MoE training?▼

To configure miles and slime for TIS and MIS routing replay in MoE training, align your miles configuration with slime to enable deterministic expert behavior and apply R3 routing replay during SGLang inference.

Related Skills

Explore More Education & Research Skills→
hhhi21ghhhi21g

miles-rl-training

Optimize reinforcement learning training for large MoE models with low-precision and speculative RL.

Community
Advanced
ovachieverovachiever
52

grpo-rl-training

Apply GRPO/RL training with TRL to optimize language models.

Community
Advanced
Supporter09Supporter09

miles-rl-training

Optimize large-scale MoE reinforcement learning training with Megatron parallelism and FP8 or INT4 quantization.

Community
Advanced
MesferAliMesferAli

miles-rl-training

Trains large MoE models with FP8/INT4 quantization and R3 alignment.

Community
Advanced