skills.rest
MedTiLab
MedTiLabCOMMUNITY
Explore all skills by this author →
GitHub

miles-rl-training

Configures RL post-training for large MoE models with FP8, INT4, and speculative decoding.

5|2|Updated Aug 10, 2026
Search Tags:#reinforcement learning#MoE#speculative decoding#GRPO#INT4#FP8#megatron-lm
One-click install
npx skills add https://github.com/MedTiLab/Auto-meta-analysis --skill miles-rl-training-medtilab
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: miles-rl-training
Source: https://github.com/MedTiLab/Auto-meta-analysis/tree/main/skills/post-training/miles
Command: npx skills add https://github.com/MedTiLab/Auto-meta-analysis --skill miles-rl-training-medtilab

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve? Training large MoE models like DeepSeek V3 or Qwen3-MoE with reinforcement learning often fails due to FP8 instability, train-inference mismatch, and slow rollout throughput. This Skill provides structured guidance for configuring miles, a production fork of slime, to address these challenges. ## Core Features & Use Cases - Low-Precision Training: Guides unified FP8 pipelines and INT4 quantization-aware training so 1TB models fit on a single H200 machine. - Train-Inference Alignment: Covers Rollout Routing Replay (R3) and TIS/MIS importance sampling to keep Megatron training bit-wise aligned with SGLang inference. - Speculative RL: Configures EAGLE speculative decoding with online MTP draft training for 25-40% faster rollouts. - Use Case: A team post-training Qwen3-30B-A3B with GRPO can follow the MoE workflow to set expert parallelism, enable FP8 block scaling, and verify routing consistency before launching 3000 rollouts. ## Quick Start Ask the assistant to set up a miles GRPO training run for a Qwen3 MoE model with FP8 and speculative decoding enabled.

Frequently Asked Questions about miles-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a large MoE model with RL using miles?▼

Run train.py with --advantage-estimator grpo, set --expert-model-parallel-size for MoE expert parallelism, and point --hf-checkpoint at your model. Enable FP8 block scaling via the NVTE_FP8_BLOCK_SCALING_FP32_SCALES environment variable for stability.

miles vs slime vs verl for RL training, which should I use?▼

miles is a production fork of slime suited for 1TB+ MoE models needing FP8/INT4 and train-inference alignment. Use slime for the research-grade original, verl for flexible backend swapping, or torchforge for PyTorch-native abstractions.

Does miles support speculative decoding for faster rollouts?▼

Yes, miles supports EAGLE speculative decoding through SGLang flags like --sglang-speculative-algorithm EAGLE and --sglang-speculative-num-steps. Online MTP training keeps the draft model aligned, yielding 25-40% faster rollouts.

Why does FP8 RL training collapse with NaN losses?▼

FP8 collapse usually stems from quantization-induced discrepancy between training and inference. Enable block scaling, reduce the learning rate to around 5e-7, and turn on Rollout Routing Replay so MoE expert routing matches between SGLang and Megatron.

Can INT4 quantization fit a 1TB model on one machine?▼

Yes, INT4 QAT reduces VRAM roughly 3.1x compared to BF16, so a 671B model drops from 1.3TB to about 420GB, fitting on a single H200. Sensitive layers like lm_head and embeddings can be kept in higher precision.

What models does miles support for RL post-training?▼

miles supports DeepSeek R1/V3, Qwen 2/2.5/3 including MoE variants, Llama 3/4, Gemma 2/3, GLM 4.5+, and MiniMax M2. Full MoE support covers DeepSeek, Qwen MoE, and MiniMax; Llama, Gemma, and GLM are dense-only.

Related Skills

Explore More Software Engineering Skills→
Clay-HHKClay-HHK
1

ray-train

Orchestrates distributed PyTorch, TensorFlow, and HuggingFace training across multi-node Ray clusters.

Community
Advanced
yo-stevenyo-steven

deployment

Deploy HF LLM checkpoints as OpenAI-compatible endpoints via vLLM, SGLang, or TRT-LLM.

Community
Advanced
plurigridplurigrid
60

fokker-planck-analyzer

Compare empirical weight distributions to Gibbs distributions for Langevin training convergence.

Official
Advanced
tadod12tadod12

verl-rl-training

Configure scalable Verl RL training workflows for LLMs across vLLM and Megatron backends.

Community
Advanced