skills.rest
Orchestra-Research
Orchestra-ResearchOFFICIAL
Explore all skills by this author →
GitHub

miles-rl-training

Train large-scale MoE models with FP8 and INT4 low-precision RL.

11.5k|842|Updated Nov 3, 2025
Search Tags:#enterprise#reinforcement learning#fp8#sglang#megatron-lm#moe#int4
One-click install
npx skills add https://github.com/Orchestra-Research/AI-Research-SKILLs --skill miles-rl-training
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: miles-rl-training
Source: https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/06-post-training/miles
Command: npx skills add https://github.com/Orchestra-Research/AI-Research-SKILLs --skill miles-rl-training

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes references (resource) components.

What problem does it solve?

This Skill provides an enterprise-grade Reinforcement Learning (RL) framework optimized for training large-scale Mixture-of-Experts (MoE) models, addressing challenges in stability, low-precision training, and train-inference alignment.

Core Features & Use Cases

  • Low-Precision Training: Supports FP8 and INT4 quantization-aware training for massive models.
  • Train-Inference Alignment: Ensures bit-wise identical alignment between training and inference.
  • Speculative RL: Achieves maximum throughput via speculative decoding for faster rollouts.
  • Use Case: Train a 1TB+ MoE model like DeepSeek V3 or Qwen3-MoE using FP8 quantization for reduced memory footprint and faster training, while ensuring the final model behaves identically during inference.

Quick Start

Use the miles skill to train a Qwen3-30B-a3b model using the GRPO advantage estimator and the specified HuggingFace checkpoint.

Frequently Asked Questions about miles-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train large-scale MoE models like DeepSeek V3 using FP8 quantization?▼

To train large-scale MoE models using FP8 quantization, you can use this framework to achieve reduced memory footprint and faster training while ensuring bit-wise identical train-inference alignment. It supports models like DeepSeek V3 and Qwen3-MoE.

What is train-inference alignment and why is it critical for reinforcement learning?▼

Train-inference alignment ensures bit-wise identical behavior between the training and inference phases. This RL framework solves the alignment problem to prevent performance degradation when deploying trained large-scale Mixture-of-Experts models.

Can I use Ray and Megatron-LM for distributed reinforcement learning with INT4 quantization?▼

Yes, this framework supports INT4 quantization-aware training for massive models and relies on Ray for distributed execution. It addresses stability and throughput challenges specific to low-precision enterprise RL.

Does speculative RL improve throughput for Qwen3-MoE rollouts?▼

Speculative RL achieves maximum throughput by using speculative decoding to generate faster rollouts for models like Qwen3-MoE. This optimizes the reinforcement learning cycle for large-scale architectures.

What dependencies do I need to set up enterprise RL training for Mixture-of-Experts models?▼

You need PyTorch 2.0.0 or higher, Transformers 4.40.0 or higher, Ray, and sglang-router 0.2.3 or higher. These libraries provide the distributed computing and advanced quantization capabilities required for the training environment.

How do I start training a Qwen3-30B-a3b model using the GRPO advantage estimator?▼

You can use the miles skill to train a Qwen3-30B-a3b model by specifying your HuggingFace checkpoint and applying the GRPO advantage estimator. This initiates the enterprise-grade reinforcement learning process.

Related Skills

Explore More Education & Research Skills→
MesferAliMesferAli

ray-train

Orchestrate distributed machine learning training across clusters with Ray Train.

Community
Advanced
ihatesea69ihatesea69
3

miles-rl-training

Train large-scale Mixture-of-Experts models with FP8/INT4 quantization-aware training.

Community
Advanced
ovachieverovachiever
52

grpo-rl-training

Apply GRPO/RL training with TRL to optimize language models.

Community
Advanced
zechenzhangAGIzechenzhangAGI
11.5k

moe-training

Train Mixture-of-Experts models with DeepSpeed MoE and HuggingFace Transformers.

Community
Advanced