skills.rest
hhhi21g
hhhi21gCOMMUNITY
Explore all skills by this author →
GitHub

miles-rl-training

Optimize reinforcement learning training for large MoE models with low-precision and speculative RL.

Updated Apr 11, 2026
Search Tags:#enterprise#reinforcement learning#MoE#INT4#FP8
One-click install
npx skills add https://github.com/hhhi21g/HealthCenter --skill miles-rl-training-hhhi21g
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: miles-rl-training
Source: https://github.com/hhhi21g/HealthCenter/tree/main/.codex/skills/miles
Command: npx skills add https://github.com/hhhi21g/HealthCenter --skill miles-rl-training-hhhi21g

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill addresses the challenges of training large MoE models with FP8/INT4, needing train-inference alignment, or requiring speculative RL for maximum throughput.

Core Features & Use Cases

  • Low-Precision Training: Supports unified FP8 and INT4 QAT for efficient training on large models.
  • Performance Optimizations: Implements speculative RL and zero-copy weight sync for enhanced throughput.
  • Train-Inference Alignment: Achieves bit-wise identical alignment with TIS/MIS and kernel-level optimizations.
  • Use Case: Ideal for training large MoE models like DeepSeek V3 or Qwen3-MoE, requiring stability, low-precision, and alignment.

Quick Start

Train a large MoE model using miles with the following command:

python train.py \
    --advantage-estimator grpo \
    --model-name qwen3-30b-a3b \
    --hf-checkpoint /path/to/qwen3-30b-a3b-hf \
    --rollout-batch-size 512 \
    --n-samples-per-prompt 8

Frequently Asked Questions about miles-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train large MoE models with FP8 or INT4 low-precision reinforcement learning?▼

You can train large MoE models with FP8 or INT4 low-precision reinforcement learning using miles, which supports unified FP8 and INT4 QAT for efficient, stable training on architectures like DeepSeek V3.

How do I achieve bit-wise train-inference alignment during RL training?▼

To achieve bit-wise train-inference alignment during RL training, miles implements TIS/MIS alongside kernel-level optimizations to ensure the training and inference phases remain perfectly synchronized.

What's the best way to maximize throughput for RL training on large models?▼

To maximize throughput for RL training on large models, this Skill implements speculative RL and zero-copy weight sync, significantly enhancing performance and reducing overhead during the training cycle.

Does this reinforcement learning training approach work with Ray and sglang-router?▼

Yes, this reinforcement learning training approach works with Ray and sglang-router, explicitly requiring sglang-router>=0.2.3, Ray, torch>=2.0.0, and transformers>=4.40.0 as core dependencies for execution.

Can I use miles to train Qwen3-MoE models with GRPO advantage estimation?▼

Yes, you can use miles to train Qwen3-MoE models with GRPO advantage estimation by specifying the advantage estimator and model checkpoint in the command line to initiate the rollout process.

When do I need speculative RL for enterprise-grade reinforcement learning?▼

You need speculative RL for enterprise-grade reinforcement learning when training large MoE models like DeepSeek V3 or Qwen3-MoE that require maximum throughput, low-precision support, and strict train-inference alignment.

Related Skills

Explore More Software Engineering Skills→
aayushsoamaayushsoam
1

slime-rl-training

Automate LLM post-training with slime and Megatron-LM integration.

Community
Advanced
ccfccf

vector-index-tuning

Tune HNSW parameters and quantization for large-scale vector search indexes.

Community
Advanced
annealing-inversionannealing-inversion

sglang-diffusion-ako4all-kernel

Automate AKO4ALL-based optimization of SGLang diffusion kernels with benchmarking and validation.

Community
Advanced
ihatesea69ihatesea69
3

miles-rl-training

Train large-scale Mixture-of-Experts models with FP8/INT4 quantization-aware training.

Community
Advanced