skills.rest
jonnabio
jonnabioCOMMUNITY
Explore all skills by this author →
𝕏@jonnabioGitHub

miles-rl-training

Optimize large-scale MoE model training with low-precision and speculative RL.

6|3|Updated Jan 29, 2026
Search Tags:#MoE#INT4#FP8#Reinforcement Learning#Enterprise
One-click install
npx skills add https://github.com/jonnabio/ace-framework --skill miles-rl-training-jonnabio
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: miles-rl-training
Source: https://github.com/jonnabio/ace-framework/tree/main/.ace/packs/ai-research/miles
Command: npx skills add https://github.com/jonnabio/ace-framework --skill miles-rl-training-jonnabio

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill addresses the challenges of training large MoE models with features like low-precision training, train-inference alignment, and speculative RL for maximum throughput.

Core Features & Use Cases

  • Low-Precision Training: Supports FP8 and INT4 quantization for efficient memory usage.
  • Speculative RL: Enhances rollout throughput with online SFT draft models.
  • Train-Inference Alignment: Achieves exact alignment through TIS/MIS and kernel-level optimizations.
  • Use Case: Ideal for training MoE models like DeepSeek V3 or Qwen3-MoE, requiring stability, low-precision, and high throughput.

Quick Start

Train your MoE model using the miles skill with the following command:

python train.py --hf-checkpoint /path/to/model --rollout-batch-size 512 --n-samples-per-prompt 8

Frequently Asked Questions about miles-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize large-scale MoE model training with reinforcement learning?▼

Optimize large-scale MoE model training by applying low-precision quantization, speculative RL, and train-inference alignment to maximize throughput and stability for enterprise-grade models.

Can I use FP8 and INT4 quantization for MoE training?▼

Yes, FP8 and INT4 low-precision training are supported to enable efficient memory usage during large-scale MoE model training, significantly reducing the hardware footprint for enterprise scenarios.

What dependencies do I need for enterprise-grade RL training of MoE models?▼

You need Ray, Torch, Transformers, and sglang-router to establish the distributed environment required for stable, high-throughput MoE model training with reinforcement learning.

What's the best way to enhance rollout throughput in MoE reinforcement learning?▼

Speculative RL enhances rollout throughput by utilizing online SFT draft models, accelerating the reinforcement learning phase for large MoE architectures like DeepSeek V3 or Qwen3-MoE.

How do I achieve train-inference alignment for MoE models?▼

Achieve exact train-inference alignment for MoE models through TIS/MIS and kernel-level optimizations, ensuring consistent behavior between training and deployment phases.

Does this approach work with models like DeepSeek V3 or Qwen3-MoE?▼

Yes, this approach is ideal for training large MoE models like DeepSeek V3 and Qwen3-MoE, providing the required stability, low-precision support, and high throughput for enterprise scenarios.

Related Skills

Explore More Software Engineering Skills→
hhhi21ghhhi21g

miles-rl-training

Optimize reinforcement learning training for large MoE models with low-precision and speculative RL.

Community
Advanced
OpenLAIROpenLAIR
1.0k

miles-rl-training

Configure miles for large MoE RL training with FP8/INT4 and train-inference alignment.

Official
Advanced
qcmuuqcmuu

miles-rl-training

Run stable FP8/INT4 reinforcement learning post-training for large MoE models.

Community
Advanced
zechenzhangAGIzechenzhangAGI
11.5k

moe-training

Train Mixture-of-Experts models with DeepSpeed MoE and HuggingFace Transformers.

Community
Advanced