skills.rest
superfhp
superfhpCOMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Automate RL training with Megatron-LM and SGLang rollout generation.

Updated May 8, 2026
Search Tags:#reinforcement learning#LLM training#large language models#SGLang#Megatron-LM#RL scaling
One-click install
npx skills add https://github.com/superfhp/lumi-agent --skill slime-rl-training-superfhp
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/superfhp/lumi-agent/tree/main/optional-skills/mlops/slime
Command: npx skills add https://github.com/superfhp/lumi-agent --skill slime-rl-training-superfhp

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of scaling reinforcement learning (RL) with large language models (LLMs), specifically focusing on the integration of Megatron-LM for training and SGLang for high-throughput rollout generation.

Core Features & Use Cases

  • Megatron-LM Integration: Leverages Megatron-LM for full parallelism support in actor model training.
  • SGLang Rollout Generation: Utilizes SGLang for high-throughput rollout generation with router support.
  • Data Buffer Management: Offers flexible prompt management and sample storage for data buffers.
  • Model Support: Supports training of various LLM models like GLM, Qwen3, DeepSeek V3/R1, and Llama 3.
  • Use Case: Ideal for researchers and practitioners looking to implement custom data generation workflows or require tight integration with Megatron-LM for RL scaling.

Quick Start

Run the following command to start a GRPO training session:

python train.py \
    --actor-num-nodes 1 \
    --actor-num-gpus-per-node 4 \
    --rollout-num-gpus 4 \
    --advantage-estimator grpo \
    --use-kl-loss --kl-loss-coef 0.001 \
    --rollout-batch-size 32 \
    --n-samples-per-prompt 8 \
    --global-batch-size 256 \
    --num-rollout 3000 \
    --prompt-data /path/to/data.jsonl \
    ${MODEL_ARGS[@]} ${CKPT_ARGS[@]}

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scale reinforcement learning with large language models using Megatron-LM?▼

You can scale reinforcement learning with large language models by integrating Megatron-LM for full parallelism support during actor model training and using SGLang for high-throughput rollout generation. This provides a streamlined workflow for complex RL tasks.

What is the best way to generate high-throughput rollouts for LLM training?▼

The best way to generate high-throughput rollouts for LLM training is to utilize SGLang with router support. This approach manages data buffers effectively while providing the high-throughput rollout generation required for scaling reinforcement learning workflows.

Can I train Qwen3 or DeepSeek V3/R1 models with GRPO advantage estimation?▼

Yes, you can train Qwen3, DeepSeek V3/R1, GLM, and Llama 3 models using GRPO advantage estimation. The framework supports custom data generation workflows with configurable parameters like KL loss coefficients and rollout batch sizes.

Do I need Ray and PyTorch to run distributed RL training with SGLang?▼

Yes, you need Ray and PyTorch (version 2.0.0 or higher) along with sglang-router and transformers. These dependencies provide the distributed computing environment and model handling capabilities required for the training process.

How do I start a GRPO training session with a custom prompt dataset?▼

You start a GRPO training session by executing the train.py script with parameters for actor nodes, GPUs, advantage estimator, and prompt data path. This automates the reinforcement learning pipeline using your specified data buffers.

Why use SGLang router instead of other category-level tools for rollout generation?▼

You use SGLang router because it provides specialized high-throughput rollout generation with flexible prompt management and sample storage. This tight integration with Megatron-LM training is specifically optimized for scaling reinforcement learning tasks.

Related Skills

Explore More Software Engineering Skills→
tadod12tadod12

ray-train

Orchestrate distributed ML training across multi-node clusters with Ray Train.

Community
Advanced
BirgerMoellBirgerMoell

megatron-hf-converter — Claude Code skill guide

Convert HuggingFace and Megatron-LM checkpoints with numerical equivalence validation.

Community
Intermediate
qcmuuqcmuu

slime-rl-training

Coordinate Megatron-LM training with SGLang rollouts for LLM reinforcement-learning post-training.

Community
Advanced
Tommy-ywTommy-yw
539

slime-rl-training

Optimize large language models with Megatron-LM training and SGLang rollouts.

Community
Advanced