skills.rest
richardnguyen0715
richardnguyen0715COMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Train LLMs with reinforcement learning using Megatron-LM and SGLang.

Updated May 11, 2026
Search Tags:#LLM#GLM#RL#SGLang#Megatron-LM#Reinforcement Learning#Post-Training
One-click install
npx skills add https://github.com/richardnguyen0715/keep-it-real --skill slime-rl-training-richardnguyen0715
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/richardnguyen0715/keep-it-real/tree/main/refer-projects/hermes-agent/optional-skills/mlops/slime
Command: npx skills add https://github.com/richardnguyen0715/keep-it-real --skill slime-rl-training-richardnguyen0715

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides a framework for LLM post-training with reinforcement learning (RL), optimizing Megatron-LM for scalable RL tasks.

Core Features & Use Cases

  • Megatron-LM Integration: Connects with Megatron-LM for training and SGLang for high-throughput rollout generation.
  • Flexible Data Buffers: Manages prompt initialization, data generation, and rollout sample storage.
  • Model Support: Supports GLM, Qwen3, DeepSeek V3/R1, Llama 3, and more.
  • Use Case: Ideal for researchers and developers implementing custom data generation workflows or seeking tight integration with Megatron-LM for RL.

Quick Start

Execute the following command to start the GRPO training workflow:

python train.py \
  --actor-num-nodes 1 \
  --actor-num-gpus-per-node 8 \
  --rollout-num-gpus 8 \
  --advantage-estimator grpo \
  --use-kl-loss --kl-loss-coef 0.001 \
  --rollout-batch-size 32 \
  --n-samples-per-prompt 8 \
  --global-batch-size 256 \
  --num-rollout 3000 \
  --prompt-data /path/to/data.jsonl \
  ${MODEL_ARGS[@]} ${CKPT_ARGS[@]}

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I use reinforcement learning for LLM post-training with Megatron-LM?▼

LLM post-training with reinforcement learning using Megatron-LM is achieved by integrating it for training and SGLang for high-throughput rollout generation. This framework provides scalable RL task optimization and manages flexible data buffers for prompts and rollout samples.

Can I use SGLang for rollout generation in a distributed RL training setup?▼

SGLang is used for rollout generation in this distributed RL training setup, working in conjunction with Megatron-LM. It enables high-throughput generation and connects with Ray to support scalable reinforcement learning workflows across multiple GPUs and nodes.

Does this RL training framework support models like GLM, Qwen3, and DeepSeek?▼

This RL training framework supports models including GLM, Qwen3, DeepSeek V3/R1, and Llama 3. It allows researchers and developers to implement custom data generation workflows and perform GRPO training using advantage estimators and KL loss coefficients.

What dependencies do I need to run GRPO training workflows for large language models?▼

To run GRPO training workflows for large language models, you need dependencies including sglang-router, Ray, Torch, and Transformers. You also need to configure actor nodes, GPUs, rollout batch sizes, and provide a JSONL file for prompt data initialization.

How do I configure batch sizes and GPU allocation for scalable RL post-training?▼

Configuring batch sizes and GPU allocation for scalable RL post-training involves setting actor-num-nodes, actor-num-gpus-per-node, and rollout-num-gpus parameters. You must also define global-batch-size, rollout-batch-size, and n-samples-per-prompt to manage rollout generation effectively.

Why use Megatron-LM instead of other frameworks for RL scaling?▼

Using Megatron-LM for RL scaling provides tight integration for large-scale model training, paired with SGLang for efficient rollout generation. This combination offers optimized performance for researchers needing flexible data buffers and custom data generation workflows in distributed environments.

Related Skills

Explore More Software Engineering Skills→
aayushsoamaayushsoam
1

slime-rl-training

Automate LLM post-training with slime and Megatron-LM integration.

Community
Advanced
hhhi21ghhhi21g

slime-rl-training

Scale RL post-training for LLMs with Megatron-LM and SGLang.

Community
Advanced
heysuhasheysuhas
2

slime-rl-training

Integrate Megatron-LM and SGLang for large-scale LLM reinforcement learning training.

Community
Advanced
projectedanxprojectedanx
1

slime-rl-training

Train reasoning models with Megatron-LM and SGLang for RL post-training.

Community
Advanced