skills.rest
qcmuu
qcmuuCOMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Coordinate Megatron-LM training with SGLang rollouts for LLM reinforcement-learning post-training.

Updated May 2, 2026
Search Tags:#reinforcement learning#sglang#grpo#post-training#megatron-lm#rl rollout#multi-turn training
One-click install
npx skills add https://github.com/qcmuu/AI-Research-Skills --skill slime-rl-training-qcmuu
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/qcmuu/AI-Research-Skills/tree/main/06-post-training/slime
Command: npx skills add https://github.com/qcmuu/AI-Research-Skills --skill slime-rl-training-qcmuu

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve?

It streamlines LLM reinforcement-learning post-training by coordinating Megatron-LM training with high-throughput SGLang rollouts so you can train faster and at scale without building the systems yourself.

Core Features & Use Cases

  • Megatron-LM RL training: Runs actor/critic (optional) training with full parallelism support (TP/PP/DP/SP) for stable, scalable policy optimization.
  • SGLang-based rollout generation: Produces responses efficiently via SGLang router for GRPO/PPO-style workflows and multi-turn agent training.
  • Data buffer for RL loops: Manages prompts, custom filtering, and off-policy buffering to support research-grade training pipelines.
  • Use Case: Train a GLM-4.x or Qwen/DeepSeek/Llama reasoning model with GRPO by streaming JSONL prompts into SGLang rollouts and feeding generated samples into Megatron-LM for iterative policy updates.

Quick Start

Train a GRPO run by executing: python train.py --actor-num-nodes 1 --actor-num-gpus-per-node 4 --rollout-num-gpus 4 --advantage-estimator grpo --use-kl-loss --kl-loss-coef 0.001 --rollout-batch-size 32 --n-samples-per-prompt 8 --global-batch-size 256 --num-rollout 3000 --prompt-data /path/to/data.jsonl

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I coordinate Megatron-LM training with SGLang rollouts for LLM reinforcement learning?▼

To coordinate Megatron-LM training with SGLang rollouts for LLM reinforcement learning, this framework links Megatron-LM policy optimization with high-throughput SGLang router rollout generation using an off-policy buffer and configurable training loop arguments.

Can I use GRPO for multi-turn agentic training with Qwen or Llama models?▼

Yes, you can use GRPO for multi-turn agentic training with Qwen or Llama models. The framework applies GRPO and PPO-style optimization to GLM, Qwen, DeepSeek, and Llama reasoning workflows requiring tight training-inference integration.

What do I need to set up before starting RL post-training with Megatron-LM and SGLang?▼

Before starting RL post-training with Megatron-LM and SGLang, you need Ray, Torch, Transformers, and sglang-router dependencies installed, plus a JSONL prompt dataset to stream into the SGLang rollout generation pipeline.

How does the data buffer manage prompts and off-policy buffering during PPO-style training?▼

The data buffer manages prompts and off-policy buffering during PPO-style training by handling custom filtering and storing generated samples from SGLang rollouts, feeding them iteratively into Megatron-LM for stable policy updates.

Does Megatron-LM parallelism support TP, PP, DP, and SP for scalable RL training?▼

Yes, Megatron-LM parallelism supports TP, PP, DP, and SP for scalable RL training. The framework runs actor and optional critic training with full parallelism support to ensure stable and scalable policy optimization.

What are the limitations of using SGLang router for high-throughput rollout generation?▼

A limitation of using SGLang router for high-throughput rollout generation is that it requires tight integration with Megatron-LM training loops and specific GPU allocations configured via arguments like rollout-num-gpus and rollout-batch-size.

Related Skills

Explore More Education & Research Skills→
hhhi21ghhhi21g

slime-rl-training

Scale RL post-training for LLMs with Megatron-LM and SGLang.

Community
Advanced
BirgerMoellBirgerMoell

megatron-hf-converter — Claude Code skill guide

Convert HuggingFace and Megatron-LM checkpoints with numerical equivalence validation.

Community
Intermediate
heysuhasheysuhas
2

slime-rl-training

Integrate Megatron-LM and SGLang for large-scale LLM reinforcement learning training.

Community
Advanced
SignmanalSignmanal
1

slime-rl-training

Coordinate distributed Megatron-LM training and SGLang inference for RL post-training.

Community
Advanced