skills.rest
box755
box755COMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Orchestrate Megatron-LM training with SGLang router rollouts for RL post-training.

Updated Aug 23, 2026
Search Tags:#reinforcement learning#sglang#grpo#megatron-lm#llm post-training#multi-turn training#router-based rollouts
One-click install
npx skills add https://github.com/box755/simlens-research --skill slime-rl-training-box755
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/box755/simlens-research/tree/main/skill-packs/AI-Research-SKILLs/06-post-training/slime
Command: npx skills add https://github.com/box755/simlens-research --skill slime-rl-training-box755

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes references (resource) components.

What problem does it solve?

This Skill helps you run LLM post-training using reinforcement learning at scale by connecting Megatron-LM training with SGLang rollout generation, so you can iterate on RL objectives and data pipelines without manual glue code.

Core Features & Use Cases

  • Megatron-LM RL Training: Enables actor/critic style post-training with parallelism (TP/PP/DP/SP) while keeping training tightly integrated with rollout weights.
  • SGLang High-Throughput Rollouts: Uses a router-backed SGLang engine to generate many responses efficiently and feed reward/verifier outputs back into training.
  • Flexible Data Buffer & Multi-Turn Generation: Supports custom data generation, filtering (off-policy buffering), and multi-turn/tool-calling workflows via a custom generate function.

Use case: you want to train a GLM/Qwen/DeepSeek/Llama reasoning model with GRPO/GPSO/PPO-style objectives, generating multiple candidate responses per prompt and selecting or weighting them using rewards.

Quick Start

Start GRPO training by running the repository’s GRPO example command configured for your model and dataset, and point it to your JSONL prompts and labels.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run GRPO reinforcement learning post-training with Megatron-LM and SGLang?▼

GRPO reinforcement learning post-training is executed by orchestrating Megatron-LM training with SGLang router rollouts. You configure slime's SLiME settings for Megatron parallelism alongside SGLang rollout parameters to generate candidate responses efficiently.

Can I use SGLang rollouts for multi-turn tool-calling agent training?▼

Multi-turn tool-calling agent training is supported using SGLang rollouts via a custom generate function. This enables flexible data generation, off-policy buffering, and iterative multi-turn workflows to feed reward outputs back into the training loop.

What is the best way to scale LLM post-training with asynchronous rollouts?▼

Scaling LLM post-training with asynchronous rollouts is handled by connecting Megatron-LM training with a router-backed SGLang engine. This architecture generates many high-throughput responses and tightly integrates rollout weights without manual glue code.

Do I need a specific data format for GRPO training with slime?▼

GRPO training with slime requires a JSONL dataset containing keys for prompts and labels. You must provide this dataset format alongside appropriate RL algorithm flags and SLiME configuration to start the training process.

Does slime support PPO-style objectives with actor/critic parallelism?▼

PPO-style objectives are supported through actor/critic style post-training with Megatron-LM parallelism. This includes TP, PP, DP, and SP configurations while keeping training tightly integrated with rollout generation for reward weighting.

Related Skills

Explore More Education & Research Skills→
aayushsoamaayushsoam
1

slime-rl-training

Automate LLM post-training with slime and Megatron-LM integration.

Community
Advanced
SignmanalSignmanal
1

slime-rl-training

Coordinate distributed Megatron-LM training and SGLang inference for RL post-training.

Community
Advanced
hhhi21ghhhi21g

slime-rl-training

Scale RL post-training for LLMs with Megatron-LM and SGLang.

Community
Advanced
Clay-HHKClay-HHK
1

slime-rl-training

Configures RL post-training of LLMs using Megatron-LM training with SGLang rollout generation.

Community
Advanced