Verified:GitHub

slime-rl-training

Orchestrate Megatron-LM training with SGLang rollout generation for RL post-training.

19|4|Updated Apr 22, 2026

Search Tags:#reinforcement learning #distributed training #sglang #grpo #megatron-lm #reward modeling #llm post-training

Ready to run

npx skills add https://github.com/carterwayneskhizeine/hermes-agent-windows-R --skill slime-rl-training-carterwayneskhizeine

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/carterwayneskhizeine/hermes-agent-windows-R/tree/main/optional-skills/mlops/slime
Command: npx skills add https://github.com/carterwayneskhizeine/hermes-agent-windows-R --skill slime-rl-training-carterwayneskhizeine

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

slime streamlines the painful process of post-training large language models with reinforcement learning by tightly coordinating Megatron-LM training with SGLang-based high-throughput rollout generation.

Core Features & Use Cases

RL post-training with Megatron-LM: Supports actor training (and optional critic) with full parallelism (TP/PP/DP/SP) so you can run GRPO-style pipelines efficiently.
Fast rollout generation with SGLang + router: Produces rollout samples at scale using an inference router, enabling multi-turn and tool-aware workflows.
Flexible data buffer and off-policy workflows: Provides rollout data sources and buffering/selection so you can reuse or filter generated samples before training.

Use cases include training reasoning/GLM-style models with GRPO, building custom rollout data generation pipelines, and integrating reward/verifier logic while maintaining strong Megatron-LM compatibility for RL scaling.

Quick Start

Ask the AI to help you train with GRPO by running train.py with your chosen model script, rollout/batch parameters, and a JSONL dataset path that contains prompt and label fields.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I scale GRPO reinforcement learning post-training with Megatron-LM and SGLang?▼

To scale GRPO reinforcement learning post-training, this Skill integrates Megatron-LM for actor training with SGLang for high-throughput rollout generation, coordinating both to handle large-scale LLM post-training efficiently.

Can I use custom reward modeling and multi-turn agentic rollouts for LLM post-training?▼

Yes, you can use custom reward modeling and multi-turn agentic rollouts for LLM post-training. The workflow supports custom reward logic and routed rollout sampling to build tailored data generation pipelines.

How do I start training a Qwen or Llama-family model using GRPO?▼

To start training a Qwen or Llama-family model with GRPO, run train.py with your chosen model script, rollout and batch parameters, and a JSONL dataset path containing prompt and label fields.

Does this reinforcement learning pipeline support off-policy data buffering and full tensor parallelism?▼

Yes, the reinforcement learning pipeline supports off-policy data buffering and full tensor parallelism. It includes flexible rollout data buffering with selection mechanisms and Megatron-LM training with TP/PP/DP/SP parallelism.

What is the best way to handle GPU resource allocation when co-scheduling Megatron-LM training and SGLang inference?▼

The best way to handle GPU resource allocation for co-scheduling is using the integrated orchestration layer, which manages Megatron-LM training and SGLang inference routing to satisfy complex co-scheduling requirements.

Can I configure advantage estimators and KL regularization for PPO or reinforce_plus_plus training scenarios?▼

Yes, you can configure advantage estimators and KL regularization for PPO or reinforce_plus_plus training scenarios. The pipeline supports configurable advantage estimation and KL regularization across various RL training algorithms.