skills.rest
Z43L
Z43LCOMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Integrates Megatron-LM training loops with accelerated SLang rollouts for LLM RL fine-tuning.

Updated May 5, 2026
Search Tags:#reinforcement learning#distributed training#sglang#grpo#megatron-lm#llm post-training#rollout generation
One-click install
npx skills add https://github.com/Z43L/zeus-agent --skill slime-rl-training-z43l
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/Z43L/zeus-agent/tree/main/optional-skills/mlops/slime
Command: npx skills add https://github.com/Z43L/zeus-agent --skill slime-rl-training-z43l

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes references (resource) and assets (resource) components.

What problem does it solve?

LLM developers struggle to efficiently run reinforcement learning post-training when training needs Megatron-LM parallelism while rollouts require high-throughput generation, causing slow iteration and expensive GPU usage.

Core Features & Use Cases

  • Megatron-LM Training Loop: Actor (and optional critic) training with full TP/PP/DP/SP parallelism for large-scale model updates.
  • SGLang Rollouts with Router: High-throughput response generation using SGLang, optionally routed for better utilization and multi-turn rollout support.
  • Flexible Data Buffering (Off-Policy Ready): Prompt management plus sample storage and optional buffer filtering to support GRPO/GPSO/PPO-style workflows and custom data generation.

Use case example: You want to post-train a GLM-family model using GRPO on reasoning prompts, generating many rollouts per prompt and feeding rewards back into Megatron-LM training with tight rollout/training integration.

Quick Start

Ask your AI to set up GRPO training for your model by providing the model script to source, your JSONL prompt/label data paths, and the desired batch and rollout parameters so it can launch train.py with the correct keys and estimation settings.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run GRPO reinforcement learning post-training using Megatron-LM?▼

GRPO reinforcement learning post-training is executed by coupling Megatron-LM actor training with SGLang high-throughput rollout generation. You provide your model script, JSONL prompt data paths, and batch parameters to launch the training loop.

How does SGLang router improve LLM rollout generation during distributed training?▼

SGLang router improves LLM rollout generation by providing high-throughput response generation with better GPU utilization. It supports multi-turn rollout generation, feeding responses back efficiently into the Megatron-LM training loop.

Can I use this RL post-training workflow for multi-turn agentic training?▼

Yes, this RL post-training workflow supports multi-turn agentic training. SGLang router handles the high-throughput multi-turn rollout generation, while Megatron-LM manages the actor and optional critic updates with full parallelism.

Does this reinforcement learning training skill support GLM, Qwen, DeepSeek, and Llama model families?▼

Yes, this reinforcement learning training skill supports GLM, Qwen, DeepSeek, and Llama model families. It allows custom prompt and label data generation tailored to the specific architecture of these models during post-training.

What advantage estimation and KL loss options do I need to configure for PPO-style training?▼

For PPO-style training, you must explicitly configure advantage estimation and KL loss options. The workflow requires explicit argument configuration to manage the data buffering, sample storage, and reward feedback integration properly.

Why does distributed RL post-training require both Megatron-LM and SGLang?▼

Distributed RL post-training requires Megatron-LM for large-scale model parallelism during training and SGLang for high-throughput rollout generation. Coupling them solves slow iteration and expensive GPU usage caused by separating these workloads.

Related Skills

Explore More Software Engineering Skills→
qcmuuqcmuu

slime-rl-training

Coordinate Megatron-LM training with SGLang rollouts for LLM reinforcement-learning post-training.

Community
Advanced
GarrettRoiGarrettRoi

slime-rl-training

Train LLMs with reinforcement learning using Megatron-LM and SGLang.

Community
Advanced
hhhi21ghhhi21g

slime-rl-training

Scale RL post-training for LLMs with Megatron-LM and SGLang.

Community
Advanced
InverterNetworkInverterNetwork

slime-rl-training

Orchestrate Megatron-LM training and SGLang rollout generation for large-scale RL post-training.

Official
Advanced