skills.rest
KarlinskyS
KarlinskySCOMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Coordinate Megatron-LM training with SGLang rollouts for RL-based post-training.

Updated Apr 10, 2026
Search Tags:#rollout#glm#reinforcement-learning#sglang#grpo#megatron-lm#rl-post-training
One-click install
npx skills add https://github.com/KarlinskyS/hermesSkills --skill slime-rl-training-karlinskys
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/KarlinskyS/hermesSkills/tree/main/mlops/training/slime
Command: npx skills add https://github.com/KarlinskyS/hermesSkills --skill slime-rl-training-karlinskys

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Reduces the complexity of performing reinforcement-learning-based post-training for large language models by integrating Megatron-LM training with high-throughput SGLang rollouts, flexible data buffering, and customizable reward/generation logic so teams can move from research experiments to scalable training runs.

Core Features & Use Cases

  • Megatron-LM native training with full parallelism (tensor, pipeline, data, sequence) to train large GLM, Qwen3, DeepSeek, and Llama 3 models.
  • SGLang rollouts and router for high-throughput generation, multi-turn agent workflows, and tool-calling scenarios.
  • Flexible data buffer and off-policy reuse enabling async training, buffered rollouts, prioritized sampling, and GRPO-style advantage estimators.
  • Custom hooks for generate functions and reward models to support multi-turn tool-enabled agents and bespoke evaluation metrics.
  • Operational guidance for Docker/source installs, async vs synchronous modes, colocated inference, and common troubleshooting for OOMs, weight sync, and router imbalance.

Quick Start

Run a GRPO post-training run that uses Megatron-LM for actor updates and SGLang for rollouts on your GLM checkpoint with a buffered prompt dataset.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run GRPO post-training for large language models using Megatron-LM and SGLang?▼

To run GRPO post-training, coordinate Megatron-LM for actor updates with SGLang for high-throughput rollouts on your model checkpoint using a buffered prompt dataset. This setup supports asynchronous rollout buffering and group-relative advantage training.

Does this RL post-training workflow support multi-turn agentic training with tool calls?▼

Yes, the workflow supports multi-turn agentic training with tool calls. It utilizes SGLang rollouts and routers to handle generation, multi-turn agent workflows, and custom hooks for generate functions and reward models.

Can I apply reinforcement learning post-training to Qwen3, DeepSeek, and Llama 3 models?▼

Yes, you can apply RL post-training to Qwen3, DeepSeek, GLM, and Llama 3 models. The integration uses Megatron-LM native training with full parallelism, including tensor, pipeline, data, and sequence parallelism for large models.

What is the best way to handle GPU cluster resource allocation for RL post-training?▼

The best way to handle GPU cluster resources is using Ray orchestration for allocation. This approach satisfies requirements for distributed training, colocated inference, and managing asynchronous versus synchronous training modes.

Why does my RL training run encounter OOMs and router imbalance during SGLang rollouts?▼

OOMs and router imbalance during SGLang rollouts occur due to memory constraints and uneven traffic distribution. Operational guidance includes troubleshooting these issues alongside weight synchronization and configurable data buffer adjustments.

Do I need Ray orchestration to scale RL post-training for large language models?▼

Yes, Ray orchestration is required to scale RL post-training. It manages GPU cluster resource allocation and coordinates asynchronous rollout buffering alongside Megatron-LM training and SGLang inference.

Related Skills

Explore More Software Engineering Skills→
JuanMS20JuanMS20

huggingface-accelerate

Configure distributed PyTorch training across multi-GPU and multi-node devices.

Community
Advanced
brittaniebuffiecsubrittaniebuffiecsu
1

slime-rl-training

Train large language models with reinforcement learning using Megatron-LM and SGLang.

Community
Advanced
gagan114662gagan114662

slime-rl-training

Integrates Megatron-LM and SGLang for RL-based LLM post-training with flexible data buffering.

Community
Advanced
devMoezdevMoez
1

slime-rl-training

Coordinate Megatron-LM training with SGLang rollouts for GRPO-style RL post-training.

Community
Intermediate