skills.rest
Rheasilvia
RheasilviaCOMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Orchestrate Megatron-LM training and SGLang rollouts for large-scale RL post-training.

Updated Jun 25, 2026
Search Tags:#llm#reinforcement learning#sglang#grpo#post-training#megatron-lm
One-click install
npx skills add https://github.com/Rheasilvia/hermes-desktop --skill slime-rl-training-rheasilvia
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/Rheasilvia/hermes-desktop/tree/main/optional-skills/mlops/slime
Command: npx skills add https://github.com/Rheasilvia/hermes-desktop --skill slime-rl-training-rheasilvia

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This skill addresses the complexity of scaling Reinforcement Learning (RL) for Large Language Models by integrating high-throughput rollout generation with distributed training.

Core Features & Use Cases

  • Distributed RL Training: Supports GRPO, PPO, and other advantage estimators using Megatron-LM for model parallelism.
  • High-Throughput Rollout: Leverages SGLang for efficient response generation and multi-turn agentic interactions.
  • Use Case: Researchers can use this framework to train reasoning models like DeepSeek-R1 or GLM-4 by automating the generation of training samples and reward-based optimization.

Quick Start

Execute the slime training script by providing the path to your prompt data and sourcing the appropriate model configuration file.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scale reinforcement learning post-training for LLMs with Megatron-LM?▼

Scale LLM reinforcement learning post-training by orchestrating Megatron-LM for distributed training and SGLang for high-throughput rollout generation. It supports GRPO and PPO across multi-node GPU clusters using Ray for distributed task management.

Can I use SGLang for multi-turn rollout generation during PPO training?▼

Yes, SGLang facilitates high-throughput response generation and multi-turn agentic interactions during PPO training, automating the generation of training samples required for reward-based optimization.

Do I need Ray to manage distributed tasks for GRPO training?▼

Yes, Ray is required for distributed task management when running GRPO training across multi-node GPU clusters to ensure training stability.

What's the best way to train reasoning models like DeepSeek-R1 using reinforcement learning?▼

Train reasoning models like DeepSeek-R1 or GLM-4 by automating training sample generation and reward-based optimization, integrating Megatron-LM model parallelism with SGLang rollouts for efficient RL.

Does this skill support advantage estimators other than PPO for LLM post-training?▼

Yes, it supports complex RL algorithms including GRPO, PPO, and other advantage estimators facilitated through Megatron-LM for model parallelism during post-training.

How do I start GRPO training using a specific model configuration?▼

Execute the training script by providing the path to your prompt data and sourcing the appropriate model configuration file required for maintaining training stability.

Related Skills

Explore More Software Engineering Skills→
InverterNetworkInverterNetwork

slime-rl-training

Orchestrate Megatron-LM training and SGLang rollout generation for large-scale RL post-training.

Official
Advanced
Orchestra-ResearchOrchestra-Research
11.5k

sglang

Serve LLMs with RadixAttention prefix caching and structured JSON outputs.

Official
Advanced
DaddyElonMusk69DaddyElonMusk69

slime-rl-training

Scale RL-driven LLM fine-tuning with Megatron-LM and SGLang rollouts.

Community
Advanced
devMoezdevMoez
1

slime-rl-training

Coordinate Megatron-LM training with SGLang rollouts for GRPO-style RL post-training.

Community
Intermediate