skills.rest
Orchestra-Research
Orchestra-ResearchOFFICIAL
Explore all skills by this author →
GitHub

slime-rl-training

Integrate Megatron-LM and SGLang for large-scale LLM RL post-training.

11.5k|842|Updated Nov 3, 2025
Search Tags:#llm#reinforcement learning#distributed training#sglang#post-training#megatron-lm
One-click install
npx skills add https://github.com/Orchestra-Research/AI-Research-SKILLs --skill slime-rl-training-orchestra-research
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/06-post-training/slime
Command: npx skills add https://github.com/Orchestra-Research/AI-Research-SKILLs --skill slime-rl-training-orchestra-research

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines and scales the process of post-training Large Language Models (LLMs) using Reinforcement Learning (RL), particularly for models like GLM, by integrating powerful frameworks like Megatron-LM and SGLang.

Core Features & Use Cases

  • Megatron-LM Integration: Leverages Megatron-LM for efficient, distributed training with full parallelism support (TP, PP, DP, SP).
  • SGLang Rollout: Utilizes SGLang for high-throughput response generation during the RL process.
  • Flexible Data Handling: Supports custom data generation workflows and flexible prompt management.
  • Use Case: You are training a new LLM and need to fine-tune its reasoning capabilities using RLHF. This Skill provides the robust infrastructure to handle the complex training pipeline, from data preparation to model rollout and weight synchronization, ensuring efficient scaling across multiple GPUs.

Quick Start

Use the slime skill to launch GRPO training for a Qwen3-4B model using a provided JSONL data file.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scale LLM reinforcement learning training across multiple GPUs?▼

Scale LLM reinforcement learning by integrating Megatron-LM for distributed training with full parallelism support and SGLang for efficient inference rollouts. This combination provides a unified framework to manage training, generation, and weight synchronization across multiple GPUs.

Can I use Megatron-LM and SGLang for asynchronous RLHF post-training?▼

Yes, Megatron-LM and SGLang support asynchronous RLHF post-training. The framework facilitates custom data generation, asynchronous training workflows, and multi-turn agentic workflows to fine-tune LLM reasoning capabilities efficiently.

Does this distributed RL training framework support custom model architectures like Qwen and DeepSeek?▼

Yes, the distributed RL training framework supports various model architectures including GLM, Qwen, DeepSeek, and Llama. It handles the complex training pipeline from data preparation to model rollout for these specific architectures.

What is the best way to generate high-throughput rollouts during LLM reinforcement learning?▼

The best way to generate high-throughput rollouts during LLM reinforcement learning is using SGLang. It is integrated specifically for efficient, high-throughput response generation and flexible prompt management during the RL process.

Do I need Ray and PyTorch to run large-scale LLM post-training with Megatron-LM?▼

Yes, you need Ray, PyTorch, Transformers, and sglang-router to run large-scale LLM post-training. These dependencies establish the robust infrastructure required to handle the complex RL training pipeline and distributed execution.

How to start GRPO training for a Qwen3-4B model using a JSONL dataset?▼

Start GRPO training for a Qwen3-4B model by providing a JSONL data file to the slime skill. The framework manages the robust infrastructure for the complex training pipeline, from data preparation to model rollout and weight synchronization.

Related Skills

Explore More Education & Research Skills→
hhhi21ghhhi21g

verl-rl-training

Train large language models with reinforcement learning on distributed infrastructure.

Community
Advanced
GarrettRoiGarrettRoi

slime-rl-training

Train LLMs with reinforcement learning using Megatron-LM and SGLang.

Community
Advanced
evalstateevalstate

trl

Submit TRL training jobs on Hugging Face Jobs and convert models to GGUF.

Community
Advanced
zangjeicyzangjeicy

slime-rl-training

Train LLMs with GRPO and PPO using Megatron-LM and SGLang.

Community
Advanced