skills.rest
big4council-prog
big4council-progCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

slime-rl-training

Orchestrate Megatron-LM training with SGLang rollouts for GRPO optimization.

Updated Aug 27, 2026
Search Tags:#reinforcement learning#distributed training#sglang#megatron-lm#reward modeling#llm post-training#grpo training
One-click install
npx skills add https://github.com/big4council-prog/b4c-agent --skill slime-rl-training-big4council-prog
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/big4council-prog/b4c-agent/tree/main/optional-skills/mlops/slime
Command: npx skills add https://github.com/big4council-prog/b4c-agent --skill slime-rl-training-big4council-prog

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes references (resource) components.

What problem does it solve?

LLM RL post-training is hard to scale and wire together correctly when you need efficient rollouts plus tightly integrated Megatron-LM training. This Skill helps you set up and run RL post-training using THUDM’s slime framework so that training and rollout generation work as a single system.

Core Features & Use Cases

  • Megatron-LM training with full parallelism: Runs actor training (and optional critic) with tensor/pipeline/data parallelism to match large-model training setups.
  • SGLang-based high-throughput rollouts: Generates rollout samples efficiently via SGLang and a router, enabling GRPO-style group-relative optimization loops.
  • Flexible data buffer and custom generation: Supports off-policy buffering and custom multi-turn/tool-calling generation workflows for research-grade agents.
  • Use Case: Train a GLM/Qwen/DeepSeek/Llama model with GRPO on JSONL prompts and labels, automatically generating multiple responses per prompt and optimizing using KL-regularized RL.

Quick Start

Use the slime skill to run GRPO training by executing a train.py command with your sourced model args, specifying prompt-data, input-key, label-key, rollout batch sizing, and rollout sample counts.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scale LLM RL post-training with Megatron-LM and SGLang?▼

You scale LLM RL post-training by orchestrating Megatron-LM training with SGLang-based rollout generation, running end-to-end rollout-to-update loops for group-relative optimization.

Can I use GRPO training for multi-turn and tool-calling data generation?▼

Yes, GRPO training supports custom multi-turn and tool-calling generation workflows. It uses an off-policy data buffer to generate multiple responses per prompt for research-grade agent optimization.

Do I need Ray and Torch to run distributed reinforcement learning for LLMs?▼

Yes, you need a Megatron-LM, SGLang, Ray, Torch, and Transformers environment to run distributed reinforcement learning, utilizing tensor, pipeline, and data parallelism for large-model training.

What is the best way to train a Qwen or DeepSeek model with KL-regularized RL?▼

The best way to train Qwen or DeepSeek models with KL-regularized RL is using GRPO estimators on JSONL prompts, automatically generating rollout samples and optimizing via SGLang routing.

Does this reinforcement learning setup support tensor and pipeline parallelism for large models?▼

Yes, the reinforcement learning setup runs actor training and optional critic models with full tensor, pipeline, and data parallelism to match large-model training configurations.

Why use SGLang for rollout generation in LLM post-training?▼

SGLang generates high-throughput rollout samples via a router, enabling efficient GRPO-style group-relative optimization loops tightly integrated with Megatron-LM training synchronization.

Related Skills

Explore More Software Engineering Skills→
blueskies1818blueskies1818
1

slime-rl-training

Guide LLM post-training with RL using Megatron-LM and SGLang.

Community
Advanced
niznik-devniznik-dev
16

Run Experiment

Orchestrate SLURM fine-tuning jobs with torchtune and evaluate with inspect-ai.

Community
Advanced
ovachieverovachiever
52

grpo-rl-training

Apply GRPO/RL training with TRL to optimize language models.

Community
Advanced
GarrettRoiGarrettRoi

fine-tuning-with-trl

Fine-tune LLMs with TRL using SFT, DPO, PPO, and GRPO.

Community
Advanced