skills.rest
ZardLi1115
ZardLi1115COMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Integrate Megatron-LM training with SGLang rollout generation for GRPO-style RL post-training.

4|Updated May 18, 2026
Search Tags:#reinforcement learning#distributed training#sglang#grpo#megatron-lm#llm post-training#data buffering
One-click install
npx skills add https://github.com/ZardLi1115/zedclaw --skill slime-rl-training-zardli1115
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/ZardLi1115/zedclaw/tree/main/optional-skills/mlops/slime
Command: npx skills add https://github.com/ZardLi1115/zedclaw --skill slime-rl-training-zardli1115

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes references (resource) components.

What problem does it solve?

slime streamlines LLM reinforcement-learning post-training by connecting Megatron-LM training with SGLang rollout generation, reducing the complexity of building an RL system that needs high-throughput sampling and tight training/inference integration.

Core Features & Use Cases

  • Megatron-LM + SGLang orchestration: Runs actor (and optional critic) training in Megatron-LM while generating rollouts with SGLang and a router for throughput.
  • GRPO and related RL algorithms: Supports group-relative advantage estimation and common RL objective variants with KL loss control for stability.
  • Flexible data buffer and rollout reuse: Uses a data buffer system to manage prompts, store rollouts, and optionally filter/reselect samples for off-policy style training.
  • Use Case: Post-train a GLM/Qwen/DeepSeek/Llama-style model on reasoning or preference signals by repeatedly generating candidate responses, scoring them with a verifier/reward function, and training with bounded rollout/fix loops until performance improves.

Quick Start

Run slime with GRPO training by providing your JSONL dataset and selecting a model script (e.g., qwen3-4B) to start training and rollout generation in a single coordinated workflow.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scale LLM reinforcement learning post-training with Megatron-LM?▼

It connects Megatron-LM training with SGLang rollout generation to streamline LLM reinforcement learning post-training, reducing the complexity of building an RL system that needs high-throughput sampling and tight training/inference integration.

What is the best way to generate custom rollout data for GRPO training?▼

The best way to generate custom rollout data is using SGLang with a router for throughput, storing rollouts in a flexible data buffer system that optionally filters and reselects samples for off-policy style training.

Can I use this Skill for multi-turn or tool-augmented agent training?▼

Yes, the Skill explicitly applies to GRPO-style and related RL scaling scenarios such as multi-turn or tool-augmented agent training, supporting custom rollout data generation and bounded rollout and fix loops.

Does the distributed training workflow support models like Qwen and DeepSeek?▼

Yes, the workflow applies to GLM, Qwen, DeepSeek, and Llama-style post-training, allowing you to train models on reasoning or preference signals by repeatedly generating, scoring, and training with bounded rollout and fix loops.

Do I need Ray and Torch to configure Megatron parallelism for RL?▼

Yes, you need Ray, torch>=2.0.0, transformers>=4.40.0, and sglang-router>=0.2.3 to configure Megatron parallelism and properly set rollout and training batch constraints using the specified command-line arguments.

Why does my RL training loop require a data buffer system for rollouts?▼

Your RL training loop requires a data buffer system to manage prompts, store rollouts, and optionally filter or reselect samples for off-policy style training, ensuring stable GRPO training with KL loss control.

Related Skills

Explore More Education & Research Skills→
richardnguyen0715richardnguyen0715

slime-rl-training

Train LLMs with reinforcement learning using Megatron-LM and SGLang.

Community
Advanced
kaminocorpkaminocorp
228

grpo-rl-training

Implement GRPO training with TRL for language-model fine-tuning.

Official
Advanced
qcmuuqcmuu

slime-rl-training

Coordinate Megatron-LM training with SGLang rollouts for LLM reinforcement-learning post-training.

Community
Advanced
KarlinskySKarlinskyS

slime-rl-training

Coordinate Megatron-LM training with SGLang rollouts for RL-based post-training.

Community
Advanced