skills.rest
NousResearch
NousResearchOFFICIAL
Explore all skills by this author →
Website𝕏@nousresearchGitHub

slime

Train LLMs with reinforcement learning using Megatron and SGLang.

239k|48.8k|Updated Jul 22, 2025
Search Tags:#Ray#SGLang#grpo#Megatron-LM#Reinforcement Learning#Distributed training#LLM post-training
One-click install
npx skills add https://github.com/NousResearch/hermes-agent --skill slime
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime
Source: https://github.com/NousResearch/hermes-agent/tree/main/optional-skills/mlops/slime
Command: npx skills add https://github.com/NousResearch/hermes-agent --skill slime

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve?

Training large language models with reinforcement learning requires coordinating high-throughput rollout generation with distributed training, which is complex and error-prone when done from scratch.

Core Features & Use Cases

  • Megatron-LM Training with SGLang Rollouts: Combines Megatron-LM for distributed training with SGLang for fast inference, supporting TP, PP, DP, and SP parallelism.
  • GRPO and Other RL Algorithms: Implements GRPO, GSPO, PPO, and REINFORCE++ advantage estimators for post-training reasoning and agentic models.
  • Flexible Data Buffer: Manages prompts, custom generation workflows, and off-policy sample reuse for research and production training.
  • Use Case: Train a Qwen3-4B model on math reasoning data using GRPO with 8 samples per prompt, monitoring reward curves via TensorBoard across multiple GPU nodes.

Quick Start

Use the slime skill to set up a GRPO training run for a Qwen3-4B model with custom prompt data and KL regularization.

Frequently Asked Questions about slime

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train an LLM with GRPO using Megatron and SGLang?▼

Use slime's train.py with --advantage-estimator grpo, configure rollout and actor GPU counts, and source a pre-configured model script from scripts/models/. The framework handles weight synchronization between Megatron training and SGLang inference automatically.

What is the difference between slime and verl for RL post-training?▼

slime uses Megatron-LM as its training backend with native SGLang integration, making it suitable for GLM and Qwen3 models. verl offers more flexible backend swapping with PyTorch-native abstractions, while slime focuses on research-grade Megatron parallelism.

Does slime support asynchronous training for higher throughput?▼

Yes, slime provides train_async.py with --async-buffer-size and --update-weights-interval parameters to overlap rollout generation with training. Note that colocated mode is not supported with async training.

Why does SGLang engine crash during slime training?▼

SGLang crashes typically result from insufficient GPU memory or batch size pressure. Enable --use-fault-tolerance, increase --sglang-mem-fraction-static to 0.85, or reduce --rollout-batch-size to stabilize inference.

Which models does slime support for RL post-training?▼

slime supports GLM-4.5, GLM-4.6, GLM-4.7, Qwen3 (4B, 8B, 30B-A3B), DeepSeek V3/R1, Llama 3, Kimi K2, and Moonlight-16B. Each model has pre-configured scripts in the scripts/models/ directory.

When should I use miles instead of slime for RL training?▼

Choose miles when you need enterprise-grade stability features and production hardening. slime is better suited for research workflows requiring Megatron-LM native parallelism and custom data generation pipelines.

Related Skills

Explore More Software Engineering Skills→
helix4uhelix4u

slime-rl-training

Guide RL-based post-training of LLMs with Megatron-LM and SGLang.

Community
Advanced
qcmuuqcmuu

evaluating-llms-harness

Run 60+ academic benchmarks on LLMs with lm-evaluation-harness.

Community
Intermediate
qcmuuqcmuu

openrlhf-training

Run distributed OpenRLHF RLHF training on Ray with vLLM acceleration.

Community
Advanced
tianhao909tianhao909
1

torchforge-rl-training

Train PyTorch-native RL agents with GRPO, DAPO, and SAPO algorithms.

Community
Advanced