skills.rest
xu1713
xu1713COMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Guides LLM post-training with reinforcement learning using Megatron-LM and SGLang.

Updated Jun 5, 2026
Search Tags:#reinforcement learning#distributed training#sglang#rlhf#grpo#megatron-lm#llm post-training
One-click install
npx skills add https://github.com/xu1713/openhorse --skill slime-rl-training-xu1713
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/xu1713/openhorse/tree/main/openhorse/openhorse/optional-skills/mlops/slime
Command: npx skills add https://github.com/xu1713/openhorse --skill slime-rl-training-xu1713

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve? Setting up reinforcement learning post-training for large language models requires coordinating distributed training, high-throughput rollout generation, and data management across frameworks, which is complex and error-prone to configure manually. ## Core Features & Use Cases - GRPO and RL Training Workflows: Step-by-step guidance for GRPO, PPO, GSPO, and REINFORCE++ training with Megatron-LM parallelism and SGLang rollout generation. - Async and Multi-Turn Training: Instructions for asynchronous training with buffered rollouts and multi-turn agentic training with custom tool-calling generate functions. - Troubleshooting Reference: Solutions for SGLang crashes, weight sync timeouts, OOM errors, reward collapse, and data loading bottlenecks. - Use Case: Train a Qwen3-4B model with GRPO on math reasoning data by sourcing a pre-configured model script, preparing JSONL prompt data, and launching train.py with the documented argument set. ## Quick Start Ask the assistant to set up a GRPO training run with slime for a Qwen3 model using your JSONL prompt dataset.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train an LLM with GRPO using slime?▼

Prepare a JSONL dataset with prompt and label fields, source a pre-configured model script from scripts/models/, then launch train.py with --advantage-estimator grpo, rollout batch size, and samples per prompt. Ensure rollout_batch_size times n_samples_per_prompt equals global_batch_size times num_steps_per_rollout.

What models does slime support for RL training?▼

slime supports GLM-4.5 through GLM-4.7, Qwen3 including MoE variants, DeepSeek V3 and R1, Llama 3, Kimi K2, and Moonlight-16B. Each model family has pre-configured shell scripts in the scripts/models/ directory defining architecture and checkpoint arguments.

slime vs verl vs miles for RL post-training?▼

Choose slime for Megatron-LM native training with SGLang inference and custom data generation workflows. Use miles for enterprise-grade stability features, verl for flexible backend swapping, or torchforge for PyTorch-native abstractions.

Can I use async training with colocated GPUs in slime?▼

No, colocated mode is not supported with async training in slime. When using train_async.py, allocate separate GPUs for training and rollout instead of passing the --colocate flag.

Why does slime training OOM during the backward pass?▼

CUDA OOM during training typically results from large batch sizes or long sequences. Enable gradient checkpointing with --recompute-activations, reduce --micro-batch-size to 1, or enable --sequence-parallel to distribute memory load.

How do I train multi-turn agents with tool calling in slime?▼

Write a custom async generate function that loops through turns, executes tool calls, and appends results to the conversation. Pass it via --custom-generate-function-path and set a proper loss mask so tool responses are excluded from loss computation.

Related Skills

Explore More Software Engineering Skills→
box755box755

slime-rl-training

Orchestrate Megatron-LM training with SGLang router rollouts for RL post-training.

Community
Advanced
swcstudiospaceswcstudiospace

evaluating-llms-harness

Benchmarks LLMs across 60+ academic tasks using the lm-evaluation-harness framework.

Official
Advanced
JKhyroJKhyro

fine-tuning-with-trl

Fine-tune language models with TRL for SFT, DPO, PPO, and GRPO training.

Community
Advanced
NousResearchNousResearch
239.1k

axolotl

Configure and execute LLM fine-tuning workflows using Axolotl YAML configs.

Official
Advanced