skills.rest
Chensihakniroth
ChensihaknirothCOMMUNITY
Explore all skills by this author →
GitHub

slime-rl-training

Guides LLM post-training with reinforcement learning using Megatron-LM and SGLang.

Updated Jun 7, 2026
Search Tags:#reinforcement learning#LLM training#distributed training#GRPO#sglang#post-training#megatron-lm
One-click install
npx skills add https://github.com/Chensihakniroth/ANAKOT-AGENT --skill slime-rl-training-chensihakniroth
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/Chensihakniroth/ANAKOT-AGENT/tree/main/optional-skills/mlops/slime
Command: npx skills add https://github.com/Chensihakniroth/ANAKOT-AGENT --skill slime-rl-training-chensihakniroth

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve? Setting up reinforcement learning post-training for large language models requires coordinating distributed training, high-throughput rollout generation, and data buffering, which is complex to configure correctly. ## Core Features & Use Cases - GRPO and PPO Training Workflows: Step-by-step guidance for standard, asynchronous, and multi-turn agentic RL training with Megatron-LM and SGLang. - Custom Data and Reward Functions: Patterns for implementing custom generate functions, reward models, and buffered off-policy data sources. - Troubleshooting Reference: Solutions for SGLang crashes, weight sync timeouts, OOM errors, and training instability. - Use Case: Train a Qwen3-4B reasoning model with GRPO by preparing JSONL prompt data, sourcing a pre-configured model script, and launching distributed training across 8 GPUs. ## Quick Start Ask the agent to set up a GRPO training run for a Qwen3 model using slime with your JSONL prompt dataset.

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train an LLM with GRPO using slime?▼

Prepare a JSONL dataset with prompt and label fields, source a pre-configured model script from scripts/models/, then launch train.py with --advantage-estimator grpo, rollout batch size, and GPU allocation flags. Monitor progress through TensorBoard reward curves.

slime vs verl vs torchforge for RL training?▼

slime provides native Megatron-LM training with SGLang inference and backs GLM-4.x models in production. Choose verl for flexible backend swapping, torchforge for PyTorch-native abstractions, or miles for enterprise stability features.

Which models does slime support for RL post-training?▼

slime supports GLM-4.5/4.6/4.7, Qwen3 including MoE variants, DeepSeek V3 and R1, Llama 3, Kimi K2, and Moonlight-16B. Each model family has pre-configured shell scripts in the scripts/models/ directory.

Can slime run asynchronous RL training?▼

Yes, use train_async.py with --async-buffer-size and --update-weights-interval to overlap rollout and training for higher throughput. Note that colocated mode sharing GPUs between training and inference is not supported in async mode.

Why does slime training hit CUDA OOM errors?▼

OOM occurs from large batch sizes or insufficient memory allocation between training and inference. Enable --recompute-activations, reduce --micro-batch-size to 1, lower --sglang-mem-fraction-static in colocated mode, or enable sequence parallelism.

How do I train multi-turn agents with tool calling in slime?▼

Write a custom async generate function that loops through turns, executes tool calls, and appends results to the conversation. Pass it via --custom-generate-function-path and set a proper loss mask so tool responses are excluded from training loss.

Related Skills

Explore More Software Engineering Skills→
vivekgoquestvivekgoquest

slime

Trains LLMs with reinforcement learning using Megatron-LM and SGLang rollout generation.

Community
Advanced
Jensen-YaoJensen-Yao
2

verl-rl-training

Configures and launches reinforcement learning training for LLMs using verl with PPO, GRPO, and distributed backends.

Community
Advanced
Jensen-YaoJensen-Yao
2

torchforge-rl-training

Implements PyTorch-native agentic RL training with GRPO, SFT, Monarch, and TorchTitan.

Community
Advanced
kryptonum-devkryptonum-dev
1

10x-research

Decompose codebase research questions into parallel sub-agent tasks with file:line evidence.

Official
Advanced