skills.rest
loteiron
loteironCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

slime

Trains LLMs with reinforcement learning using Megatron-LM and SGLang rollout generation.

Updated Sep 10, 2026
Search Tags:#reinforcement learning#GLM#distributed training#GRPO#sglang#megatron-lm#llm post-training
One-click install
npx skills add https://github.com/loteiron/ZeusAgent --skill slime-loteiron
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime
Source: https://github.com/loteiron/ZeusAgent/tree/main/optional-skills/mlops/slime
Command: npx skills add https://github.com/loteiron/ZeusAgent --skill slime-loteiron

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve? Setting up RL post-training pipelines for large language models requires coordinating distributed training, high-throughput inference, and data management across frameworks, which is complex and error-prone to build from scratch. ## Core Features & Use Cases - Megatron-LM + SGLang Integration: Combines Megatron-LM training parallelism (TP, PP, DP, SP) with SGLang-based high-throughput rollout generation and weight synchronization. - Multiple RL Workflows: Supports standard GRPO training, asynchronous training with buffered rollouts, and multi-turn agentic training with custom generate functions and tool calling. - Flexible Data Buffer: Provides prompt management, rollout sample storage, and off-policy buffered data sources with custom filtering logic. - Use Case: Train a Qwen3-4B reasoning model with GRPO by sourcing a pre-configured model script, preparing JSONL prompt-label data, and launching train.py with rollout and batch parameters. ## Quick Start Ask the agent to set up a GRPO training run for a Qwen3-4B model using slime with a JSONL prompt dataset and 8 GPUs split between training and rollout.

Frequently Asked Questions about slime

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run GRPO training with slime?▼

Source a pre-configured model script from scripts/models/, prepare JSONL data with prompt and label fields, then launch train.py with --advantage-estimator grpo, rollout batch size, samples per prompt, and GPU allocation flags. Monitor progress through TensorBoard reward curves.

What models does slime support for RL post-training?▼

slime supports GLM-4.5/4.6/4.7, Qwen3 (4B, 8B, 30B-A3B), DeepSeek V3/V3.1/R1, Llama 3 (8B, 70B), Kimi K2, and Moonlight-16B. Each model family has pre-configured shell scripts in scripts/models/ defining architecture and checkpoint arguments.

slime vs verl for LLM reinforcement learning?▼

slime provides Megatron-LM native training tightly integrated with SGLang inference and backs production GLM models, while verl offers more flexible backend swapping. Choose slime for Megatron-SGLang workflows and verl when you need interchangeable training or inference backends.

Can slime do multi-turn agent training with tool calls?▼

Yes, slime supports multi-turn agentic training through a custom generate function passed via --custom-generate-function-path. The function handles tool execution loops, conversation history, and loss masks that exclude tool responses from training loss.

Why does slime training OOM during the backward pass?▼

CUDA OOM during training typically results from large batch sizes or long sequences. Enable gradient checkpointing with --recompute-activations, reduce --micro-batch-size to 1, enable --sequence-parallel, or lower --global-batch-size.

Does async training work with colocated GPUs in slime?▼

No, colocated mode is not supported with train_async.py. Async training requires separate GPU allocations for training and rollout, using --async-buffer-size and --update-weights-interval to control buffering and weight sync frequency.

Related Skills

Explore More Software Engineering Skills→
NVlabsNVlabs
108

alpamayo1-x-rl

Automate RL post-training for Alpamayo VLM action models with Cosmos-RL and GRPO.

Official
Advanced
ScientiaCapitalScientiaCapital
1

model-deployment

Export fine-tuned models to GGUF and deploy with vLLM or Ollama.

Community
Advanced
Jensen-YaoJensen-Yao
2

openrlhf-training

Trains large language models with distributed PPO, GRPO, RLOO, and DPO using Ray and vLLM.

Community
Advanced
MesferAliMesferAli

torchforge-rl-training

Guide PyTorch-native agentic RL training with GRPO using torchforge.

Community
Advanced