skills.rest
vivekgoquest
vivekgoquestCOMMUNITY
Explore all skills by this author →
GitHub

slime

Trains LLMs with reinforcement learning using Megatron-LM and SGLang rollout generation.

Updated Aug 22, 2026
Search Tags:#reinforcement learning#RLHF#distributed training#GRPO#sglang#megatron-lm#llm post-training
One-click install
npx skills add https://github.com/vivekgoquest/hermes-agent-stable --skill slime-vivekgoquest
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime
Source: https://github.com/vivekgoquest/hermes-agent-stable/tree/main/optional-skills/mlops/slime
Command: npx skills add https://github.com/vivekgoquest/hermes-agent-stable --skill slime-vivekgoquest

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router, ray, torch, transformers, and includes references (resource) components.

What problem does it solve? Setting up RL post-training pipelines for large language models requires coordinating distributed training, high-throughput inference, and data management across many GPUs, which is complex to build from scratch. ## Core Features & Use Cases - GRPO and PPO Training: Run group-relative policy optimization and other RL algorithms with Megatron-LM parallelism (TP, PP, DP, SP) and SGLang-based rollout generation. - Multi-Turn Agentic Training: Define custom generate functions for tool-calling and multi-step reasoning workflows with proper loss masking. - Async and Colocated Modes: Overlap rollout and training for higher throughput, or share GPUs between training and inference to reduce memory. - Use Case: Train a Qwen3-4B reasoning model on math problems by preparing JSONL prompts with labels, sourcing a pre-configured model script, and launching GRPO training across 8 GPUs. ## Quick Start Use the slime skill to launch GRPO training on my Qwen3-4B model with the math dataset at /data/train.jsonl across 8 GPUs.

Frequently Asked Questions about slime

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run GRPO training on an LLM with Megatron and SGLang?▼

Prepare a JSONL dataset with prompt and label fields, source a pre-configured model script from scripts/models/, then launch train.py with --advantage-estimator grpo, GPU allocation flags, and batch size settings. Ensure rollout_batch_size times n_samples_per_prompt equals global_batch_size times num_steps_per_rollout.

slime vs verl vs torchforge for RL post-training?▼

slime provides Megatron-LM native training with SGLang inference and powers GLM-4.x models. Choose verl for flexible backend swapping, torchforge for PyTorch-native abstractions, or miles for enterprise-grade stability features.

What models does slime support for RL training?▼

slime supports GLM-4.5/4.6/4.7, Qwen3 (4B, 8B, 30B-A3B), DeepSeek V3/V3.1/R1, Llama 3 (8B, 70B), Kimi K2, and Moonlight-16B. Each model has pre-configured scripts in the scripts/models/ directory.

Can I use colocated mode with async training in slime?▼

No, colocated mode is not supported with train_async.py. Async training requires separate GPUs for training and rollout; remove the --colocate flag and allocate distinct GPU counts for actor and rollout workers.

Why does SGLang crash during slime training?▼

SGLang engine crashes mid-training usually stem from memory pressure or large batches. Enable --use-fault-tolerance, raise --sglang-mem-fraction-static to 0.85, or reduce --rollout-batch-size to 16.

How do I fix CUDA OOM during slime RL training?▼

Enable gradient checkpointing with --recompute-activations, reduce --micro-batch-size to 1, turn on --sequence-parallel, or lower --global-batch-size. In colocated mode, also reduce --sglang-mem-fraction-static to 0.4.

Related Skills

Explore More Software Engineering Skills→
musical-basicsmusical-basics

grpo-rl-training

Fine-tune language models with Group Relative Policy Optimization and composable reward functions.

Community
Advanced
MLT-OSSMLT-OSS
14

evaluating-llms-harness

Evaluates LLMs across 60+ academic benchmarks using standardized prompts and metrics.

Official
Advanced
ovachieverovachiever
52

grpo-rl-training

Apply GRPO/RL training with TRL to optimize language models.

Community
Advanced
tadod12tadod12

verl-rl-training

Configure scalable Verl RL training workflows for LLMs across vLLM and Megatron backends.

Community
Advanced