skills.rest
jonnabio
jonnabioCOMMUNITY
Explore all skills by this author →
𝕏@jonnabioGitHub

slime-rl-training

Optimize LLMs post-training with RL using slime, Megatron-LM, and SGLang.

6|3|Updated Jan 29, 2026
Search Tags:#LLM#GLM#SGLang#Megatron-LM#Qwen3#Reinforcement Learning
One-click install
npx skills add https://github.com/jonnabio/ace-framework --skill slime-rl-training-jonnabio
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/jonnabio/ace-framework/tree/main/.ace/packs/ai-research/slime
Command: npx skills add https://github.com/jonnabio/ace-framework --skill slime-rl-training-jonnabio

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill addresses the challenge of optimizing Large Language Models (LLMs) post-training using Reinforcement Learning (RL) with the slime framework, enabling scalable and efficient model enhancements.

Core Features & Use Cases

  • Megatron-LM Integration: Seamless integration with Megatron-LM for training with full parallelism support.
  • SGLang Rollout: High-throughput rollout generation using SGLang for efficient inference.
  • Data Buffer Management: Flexible prompt management and sample storage for data handling.
  • Model Support: Supports training for various models including GLM, Qwen3, DeepSeek V3/R1, and Llama 3.
  • Use Case: Ideal for research-grade frameworks requiring production backing, such as Z.ai, for implementing custom data generation workflows and tight integration with Megatron-LM for RL scaling.

Quick Start

Run the following command to start the training process for a GRPO model using slime:

python train.py \
    --actor-num-nodes 1 \
    --actor-num-gpus-per-node 4 \
    --rollout-num-gpus 4 \
    --advantage-estimator grpo \
    --use-kl-loss --kl-loss-coef 0.001 \
    --rollout-batch-size 32 \
    --n-samples-per-prompt 8 \
    --global-batch-size 256 \
    --num-rollout 3000 \
    --prompt-data /path/to/data.jsonl \
    ${MODEL_ARGS[@]} ${CKPT_ARGS[@]}

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize LLM post-training with reinforcement learning using slime?▼

To optimize LLM post-training with reinforcement learning using slime, you run the provided train.py script. It integrates with Megatron-LM for training and SGLang for rollout generation, allowing you to scale models efficiently.

Can I train a Qwen3 or DeepSeek model with Megatron-LM and SGLang?▼

Yes, you can train Qwen3 or DeepSeek models. The framework supports various models including GLM, DeepSeek V3/R1, and Llama 3, integrating Megatron-LM for parallel training and SGLang for high-throughput rollout inference.

How do I start GRPO training for a scalable LLM?▼

You start GRPO training by executing the train.py script with parameters like advantage-estimator set to grpo. You must specify actor nodes, GPU counts, rollout batch size, and your prompt data path.

Do I need Ray and PyTorch to run slime reinforcement learning?▼

Yes, you need Ray and PyTorch to run slime reinforcement learning. The framework requires specific dependencies including ray, torch>=2.0.0, transformers>=4.40.0, and sglang-router>=0.2.3 to function properly.

What is the best way to manage prompts for LLM reinforcement learning?▼

The best way to manage prompts is using the framework's data buffer management. It offers flexible prompt management and sample storage, allowing you to handle data efficiently via a specified JSONL data path.

Related Skills

Explore More Software Engineering Skills→
aayushsoamaayushsoam
1

slime-rl-training

Automate LLM post-training with slime and Megatron-LM integration.

Community
Advanced
vasilyu1983vasilyu1983
73

ai-llm-inference

Optimize LLM inference performance and cost with vLLM and quantization.

Community
Advanced
Tommy-ywTommy-yw
539

slime-rl-training

Optimize large language models with Megatron-LM training and SGLang rollouts.

Community
Advanced
ovachieverovachiever
52

grpo-rl-training

Apply GRPO/RL training with TRL to optimize language models.

Community
Advanced