skills.rest
ragnarokhaa
ragnarokhaaCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

slime-rl-training

Integrate Megatron-LM and SGLang for LLM post-training reinforcement learning.

4|Updated Apr 19, 2026
Search Tags:#LLM#large language models#RL#SGLang#post-training#Megatron-LM
One-click install
npx skills add https://github.com/ragnarokhaa/hermes --skill slime-rl-training-ragnarokhaa
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: slime-rl-training
Source: https://github.com/ragnarokhaa/hermes/tree/main/hermes-cerul-tech-news-package/hermes-cerul-tech-news-package/hermes-agent/optional-skills/mlops/slime
Command: npx skills add https://github.com/ragnarokhaa/hermes --skill slime-rl-training-ragnarokhaa

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the complexity and scalability challenges of post-training reinforcement learning (RL) for large language models (LLMs), specifically targeting the tight integration of Megatron-LM and SGLang frameworks.

Core Features & Use Cases

  • Megatron-LM Training: Full parallelism support for actor model training with Megatron-LM.
  • Rollout Generation: High-throughput rollout generation using SGLang-based router.
  • Data Buffer: Flexible prompt management and sample storage for custom data generation workflows.
  • Model Support: Compatibility with GLM, Qwen3, DeepSeek V3/R1, Llama 3, and other large models.
  • Use Case: Ideal for research-grade frameworks seeking production backing, especially for custom data generation workflows and tight integration with Megatron-LM.

Quick Start

Run the following command to begin a standard GRPO training session with the provided model and data:

python train.py --actor-num-nodes 1 --actor-num-gpus-per-node 4 --rollout-num-gpus 4 --advantage-estimator grpo --use-kl-loss --kl-loss-coef 0.001 --rollout-batch-size 32 --n-samples-per-prompt 8 --global-batch-size 256 --num-rollout 3000 --prompt-data /path/to/data.jsonl

Frequently Asked Questions about slime-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run LLM post-training reinforcement learning with Megatron-LM and SGLang?▼

LLM post-training reinforcement learning integrates Megatron-LM for actor model training and SGLang for high-throughput rollout generation. You launch a standard GRPO training session via command line, specifying GPU nodes, batch sizes, and prompt data paths.

Can I use SGLang for high-throughput rollout generation with large models like DeepSeek V3/R1 and Llama 3?▼

SGLang supports high-throughput rollout generation for large models including DeepSeek V3/R1, Llama 3, GLM, and Qwen3. It uses an SGLang-based router to manage and streamline the generation workflow during reinforcement learning.

What is the best way to manage prompts and samples for custom data generation workflows in RL post-training?▼

The best way to manage prompts and samples for custom data generation workflows is using a flexible data buffer. This buffer handles prompt management and sample storage, ensuring seamless integration with Megatron-LM training and SGLang rollouts.

Do I need Ray and PyTorch to run GRPO training sessions for large language models?▼

Yes, you need Ray, PyTorch, Transformers, and an SGLang router to run GRPO training sessions. These dependencies provide the distributed computing, tensor operations, and model handling required for Megatron-LM and SGLang integration.

Does Megatron-LM training support full parallelism for actor models during reinforcement learning?▼

Megatron-LM training provides full parallelism support for actor model training during reinforcement learning. This ensures scalable and efficient computation when processing large models like Qwen3, DeepSeek, and Llama 3.

Why use slime for LLM post-training instead of other RL frameworks?▼

You use this approach to streamline complex post-training RL processes by tightly integrating Megatron-LM and SGLang. It is ideal for research-grade frameworks seeking production backing, offering custom data generation workflows and high-throughput rollouts.

Related Skills

Explore More Software Engineering Skills→
richardnguyen0715richardnguyen0715

slime-rl-training

Train LLMs with reinforcement learning using Megatron-LM and SGLang.

Community
Advanced
Jonah-1Jonah-1

eval

Run inference on neural network checkpoints and generate PSNR, SSIM, and LPIPS metrics.

Community
Intermediate
Z43LZ43L

slime-rl-training

Integrates Megatron-LM training loops with accelerated SLang rollouts for LLM RL fine-tuning.

Community
Advanced
jason660519jason660519

slime-rl-training

Integrate Megatron-LM training with SGLang rollout for RL post-training.

Community
Advanced