skills.rest
qcmuu
qcmuuCOMMUNITY
Explore all skills by this author →
GitHub

torchforge-rl-training

Train PyTorch-native reinforcement learning with GRPO objectives and vLLM rollouts.

Updated May 2, 2026
Search Tags:#reinforcement learning#pytorch#distributed training#vllm#torchforge#monarch#grpo training
One-click install
npx skills add https://github.com/qcmuu/AI-Research-Skills --skill torchforge-rl-training-qcmuu
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: torchforge-rl-training
Source: https://github.com/qcmuu/AI-Research-Skills/tree/main/06-post-training/torchforge
Command: npx skills add https://github.com/qcmuu/AI-Research-Skills --skill torchforge-rl-training-qcmuu

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch>=2.9.0, torchtitan>=0.2.0, vllm, monarch, and includes references (resource) components.

What problem does it solve?

This Skill helps you train reinforcement learning models in a PyTorch-native, scalable way by separating RL algorithm code from distributed training and inference infrastructure.

Core Features & Use Cases

  • PyTorch-native agentic RL architecture: Use torchforge to keep algorithms focused while Monarch and TorchTitan handle distributed coordination and training.
  • Async RL pipeline with vLLM generation: Run an asynchronous loop where a generator (vLLM) produces rollouts, a reference model computes KL baselines, and trainer actors optimize updates.
  • Built-in training objectives and easy customization: Apply GRPO-style training workflows (plus other loss variants) and swap in custom loss functions or rewards.

Use cases: training reasoning models with group-relative advantages (GRPO) on math datasets (e.g., GSM8K/MATH), experimenting with new RL loss functions, and scaling multi-GPU distributed runs using actor-based services.

Quick Start

Use the torchforge-rl-training skill to launch GRPO math training with your chosen config by running: python -m apps.grpo.main --config config/grpo_math.yaml.

Frequently Asked Questions about torchforge-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train reinforcement learning models with PyTorch using GRPO and vLLM?▼

To train reinforcement learning models with PyTorch, this Skill uses torchforge to separate GRPO algorithm logic from distributed infrastructure, running an async pipeline where vLLM generates rollouts and trainer actors apply updates. Launch via a config-driven entrypoint like python -m apps.grpo.main.

Does distributed reinforcement learning training work with Monarch and TorchTitan FSDP across multiple GPUs?▼

Distributed reinforcement learning training works with Monarch and TorchTitan FSDP across 2+ GPUs by using actor-based services to coordinate the separated training infrastructure, allowing scalable multi-GPU runs while keeping the algorithm code focused.

What is the best way to customize loss functions and rewards for agentic RL workflows?▼

The best way to customize agentic RL workflows is by using the built-in reward and loss interfaces provided by torchforge, allowing you to apply GRPO-style objectives and swap in custom loss functions without modifying the underlying distributed training infrastructure.

Can I use vLLM for asynchronous rollout generation in PyTorch-native RL training?▼

Yes, you can use vLLM for asynchronous rollout generation in PyTorch-native RL training. The Skill runs an async loop where vLLM acts as the generator producing rollouts for the trainer actors to optimize using KL-regularized updates.

Do I need specific torch and vLLM dependencies to run a GRPO math training pipeline?▼

Yes, you need torch>=2.9.0, torchtitan>=0.2.0, vllm, and monarch dependencies installed. You also need a compatible torchforge setup to run the GRPO math training pipeline on datasets like GSM8K or MATH.

Related Skills

Explore More Software Engineering Skills→
tadod12tadod12

torchforge-rl-training

Train RL experiments with torchforge separating infrastructure from algorithms in PyTorch.

Community
Advanced
nishide-devnishide-dev

ml-train

Execute PyTorch Lightning and Hydra training runs with monitoring, checkpointing, and W&B integration.

Community
Intermediate
zhuangbiaoweizhuangbiaowei
2

torchforge-rl-training

Train PyTorch-native reinforcement learning algorithms with distributed infrastructure.

Community
Advanced
ihatesea69ihatesea69
3

torchforge-rl-training

Train PyTorch-native agentic reinforcement learning algorithms with torchforge.

Community
Advanced