skills.rest
box755
box755COMMUNITY
Explore all skills by this author →
GitHub

torchforge-rl-training

Coordinate distributed PyTorch-native reinforcement learning training with TorchTitan and vLLM.

Updated Aug 23, 2026
Search Tags:#reinforcement learning#distributed training#vllm#torch#grpo#torchtitan#pytorch-native
One-click install
npx skills add https://github.com/box755/simlens-research --skill torchforge-rl-training-box755
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: torchforge-rl-training
Source: https://github.com/box755/simlens-research/tree/main/skill-packs/AI-Research-SKILLs/06-post-training/torchforge
Command: npx skills add https://github.com/box755/simlens-research --skill torchforge-rl-training-box755

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

torchforge-rl-training streamlines building and scaling PyTorch-native reinforcement learning pipelines by separating RL algorithm code from distributed training and inference infrastructure.

Core Features & Use Cases

  • Algorithm-first RL development: implement and iterate on losses and reward logic (e.g., GRPO-style objectives) without rewriting infrastructure for distribution and weight sync.
  • Scalable distributed training: coordinate trainer, generator, and reference policy components via Monarch and integrate with TorchTitan for efficient training and parallelism.
  • Practical RL workflows: use GRPO training configurations for reasoning/math tasks and extend with custom loss functions and multi-GPU launch patterns.

Use case: Train a reasoning model for math (e.g., GSM8K/MATH) by sampling multiple responses per prompt, computing advantages using a frozen reference policy, and optimizing with a GRPO-like clipped objective.

Quick Start

Run GRPO training using the provided torchforge entrypoint by executing: python -m apps.grpo.main --config config/grpo_math.yaml.

Frequently Asked Questions about torchforge-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scale PyTorch-native reinforcement learning across multiple GPUs?▼

You can scale PyTorch-native reinforcement learning by coordinating distributed trainer, generator, and frozen reference policy services via Monarch, integrating TorchTitan for efficient multi-GPU parallelism and weight synchronization.

What is GRPO-style training and how does it work with a frozen reference policy?▼

GRPO-style training optimizes models by sampling multiple responses per prompt, computing advantages via a frozen reference policy, and applying a clipped objective. This Skill separates that algorithm logic from distributed infrastructure for seamless iteration.

Can I use vLLM for generation in a distributed GRPO training loop?▼

Yes, vLLM is supported for generation. This Skill coordinates the generator service alongside the trainer and frozen reference policy to handle RL batches structured around logprobs, ref_logprobs, advantages, and padding masks.

How do I implement custom loss functions for reinforcement learning without rewriting distributed code?▼

Implement custom loss functions by applying your algorithm logic directly to RL batch targets like logprobs and advantages. This Skill manages the distributed training and weight synchronization separately via Monarch and TorchTitan.

Do I need PyTorch and Monarch to run GRPO training for math reasoning tasks?▼

Yes, you need PyTorch, Monarch, TorchTitan, and vLLM. This Skill provides a torchforge entrypoint to run GRPO training configurations for math reasoning tasks using a frozen reference policy and clipped objective.

Related Skills

Explore More Education & Research Skills→
tadod12tadod12

torchforge-rl-training

Train RL experiments with torchforge separating infrastructure from algorithms in PyTorch.

Community
Advanced
ricablericable

torchforge-rl-training

Coordinate distributed PyTorch-native RL training for agentic language models.

Community
Advanced
tianhao909tianhao909
1

torchforge-rl-training

Train PyTorch-native RL agents with GRPO, DAPO, and SAPO algorithms.

Community
Advanced
OpenLAIROpenLAIR
1.0k

torchforge-rl-training

Automate distributed RL training orchestration for PyTorch-native workflows.

Official
Advanced