skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

torchforge-rl-training

Orchestrate PyTorch-native distributed RL training and vLLM inference for agentic models.

2|Updated Apr 12, 2026
Search Tags:#reinforcement learning#pytorch#distributed-training#vllm#grpo#torchtitan#monarch
One-click install
npx skills add https://github.com/Clay-HHK/claude-config --skill torchforge-rl-training-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: torchforge-rl-training
Source: https://github.com/Clay-HHK/claude-config/tree/main/skills/AI-research-SKILLs/06-post-training/torchforge
Command: npx skills add https://github.com/Clay-HHK/claude-config --skill torchforge-rl-training-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Enables researchers and engineers to focus on RL algorithm development by separating infrastructure from algorithm code, removing manual orchestration of distributed training, inference, and weight synchronization.

Core Features & Use Cases

  • Algorithm isolation: Implement GRPO, DAPO, SAPO and custom loss functions without modifying distributed infra.
  • Scalable multi-GPU training: Run from single-GPU experiments to multi-node Monarch actor deployments with TorchTitan FSDP.
  • High-throughput inference: Integrates vLLM-based generators and weight sync mechanisms for live evaluation and RL loops.
  • Use Case: Train a math-reasoning model with GRPO across trainer, reference model, and generator services, or prototype a new RL loss and deploy across a distributed cluster.

Quick Start

Start a GRPO training run by launching the GRPO application with your config file to provision trainer, generator, and reference services.

Frequently Asked Questions about torchforge-rl-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scale reinforcement learning training across multiple GPUs without writing distributed infrastructure code?▼

You can scale reinforcement learning training by using this Skill to isolate algorithm code from distributed infrastructure. It orchestrates multi-GPU and multi-node deployments using Monarch actors and TorchTitan FSDP, allowing you to focus purely on RL development.

Can I use vLLM for high-throughput generation in my PyTorch RL training loop?▼

Yes, you can use vLLM for high-throughput generation in your PyTorch RL training loop. The Skill integrates vLLM-based generators and handles weight synchronization between the trainer and generator services for live evaluation.

Does PyTorch 2.9+ compatibility matter for distributed GRPO algorithm development?▼

PyTorch 2.9+ compatibility is explicitly required for distributed GRPO algorithm development here. The Skill meets technical requirements for this version to ensure proper weight synchronization and configurable multi-process GPU services.

What is the best way to implement custom loss functions for agentic model training without modifying distributed infra?▼

The best way to implement custom loss functions is by using the Skill's algorithm isolation feature. It allows you to develop GRPO, DAPO, or SAPO algorithms and plug in custom reward modules without touching the underlying distributed infrastructure.

How do I start a GRPO training run for a math-reasoning model with trainer, reference, and generator services?▼

To start a GRPO training run, launch the GRPO application with your configuration file. This provisions the trainer, reference model, and generator services automatically to begin training your math-reasoning model.

Related Skills

Explore More Education & Research Skills→
CUexterCUexter
2

pytorch-lightning

Structure PyTorch training workflows with LightningModule and Trainer.

Community
Advanced
GitnappGitnapp

serving-llms-vllm

Deploy large language model inference services with vLLM and OpenAI-compatible endpoints.

Community
Advanced
tadod12tadod12

torchforge-rl-training

Train RL experiments with torchforge separating infrastructure from algorithms in PyTorch.

Community
Advanced
hung-phanhung-phan
1

distributed-grpo

Scale GRPO/PPO reinforcement learning training across multiple GPUs with DeepSpeed.

Community
Advanced