What problem does it solve? Setting up reinforcement learning training for LLMs requires wiring together distributed training, inference, and weight synchronization infrastructure, which distracts from actual algorithm research. This Skill provides guidance for using torchforge, Meta's PyTorch-native RL library that separates infrastructure concerns from algorithm logic. ## Core Features & Use Cases - GRPO and SFT Training Workflows: Step-by-step configuration and launch instructions for math reasoning training with group-relative advantages and supervised fine-tuning. - Custom Loss Functions: Implement new RL algorithms (GRPO, DAPO, SAPO variants) in roughly 100 lines using the forge.losses module without touching infrastructure. - Distributed Scaling: Configure multi-GPU and multi-node training with Monarch actors, TorchTitan parallelism, vLLM inference, and TorchStore weight sync. - Use Case: Train a Qwen2.5-7B model on GSM8K with GRPO across 3 GPUs by writing a YAML config, defining a math reward function, and launching with a single python -m apps.grpo.main command. ## Quick Start Ask the agent to help you set up a GRPO training run with torchforge for a math reasoning model, including the YAML configuration and reward function.
