torchforge-rl-training

Automate distributed reinforcement learning training by using TorchForge to orchestrate execution across GPUs and nodes. It isolates algorithms from infrastructure, handling weight synchronization and distributed training automatically so you can focus on experimentation.

Yes, vLLM for inference and TorchTitan for training integrate seamlessly within the same RL workflow. TorchForge provides production-ready orchestration, connecting the PyTorch-native training stack with vLLM inference without manual pipeline wiring.

Algorithm isolation works by cleanly separating your RL algorithm implementation from the distributed infrastructure. TorchForge manages Monarch-based execution, weight synchronization, and node communication, letting you prototype algorithms without touching distributed systems code.

The built-in loss functions available for reinforcement learning include GRPO, DAPO, and SAPO. These are integrated natively into TorchForge alongside sampling utilities, allowing rapid prototyping of RL methods on large models.

Monarch is required for scalable, distributed execution across multiple GPUs and nodes. TorchForge relies on Monarch to manage the distributed infrastructure, ensuring your PyTorch-native RL training scales efficiently across your hardware environment.

The best way to start a GRPO training workflow is to run the provided configuration file, config/grpo_math.yaml. This quick start approach leverages TorchForge to automatically handle the orchestration, allowing immediate experimentation with minimal boilerplate.