torchforge-rl-training

You can run distributed reinforcement learning training without Ray by using TorchForge, a PyTorch-native library that handles distributed training, inference, and weight sync automatically through the Monarch actor system.

TorchForge separates RL algorithm logic from infrastructure by providing modular abstractions, allowing you to focus purely on algorithm experimentation while it natively orchestrates scalable training and inference without Ray.

Yes, you can use Monarch and TorchTitan together for scalable RL experiments. TorchForge integrates with Monarch for its actor system and TorchTitan for model parallelism to orchestrate distributed training and weight synchronization.

Built-in reinforcement learning algorithms available for experimentation include GRPO, DAPO, CISPO, GSPO, and SAPO, enabling rapid algorithm testing within the TorchForge framework.

The best way to debug weight synchronization issues in PyTorch-native RL is to follow the debugging strategies provided by TorchForge, which cleanly isolates infrastructure orchestration from your algorithm logic.

To start a GRPO training workflow, you need to install TorchForge and run it with Monarch and TorchTitan, which handle the underlying distributed training and model parallelism automatically.