distributed-grpo

What problem does it solve?

This Skill addresses the scaling challenges of GRPO/PPO reinforcement learning for LLMs, allowing training across multiple GPUs, with considerations for reward function design, vLLM generation, DeepSpeed training, and Ray orchestration.

Core Features & Use Cases

• Multi-GPU Training: Enables training of LLMs on multi-GPU setups, optimizing for performance and scalability.
• Reward Function Design: Assists in designing custom reward functions for training reinforcement learning models.
• vLLM Generation: Supports fast completion generation for large models using tensor parallelism and PagedAttention.
• Use Case: For users looking to train a GRPO/PPO model on a multi-GPU setup for building reasoning models at scale.

Quick Start

Execute the distributed-grpo skill to train a reinforcement learning model with a custom reward function and DeepSpeed on multiple GPUs.