What problem does it solve? Reinforcement learning training is unstable and error-prone: sparse rewards stall learning, unclipped policy updates cause collapse, and RLHF without KL penalties leads to reward hacking. This Skill grounds AI responses in curated reference patterns, known failure modes, and validation rules so RL code is built and reviewed correctly. ## Core Features & Use Cases - Implementation Patterns: Consult references/patterns.md for algorithm taxonomy (value-based, policy-based, actor-critic), PPO hyperparameter configs, and the three-step RLHF pipeline (SFT, reward model, PPO with KL penalty). - Failure Diagnosis: Use references/sharp_edges.md to explain critical issues like reward hacking, catastrophic policy collapse, sparse-reward failure, Q-value overestimation, and KL divergence explosion, each with symptoms and fixes. - Code Validation: Apply references/validations.md regex rules to flag PPO without clipping, missing entropy bonuses, DQN without target networks, and RLHF without KL penalties. - Use Case: When a user asks why their PPO training collapsed, the Skill identifies missing ratio clipping and provides the corrected torch.clamp implementation. ## Quick Start Ask the AI to review my PPO training loop for stability issues and explain why the policy collapsed after 10k steps.
