value-based-methods

Double DQN and Dueling DQN extend a base Deep Q-Network by splitting action evaluation and selection or separating value and advantage streams. This skill guides implementing these DQN variants for discrete action spaces, covering architectural modifications and stability techniques.

Stability techniques for training deep Q-networks on Atari-like environments include experience replay buffers, target networks, frame stacking, and reward normalization. This skill outlines the specific requirements and architectural variants for these discrete RL stability mechanisms.

Rainbow DQN integrates components from Double DQN, Dueling DQN, and other value-based reinforcement learning improvements into a single discrete action space agent. This skill covers the Rainbow architecture, its integrated components, and architectural variants.

Value-based methods are optimal for discrete control tasks when evaluating action quality directly through Q-learning rather than parameterizing a policy. This skill provides guidelines for designing Q-learning-based agents specifically for discrete action spaces and benchmarks.

This approach is limited to discrete action spaces, focusing on Q-learning foundations and DQN variants like Double DQN and Rainbow. It does not support continuous control, which requires different value-based or policy gradient methods outside this skill's scope.

Comparing DQN variants on discrete control benchmarks requires implementing a minimal base DQN first, then progressively integrating Double DQN, Dueling DQN, and Rainbow components to experiment with stability and sample efficiency improvements. This skill outlines that evaluation strategy.