rl-foundations

An MDP formally frames sequential decision making in reinforcement learning using states, actions, rewards, and the Markov property. It provides the rigorous mathematical foundation needed to reason about optimal policies.

Bellman equations recursively define a state's value based on expected returns from successor states in reinforcement learning. They provide the formal derivations necessary for rigorous policy evaluation and optimization.

You derive the Bellman backup for V(s) by applying the expectation over next states, rewards, and actions using the MDP transition dynamics. This yields the recursive formulation used for rigorous policy evaluation.

This rigorous reinforcement learning theory requires a strong mathematical foundation to understand MDPs, value functions, and Bellman equations. It targets students and professionals needing formal definitions rather than rote implementation.

Learn reinforcement learning theory when you need to design novel algorithms, prepare for interviews, or reason about policy optimization before coding. Formal MDP and Bellman equation knowledge prevents rote implementation.

Policy evaluation calculates the value function for a fixed policy using Bellman equations, while policy improvement uses those values to find better actions. Alternating these processes derives optimal policies within MDPs.