Pekiştirmeli Öğrenme

Setup

1. Agent and EnvironmentThe RL loop: states, actions, rewards, and the interface between mind and world.1–2 sa
2. Return and DiscountingCumulative reward with a horizon: what gamma says about patience.1–2 sa
3. Multi-Armed BanditsThe simplest exploration problem: epsilon-greedy, UCB, and Thompson sampling measured by regret.1–2 sa
4. Markov Decision ProcessesStates, actions, transitions, rewards, and discount: the formal problem RL solves.1–2 sa

Policies And Values

1. PoliciesState to action: stochastic and deterministic policies, and what learning changes.1–2 sa
2. Value FunctionsV and Q: predicting future return from states and state-action pairs.1–2 sa
3. Bellman IntuitionValue here equals reward plus value there: recursion and bootstrapping.1–2 sa
4. Dynamic Programming: Policy and Value IterationSolving a known MDP exactly by repeated Bellman backups.1–2 sa

Learning

1. Exploration vs ExploitationThe dilemma: earning now vs learning for later, with epsilon-greedy as the first answer.1–2 sa
2. Monte Carlo Prediction and ControlLearning values by averaging complete episode returns, without a model of the environment.1–2 sa
3. Temporal-Difference LearningBootstrapping from the next estimate: TD(0) and SARSA learn online, step by step.1–2 sa
4. Q-LearningOff-policy TD control: learning optimal values while behaving otherwise.1–2 sa
5. Policy OptimizationGradients through expected return: the REINFORCE idea and what it climbs.1–2 sa
6. Imitation Learning and Behavioral CloningLearning a policy from expert demonstrations as supervised learning, and why errors compound off-distribution.1–2 sa
7. Offline Reinforcement LearningLearning policies from fixed logged data without further interaction, and the out-of-distribution action problem.1–2 sa

Deep Rl

1. Deep Q-NetworksQ-learning with a neural function approximator, stabilised by experience replay and target networks.1–2 sa
2. Policy Gradients: REINFORCEGradient ascent on expected return using the log-derivative trick; baselines reduce variance.1–2 sa
3. Actor-Critic MethodsA learned value function (the critic) supplies the baseline and advantage for the policy (the actor).1–2 sa
4. Trust Regions and PPOLimiting how far one update moves the policy: the clipped surrogate objective behind most applied deep RL and RLHF.1–2 sa

Planning

1. Model-Based Reinforcement LearningLearning a dynamics model and planning with it, and the compounding error that limits long rollouts.1–2 sa
2. Partially Observable MDPsActing when the state is hidden: belief states and why memory or filtering becomes part of the policy.1–2 sa