Reinforcement Learning Lectures
Eight lectures, from multi-armed bandits to model-based reinforcement learning, as interactive slides.
These are the lecture slides of an introductory course on reinforcement learning. They follow Sutton and Barto's textbook, from the exploration–exploitation dilemma in bandits to policy gradients and model-based methods. The tutorials put each lecture's methods into code. The course map lists every concept with its slides and the tutorial tasks that practise it.
Video recordings
Each lecture as taught in Reinforcement Learning (WBAI061-05) at the University of Groningen, 2025–2026, on the university's video portal. The slides here were lightly edited for the web, so the recordings differ in places.
Reference text
R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed., MIT Press, 2018, free online from the authors. Each lecture lists the chapters it covers.
The lectures
-
1
Exploration Methods
The multi-armed bandit problem: action-value estimates, sample-average and constant step-size updates, greedy and ε-greedy action selection, optimistic initial values, upper confidence bounds and softmax action selection, and the gradient bandit algorithm with action preferences.
-
2
The RL Problem and Finite MDPs
Agent and environment, Markov decision processes, returns and discounting, value functions, and the Bellman expectation and optimality equations.
-
3
Dynamic Programming
Policy evaluation, policy improvement, policy iteration and value iteration; why value iteration converges (Bellman operators and the Banach fixed-point theorem); asynchronous dynamic programming.
-
4
Monte Carlo Methods and Temporal-Difference Learning
Model-free prediction from sampled experience: Monte Carlo and TD(0), bootstrapping, the bias–variance trade-off, multi-step returns and eligibility traces.
-
5
Model-Free Control
From prediction to control: GLIE Monte Carlo control, SARSA, Q-learning and Expected SARSA, maximisation bias and double Q-learning, the ideas behind DQN, and importance sampling for off-policy learning.
-
6
Function Approximation
Value-function approximation by stochastic gradient descent, linear approximation for prediction and control, and when learning converges or diverges.
-
7
Policy Gradients and Actor-Critic
Parameterised policies, the policy-gradient theorem, REINFORCE with causality and baselines, actor-critic methods and A2C.
-
8
Model-Based Reinforcement Learning
Learning a model from experience, Dyna-Q, planning with an inaccurate model, and Monte Carlo tree search.
© 2025–2026 Rafael Fernandes Cunha · Slides licensed under CC BY 4.0, except figures credited to other sources, which remain their authors'. The video recordings are not covered by this licence.
Originally developed for Reinforcement Learning (WBAI061-05), University of Groningen.