← Reinforcement Learning

Reinforcement Learning Lectures

Eight lectures, from multi-armed bandits to model-based reinforcement learning, as interactive slides.

These are the lecture slides of an introductory course on reinforcement learning. They follow Sutton and Barto's textbook, from the exploration–exploitation dilemma in bandits to policy gradients and model-based methods. The tutorials put each lecture's methods into code. The course map lists every concept with its slides and the tutorial tasks that practise it.

Video recordings

Each lecture as taught in Reinforcement Learning (WBAI061-05) at the University of Groningen, 2025–2026, on the university's video portal. The slides here were lightly edited for the web, so the recordings differ in places.

Reference text

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed., MIT Press, 2018, free online from the authors. Each lecture lists the chapters it covers.

The lectures

  1. 1

    Exploration Methods

    The multi-armed bandit problem: action-value estimates, sample-average and constant step-size updates, greedy and ε-greedy action selection, optimistic initial values, upper confidence bounds and softmax action selection, and the gradient bandit algorithm with action preferences.

    Sutton & Barto Ch. 2Slides 92 + 10 optional reviewVideoTutorial 1
  2. 2

    The RL Problem and Finite MDPs

    Agent and environment, Markov decision processes, returns and discounting, value functions, and the Bellman expectation and optimality equations.

    Sutton & Barto Ch. 3Slides 77VideoTutorial 2
  3. 3

    Dynamic Programming

    Policy evaluation, policy improvement, policy iteration and value iteration; why value iteration converges (Bellman operators and the Banach fixed-point theorem); asynchronous dynamic programming.

    Sutton & Barto Ch. 4Slides 78VideoTutorial 3
  4. 4

    Monte Carlo Methods and Temporal-Difference Learning

    Model-free prediction from sampled experience: Monte Carlo and TD(0), bootstrapping, the bias–variance trade-off, multi-step returns and eligibility traces.

    Sutton & Barto Ch. 5–7, 12Slides 68VideoTutorial 4
  5. 5

    Model-Free Control

    From prediction to control: GLIE Monte Carlo control, SARSA, Q-learning and Expected SARSA, maximisation bias and double Q-learning, the ideas behind DQN, and importance sampling for off-policy learning.

    Sutton & Barto Ch. 5–6, §16.5Slides 62VideoTutorial 5
  6. 6

    Function Approximation

    Value-function approximation by stochastic gradient descent, linear approximation for prediction and control, and when learning converges or diverges.

    Sutton & Barto Ch. 9–11Slides 65VideoTutorial 6
  7. 7

    Policy Gradients and Actor-Critic

    Parameterised policies, the policy-gradient theorem, REINFORCE with causality and baselines, actor-critic methods and A2C.

    Sutton & Barto Ch. 13Slides 58VideoTutorial 7
  8. 8

    Model-Based Reinforcement Learning

    Learning a model from experience, Dyna-Q, planning with an inaccurate model, and Monte Carlo tree search.

    Sutton & Barto Ch. 8Slides 45Video

© 2025–2026 Rafael Fernandes Cunha · Slides licensed under CC BY 4.0, except figures credited to other sources, which remain their authors'. The video recordings are not covered by this licence.
Originally developed for Reinforcement Learning (WBAI061-05), University of Groningen.