Reinforcement Learning Tutorials

Seven hands-on tutorials, from multi-armed bandits to policy gradients. The code runs in your browser.

These tutorials accompany an introductory course on reinforcement learning. Each one alternates short explanations, coding tasks with hints, and experiments followed by questions about what the results show. You implement the methods yourself, then test where they work and where they fail.

Before you start

  • Python: functions and classes
  • NumPy arrays
  • Probability: random variables, expected value

How the pages work

Tutorials 1–5 run Python in your browser: no installation and no account. Run the cells in order. Your edits are kept in your browser, and Download .ipynb saves your work as a Jupyter notebook. Tutorials 6–7 use PyTorch, which does not run in the browser: Tutorial 7 runs on a free mybinder.org server started from the page, and Tutorial 6 opens in Google Colab.

Reference text

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed., MIT Press, 2018. Each tutorial lists the chapters it builds on.

The tutorials

  1. 1

    The Bandit Lab

    Rewards as random variables; sample-average and constant step-size updates in stationary and non-stationary bandits; greedy and ε-greedy action selection.

    Sutton & Barto Ch. 2Setting k-armed bandit
    Runs in browser
  2. 2

    The RL Detective's Guide

    Returns of a policy in an MDP: estimating them from simulation, and tracing where their variance comes from.

    Sutton & Barto Ch. 3Environment FrozenLake
    Runs in browser
  3. 3

    The RL Detective's Guide, Part 2

    Value functions from the Bellman equation, policy improvement, and checking the theory against simulation.

    Sutton & Barto Ch. 3–4Environment FrozenLake
    Runs in browser
  4. 4

    The Detective's Race

    Monte Carlo and temporal-difference prediction compared on the same task: the bias–variance trade-off.

    Sutton & Barto Ch. 5–6Environment Random Walk
    Runs in browser
  5. 5

    The RL Detective's Field Manual

    From prediction to control: reading Q-tables, implementing Q-learning, behaviour versus target policy, and how the discount factor shapes what the agent learns.

    Sutton & Barto Ch. 6Environments CliffWalking, FrozenLake, Taxi
    Runs in browser
  6. 6

    The Modern RL Practitioner's Toolkit

    Deep Q-Networks with Stable-Baselines3, what the hyperparameters do, and automated tuning with Optuna.

    Sutton & Barto Ch. 9–10, §16.5Environment CartPole
    Opens in Colab
  7. 7

    Policy Gradients and PyTorch Foundations

    Learning a policy directly with a PyTorch policy network and probability distributions, for continuous actions.

    Sutton & Barto Ch. 13Environment MountainCarContinuous
    Runs on Binder

© 2025–2026 Rafael Fernandes Cunha · Text licensed under CC BY 4.0 · Code licensed under the MIT License
Originally developed for Reinforcement Learning (WBAI061-05), University of Groningen.