Exercise 5.5.3

Expected SARSA: Double Expected SARSA update equations

Reinforcement Learning › Exercises › Lecture 5 · Model-Free Control › 5.5 Maximization bias and double Q-learning
MediumWritten

What are the update equations for Double Expected Sarsa with an \(\epsilon\)-greedy target policy? Hint: use the Double \(Q\)-learning algorithm as a reference.

Source: Sutton & Barto (2018), Reinforcement Learning: An Introduction, 2nd ed., Exercise 6.13. Licensed CC BY-NC-ND 2.0.