Exercise 5.3.6

Q-learning vs SARSA: equivalence under greedy action selection

Reinforcement Learning › Exercises › Lecture 5 · Model-Free Control › 5.3 Q-learning (off-policy learning)
HardWritten

Suppose action selection is greedy. Is \(Q\)-learning then exactly the same algorithm as Sarsa? Will they make exactly the same action selections and weight updates?

Source: Sutton & Barto (2018), Reinforcement Learning: An Introduction, 2nd ed., Exercise 6.12. Licensed CC BY-NC-ND 2.0.