Exercise 6.4.5

Semi-gradient SARSA: on-policy vs off-policy closeness to gradient descent

Reinforcement Learning › Exercises › Lecture 6 · Function Approximation › 6.4 Convergence, divergence and experience replay
MediumWritten

Explain why on-policy control algorithms, such as Semi-gradient SARSA, tend to follow the true gradient of the value function error more closely than off-policy methods.