Exercise 4.3.1
Temporal-difference learning: why TD estimates are biased (MCQ)
Reinforcement Learning › Exercises › Lecture 4 · Monte Carlo Methods and TD Learning › 4.3 MC vs TD: bias and variance, batch updates
Temporal Difference (TD) learning is a fundamental method in reinforcement learning that combines ideas from both Monte Carlo and dynamic programming methods. TD learning is known to introduce some bias during the estimation process. What is the primary reason for this bias in TD learning methods?