Exercise 1.3.4

Weighted-average method: Q_{n+1} in terms of Q_1 and past rewards

Reinforcement Learning › Exercises › Lecture 1 · Exploration Methods › 1.3 Weighted-average method (nonstationary problems)
MediumWritten

Starting from the incremental update rule of \(Q_t(a)\) with \(\alpha\) constant, express \(Q_{n+1}(a)\) in terms of the initial estimate \(Q_1(a)\) and past rewards \(R_i\) where \(R_i\) is the reward at step \(i\), with \(i \in [1, n]\).