Exercise 1.3.2
Weighted-average method: step size α from the target and estimate plot
Reinforcement Learning › Exercises › Lecture 1 · Exploration Methods › 1.3 Weighted-average method (nonstationary problems)
The weight-average method updates the action-value estimate using the formula: \[\begin{align*} q_{n+1} = q_n + \alpha [R_n - q_n] \end{align*}\] where \(q_n\) is the action-value estimate, \(\alpha\) is the step-size, and \(R_n\) is the reward received, all at time step \(n\). Given the estimated update, what do you think was the value of the step size parameter used to update the estimate on each time step?
