Exercise 1.3.2

Weighted-average method: step size α from the target and estimate plot

Reinforcement Learning › Exercises › Lecture 1 · Exploration Methods › 1.3 Weighted-average method (nonstationary problems)
EasyMultiple choice

The weight-average method updates the action-value estimate using the formula: \[\begin{align*} q_{n+1} = q_n + \alpha [R_n - q_n] \end{align*}\] where \(q_n\) is the action-value estimate, \(\alpha\) is the step-size, and \(R_n\) is the reward received, all at time step \(n\). Given the estimated update, what do you think was the value of the step size parameter used to update the estimate on each time step?

target_estimate2.png