Exercise 1.4.1
Greedy: arm selected in the long run, from the estimates (figure)
In the k-armed bandit problem depicted in the figure below, the black bars represent the true value of each arm’s reward distribution, labeled with \(q_*(i)\), \(i\) corresponds to the arm number (action). The current action-value estimates are denoted by the dashed bars. You are employing a greedy policy and updating your action-value estimates with the data obtained from sampling and a small step size. Considering the current estimates, which arm is the greedy policy most likely to continue selecting in the long run?
