Exercise 1.5.4

ε-greedy: long-run best method in the 10-armed testbed figure

MediumWritten

In the comparison shown in Figure 1, which method will perform best in the long run in terms of cumulative reward and probability of selecting the best action? How much better will it be? Express your answer quantitatively.

fig2.2.png
Figure 1: Average performance of \(\epsilon\)-greedy action-value methods on the 10-armed testbed. These data are averages over 2000 runs with different bandit problems (see chapter 2.3 of Sutton, R., & Barto, A. Reinforcement Learning: An Introduction). All methods used sample averages as their action-value estimates.

Source: Sutton & Barto (2018), Reinforcement Learning: An Introduction, 2nd ed., Exercise 2.3. Figure 2.2 is redrawn from a new run of the book's experiment. Licensed CC BY-NC-ND 2.0.