Exercise 4.5.1
λ-return: half-life τ_λ in terms of λ
Consider the \(\lambda\)-return of the TD(\(\lambda\)) algorithm: \[G^{\lambda}_t = (1- \lambda) \sum^{\infty}_{n=1} \lambda^{n-1} G_{t:t+n}.\] The parameter \(\lambda\) characterizes how fast the exponential weighting in Figure 1 falls off, and thus how far into the future the \(\lambda\)-return algorithm looks in determining its update. But a rate factor such as \(\lambda\) is sometimes an awkward way of characterizing the speed of the decay. For some purposes it is better to specify a time constant, or half-life. What is the equation relating \(\lambda\) and the half-life, \(\tau_{\lambda}\), the time by which the weighting sequence will have fallen to half of its initial value?
Source: Sutton & Barto (2018), Reinforcement Learning: An Introduction, 2nd ed., Exercise 12.2. Figure 12.2 is redrawn from the book. Licensed CC BY-NC-ND 2.0.