Exercise 2.3.4
Returns and discounting: pole balancing, episodic vs continuing return
Reinforcement Learning › Exercises › Lecture 2 · The RL Problem and Finite MDPs › 2.3 Returns and discounting
The pole-balancing problem (also called inverted pendulum) is a classic reinforcement learning task in which an agent must apply forces to a cart so that a pole hinged to it stays upright. The task ends (failure) if the pole falls too far or the cart moves out of bounds. Suppose you treated pole-balancing as an episodic task but also used discounting, with all rewards zero except for \(-1\) upon failure. What then would the return be at each time? How does this return differ from that in the discounted, continuing formulation of this task?
Source: Sutton & Barto (2018), Reinforcement Learning: An Introduction, 2nd ed., Exercise 3.6. Licensed CC BY-NC-ND 2.0.