Exercise 7.1.5

Policy-based vs value-based RL: continuous action spaces

Reinforcement Learning › Exercises › Lecture 7 · Policy Gradients and Actor-Critic › 7.1 Policy-based RL: parameterized policies
EasyWritten

Policy gradient methods are often the preferred choice for certain types of reinforcement learning problems where value-based methods like Q-learning are difficult to apply.

Consider the challenge of solving a control problem in a continuous action space (e.g., setting the angle of a robotic arm).

Contrast the fundamental approach of a policy gradient method with that of a value-based method (like Q-learning) for selecting an action. Based on this contrast, explain why one of these methods is naturally suited for continuous action spaces, while the other faces a significant practical and computational challenge.