Exercise 1.9.2

Exploration methods: deterministic versus stochastic policies, hyperparameters, baseline

Reinforcement Learning › Exercises › Lecture 1 · Exploration Methods › 1.9 Policy gradient methods: action preferences
EasyWritten

Answer the following theoretical questions about exploration methods.

  • 1. What is the main difference between a deterministic policy and a stochastic policy?

  • 2. How is exploration driven in a stochastic policy?

  • 3. Classify the policies greedy, \(\epsilon\)-greedy, UCB, Softmax, and Action Preference as stochastic or deterministic.

  • 4. Explain the function of the hyperparameters below. What effect does a high and low hyperparameter value have for each of these policies?

    • a) \(\epsilon\) in \(\epsilon\)-greedy.

    • b) \(c\) in UCB.

    • c) \(\tau\) in Softmax.

    • d) \(\alpha\) in action preference.

  • 5. Why does the use of a baseline for the Action Preference policy improve performance?