Exercise R.3.2

Gradients: simplify five expressions with expectations

Reinforcement Learning › Exercises › Background review › R.3 Expected value
EasyWritten

Compute the following gradients. Try to push the gradient as much as possible, which should be the final answer. Leave expected values and fractions as such. Note that some examples might already be maximally simplified, and if so, it is enough to just recognize it.

  • \(\nabla 100\)

  • \(\nabla f(s)\)

  • \(\nabla_x f(s)\)

  • \(\nabla_a \mathbb{E}(\pi(a) + b(s))\)

  • \(\nabla_a \log \mathbb{E}(\pi(a))\)