Exercise R.4.3

Expected value: r(s,a) from the four-argument p

Reinforcement Learning › Exercises › Background review › R.4 Probability mass functions
EasyWritten

Using the definition of expected value and the conversion from the 3-argument to the 4-argument transition function, show that \[\begin{align*} r(s,a) = \sum_r r \sum_{s'} p(r, s' | s,a), \end{align*}\] where \[\begin{align*} r(s,a) \doteq \mathbb{E}(R_t | S_{t-1}=s, A_{t-1}=a). \end{align*}\]