Exercise 5.3.4
SARSA/Q-learning: numeric update of Q(s1,1) from a Q-table
Reinforcement Learning › Exercises › Lecture 5 · Model-Free Control › 5.3 Q-learning (off-policy learning)
Given the following table of Q-values for three states (\(s_1, s_2, s_3\)) and two actions (1, 2), calculate the updated Q-value \(Q(s, a)\) for the state-action pair \(Q(s_1, 1)\) after a transition to state \(s_2\) with a reward of 5. Assume \(\alpha = 0.1\) and \(\gamma = 0.9\). Consider that the behavior policy is the greedy policy.
| State | Action 1 | Action 2 |
|---|---|---|
| \(s_1\) | 4.0 | 4.5 |
| \(s_2\) | 8.0 | 1.0 |
| \(s_3\) | 3.5 | 2.0 |