Exercise R.1.2

Conditional probability: prove Bayes' theorem

Reinforcement Learning › Exercises › Background review › R.1 Random variables
EasyWritten

An event is a set of outcomes, e.g. \(A=\{\)we draw a king from a deck of cards\(\}\) or \(B=\{X=x\}\), where \(X\) is a random variable. \(A, B\) is also an event that would mean that simultaneously we draw a king from a deck of cards and \(X\) takes the value \(x\).

Consider events \(A\) and \(B\) such that \(\mathbb{P}(B)>0\). Bayes’ theorem states that \[\begin{align*} \mathbb{P}(A \mid B)=\frac{\mathbb{P}(B \mid A)\mathbb{P}(A)}{\mathbb{P}(B)}. \end{align*}\] Prove it.

Hint: Write out the conditional probabilities \(\mathbb{P}(A \mid B)\) and \(\mathbb{P}(B \mid A)\) and get an expression for \(\mathbb{P}(A , B)\) from the latter.