Exercise R.1.2
Conditional probability: prove Bayes' theorem
An event is a set of outcomes, e.g. \(A=\{\)we draw a king from a deck of cards\(\}\) or \(B=\{X=x\}\), where \(X\) is a random variable. \(A, B\) is also an event that would mean that simultaneously we draw a king from a deck of cards and \(X\) takes the value \(x\).
Consider events \(A\) and \(B\) such that \(\mathbb{P}(B)>0\). Bayes’ theorem states that \[\begin{align*} \mathbb{P}(A \mid B)=\frac{\mathbb{P}(B \mid A)\mathbb{P}(A)}{\mathbb{P}(B)}. \end{align*}\] Prove it.
Hint: Write out the conditional probabilities \(\mathbb{P}(A \mid B)\) and \(\mathbb{P}(B \mid A)\) and get an expression for \(\mathbb{P}(A , B)\) from the latter.