Seminar 2

\[\newcommand{\st}{\, : \:} \newcommand{\ind}[1]{\mathbf{1}_{#1}} \newcommand{\dd}{\mathrm{d}}\]

Exercise 1

  1. Is the equality \(\mathbb{P}(B|A) + \mathbb{P}(C|A) = \mathbb{P}(B\cup C|A)\) true?
  2. Provide examples showing that the following equalities are generally not true:
    1. \(\mathbb{P}(A|B \cup C) = \mathbb{P}(A|B) + \mathbb{P}(A|C)\).
    2. \(\mathbb{P}(B |A) + \mathbb{P}(B |\overline A) = 1\).

  1. No, it is not true. The correct formula, analogous to the formula for unconditional probabilities, is: \[ \mathbb{P}(B \cup C | A) = \mathbb{P}(B|A) + \mathbb{P}(C|A) - \mathbb{P}(B \cap C | A) \] The equality will hold only if \(\mathbb{P}(B \cap C | A) = 0\), i.e., events \(B\) and \(C\) are disjoint under the condition \(A\).

  2. Let \(\Omega\) be a set containing at least two points.

    1. Take \(A \supset B\cup C\), e.g. \(A=\Omega\), with \(\mathbb{P}(B),\mathbb{P}(C)>0\).
    2. Take \(B=\Omega\) and \(\mathbb P(A) \in (0,1)\).

Exercise 2

Let \((\Omega, \mathcal{F}, \mathbb{P})\) be a probability space and assume the events \(H_1, H_2\in\mathcal{F}\) have positive probabilities. Let us denote \(\mathbb{P}_{H_i}:=\mathbb{P}(\cdot|H_i)\), \(i=1,2\). Prove that \[ \mathbb{P}_{H_1}(\cdot|H_2) = \mathbb{P}(\cdot|H_1\cap H_2) = \mathbb{P}_{H_2}(\cdot| H_1). \] That is, for any \(A\in\mathcal{F}\), \[ \mathbb{P}_{H_1}(A|H_2) = \mathbb{P}(A|H_1\cap H_2) = \mathbb{P}_{H_2}(A| H_1). \]

Let’s prove the first equality. By the definition of conditional probability: \[ \mathbb{P}_{H_1}(A|H_2) = \frac{\mathbb{P}_{H_1}(A \cap H_2)}{\mathbb{P}_{H_1}(H_2)} \] Now let’s expand the probabilities in the numerator and denominator using the definition of the measure \(\mathbb{P}_{H_1}\): \[ \mathbb{P}_{H_1}(A \cap H_2) = \mathbb{P}(A \cap H_2 | H_1) = \frac{\mathbb{P}(A \cap H_2 \cap H_1)}{\mathbb{P}(H_1)} \] \[ \mathbb{P}_{H_1}(H_2) = \mathbb{P}(H_2 | H_1) = \frac{\mathbb{P}(H_2 \cap H_1)}{\mathbb{P}(H_1)} \] Substituting this back into the original expression, we get: \[ \mathbb{P}_{H_1}(A|H_2) = \frac{\frac{\mathbb{P}(A \cap H_1 \cap H_2)}{\mathbb{P}(H_1)}}{\frac{\mathbb{P}(H_1 \cap H_2)}{\mathbb{P}(H_1)}} = \frac{\mathbb{P}(A \cap H_1 \cap H_2)}{\mathbb{P}(H_1 \cap H_2)} \] This is exactly the definition of \(\mathbb{P}(A|H_1 \cap H_2)\). The second equality \(\mathbb{P}(\cdot|H_1\cap H_2) = \mathbb{P}_{H_2}(\cdot| H_1)\) is proven in exactly the same way, just by swapping the roles of \(H_1\) and \(H_2\).

Exercise 3

A deck of \(52\) cards is dealt to \(4\) players. One of the players announces that he has an ace.

  1. What is the probability that he has at least one more ace?
  2. What is the probability that he has at least one more ace if he announced that he has the ace of spades?

First, solve this problem directly, without using the concept of conditional probability (by redefining the set of elementary outcomes), and then by the definition of conditional probability.

This might be a bit surprising, because why should the suit of the ace change the probability? Of course, it must, because having the ace of spades is not as easy as having any ace, and this correlates more strongly with having any other ace. Let’s calculate this quantitatively.

  1. Let \(A_{\ge 1}\) be the event that the player has at least one ace, and \(A_{\ge 2}\) be the event that he has at least two aces. We are looking for \(\mathbb{P}(A_{\ge 2} | A_{\ge 1})\). Let’s count:
    • The number of possible 13-card hands: \(|\Omega|=\binom{52}{13}\).
    • Number of hands with no aces: \(\binom{48}{13}\).
    • Number of hands with at least one ace: \(\binom{52}{13} - \binom{48}{13}\).
    • Number of hands with exactly one ace: \(\binom{4}{1}\binom{48}{12}\).
    • Number of hands with at least two aces: \((\binom{52}{13} - \binom{48}{13}) - \binom{4}{1}\binom{48}{12}\). The required probability is thus: \[ \mathbb{P}(A_{\ge 2} | A_{\ge 1}) = \frac{\mathbb{P}(A_{\ge 2})}{\mathbb{P}(A_{\ge 1})} = \frac{\binom{52}{13} - \binom{48}{13} - \binom{4}{1}\binom{48}{12}}{\binom{52}{13} - \binom{48}{13}} \approx 0.37 \]
  2. Let \(A_S\) be the event that the player has the ace of spades. We are looking for \(\mathbb{P}(A_{\ge 2} | A_S)\). The new sample space is all hands containing the ace of spades. Their number is \(\binom{51}{12}\). Among these hands, those that do not contain other aces (i.e., contain exactly one ace - the ace of spades) consist of the ace of spades and 12 cards from the 48 non-aces. Their number is \(\binom{48}{12}\). The number of hands containing the ace of spades and at least one more ace is \(\binom{51}{12} - \binom{48}{12}\). Therefore \[ \mathbb{P}(A_{\ge 2} | A_{\ge 1}) = \frac{\binom{51}{12} - \binom{48}{12}}{\binom{51}{12}} = 1 - \frac{\binom{48}{12}}{\binom{51}{12}} = 1 - \frac{48! \cdot 39!}{51! \cdot 36!} = 1 - \frac{39 \cdot 38 \cdot 37}{51 \cdot 50 \cdot 49} \approx 0.56 \] This value is noticeably larger than in part a..

Exercise 4

From the set \({1, 2, . . . , n}\), three distinct numbers are chosen in sequence without replacement. Find the conditional probability that the third number lies between the first and the second, given that the first number is less than the second.

Let the chosen numbers be \(x, y, z\). They are distinct. Consider any three distinct numbers \({a, b, c}\) from the set. There are \(3! = 6\) equally likely ways to order them when choosing. Let \(A\) be the event that the first number is less than the second (\(x < y\)). Let \(B\) be the event that the third number lies between the first and the second (\(x < z < y\) or \(y < z < x\)). We are looking for \(\mathbb{P}(B|A)\).

By symmetry, for any pair of distinct numbers \((x,y)\), it is equally likely that \(x<y\) or \(y<x\). Therefore, \(\mathbb{P}(A) = 1/2\).

The event \(A \cap B\) means that \(x < z < y\). Out of the 6 possible permutations of the three numbers \({a, b, c}\), only one satisfies this condition. Consequently, \(\mathbb{P}(A \cap B) = 1/6\).

Then the conditional probability is: \[ \mathbb{P}(B|A) = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(A)} = \frac{1/6}{1/2} = 1/3. \]

Exercise 5

  1. There was one white ball and one black ball in a bag. One ball was drawn from it and placed in an empty box. Another white ball was also placed in the box. Finally, one ball was drawn from the box, and it turned out to be white. What is the probability that the remaining ball in the box is also white?
  2. Solve the previous problem assuming that initially there were \(10\) black and \(7\) white balls in the bag.

  1. Let \(W_1\) be the event that a white ball was drawn from the bag (first extraction), and \(W_2\) the event that a white ball was next drawn from the box (second extraction). We interpret the problem as follows:

    • \(\mathbb{P}(W_1)=\mathbb{P}(W_1^c)=1/2\), since there is a white ball and a black ball in the bag.
    • \(\mathbb{P}(W_2 \mid W_1)=1\), since if we sampled a white ball in the bag, there will be two white balls in the box before sampling.
    • \(\mathbb{P}(W_2 \mid W_1^c)=1/2\), since if we sampled a black ball in the bag, there will be a white and a black ball in the box before sampling.
    • We want to know \(\mathbb{P}(W_1 \mid W_2)=1/2\). Indeed, the remaining ball in the box will be white iff the first sample gave a white ball.

With this set, the problem is easily solved using Bayes formula: \[ \mathbb{P}(W_1 | W_2) = \frac{\mathbb{P}(W_2 \mid W_1) \mathbb{P}(W_1)}{\mathbb{P}(W_2)} = \frac{\mathbb{P}(W_2 \mid W_1) \mathbb{P}(W_1)}{ \mathbb{P}(W_2| W_1) \mathbb{P}(W_1)+ \mathbb{P}(W_2| W_1^c) \mathbb{P}(W_1^c)} = \frac{1 \cdot \tfrac{1}{2}}{1 \cdot \tfrac{1}{2} + \tfrac{1}{2} \cdot \frac{1}{2}}= \frac{2}{3} \]

  1. Now there are \(10\) black and \(7\) white balls in the bag. The only change is that \(\mathbb{P}(W_1)=7/17\) and thus \(\mathbb{P}(W_1^c)=10/17\), to get \(\tfrac{7}{12}\) as answer.

Exercise 6

(Pólya’s Urn Scheme) An urn contains \(a\) white and \(b\) black balls. We perform \(n\) random draws, and immediately after each draw, the ball is returned to the urn along with \(m\) other balls of the same color. (\(m\ge -1\) and \(n\le a+ b\) if \(m=-1\)).

  1. What is the probability that out of \(n=n_1+n_2\) chosen balls, \(n_1\) will be white and \(n_2\) will be black?
  2. Prove that the probability of drawing a white ball on the \(i\)-th step is \(a/(a+b)\).

  1. The probability of any specific sequence of draws containing \(n_1\) white and \(n_2\) black balls is: \[ \frac{\left(a(a+m)\dots(a+(n_1-1)m)\right) \cdot \left(b(b+m)\dots(b+(n_2-1)m)\right)}{(a+b)(a+b+m)\dots(a+b+(n-1)m)} \] The number of such sequences is equal to the multinomial coefficient \(\binom{n}{n_1, n_2} = \binom{n}{n_1}\). The final probability: \[ \mathbb{P}(n_1 \text{ white}, n_2 \text{ black}) = \binom{n}{n_1} \frac{\prod_{i=0}^{n_1-1}(a+im) \prod_{j=0}^{n_2-1}(b+jm)}{\prod_{k=0}^{n-1}(a+b+km)} \]

  2. Since the probability of sequences of white and black samples is invariant by permutations, the probability of sampling white is the same at each sample, namely \(a/(a+b)\) as in the first sample.

Exercise 7

(Monty Hall Paradox). Imagine you are a contestant in a game where you have to choose one of three doors. Behind one of the doors is a car; behind the other two are goats. You choose one of the doors, for example, number 1. After this, the host, who knows where the car is and where the goats are, opens one of the remaining doors, for example, number 3, which has a goat behind it. He then asks you if you would like to change your choice to door number 2. Will your chances of winning the car increase if you accept the host’s offer and change your choice? Clarifications: the car is placed behind any of the three doors with equal probability; the host knows where the car is; regardless of which door you choose, the host must in any case open a door with a goat (but not the one you chose) and offer to change the choice; if the host has a choice of which of the two doors to open (that is, you pointed to the correct door, and behind both remaining doors are goats), he chooses any of them with equal probability.

Yes, the chances will increase. Indeed, if you do not switch, you win iff your initial choice was correct, namely with probability \(1/3\). If you switch, you win iff the initial choice was wrong, namely with probability \(2/3\).

Exercise 8

Covid is back. But scientists are not asleep either: a new test has been developed with a sensitivity of 99% (i.e., it correctly diagnoses a sick person in 99% of cases) and a specificity of 99% (only 1% of healthy people are declared sick). It is known that in a certain happy village, 1 out of every 1000 inhabitants suffers from Covid. What is the probability that a resident of this village, who tested positive, is actually sick?

Let \(D\) be the event that a person is sick, and \(T\) be the event that the test is positive. We are given:

  • \(\mathbb{P}(D) = 1/1000 = 0.001\) (prior probability of being sick).
  • \(\mathbb{P}(T|D) = 0.99\) (sensitivity).
  • \(\mathbb{P}(\overline{T}|\overline{D}) = 0.99\) (specificity), hence \(\mathbb{P}(T|\overline{D}) = 1 - 0.99 = 0.01\) (probability of a false positive).

We want to find \(\mathbb{P}(D|T)\), i.e., the probability that a person is actually sick, given a positive test. Using Bayes’ formula: \[ \mathbb{P}(D|T) = \frac{\mathbb{P}(T|D)\mathbb{P}(D)}{\mathbb{P}(T|D)\mathbb{P}(D) + \mathbb{P}(T|\overline{D})\mathbb{P}(\overline{D})}=\frac{0.00099}{0.01098} \approx 0.09016 \] Thus, the probability that a person with a positive test is actually sick is only about 9%.

Exercise 9

Agent D. is monitoring the movements of a company director. It is known that the director is in the office with a probability of 60% and at his dacha with a probability of 40%. Agent D. has two informants; the first is wrong with a probability of 20%, and the second with a probability of 10%. The first informant claims the director is in the office, while the second informant claims he is at the dacha. Where is the director?

The key point in this problem is not the mathematics, but how we interpret the frequency of the informants’ errors. We must understand that when the director is in the office (at the dacha), the informants will report that he is at the dacha (in the office) independently with probabilities \(0.2\) and \(0.1\).

Formally, if we denote by \(O\) the event ‘the director is in the office’, and by \(I_1\) the event ‘the first informant reported that the director is in the office’ and \(I_2\) the event ‘the second informant reported that the director is at the dacha’, we assume that \(\mathbb{P}(I_1\cap I_2|O)=\mathbb{P}(I_1|O)\mathbb{P}(I_2|O)\) etc. Then by Bayes’ formula: \[ \mathbb{P}(O|I_1\cap I_2) = \frac{\mathbb{P}(I_1\cap I_2|O) \mathbb{P}(O)}{\mathbb{P}(I_1\cap I_2|O)\mathbb{P}(O)+\mathbb{P}(I_1\cap I_2|\overline{O})\mathbb{P}(\overline{O})} \] Probabilities of the reports given the location:

  • \(\mathbb{P}(I_1|O) = 1 - 0.2 = 0.8\)
  • \(\mathbb{P}(I_2|O) = 0.1\) (error)
  • \(\mathbb{P}(I_1|\overline{O}) = 0.2\) (error)
  • \(\mathbb{P}(I_2|\overline{O}) = 1 - 0.1 = 0.9\)

\[ \mathbb{P}(O|I_1\cap I_2) = \frac{(0.8 \cdot 0.1) \cdot 0.6}{(0.8 \cdot 0.1) \cdot 0.6 + (0.2 \cdot 0.9) \cdot 0.4} = \frac{0.048}{0.048 + 0.072} = \frac{0.048}{0.120} = 0.4 \] The posterior probability that the director is at the dacha is \(1 - 0.4 = 0.6\). Therefore, it is more likely that the director is at the dacha.

Additional Exercises

Exercise 10

Let \(n \ge 2\) and select a random number \(\xi\) from \(\{1, 2,\ldots,n\}\). Let \(A\) be the event on which \(\xi\) is even, and \(B\) the event on which \(\xi\) is divisible by \(7\). Find all \(n\) such that events \(A\) and \(B\) are independent.

We need to solve \[ \lfloor n/14 \rfloor / n = \mathbb{P}(A\cap B) =^? \mathbb{P}(A) \mathbb{P}(B) = \lfloor n/2 \rfloor \lfloor n/7 \rfloor / n^2 \] Write \(n=14*k+r\), with \(0 \le r \le 13\) to get \[ k r = 2k \lfloor r/2 \rfloor +7k \lfloor r/7 \rfloor + \lfloor r/7 \rfloor \lfloor r/2 \rfloor \] which is satisfied if \(k=0\) and \(n=r \le 6\), or for \(k\ge 1\) and \(r=0,2,4,6\). So the answer is \(n=0,2,4,6 \pmod 14\) or \(n=3,5\).

Exercise 11

Give an example of three events \(A\), \(B\), \(C\) that are pairwise independent but not independent. In general, give an example of \(n\) events \(A_1,\ldots A_n\) that are dependent but each subset of \(n-1\) sets is independent.

Let \(\mathbb{Z}_{n}:=\{0,1,\ldots,n-1\}\) represent the \(n\) knights seated in a rounded table. Each knight drinks randomly either wine or beer with probability \(1/2\). Let \(A_k\) be the event where the knights \(k\) and \(k+1\) have the same drink. Any subcollection of these events is independent, till we feel that the table is rounded, namely if take all of the \(A_k\), since in this case \[ \mathbb{P}(A_1\cap \cdots \cap A_n)= 2*2^{-n} \neq \prod_{k=1}^n \mathbb{P}(A_k) =2^{-n} \]