Download Basic Combinatorics - University of Toronto

Conditional Probability
• Idea – have performed a chance experiment but don’t know the outcome
(ω), but have some partial information (event A) about ω.
Question: given this partial information what’s the probability that the
outcome is in some event B?
• Example:
Toss a coin 3 times. We are interested in event B that there are 2 or more
heads. The sample space has 8 equally likely outcomes.
  HHH , HHT , HTH , THH , HTT , THT , TTH , TTT 
The probability of the event B is …
Suppose we know that the first coin came up H. Let A be the event the first
outcome is H. Then A  HHH , HHT , HTH , HTT  and A  B  HHH , HHT , HTH 
The conditional probability of B given A is 3  3 8  P A  B 
4
week 3
48
P  A
1
• Given a probability space (Ω, F, P) and events A, B  F with P(A) > 0
The conditional probability of B given the information that A has occurred is
P  B | A 
P A  B 
P  A
• Example:
We toss a die. What is the probability of observing the number 6 given that
the outcome is even?
• Does this give rise to a valid probability measure?
• Theorem
If A F and P(A) > 0 then (Ω, F, Q) is a probability space where Q : F  R
is defined by Q(B) = P(B | A).
Proof:
week 3
2
• The fact that conditional probability is a valid probability
measure allows the following:
 PB | A  1  PB | A ,
A, B  F, P(A) >0
 PB1  B2 | A  PB1 | A  PB2 | A  PB1  B2 | A
for any A, B1, B2  F, P(A) >0.
week 3
3
Multiplication rule
• For any two events A and B, P( A  B)  PB | AP A
• For any 3 events A, B and C, P A  B  C   P APB | APC | A  B
• In general,
n 1
 n 


P  Ai   P A1 P A2 | A1 P A3 | A1  A2     P An |  Ai 
i 1
 i 1 


• Example:
An urn initially contains 10 balls, 3 blue and 7 white. We draw a ball and
note its colure; then we replace it and add one more of the same colure.
We repeat this process 3 times. What is the probability that the first 2 balls
drawn are blue and the third one is white?
Solution:
week 3
4
Law of total probability
• Definition:
For a probability space (Ω, F, P), a partition of Ω is a
countable collection Bi  of events such that
Bi  F , Bi  B j   and  Bi  .
i
• Theorem:
If B1 , B2 ,... is a partition of Ω such that PBi   0  i then
P A   P A | Bi PBi 
• Proof:
for any A  F
.
i
week 3
5
Examples
1.
Calculation of PB2  for the Urn example.
2.
In a certain population 5% of the females and 8% of the males are lefthanded; 48% of the population are males. What proportion of the
population is left-handed?
Suppose 1 person from the population is chosen at random; what is the
probability that this person is left-handed?
week 3
6
Bayes’ Rule
• Let B1 , B2 ,... be a partition of Ω such that PBi   0 for all i then
PB j | A 
PA | B j PB j 
 P A | B PB 
i
for any A  F .
i
i
• Example:
A test for a disease correctly diagnoses a diseased person as having the
disease with probability 0.85. The test incorrectly diagnoses someone
without the disease as having the disease with probability 0.1 If 1% of
the people in a population have the disease, what is the probability that a
person from this population who tests positive for the disease actually
has it?
(a) 0.0085 (b) 0.0791 (c) 0.1075 (d) 0.1500 (e) 0.9000
week 3
7
Independence
• Example:
Roll a 6-sided die twice. Define the following events
A : 3 or less on first roll
B : Sum is odd.
• If occurrence of one event does not affect the probability that the other
occurs than A, B are independent.
week 3
8
• Definition
Events A and B are independent if
P A  B  P APB
• Note: Independence ≠ disjoint. Two disjoint events are independent if and
only if the probability of one of them is zero.
• Generalized to more than 2 events:
A collection of events A1 , A2 ,... An  is (mutually) independent if for any
subcollection Ai , Ai ,... Ai 

1
2
m

  
P Ai1  Ai2      Aim  P Ai1 P Ai2    P Aim 
• Note: pairwize independence does not guarantee mutual independence.
week 3
9
Example
• Roll a die twice. Define the following events;
A: 1st die odd
B: 2nd die odd
C: sum is odd.
week 3
10
Example
• Let R, S and T be independent, equally likely events with common
probability 1/3. What is PR  S  T  ?
• Solution:
week 3
11
Claim
• If A, B are independent so are A, B and A, B and A, B .
• Proof:
week 3
12