Download P(A)

Chapter 3
Conditional Probability and
Independence
Wen-Guey Tzeng
Computer Science Department
National Chiao Tung University
Conditional probability
• P(A|B) = the probability of event A given the
occurrence of event B
• Knowing that B has occurred changes the probability
of A’s occurrence
A
S
• Definition. If P(B)>0, the conditional probability of A
given B,
P( AB )
P( A | B ) 
P( B )
2014 Fall
2
•
•
•
•
A class of 80 students.
50 of them take Linear Algebra, 60 of them take Probability,
and 30 of them take both.
Randomly choose a student and find that the student takes
Probability
What is the probability that the student takes Linear Algebra
also?
Sol:
Let B be event that a selected student takes Probability.
Let A be the event that a selected student takes Linear Algebra.
P(A|B )= P(AB)/P(B) = (30/80) / (60/80)
2014 Fall
3
•
•
Draw 8 cards from a deck of 52 cards.
Given 3 of them are spades,
what is P(the remaining 5 are also spades|3 are spades)=?
Sol: B: the event that at least 3 of them are spades
A: the event that the remaining 5 are spades
P(A|B) = P(AB)/P(B) = 5.44 x 10-6
P(AB)=C(13,8)/C(52,8)
P(B)=[C(13,3)C(39,5)+C(13,4)C(39,4)+ …+C(13,8)] / C(52,8)
2014 Fall
4
• Box 1: 2 red balls and 3 green balls
• Box 2: 4 red balls and 1 green balls
• Choose a box with P(Box 1)=0.6, P(Box 2)=0.4, and then
randomly choose two balls from the box.
• What is the sample space?
• What is the probability of
• the event that Box 1 is chosen and two balls are both red?
• the event that two red balls are chosen under the condition
that Box 2 is chosen?
• the event that one red and one green are chosen?
• the event that Box 1 is chosen under the condition that two
balls are red and green?
2014 Fall
5
P(BiDj)
D1: balls are GG
D2: balls are RG
D3: balls are GG
B1: Box 1 is chosen
B2: Box 2 is chosen
2014 Fall
6
•
•
•
From the set of all families with 2 children, a family is selected
at random and is found to have a girl.
Assume that in a 2-child family all sex distributions are equally
probable.
What is the probability that the other child of the family is a
girl?
Sol:
All possible outcomes (equally likely): bb, bg, gb, gg
Let B be the event that the family has a girl.
Let A be the event that the other child of the family is a girl.
P(A|B)=P(AB)/P(B)=P({gg} / P({bg,gb,gg})=(1/4)/(3/4)=1/3.
2014 Fall
7
•
•
•
From the set of all families with 2 children, a child is randomly
selected and found to be a girl.
Assume that in a 2-child family all sex distributions are equally
probable.
What is the probability that the other child of the family is a
girl?
Sol:
Let B be the event that a randomly selected child is a girl.
Let A be the event that the second child of the family is a girl.
P(A|B)= P(AB)/P(B) = (1/4)/(2/4)=1/2.
How do you define a sample space for this problem?
2014 Fall
8
• Conditional probability = reduction of sample space
S
A
C
S’ = B
2014 Fall
9
All probability theorems hold under conditional
probability, for P(B)0,
• P(Ac|B) = 1-P(A|B)
• P(EF|B) = P(E|B)+P(F|B)-P(EF|B)
• IF C  A, P(C|B)  P(A|B)
• Inclusion-exclusion principle
•…
2014 Fall
10
• A child mixes 10 good and 3 dead batteries.
• To find the dead battery, the father tests batteries one by one
and without replacement.
• If the first 4 batteries are good, what is the probability that the
fifth is dead?
Sol:
Old sample space
S = {all sequences of 10 good and 3 dead batteries}
New sample space
S’ = {all sequences of 6 good and 3 dead batteries}
P(the fifth is dead | the first 4 are good} -- in the old sample space S
= P(the first is dead) -- in the new sample space S
= 3/9
2014 Fall
11
• There are three boxes. One contains a 100-dollar bill and the
other two are empty.
• You select one of them randomly.
• What is the probability that you win 100 dollars?
---------------------• I open an empty box that is not selected by you.
• Will you switch to the other un-opened box ?
• Yes, always switch. Why?
• No, never switch. Why?
-----------------------
2014 Fall
12
Sol:
• Assume that the bill is in box 1. – reduce the sample space
• Let outcome=(x, y, z)
• x: your first choice
• y: the opened box
• z: the final choice (switched to)
• Yes: always switch
• The reduced sample space
S={ (1, 2, 3), (1, 3, 2), (3, 2, 1), (2, 3, 1)}
• P({3,2,1}) = P({2,3,1}) = P({1,2,3), (1,3,2)}) = 1/3
• P(you win)=P({(3,2,1), (2,3,1)})=2/3
• No: never switch
• The reduced sample space
S={(1,2,1), (1,3,1), (2,3,2), (3,2,3)}
• P(you win) = P({(1,2,1), (1,3,1)} = 1/3
2014 Fall
13
Law of multiplication
P( AB )
P( A | B ) 
P( B )
 P( AB)  P( B) P( A | B)
P(AB)  P(BA)  P(B)P(A|B)  P(A)P(B|A)
Extension
P(A1 A2 A3 )  P(A1 )P(A2|A1 )P( A3 | A1 A2）
2014 Fall
14
• Suppose 5 good fuses and 2 defective ones have been mixed up.
To find the defective fuses, we test them 1-by-1, at random and
without replacement.
• What is the probability that we are lucky and find both of the
defective fuses in the first two tests?
Sol:
Let D1 and D2 be the events of finding a defective fuse
in the 1st and 2nd tests, respectively.
P(D1D2) = P(D1)P(D2|D1) = 2/7x1/6 = 1/21
P(find both defective fuses in exactly 3 tests) = ?
2014 Fall
15
Law of total probability
Theorem (Law of total probability)
Let P(B)>0, and P(Bc)>0, then
P(A) = P(A|B)P(B) + P(A|Bc)P(Bc)
Proof:
P(A) = P(AB) + P(ABc)
and use the law of multiplication.
2014 Fall
16
• An insurance company rents 35% of the cars for its customers
from agency I and 65% from agency II. If 8% of the cars of
agency I and 5% of the cars of agency II break down during the
rental periods, what is the probability that a car rented by this
insurance company breaks down?
Sol:
P(A) = P(A|I)P(I)+ P(A|II)P(II)
= (0.08)(0.35)+(0.05)(0.65)
= 0.0605
Table method?
Tree diagram method?
2014 Fall
17
• Gambler’s ruin problem
2 gamblers play the game of “heads or tails,” in which each
time a fair coin lands heads up, player A wins $1 from B, and
each time it lands tails up, player B wins $1 from A. Suppose
that the player A initially has ‘a’ dollars and player B has ‘b’
dollars. If they continue to play this game successively, what is
the probability that
• A will be ruined ?
• the game goes forever with nobody winning ?
2014 Fall
18
Sol:
• Sample space ?
• Let E be the event that A will be ruined if he starts with i dollars,
and let pi=P(E).
Our aim is to calculate pa. To do so we define F to be the event
that A wins the 1st game. Then
P(E) = P(E|F)P(F) + P(E|Fc)P(Fc)
=> pi = pi+1(1/2)+pi-1(1/2)
=> pi+1 – pi = pi – pi-1
2014 Fall
19
p0=1, pa+b=0, and let p1- p0 = x
pi+1 – pi = pi – pi-1 = … = p2- p1 = p1- p0 = x
Thus p1 = p0 + x
p2 = p0 + 2x
…
pi = p0 + ix
…
and pa+b = p0 + (a+b)x => x = -1/(a+b)
So pi = 1 – i/(a+b). In particular pa = b/(a+b)
2014 Fall
20
• The same method can be used with obvious modifications to
calculate qi, the probability that B is ruined if he starts with i
dollars.
qi = 1 – i/(a+b)
Since B starts with b dollars, he will be ruined with probability
qb = a/(a+b).
• Thus the probability that the game goes on forever with nobody
winning is 1-(qb+pa)=0.
2014 Fall
21
• Extension
If {B1, B2, … , Bn} is a partition of the sample space S of an
experiment and P(Bi)>0 for all i. Then, for any event A of S,
P(A) = P(A|B1)P(B1) + P(A|B2)P(B2) + … + P(A|Bn)P(Bn)

2014 Fall
22
• Suppose that 80% of the seniors, 70% of the juniors, 50%
of the sophomores, and 30% of the freshmen of a college
use the library of their campus frequently. If 30% of all
students are freshmen, 25% are sophomores, 25% are
juniors, and 20% are seniors, what percent of all students
use the library frequently?
Sol: P(A)= P(A|F)P(F) + P(A|O)P(O) + P(A|J)P(J) + P(A|E)P(E)
= (0.30)(0.30) + (0.50)(0.25) + (0.70)(0.25) + (0.80)(0.20)
= 0.55
2014 Fall
23
Bayes’ formula
•𝑃 𝐵𝐴 =
𝑃(𝐴|𝐵)P B
P(𝐴)
=
P(A|B)P B
P(A|B)P B +P(A|𝐵𝑐 )P(𝐵c )
• A conditional probability formula with very important
applications on Statistics.
• Prior probability: P(B), P(Bc)
– the probability of event (hypothesis) B is believed originally
• Likelihoods (observation probability) : P(A|B), P(A|Bc)
• Posterior probability: P(B|A)
– the probability of B under that A is observed
2014 Fall
24
• In a bolt factory, 30% and 70% of production is manufactured by
machines I and II, respectively.
• 4% and 6% of the output of machines I and II are defective.
• What is the probability that a randomly selected bolt that is
found to be defective is manufactured by machine I?
probability
A: defective
Ac: non-defective
B: manufactured by I
Bc: manufactured by II
We want to compute P(B|A)?
Prior probability P(B)=0.3
Likelihoods: P(A|B)=0.04, P(A|Bc)=0.06
Post probability: P(B|A)=0.04*0.3/(0.04*0.3+0.06*0.7)=0.222
2014 Fall
25
Tree diagram method ?
2014 Fall
26
•
•
•
•
In a double homicide, a suspect, John, is caught.
The jury believes that John is guilty with 15%。
Later, a DNA sample is found and match John’s DNA
By forensic estimation, the probability that the DNA does not
come from John is 10-9.
• How certain should the jury be about that John is guilty?
Sol:
• G: the event that John is guilty, P(G) = 0.15
I: the event that John is innocent, P(I)=0.85
• D: the event that the DNA matches John’s DNA
P(D|G)=1, P(D|I)=10-9
• Compute P(G|D)=
2014 Fall
27
• A car insurance company needs to decide whether to accept a
car insurance from a driver.
• By estimation, the percentage of good drivers is 80%
• The probability that a good driver has a car accident within a
year is 10%, and the probability that a bad driver has a car
accident within a year is 40%
2014 Fall
28
• What is the probability that a driver who has a car accident in
the first year is a good driver?
2014 Fall
29
• What is the probability that a driver who has a car accident in
the first year and no car accident in the second year is a good
driver?
2014 Fall
30
• What is the probability that a driver who has no car accidents in
two consecutive years is a bad driver?
2014 Fall
31
Extension:
B1, B2, …, Bn partition S.
P A 𝐵𝑘 P 𝐵𝑘
𝑃(𝐵𝑘 |𝐴) =
P 𝐴 𝐵1 P 𝐵1 + ⋯ + P A 𝐵𝑛 P(𝐵𝑛 )
• Prior probability: P(Bk), 1kn
• Likelihoods: P(A|Bi), 1in
• Post probability: P(Bk|A )
2014 Fall
32
• A box contains 7 red and 13 blue balls.
• 2 balls are selected at random and are discarded without their
colors being seen.
• A 3rd ball is drawn randomly and observed to be red,
• What is the probability that both of the discarded balls were
blue?
Sol:
• B1: discarded balls are RR, P(B1) =
B2: discarded balls are RB, P(B2) =
B3: discarded balls are BB, P(B3) =
• A: the third ball is R
• P(A|B1)=
P(A|B2)=
P(A|B3)=
• Want to compute P(B3|A)
2014 Fall
33
Independence
Two events A and B are independent if P(AB)=P(A)P(B)
A
B
A
B
A B
2014 Fall
34
• Theorem：A and B are independent if either one of the
following holds
• P(AB) = P(A)P(B)
• P(A|B) = P(A)
• P(B|A) = P(B)
• Theorem: If A and B are independent, so are
• A and Bc
• Ac and B
• Ac and Bc
• What is good for independence?
• easy to compute
2014 Fall
35
•
•
•
•
A card is drawn from a deck of 52 cards.
A: the event that an ace is drawn.
B: the event that a heart is drawn.
Are A and B independent?
• P(A) =
• P(B)=
• P(AB) =
2014 Fall
36
• An urn contains 5 red and 7 blue balls. Suppose that 2 balls are
selected at random with/without replacement. Let A and B be the
events that the first and the second ball are red, respectively. Are A
and B independent?
• With replacement
•
•
•
•
P(A) =
P(B) =
P(AB) = P(B|A)P(A) =
Independent ?
• Without replacement
• P(B|A) =
• P(B) = P(B|A)P(A)+P(B|Ac)P(Ac) =
• Dependent ?
2014 Fall
37
Jailer’s paradox
• Three prisoners Alex, Bill, and Tim. One of them is condemned
to death. The other two will be freed.
• The jailor and the judge know who is condemned to death.
• Alex has written a letter to his fiancée and wants to give it to
either Bill or Tim, whoever goes free, to deliver.
• Alex asks the jailer to tell him which of the two will be freed.
• The jailer refuses to give that information to Alex, explaining
that, if he does, the probability of Alex dying increases from 1/3
to 1/2. Really?
2014 Fall
38
Sol-1:
• Let A, B, and T be the events that “Alex dies,”
”Bill dies,” and “Tim dies.”
• Let
w1=(T, the jailer tells Alex that Bill goes free)
w2=(B, the jailer tells Alex that Tim goes free)
w3=(A, the jailer tells Alex that Bill goes free)
w4=(A, the jailer tells Alex that Tim goes free)
• The sample space S = {w1, w2, w3, w4}.
P(w1)=P(w2)=1/3, P(w3)=P(w4)=1/6
• Let J be the event that “the jailer tells Alex that Tim goes free”
• Then,
1
P( A | J ) 
2014 Fall
P ( AJ )
P ( w4 )
1

 6 
1 1
P( J )
P ( w2 )  P ( w4 )
3

3 6
39
Sol-2:
• Zweifel, in June 1986 issue of Mathematics Magazine, page 156
• He analyzes this paradox by using Bayes’ formula:
P( A | J )
P ( J | A) P ( A)
P ( J | A) P ( A)  P ( J | B ) P ( B )  P ( J | C ) P (C )
1 1

2 3

1 1
1
1
  1  0 
2 3
3
3
1

3

2014 Fall
40
Remark:
• If A and B are disjoint (mutually exclusive),
then A and B are dependent.
• If A  B,
then A and B are dependent
2014 Fall
41
Extension
• Events A, B, and C are independent if the following all
hold:
•
•
•
•
P(AB) = P(A)P(B)
P(AC) = P(A)P(C)
P(BC) = P(B)P(C)
P(ABC) = P(A)P(B)P(C)
2014 Fall
42
• We draw cards, one at a time, at random and successively from
an ordinary deck of 52 cards with replacement. What is the
probability that an ace appears before a face card?
Sol:
• Let E be the event of an ace appearing before a face card.
• Let A, F, and B be the events of ace, face card, and neither in the
first experiment, respectively. Then
• P(E) = P(E|A)P(A) + P(E|F)P(F) + P(E|B)P(B)
= 1(4/52) + 0(12/52)+ P(E|B)(36/52)
P(E|B) = P(E)
P(E)=4/52+P(E)(36/52)
• So, P(E)=1/4
2014 Fall
43