Download Lecture 5. Reminder + Warming-up Conditional Probability

Lecture 5.
Reminder + Warming-up
Conditional Probability.
Independence
1
Reminder
B
2
A few rules:
Complement Rule: The probability that A does not occur is equal to the probability that
the complement of event A occurs. P(Ac) = 1 - P(A).
Difference Rule: If A is a subset of B, then the probability of B occurring but not A is
P(B) - P(A) = P(B Ac).
Inclusion-Exclusion Rule: The probability of either A or B (or both) occurring is
P(A U B) = P(A) + P(B) - P(AB).
Solve it:
If the probability of having blew eyes is 10%, the probability of having dark
hair is 75%, and the probability of being a blue-eyed dark haired person is 9%,
what is the probability of
•not having blue eyes? [find P(Ac)]
•having blue eyes but not dark hair? [find P(A) - P(AB)]
•having blue eyes and/or dark hair? [find P(A U B)]
3
Warming up
4
Consider three problems:
(a) You make a new friend and you ask if he has any children. Yes, he
says, two. Any girls? Yes, he says. What is the probability that both
are girls?
Answer: 1/3.
(b) You make a new friend and you ask if she has any children.
Yes, she says, two – ages 6 and 9. Is the oldest a girl? You
ask. Yes, she says. What is the probability that both of her
children are girls?
Answer: 1/2
(c) You make a new friend and you ask if he has any children. Yes,
he says, two. Any girls? Yes. Next day you meet her with a
young girl. Is this your daughter? Yes, she says.
What’s the probability that both of her children are girls?
Answer: 1/2
5
Seemingly, the problem (c) is not different from ( a).
Why are the answers different?
At Home:
Please, try to figure out the nature of this apparent
paradox.
6
Solution of the extra credit question (HW4)
Here is a solution to the test problem 2.1 offered by one of the students.
Problem 2.1 A committee of 5 is chosen from a group of 8 men and 4 women.
What is the probability the group contains a majority of women?
Suggested Solution
For a group to contain a majority of women there are two possibilities:
wwwwm and wwwmm.
The probabilities are:
P(wwwwm) = (4/12)(3/11)(2/10)(1/9)(8/8) = 0.002.
P(wwwmm) = (4/12)(3/11)(2/10)(8/9)(7/8) = 0.014.
P(majority) = 0.002 + 0.014 = 0.016.
Question
You can see that this answer differs from the solution discussed in the class.
At the same time, the calculation of P(wwwmm) and P(wwwmm) is apparently
correct.
Please, fix the solution using the same approach.
7
Solution:
The solution correctly described two of many contributions to
the event A (majority of women),
namely wwwwm and wwwmm.
The total number the contributions are C(5,1) =5 for the first
group (4 women, 1 man) and C(5,2) = 10 for the second group
(3 women, 2 men).
It’s easy to verify that, regardless the order the probabilities
within each group are the same.
For instance, P(wwwwm) = P(wwmww) and P(wwwmm)=
P(wmwwmw). As a result,
P(A) = C(5,1) P(wwwwm) + C(5,2) P(wwwmm)== 5/33.
8
Some problems involving conditioning
Here is the classical “hit and run” problem
A cab was involved in a hit and run accident at night. Two cab companies,
Green Cab and Blue Cab, operate in the city. You are given the following
data:
(a) 85% of cabs in the city are Green and 15% are Blue.
(b) A witness identified the cab as Blue.
(c) The court tested the reliability of the witness and found that he correctly
identified one of the two colors 80% of the time and failed 20%.
What is the probability that the cab involved in the accident was Blue rather
than Green?
Make your best guess.
9
Consider a simpler problem
Bill's sister Debby lives in Canada. Learning that Debby was pregnant,
Bill drew a sample space diagram based on (his interpretation of ) their
family history . He was eager to have a niece because he already had two nephews
1/12
1/3
1/3
B
G
1/6
1/12
BB
BG
GG
Using this map, he estimated a probability of the event F= "at least one of newborns will be a
girl“. What was P(F)?
10
He found that P(F)=7/12. Later on it turned out that Debby was
expecting twins. Based on this new information, Bill redrew the
sample diagram, First, he noticed that the probabilities of singletons
became 0. This fact turned the area marked by a green rectangle into
the new sample space. The probabilities of the twins should now be
rescaled, thought Bill, so that their ratios will stay the same while they should
now sum up to 1. This resulted in the following distribution of probabilities:
1/4
0 B
0 G
1/2
1/4
BB
BG
GG
Using this diagram Bill found: P(F)=P(BG)+P(GG)=3/4. Its increase by 1/6
relative to the initial estimate made Bill even happier.
11
The probability of F under the condition (the hypothesis) that B has occurred is
called a conditional probability P(F|B).
In our example, Bill was considering a
probability P(F|T) that at least one of the
newborns will be a girl under the condition that
twins will be born.
His “rescaling” approach is correct but too
clumsy.
Bill could’ve recognized that the “rescaling”
was not necessary.
The same result could be derived as
T
1/3
B
1/3 G
F
1/12
BB
1/6
BG
1/12
GG
FT
P(F/T) = (1/6 + 1/12)/(1/12 + 1/6 + 1/12)=
(3/12) / (1/3) =3/4.
But what is the meaning of the nominator?
This is exactly the probability of FT,
intersection between the event F and the
condition T
12
It is also obvious that the denominator is exactly the probability of T, the
condition itself. In other words, Bill couild have avoided rescaling by using
the universal formula
T
1/3
P(F|T)= P(FT) / P(T)
B
(1)
1/3 G
F
1/12
BB
1/6
BG
1/12
GG
FT
The probability of F given T is the probability of the intersection
between F and T divided by the probability of T.
13
The formula (1) can also be rewritten as
P(FT) = P(F|T)P(T)= P(T|F)P(F))
(2)
In this representation it is called "the multiplication rule“ for the
conditioning.
14
Examples:
1.
A fair coin is flipped twice. What is the conditional probability that both flips
result in heads, given that the first flip results in heads?
Solution: If E = {HH} and F = {HH, HT}, then
P(E|F) = P(EF)/P(F) = [P{HH}]/[P{HH, HT}]
= (1/4)/(2/4) = 1/2.
Notice that the probability of two heads ( 1 / 4 ) is less than the probability of two
heads given that the first is head.
15
2. Suppose we draw two cards out of deck of 52 without replacement. Let A =“The
first card is a spade”, and B= “The second card is a spade”. What is the
probability that both A and B occur.
P(AB) = P(B|A)P(A)=(12/51)*( 1/4)
Note that we computed P(B|A) by thinking about the situation that exists after A has
occurred rather than using the definition (1). It’s more common to use P(A) and
P(B|A) to compute P(AB) rather than using P(AB) and P(A) to compute P(B|A).
You probably noticed that we already used the conditional probability when we solved
similar problems before, without calling it by name.
"Good Heavens! For more than forty years I have been speaking prose without
knowing it." Jean Baptiste Moliere, “The Middle Class Gentleman “
16
3 Two fair coins are tossed and you are told that at least one coin came
up heads.What is the probability that both coins came up heads?
Warning : the two “instinctive” answers, ½ and ¼ are both incorrect. Use
the conditional probability formula (1)
Solution:
(1) introduce two events:
B=“ At least one coin came up heads”; A= “Both coins came up heads”
(2) choose the relevant formula:
P(A|B) = P(AB)/P(B).
(3) to find P(AB) and P(B), notice that B= {HT,TH,HH}; AB = {HH}
(intersection);
(4) As a result, P(B) = ¾; P(AB)= ¼; P(A|B)=(1/4)/(3/4) = 1/3.
17
4. And here is one of many examples showing that the results of
conditioning are often not too obvious.
A person picks 13 cards out of a deck of 52.
Let A2=“He receives at least two Aces”.
A1=“He receives at least one Ace”, and
H=“He has the Ace of Hearts”.
Question Which probability is larger, P(A2|A1) or P(A2 |H)?
In other words, which is more likely: having two aces given that he has at
least one Ace, or having two aces given that he has an Ace of hearts.
Make your best guess.
Now, try to solve this problem (extra credit)
18
5. “DNA Fingerprinting in the Courts” by Bruce Weir,
State University
North Carolina
As a part of your home assignment, please watch this video. You can find
it here . Use Ctr+F to search for the author’s name, Weir, or the title to
locate the video.
It runs about an hour. You can browse the site to find a lot of other interesting
lectures.
19
6. It’s time now to discuss the “hit and run” problem.
A cab was involved in a hit and run accident at night. Two cab
companies, Green Cab and Blue Cab, operate in the city. You are
given the following data:
(a) 85% of cabs in the city are Green and 15% are Blue.
(b) A witness identified the cab as Blue.
(c) The court tested the reliability of the witness and found that he
correctly identified one of the two colors 80% of the time and
failed 20%.
What is the probability that the cab involved in the accident was
Blue rather than Green?
Let’s firs express the question in terms of a conditional
probability.
What event plays a role of the condition?
20
SB="Witness says it's blue", B="The car was Blue“. What is P(B|SB)=?
Let’s make now a diagram of everything that we know.
First, I drew these two rectangular areas representing the shares of the Green and
Blue cabs. But then I was confused trying to figure out how to depict the A event in
the same picture. Let’s try it together. First, notice that due to a possible witness
error, the A event contains not only the blue area (which would be the case if the
witness was absolutely reliable) but also the green area. What are their relative
portions? SBGreen= 0.2*0.85=0.17. A  Blue = 0.8*0.15=0.12.
P(B|SB)=P(BSB) / P(SB)=
0.12/(0.17+0.12)=0.12/0.29=0.41
A
Btw, check that the total
P (SB OR SG) = 1
21
A question for the self-test:
What would be the probability that the car was green What
if the witness testified that it was green?
22
7. A laboratory blood test is 95% effective in detecting a certain disease when it
is, in fact, present (sensitivity). However, the test also yields a "false positive"
result for 1% of the healthy persons tested. (That is, if a healthy person is tested,
then, with probability .01 the test result will be positive.) If .5%
of the population actually has the disease (prevalence), what is the probability a
person has the disease given that the test result is positive?
Solution.
Let D = “the tested person has the disease” and Pos = “the test result is positive”.
The desired probability P(D|Pos) is given by
P(D|Pos)=P(D  Pos)/P(Pos)
=[P(Pos|D) P(D)]/[P(Pos|D) P(D)+P(Pos|DC])P(DC)]
=(.95) (.005)/[(.95) (.005) + (.01) (.995)] =.323.
23
8. The Monty Hall problem (MHP)
This problem is named for the host of the TV
show “let’s make a deal”.
Three curtains are numbered 1,2 and 3. Behind one curtain is a
car, behind two others are donkeys.
You pick a curtain, say #1. Then host opens one of the two
remaining curtains, say # 3, to reveal a donkey.
What is the probability that you will win given that there is a
donkey behind #3? Should you switch curtains and pick #2 if
you are given a chance?
24
A few historical notes
( http://www.mathpuzzle.com/MontyHall.txt, The man who loved only
numbers , etc). Btw, does anyone want to make a special presentation,
and to discuss the solution using the Mathematica modeling? I will
supply the volunteer with required materials 
An earlier version of MHP, the Three Prisoner Problem, was
analyzed in 1959 by Martin Gardner in the journal Scientific
American. He called it "a wonderfully confusing little
problem" and presciently noted that "in no other branch of
mathematics is it so easy for experts to blunder as in
probability theory."
25
The fierce debates around the MHP erupted after Marylin vos
Savant *, the columnist for the Parade magazine, answered
this brain teaser advising her correspondent to
switch the door. Sticking with the first choice
gives a 1/3 chance of winning, but switching
doubles it to 2/3.
*Marylin von Savant
was pronounced by the Guinness Book of World Records as the
“person with Highest IQ ever recorded” (228).
She sported a wedding ring of pyrolytic carbon, a special material that was invented by
her husband, Robert Jarvik, creator of the First Artificial (“Jarvik”) Heart . 
26
The experts responded in force to Ms. Savant's column. Of the critical
letters she received, close to 1,000 carried signatures with Ph.D.'s,
and many were on letterheads of mathematics and science
departments.
"Our math department had a good, self-righteous laugh at your
expense," wrote Mary Jane Still, a professor at Palm Beach Junior
College.
". . . I am sure you will receive many letters from high school and
college students. Perhaps you should keep a few addresses for help
with future columns." -W. Robert Smith, Ph.D., Georgia State
University
"You are utterly incorrect. . . How many irate mathematicians are
needed to get you to change your mind? -E. Ray Bobo, Ph.D.,
Georgetown University
". . . If all those Ph.D.s were wrong, the country would be in very
serious trouble." -Everett Harman, Ph.D., U.S. Army Research
Institute
27
Robert Sachs, a professor of mathematics at George Mason
University:
"You blew it! Let me explain: If one door is shown to be a loser, that
information changes the probability of either remaining choice -neither of which has any reason to be more likely -- to 1/2. As a
professional mathematician, I'm very concerned with the general
public's lack of mathematical skills. Please help by confessing your
error and, in the future, being more careful." 
Later, Dr. Sachs, was one of the few with the grace to concede his
mistake.
"I wrote her another letter," Dr. Sachs said last week, "telling her that
after removing my foot from my mouth I'm now eating humble pie. I
vowed as penance to answer all the people who wrote to castigate me.
It's been an intense professional embarrassment." 
28
Many people, including some professional mathematicians, argue that
two unopened curtains are the same, so they each will contain car with
probability ½ , and hence there is no point in switching. This answer is
incorrect.
To compute an answer, we should first make an assumption about how
the host behaves.
Suppose that he always chooses to show you a donkey, and picks at
random if there are two unchosen curtains with donkeys.
Assuming you pick curtain # 1, there are three possibilities:
29
#1
#2
#3
Host’s action
Case 1
donkey
donkey
car
Opens #2
Case 2
donkey
car
donkey
Opens #3
Case 3
car
donkey
donkey
Opens #2 or #3
What is the probability you will win given that there is donkey behind #3 (or the
probability of winning given that MH opens #3)
P(case 3 | open door #3) 
P(case 3  open door #3)
P(open door #3)
Now, P(open door #3) = P(case 2, open door#3) + P(case 3, open door 3);
(note that “,” also means “and” or “”)
P(case 2, open door#3) =1*1/3 = 1/3; P(case 3, open door#3)= 1/3*1/2=1/6.
It gives:
P(open door #3) = 1/3+1/6=1/2.
30
And it follows that
P(case 3|open door 3) = P(case 3,Open door 3)/P(open door #3) =1/6/1/2 = 1/3.
P(case3| open door#3) 
P(case 3, Open door#3) 1 / 6

 1/ 3
P(open door #3)
1/ 2
This answer is “obvious”. Initially we have 1/3 probability of picking a
winning door. Opening the door with a donkey does not change this
probability. It means that switching the door increases the probability
from 1/3 to 2/3.
31
Independent events
Definition: Two events E and F are independent if
P(E|F)=P(E) and P(F|E) = P(F)
(3)
The equivalent definition: The events A and B are independent if
P(AB)=P(A)P(B)
(4)
Assignment: prove that these definitions are equivalent
32
Two examples of the independent events
1. Flip two coins. A =“The first coin shows H”, B=“The second coin shows
H”. Prove that A and B are independent.
Hint: Omega={{HH},{HT},{TH},{TT}}
2.
Roll two dice. A="The first die shows 5”, B="The second die shows 2".
P(A)=1/6, P(B) = 1/6, P(AB)= P({52})=1/36.
Dependent events
Roll two dice. A="The sum of the two dice is 9". B="The first die is 2".
P(B)=1/6. A = {{6,3},{5,4},{4,5},{3,6}}. P(A)=4/36=1/9.
P(AB)=0.
33
Suppose an unfair die has the property that the probability of number i turning up is
i/21. Are the events A=“ an even number turns up” and B= “a number 4 or less turns
up” independent. What is the corresponding result for a fair die?
Solution 1 (1) describe the events: A= {2,4,6}; B={1,2,3,4}. AB= {2,4},
(2) find the probabilities of these events using the probability
function
P(A)= 2/21+4/21 + 6/21= 12/21; P(B)= 1/21 + 2/21 + 3/21 +
4/21=10/21;
P(AB) = 2/21 + 4/21 = 6/21.
(3) find P(A)P(B) = 120/21*21=40/147 < P(AB)=6/21= 42/147
Answer: the events are not independent
Solution 2 (1) the events are the same as above
(2) find the probabilities of these events using the probability function
P(A)= 1/6+1/6 + 1/6= 1/2; P(B)= 4*1/6= 2/3.
P(AB) = 2*1/6 = 1/3.
(3) find P(A)P(B) = 1/3 = P(AB)
Answer: the events are independent
34
Some more examples.
It is not always self-evident if the events are independent
Try to analyze the following example (group work):
Roll two dice. Let S7="The sum of the two dice is 7", F4="The first die is 4",
DE="The difference of two dice is even".
Are S7and F4 independent? S7 and DE? F4 and DE?
35
Solution:
(a) Describe the events and their probabilities
S7={{1,6},{2,5}{3,4},{4,3},{5,2},{6,1}}; P(S7)=1/6; P(F4)=1/6.
DE= {{1,3},{1,5},{2,4},{2,6},{3,5},{4,6},{6,4},{5,3},{6,2},{4,2},{5,1},{3,1}}; P(DE)=1/3
(b) Find the intersections and their probabilities.
S7*F4= {4,3}, P(S7*F4)= 1/36 = P(S7)*P(F4); S7 and F4 are independent.
S7*DE=, P(S7*DE)=0  (non-equal) P(S7)*P(DE):
S7 and DE are not
independent.
They are disjoined. They can never occur together. Occurrence of S7 makes
simultaneous occurrence of DE impossible.
F4*DE={4,6}, P(F4*DE)=P({42} OR {46}) =2/36 = 1/18 = P(F4)*P(DE)
Please finish this problem. We made an error in the class. Find P(DE)!
36
Definition: A set of events {A1, A2, …An} is said to be mutually
independent if for any subset {Ai, Aj, … Am} of these events we
have
P(Ai  Aj  …  Am)=P(Ai)P( Aj) … P(Am)
(5)
It is natural to ask: if all pairs of a set of events are independent, is
the whole set mutually independent?
The answer is “not necessary”.
Building various events with two dice experiments.
The sample space can be presented through the table 6×6.Different groups
of cells correspond to different events. Two events are shown in the picture:
Green= "Sum of two throws <5" and Grey="Sum of two throws = 9".
37
Are they independent?
Now, try to name the events shown in Figures a through d and
evaluate their (in)dependence. Colored black are the intersection
points between different events. In the case(d), be careful
evaluating the mutual (in)dependence of three events.
38
39
11
12
13
14
15
16
21
22
23
24
25
26
31
32
33
34
35
36
41
42
42
44
45
46
51
52
53
54
55
56
61
62
63
64
65
66
Here is another example of two dependent events:
S4= “Sum = 4”, X2 = “Second is 2”;
P(S4 X2)= 1/36;
P(S4)*P(X2)= 1/6*1/12=1/72.
40
Suppose we roll a green and a red die. Let A="The red die shows 2 or 5.
B="The some of the two dice is at least 7". Are A and B independent?
P(A)=P(Grey+Black)=1/3;
P(B)=P(yellow+Black)=21/36. P(AB)=P(black)=7/36. The events are
independent
41
Home assignment (see also WebCT).
•
•
Watch the video (p.19).
Try to experiment with the Non-Linear Regression file:
Change the positions of the detectors, position and the intensity of the source. Try
generating the data and solving the problem of locating the source based on
those data.
Familiarize yourself with the functions used for the regression
analysis. Be prepared to describe the Least Squares Approximation
Extra credit (4 problems, but you can solve only a part of them) :
•
Cards problem, page 18.
•
Solve the additional cab problem, p. 22
•
Prove the equivalence of two definitions of the
independence (p. 32).
•
Finish the last problem in page 36.
42