Download Conditional Probability and Independence - Penn Math

Conditional Probability and Independence
Christopher Croke
University of Pennsylvania
Math 115
Christopher Croke
Calculus 115
Probability
The probability of an event depends on the sample space.
Christopher Croke
Calculus 115
Probability
The probability of an event depends on the sample space.
Problem: In a group of 30 athletes 12 are women, 18 are
swimmers, and 10 are neither. A person is chosen at random.
Christopher Croke
Calculus 115
Probability
The probability of an event depends on the sample space.
Problem: In a group of 30 athletes 12 are women, 18 are
swimmers, and 10 are neither. A person is chosen at random.
What is the probability that it is a female swimmer?
Christopher Croke
Calculus 115
Probability
The probability of an event depends on the sample space.
Problem: In a group of 30 athletes 12 are women, 18 are
swimmers, and 10 are neither. A person is chosen at random.
What is the probability that it is a female swimmer?
Here we find (from inclusion/exclusion) that #(S ∩ W ) = 10 so
Pr (S ∩ W ) =
#(S ∩ W )
10
1
=
= .
total#
30
3
Christopher Croke
Calculus 115
Probability
The probability of an event depends on the sample space.
Problem: In a group of 30 athletes 12 are women, 18 are
swimmers, and 10 are neither. A person is chosen at random.
What is the probability that it is a female swimmer?
Here we find (from inclusion/exclusion) that #(S ∩ W ) = 10 so
Pr (S ∩ W ) =
#(S ∩ W )
10
1
=
= .
total#
30
3
Suppose we know we have chosen a woman. What is the
probability that she is a swimmer?
Christopher Croke
Calculus 115
Probability
The probability of an event depends on the sample space.
Problem: In a group of 30 athletes 12 are women, 18 are
swimmers, and 10 are neither. A person is chosen at random.
What is the probability that it is a female swimmer?
Here we find (from inclusion/exclusion) that #(S ∩ W ) = 10 so
Pr (S ∩ W ) =
#(S ∩ W )
10
1
=
= .
total#
30
3
Suppose we know we have chosen a woman. What is the
probability that she is a swimmer? Probability is
#(S ∩ W )
#(S ∩ W )
=
=
#Sample Space
#W
Christopher Croke
Calculus 115
Probability
The probability of an event depends on the sample space.
Problem: In a group of 30 athletes 12 are women, 18 are
swimmers, and 10 are neither. A person is chosen at random.
What is the probability that it is a female swimmer?
Here we find (from inclusion/exclusion) that #(S ∩ W ) = 10 so
Pr (S ∩ W ) =
#(S ∩ W )
10
1
=
= .
total#
30
3
Suppose we know we have chosen a woman. What is the
probability that she is a swimmer? Probability is
#(S ∩ W )
#(S ∩ W )
=
=
#Sample Space
#W
10
5
= .
12
6
Christopher Croke
Calculus 115
Conditional Probability
The conditional probability of E given F denoted Pr (E |F )
Christopher Croke
Calculus 115
Conditional Probability
The conditional probability of E given F denoted Pr (E |F ) is
given by
Pr (E ∩ F )
Pr (E |F ) =
.
Pr (F )
Christopher Croke
Calculus 115
Conditional Probability
The conditional probability of E given F denoted Pr (E |F ) is
given by
Pr (E ∩ F )
Pr (E |F ) =
.
Pr (F )
In the previous example (where there were equally likely outcomes)
we saw:
#(E ∩ F )
Pr (E |F ) =
#F
Christopher Croke
Calculus 115
Conditional Probability
The conditional probability of E given F denoted Pr (E |F ) is
given by
Pr (E ∩ F )
Pr (E |F ) =
.
Pr (F )
In the previous example (where there were equally likely outcomes)
we saw:
#(E ∩ F )
Pr (E |F ) =
#F
But this is equal to
#(E ∩ F ) (total#)
Pr (E ∩ F )
=
.
(total#)
#F
Pr (F )
Christopher Croke
Calculus 115
Why does this make sense?
Christopher Croke
Calculus 115
Why does this make sense?
Consider a large number N of trials. We expect the number of
outcomes that shows up in F to be about NPr (F ).
Christopher Croke
Calculus 115
Why does this make sense?
Consider a large number N of trials. We expect the number of
outcomes that shows up in F to be about NPr (F ). Of these we
expect about NPr (E ∩ F ) also to land in E .
Christopher Croke
Calculus 115
Why does this make sense?
Consider a large number N of trials. We expect the number of
outcomes that shows up in F to be about NPr (F ). Of these we
expect about NPr (E ∩ F ) also to land in E . So the fraction of
those that landed in F that also landed in E is
NPr (E ∩ F )
Pr (E ∩ F )
=
= Pr (E |F ).
NPr (F )
Pr (F )
Christopher Croke
Calculus 115
Why does this make sense?
Consider a large number N of trials. We expect the number of
outcomes that shows up in F to be about NPr (F ). Of these we
expect about NPr (E ∩ F ) also to land in E . So the fraction of
those that landed in F that also landed in E is
NPr (E ∩ F )
Pr (E ∩ F )
=
= Pr (E |F ).
NPr (F )
Pr (F )
PRODUCT RULE: Pr (E ∩ F ) = Pr (F ) · Pr (E |F ).
Christopher Croke
Calculus 115
Why does this make sense?
Consider a large number N of trials. We expect the number of
outcomes that shows up in F to be about NPr (F ). Of these we
expect about NPr (E ∩ F ) also to land in E . So the fraction of
those that landed in F that also landed in E is
NPr (E ∩ F )
Pr (E ∩ F )
=
= Pr (E |F ).
NPr (F )
Pr (F )
PRODUCT RULE: Pr (E ∩ F ) = Pr (F ) · Pr (E |F ).
5
In our example we see 13 = 12
30 · 6 .
Christopher Croke
Calculus 115
Problem: Two students are chosen, one after the other, from a
group of 50 students, 20 of who are freshmen and 30 of who are
sophomores.
Christopher Croke
Calculus 115
Problem: Two students are chosen, one after the other, from a
group of 50 students, 20 of who are freshmen and 30 of who are
sophomores.
a) What is the probability that the first is a freshman and the
second is a sophomore?
Christopher Croke
Calculus 115
Problem: Two students are chosen, one after the other, from a
group of 50 students, 20 of who are freshmen and 30 of who are
sophomores.
a) What is the probability that the first is a freshman and the
second is a sophomore?
b) If three are chosen, what is the probability that the first is a
sophomore, and the next two are freshmen?
Christopher Croke
Calculus 115
Problem: Two students are chosen, one after the other, from a
group of 50 students, 20 of who are freshmen and 30 of who are
sophomores.
a) What is the probability that the first is a freshman and the
second is a sophomore?
b) If three are chosen, what is the probability that the first is a
sophomore, and the next two are freshmen?
we will need the Generalized Product Rule:
Pr (E1 ∩ E2 ∩ E3 ) = Pr (E1 ) · Pr (E2 |E1 ) · Pr (E3 |E1 ∩ E2 ).
Christopher Croke
Calculus 115
Independence
Two events E and F are said to be independent if
Pr (E ) = Pr (E |F ) (as long as Pr (F ) 6= 0).
Christopher Croke
Calculus 115
Independence
Two events E and F are said to be independent if
Pr (E ) = Pr (E |F ) (as long as Pr (F ) 6= 0).
This is the same as (the official definition):
Pr (E ∩ F ) = Pr (E ) · Pr (F ).
Christopher Croke
Calculus 115
Independence
Two events E and F are said to be independent if
Pr (E ) = Pr (E |F ) (as long as Pr (F ) 6= 0).
This is the same as (the official definition):
Pr (E ∩ F ) = Pr (E ) · Pr (F ).
Note this also means Pr (F ) = Pr (F |E ).
Christopher Croke
Calculus 115
Independence
Two events E and F are said to be independent if
Pr (E ) = Pr (E |F ) (as long as Pr (F ) 6= 0).
This is the same as (the official definition):
Pr (E ∩ F ) = Pr (E ) · Pr (F ).
Note this also means Pr (F ) = Pr (F |E ).
Example: Roll a die two times.
Let E be “got a 1 on first roll”.
Let F be “got a 3 on second roll”.
Check that these are independent.
Christopher Croke
Calculus 115
Problem: A card is to be drawn from a full deck. Consider the
events:
E=“the card is a 4”.
F=“the card is a spade”
a) Are these independent events?
Christopher Croke
Calculus 115
Problem: A card is to be drawn from a full deck. Consider the
events:
E=“the card is a 4”.
F=“the card is a spade”
a) Are these independent events?
b) Answer the same question when the original deck was missing
the 7 of clubs.
Christopher Croke
Calculus 115
Problem: A card is to be drawn from a full deck. Consider the
events:
E=“the card is a 4”.
F=“the card is a spade”
a) Are these independent events?
b) Answer the same question when the original deck was missing
the 7 of clubs.
c) Answer the same question when the original deck was missing
the ace of spades and all the clubs and the ace and king of
diamonds.
Christopher Croke
Calculus 115
Problem: A card is to be drawn from a full deck. Consider the
events:
E=“the card is a 4”.
F=“the card is a spade”
a) Are these independent events?
b) Answer the same question when the original deck was missing
the 7 of clubs.
c) Answer the same question when the original deck was missing
the ace of spades and all the clubs and the ace and king of
diamonds.
If E and F are independent then so are E c and F c .
Christopher Croke
Calculus 115
Problem: A card is to be drawn from a full deck. Consider the
events:
E=“the card is a 4”.
F=“the card is a spade”
a) Are these independent events?
b) Answer the same question when the original deck was missing
the 7 of clubs.
c) Answer the same question when the original deck was missing
the ace of spades and all the clubs and the ace and king of
diamonds.
If E and F are independent then so are E c and F c . Also E and F c
are independent, etc.
Christopher Croke
Calculus 115
Problem: A card is to be drawn from a full deck. Consider the
events:
E=“the card is a 4”.
F=“the card is a spade”
a) Are these independent events?
b) Answer the same question when the original deck was missing
the 7 of clubs.
c) Answer the same question when the original deck was missing
the ace of spades and all the clubs and the ace and king of
diamonds.
If E and F are independent then so are E c and F c . Also E and F c
are independent, etc.
Pr (E ∩F c ) = Pr (E −E ∩F )
Christopher Croke
Calculus 115
Problem: A card is to be drawn from a full deck. Consider the
events:
E=“the card is a 4”.
F=“the card is a spade”
a) Are these independent events?
b) Answer the same question when the original deck was missing
the 7 of clubs.
c) Answer the same question when the original deck was missing
the ace of spades and all the clubs and the ace and king of
diamonds.
If E and F are independent then so are E c and F c . Also E and F c
are independent, etc.
Pr (E ∩F c ) = Pr (E −E ∩F ) = Pr (E )−Pr (E ∩F )
Christopher Croke
Calculus 115
Problem: A card is to be drawn from a full deck. Consider the
events:
E=“the card is a 4”.
F=“the card is a spade”
a) Are these independent events?
b) Answer the same question when the original deck was missing
the 7 of clubs.
c) Answer the same question when the original deck was missing
the ace of spades and all the clubs and the ace and king of
diamonds.
If E and F are independent then so are E c and F c . Also E and F c
are independent, etc.
Pr (E ∩F c ) = Pr (E −E ∩F ) = Pr (E )−Pr (E ∩F ) = Pr (E )−Pr (E )Pr (F )
Christopher Croke
Calculus 115
Problem: A card is to be drawn from a full deck. Consider the
events:
E=“the card is a 4”.
F=“the card is a spade”
a) Are these independent events?
b) Answer the same question when the original deck was missing
the 7 of clubs.
c) Answer the same question when the original deck was missing
the ace of spades and all the clubs and the ace and king of
diamonds.
If E and F are independent then so are E c and F c . Also E and F c
are independent, etc.
Pr (E ∩F c ) = Pr (E −E ∩F ) = Pr (E )−Pr (E ∩F ) = Pr (E )−Pr (E )Pr (F ) =
Pr (E )(1 − Pr (F )))
Christopher Croke
Calculus 115
Problem: A card is to be drawn from a full deck. Consider the
events:
E=“the card is a 4”.
F=“the card is a spade”
a) Are these independent events?
b) Answer the same question when the original deck was missing
the 7 of clubs.
c) Answer the same question when the original deck was missing
the ace of spades and all the clubs and the ace and king of
diamonds.
If E and F are independent then so are E c and F c . Also E and F c
are independent, etc.
Pr (E ∩F c ) = Pr (E −E ∩F ) = Pr (E )−Pr (E ∩F ) = Pr (E )−Pr (E )Pr (F ) =
Pr (E )(1 − Pr (F ))) = Pr (E )Pr (F c )
Christopher Croke
Calculus 115
Independence of a collection of events
A collection A1 , A2 , A3 , ..., An of events are independent if for any
subcollection Ai1 , Ai2 , Ai3 , ..., Aik we have
Pr (Ai1 ∩ Ai2 ∩ Ai3 ∩ ... ∩ Aik ) = Pr (Ai1 )Pr (Ai2 )Pr (Ai3 )...Pr (Aik ).
Christopher Croke
Calculus 115
Independence of a collection of events
A collection A1 , A2 , A3 , ..., An of events are independent if for any
subcollection Ai1 , Ai2 , Ai3 , ..., Aik we have
Pr (Ai1 ∩ Ai2 ∩ Ai3 ∩ ... ∩ Aik ) = Pr (Ai1 )Pr (Ai2 )Pr (Ai3 )...Pr (Aik ).
For example if our collection has 3 events E , F , and G then we
need:
Pr (E ∩ F ) = Pr (E )Pr (F )
Pr (E ∩ G ) = Pr (E )Pr (G )
Pr (F ∩ G ) = Pr (F )Pr (G )
Christopher Croke
Calculus 115
Independence of a collection of events
A collection A1 , A2 , A3 , ..., An of events are independent if for any
subcollection Ai1 , Ai2 , Ai3 , ..., Aik we have
Pr (Ai1 ∩ Ai2 ∩ Ai3 ∩ ... ∩ Aik ) = Pr (Ai1 )Pr (Ai2 )Pr (Ai3 )...Pr (Aik ).
For example if our collection has 3 events E , F , and G then we
need:
Pr (E ∩ F ) = Pr (E )Pr (F )
Pr (E ∩ G ) = Pr (E )Pr (G )
Pr (F ∩ G ) = Pr (F )Pr (G )
Pr (E ∩ F ∩ G ) = Pr (E )Pr (F )Pr (G )
Christopher Croke
Calculus 115
Independence of a collection of events
A collection A1 , A2 , A3 , ..., An of events are independent if for any
subcollection Ai1 , Ai2 , Ai3 , ..., Aik we have
Pr (Ai1 ∩ Ai2 ∩ Ai3 ∩ ... ∩ Aik ) = Pr (Ai1 )Pr (Ai2 )Pr (Ai3 )...Pr (Aik ).
For example if our collection has 3 events E , F , and G then we
need:
Pr (E ∩ F ) = Pr (E )Pr (F )
Pr (E ∩ G ) = Pr (E )Pr (G )
Pr (F ∩ G ) = Pr (F )Pr (G )
Pr (E ∩ F ∩ G ) = Pr (E )Pr (F )Pr (G )
It is not enough that the events are pairwise independent.
Christopher Croke
Calculus 115
Problem: If E, F and G are three independent events with
Pr (E ) = .5, Pr (F ) = .4, and Pr (G ) = .3 calculate:
a) Pr (E ∩ F ∩ G ).
b) Pr (E ∩ G c ).
c) Pr (E ∩ (F ∪ G )c ).
Christopher Croke
Calculus 115
Problem: If E, F and G are three independent events with
Pr (E ) = .5, Pr (F ) = .4, and Pr (G ) = .3 calculate:
a) Pr (E ∩ F ∩ G ).
b) Pr (E ∩ G c ).
c) Pr (E ∩ (F ∪ G )c ).
d) Pr (E ∪ (F ∪ G )c ).
Christopher Croke
Calculus 115
Problem: If E, F and G are three independent events with
Pr (E ) = .5, Pr (F ) = .4, and Pr (G ) = .3 calculate:
a) Pr (E ∩ F ∩ G ).
b) Pr (E ∩ G c ).
c) Pr (E ∩ (F ∪ G )c ).
d) Pr (E ∪ (F ∪ G )c ).
Problem:(Inspecting) A machine produces defective items with a
probability p.
a) If 10 items are chosen at random what is the probability that
exactly 3 are defective?
Christopher Croke
Calculus 115
Problem: If E, F and G are three independent events with
Pr (E ) = .5, Pr (F ) = .4, and Pr (G ) = .3 calculate:
a) Pr (E ∩ F ∩ G ).
b) Pr (E ∩ G c ).
c) Pr (E ∩ (F ∪ G )c ).
d) Pr (E ∪ (F ∪ G )c ).
Problem:(Inspecting) A machine produces defective items with a
probability p.
a) If 10 items are chosen at random what is the probability that
exactly 3 are defective?
b) What is the probability of finding at least one defective item in
the 10 chosen.
Christopher Croke
Calculus 115
Problem: If E, F and G are three independent events with
Pr (E ) = .5, Pr (F ) = .4, and Pr (G ) = .3 calculate:
a) Pr (E ∩ F ∩ G ).
b) Pr (E ∩ G c ).
c) Pr (E ∩ (F ∪ G )c ).
d) Pr (E ∪ (F ∪ G )c ).
Problem:(Inspecting) A machine produces defective items with a
probability p.
a) If 10 items are chosen at random what is the probability that
exactly 3 are defective?
b) What is the probability of finding at least one defective item in
the 10 chosen.
c) If we observe the items one at a time as they come off the line,
what is the probability that the 3rd defective item is the 10th item
observed?
Christopher Croke
Calculus 115
Using tree diagrams
You can use tree diagrams when performing a sequence of
experiments.
Christopher Croke
Calculus 115
Using tree diagrams
You can use tree diagrams when performing a sequence of
experiments.
Christopher Croke
Calculus 115
Using tree diagrams
You can use tree diagrams when performing a sequence of
experiments.
So the probabilities on the edges are conditional probabilities.
Christopher Croke
Calculus 115
Using tree diagrams
You can use tree diagrams when performing a sequence of
experiments.
So the probabilities on the edges are conditional probabilities.
From the diagram we see
Pr (O2 ∩ Oa) = 0.3 · 0.5 = 0.15.
Christopher Croke
Calculus 115
Problem:
A city of 100,000 people is broken into 4 police precincts of unequal
size (call them Pre1, Pre2, Pre3, Pre4). The population of Pr1 is
10,000, of Pr2 is 20,000, of Pr3 is 30,000, and of Pr4 is 40,000. A
review of crime recording shows that mistakes have been made.
Christopher Croke
Calculus 115
Problem:
A city of 100,000 people is broken into 4 police precincts of unequal
size (call them Pre1, Pre2, Pre3, Pre4). The population of Pr1 is
10,000, of Pr2 is 20,000, of Pr3 is 30,000, and of Pr4 is 40,000. A
review of crime recording shows that mistakes have been made.
20% of records in Pr1 contain errors.
5% of records in Pr2 contain errors.
10% of records in Pr3 contain errors.
5% of records in Pr4 contain errors.
a) Draw a tree diagram to describe these results.
Christopher Croke
Calculus 115
Problem:
A city of 100,000 people is broken into 4 police precincts of unequal
size (call them Pre1, Pre2, Pre3, Pre4). The population of Pr1 is
10,000, of Pr2 is 20,000, of Pr3 is 30,000, and of Pr4 is 40,000. A
review of crime recording shows that mistakes have been made.
20% of records in Pr1 contain errors.
5% of records in Pr2 contain errors.
10% of records in Pr3 contain errors.
5% of records in Pr4 contain errors.
a) Draw a tree diagram to describe these results.
b) What is the probability that a record has an error and is in Pre3?
Christopher Croke
Calculus 115
Problem:
A city of 100,000 people is broken into 4 police precincts of unequal
size (call them Pre1, Pre2, Pre3, Pre4). The population of Pr1 is
10,000, of Pr2 is 20,000, of Pr3 is 30,000, and of Pr4 is 40,000. A
review of crime recording shows that mistakes have been made.
20% of records in Pr1 contain errors.
5% of records in Pr2 contain errors.
10% of records in Pr3 contain errors.
5% of records in Pr4 contain errors.
a) Draw a tree diagram to describe these results.
b) What is the probability that a record has an error and is in Pre3?
c) What is the probability that a record chosen at random has an
error?
Christopher Croke
Calculus 115
Problem:
A city of 100,000 people is broken into 4 police precincts of unequal
size (call them Pre1, Pre2, Pre3, Pre4). The population of Pr1 is
10,000, of Pr2 is 20,000, of Pr3 is 30,000, and of Pr4 is 40,000. A
review of crime recording shows that mistakes have been made.
20% of records in Pr1 contain errors.
5% of records in Pr2 contain errors.
10% of records in Pr3 contain errors.
5% of records in Pr4 contain errors.
a) Draw a tree diagram to describe these results.
b) What is the probability that a record has an error and is in Pre3?
c) What is the probability that a record chosen at random has an
error?
d) What is the probability that a record is from Pre3 given that it
contains an error?
Christopher Croke
Calculus 115
Problem:
A city of 100,000 people is broken into 4 police precincts of unequal
size (call them Pre1, Pre2, Pre3, Pre4). The population of Pr1 is
10,000, of Pr2 is 20,000, of Pr3 is 30,000, and of Pr4 is 40,000. A
review of crime recording shows that mistakes have been made.
20% of records in Pr1 contain errors.
5% of records in Pr2 contain errors.
10% of records in Pr3 contain errors.
5% of records in Pr4 contain errors.
a) Draw a tree diagram to describe these results.
b) What is the probability that a record has an error and is in Pre3?
c) What is the probability that a record chosen at random has an
error?
d) What is the probability that a record is from Pre3 given that it
contains an error?
Christopher Croke
Calculus 115
Problem: There is a (very good) test for TB that will test positive
for TB 98% of the time if a person has TB while it will only test
positive 1% of the time if the person doesn’t have TB. Given that
only 0.02% of the population has TB, what is the probability that a
patient has TB if she tests positive (i.e. a so called false positive)?
Christopher Croke
Calculus 115
Problem: There is a (very good) test for TB that will test positive
for TB 98% of the time if a person has TB while it will only test
positive 1% of the time if the person doesn’t have TB. Given that
only 0.02% of the population has TB, what is the probability that a
patient has TB if she tests positive (i.e. a so called false positive)?
Trees are not always symmetric!
Christopher Croke
Calculus 115
Problem: There is a (very good) test for TB that will test positive
for TB 98% of the time if a person has TB while it will only test
positive 1% of the time if the person doesn’t have TB. Given that
only 0.02% of the population has TB, what is the probability that a
patient has TB if she tests positive (i.e. a so called false positive)?
Trees are not always symmetric!
Problem: A crate of apples contains 3 bad apples and 7 good
apples. Apples are chosen until a good one is picked. What is the
probability that it takes at least 3 picks to get a good apple?
Christopher Croke
Calculus 115
Bayes Theorem
Return to the Precincts problem with the files that had errors. The
events Pre1, Pre2, Pre3 and Pre4 were mutually exclusive events
whose union Pre1 ∪ Pre2 ∪ Pre3 ∪ Pre4 = S was the whole sample
space (i.e. they form a partition of S).
Christopher Croke
Calculus 115
Bayes Theorem
Return to the Precincts problem with the files that had errors. The
events Pre1, Pre2, Pre3 and Pre4 were mutually exclusive events
whose union Pre1 ∪ Pre2 ∪ Pre3 ∪ Pre4 = S was the whole sample
space (i.e. they form a partition of S).
We saw that Pr (E ∩ Pre3) = Pr (Pre3)Pr (E |Pre3).
Christopher Croke
Calculus 115
Bayes Theorem
Return to the Precincts problem with the files that had errors. The
events Pre1, Pre2, Pre3 and Pre4 were mutually exclusive events
whose union Pre1 ∪ Pre2 ∪ Pre3 ∪ Pre4 = S was the whole sample
space (i.e. they form a partition of S).
We saw that Pr (E ∩ Pre3) = Pr (Pre3)Pr (E |Pre3).
While Pr (E ) = Pr (Pre1)Pr (E |Pre1) + Pr (Pre2)Pr (E |Pre2) +
Pr (Pre3)Pr (E |Pre3) + Pr (Pre4)Pr (E |Pre4).
Christopher Croke
Calculus 115
Bayes Theorem
Return to the Precincts problem with the files that had errors. The
events Pre1, Pre2, Pre3 and Pre4 were mutually exclusive events
whose union Pre1 ∪ Pre2 ∪ Pre3 ∪ Pre4 = S was the whole sample
space (i.e. they form a partition of S).
We saw that Pr (E ∩ Pre3) = Pr (Pre3)Pr (E |Pre3).
While Pr (E ) = Pr (Pre1)Pr (E |Pre1) + Pr (Pre2)Pr (E |Pre2) +
Pr (Pre3)Pr (E |Pre3) + Pr (Pre4)Pr (E |Pre4).
Pr (E ∩Pre3)
=
Pr (E )
Pr (Pre3)Pr (E |Pre3)
Pr (Pre1)Pr (E |Pre1)+Pr (Pre2)Pr (E |Pre2)+Pr (Pre3)Pr (E |Pre3)+Pr (Pre4)Pr (E |Pre4) .
Hence Pr (Pre3|E ) =
Christopher Croke
Calculus 115
Bayes Theorem
Return to the Precincts problem with the files that had errors. The
events Pre1, Pre2, Pre3 and Pre4 were mutually exclusive events
whose union Pre1 ∪ Pre2 ∪ Pre3 ∪ Pre4 = S was the whole sample
space (i.e. they form a partition of S).
We saw that Pr (E ∩ Pre3) = Pr (Pre3)Pr (E |Pre3).
While Pr (E ) = Pr (Pre1)Pr (E |Pre1) + Pr (Pre2)Pr (E |Pre2) +
Pr (Pre3)Pr (E |Pre3) + Pr (Pre4)Pr (E |Pre4).
Pr (E ∩Pre3)
=
Pr (E )
Pr (Pre3)Pr (E |Pre3)
Pr (Pre1)Pr (E |Pre1)+Pr (Pre2)Pr (E |Pre2)+Pr (Pre3)Pr (E |Pre3)+Pr (Pre4)Pr (E |Pre4) .
Hence Pr (Pre3|E ) =
This works in general:
Christopher Croke
Calculus 115
Bayes’ Theorem
BAYES’ THEOREM If B1 , B2 ,....,Bn are mutually exclusive
events whose union is the whole sample space then for any event A
we have Pr (Bi |A) is
Christopher Croke
Calculus 115
Bayes’ Theorem
BAYES’ THEOREM If B1 , B2 ,....,Bn are mutually exclusive
events whose union is the whole sample space then for any event A
we have Pr (Bi |A) is
Pr (Bi )Pr (A|Bi )
.
Pr (B1 )Pr (A|B1 ) + Pr (B2 )Pr (A|B2 ) + ... + Pr (Bn )Pr (A|Bn )
Christopher Croke
Calculus 115
Bayes’ Theorem
BAYES’ THEOREM If B1 , B2 ,....,Bn are mutually exclusive
events whose union is the whole sample space then for any event A
we have Pr (Bi |A) is
Pr (Bi )Pr (A|Bi )
.
Pr (B1 )Pr (A|B1 ) + Pr (B2 )Pr (A|B2 ) + ... + Pr (Bn )Pr (A|Bn )
Problem: A refrigerator manufacturer has plants in five cities.
The following chart describes the daily production and the
rejection rate in the five cities.
City
Units output
failure rate
Atlanta
50
.03
Boston
100
.02
Chicago
400
.02
Detroit
400
.03
Eugene
50
.01
Christopher Croke
Calculus 115
Bayes’ Theorem
BAYES’ THEOREM If B1 , B2 ,....,Bn are mutually exclusive
events whose union is the whole sample space then for any event A
we have Pr (Bi |A) is
Pr (Bi )Pr (A|Bi )
.
Pr (B1 )Pr (A|B1 ) + Pr (B2 )Pr (A|B2 ) + ... + Pr (Bn )Pr (A|Bn )
Problem: A refrigerator manufacturer has plants in five cities.
The following chart describes the daily production and the
rejection rate in the five cities.
City
Units output
failure rate
Atlanta
50
.03
Boston
100
.02
Chicago
400
.02
Detroit
400
.03
Eugene
50
.01
What is the probability that a refrigerator was manufactured in
Chicago given that it was rejected?
Christopher Croke
Calculus 115
Bayes’ Theorem
Problem: It is observed that at any intersection 80% of the cars
that turn use their turn signals when turning. At a certain
intersection 85% of cars make a turn. If at this intersection you are
behind a car not using a turn signal what is the probability that it
will turn anyway? (We assume that cars going straight never use a
signal).
Christopher Croke
Calculus 115
Prior and Posterior Probabilities
Consider Pr (B) and Pr (B|A).
Christopher Croke
Calculus 115
Prior and Posterior Probabilities
Consider Pr (B) and Pr (B|A).
Pr (B) is a Prior Probability (Because it is the probability of B
with no other information.)
Christopher Croke
Calculus 115
Prior and Posterior Probabilities
Consider Pr (B) and Pr (B|A).
Pr (B) is a Prior Probability (Because it is the probability of B
with no other information.)
Pr (B|A) ia a Posterior Probability (Because it is the probability
of B *after* we know A holds.)
Christopher Croke
Calculus 115
Prior and Posterior Probabilities
Consider Pr (B) and Pr (B|A).
Pr (B) is a Prior Probability (Because it is the probability of B
with no other information.)
Pr (B|A) ia a Posterior Probability (Because it is the probability
of B *after* we know A holds.)
Example: We have two coins. Coin 1 is a fair coin while Coin 2
has two heads. We will select a coin randomly and toss it.
Christopher Croke
Calculus 115
Prior and Posterior Probabilities
Consider Pr (B) and Pr (B|A).
Pr (B) is a Prior Probability (Because it is the probability of B
with no other information.)
Pr (B|A) ia a Posterior Probability (Because it is the probability
of B *after* we know A holds.)
Example: We have two coins. Coin 1 is a fair coin while Coin 2
has two heads. We will select a coin randomly and toss it.
Let B1 be the event that the coin is fair and B2 be the event that
the coin is two headed.
Christopher Croke
Calculus 115
Prior and Posterior Probabilities
Consider Pr (B) and Pr (B|A).
Pr (B) is a Prior Probability (Because it is the probability of B
with no other information.)
Pr (B|A) ia a Posterior Probability (Because it is the probability
of B *after* we know A holds.)
Example: We have two coins. Coin 1 is a fair coin while Coin 2
has two heads. We will select a coin randomly and toss it.
Let B1 be the event that the coin is fair and B2 be the event that
the coin is two headed.
The prior probabilities are Pr (B1 ) = Pr (B2 ) = 12 .
Christopher Croke
Calculus 115
Prior and Posterior Probabilities
Consider Pr (B) and Pr (B|A).
Pr (B) is a Prior Probability (Because it is the probability of B
with no other information.)
Pr (B|A) ia a Posterior Probability (Because it is the probability
of B *after* we know A holds.)
Example: We have two coins. Coin 1 is a fair coin while Coin 2
has two heads. We will select a coin randomly and toss it.
Let B1 be the event that the coin is fair and B2 be the event that
the coin is two headed.
The prior probabilities are Pr (B1 ) = Pr (B2 ) = 12 .
Now flip the coin. Say a head comes up (event H1).
Christopher Croke
Calculus 115
Prior and Posterior Probabilities
Consider Pr (B) and Pr (B|A).
Pr (B) is a Prior Probability (Because it is the probability of B
with no other information.)
Pr (B|A) ia a Posterior Probability (Because it is the probability
of B *after* we know A holds.)
Example: We have two coins. Coin 1 is a fair coin while Coin 2
has two heads. We will select a coin randomly and toss it.
Let B1 be the event that the coin is fair and B2 be the event that
the coin is two headed.
The prior probabilities are Pr (B1 ) = Pr (B2 ) = 12 .
Now flip the coin. Say a head comes up (event H1). What are the
posterior probabilities Pr (B1 |H1) and Pr (B2 |H1)?
Christopher Croke
Calculus 115
Prior and Posterior Probabilities
Consider Pr (B) and Pr (B|A).
Pr (B) is a Prior Probability (Because it is the probability of B
with no other information.)
Pr (B|A) ia a Posterior Probability (Because it is the probability
of B *after* we know A holds.)
Example: We have two coins. Coin 1 is a fair coin while Coin 2
has two heads. We will select a coin randomly and toss it.
Let B1 be the event that the coin is fair and B2 be the event that
the coin is two headed.
The prior probabilities are Pr (B1 ) = Pr (B2 ) = 12 .
Now flip the coin. Say a head comes up (event H1). What are the
posterior probabilities Pr (B1 |H1) and Pr (B2 |H1)? Flip the coin
again and say a head comes up again (event H2). What are the
posterior probabilities?
Christopher Croke
Calculus 115
Conditional version of Bayes’ Theorem
There are two ways to solve.
Christopher Croke
Calculus 115
Conditional version of Bayes’ Theorem
There are two ways to solve. The first is like before:
Pr (B1 |H1∩H2) =
Pr (B1 )Pr (H1 ∩ H2|B1 )
=
Pr (B1 )Pr (H1 ∩ H2|B1 ) + Pr (B2 )Pr (H1 ∩ H2|B2 )
Christopher Croke
Calculus 115
Conditional version of Bayes’ Theorem
There are two ways to solve. The first is like before:
Pr (B1 |H1∩H2) =
Pr (B1 )Pr (H1 ∩ H2|B1 )
=
Pr (B1 )Pr (H1 ∩ H2|B1 ) + Pr (B2 )Pr (H1 ∩ H2|B2 )
=
1
2
·
1 1
2 · 4
1
1
4 + 2
Christopher Croke
·1
Calculus 115
Conditional version of Bayes’ Theorem
There are two ways to solve. The first is like before:
Pr (B1 |H1∩H2) =
Pr (B1 )Pr (H1 ∩ H2|B1 )
=
Pr (B1 )Pr (H1 ∩ H2|B1 ) + Pr (B2 )Pr (H1 ∩ H2|B2 )
=
1
2
·
1 1
2 · 4
1
1
4 + 2
Christopher Croke
1
= .
5
·1
Calculus 115
Conditional version of Bayes’ Theorem
There are two ways to solve. The first is like before:
Pr (B1 |H1∩H2) =
Pr (B1 )Pr (H1 ∩ H2|B1 )
=
Pr (B1 )Pr (H1 ∩ H2|B1 ) + Pr (B2 )Pr (H1 ∩ H2|B2 )
=
1
2
·
1 1
2 · 4
1
1
4 + 2
1
= .
5
·1
The other way is to use a conditional version of Bayes’ Theorem.:
Pr (Bi |A ∩ C ) =
Pr (Bi |C )Pr (A|Bi ∩ C )
.
n
Σj=1 Pr (Bj |C )Pr (A|Bj ∩ C )
Christopher Croke
Calculus 115
Conditional version of Bayes’ Theorem
There are two ways to solve. The first is like before:
Pr (B1 |H1∩H2) =
Pr (B1 )Pr (H1 ∩ H2|B1 )
=
Pr (B1 )Pr (H1 ∩ H2|B1 ) + Pr (B2 )Pr (H1 ∩ H2|B2 )
=
1
2
·
1 1
2 · 4
1
1
4 + 2
1
= .
5
·1
The other way is to use a conditional version of Bayes’ Theorem.:
Pr (Bi |A ∩ C ) =
Pr (Bi |C )Pr (A|Bi ∩ C )
.
n
Σj=1 Pr (Bj |C )Pr (A|Bj ∩ C )
in our case (C = H1 and A = H2) we get:
1
3
·
1 1
3 · 2
1
2
2 + 3
Christopher Croke
1
= .
5
·1
Calculus 115
For two events A, B they are independent if Pr (A) = Pr (A|B).
Christopher Croke
Calculus 115
For two events A, B they are independent if Pr (A) = Pr (A|B).
We can see {A1 , A2 , ..., An } independent if for any two disjoint
subsets {i1 , i2 , ..., ik } and {j1 , j2 , ..., jl } of {1, 2, ..., n} we have:
Pr (Ai1 ∩ Ai2 ∩ ... ∩ Aik ) = Pr (Ai1 ∩ Ai2 ∩ ... ∩ Aik |Aj1 ∩ Aj2 ∩ ... ∩ Ajl ).
For example:
Pr (A2 ∩ A6 ∩ A8 ) = Pr (A2 ∩ A6 ∩ A8 |A1 ∩ A3 ∩ A5 ∩ A7 ).
Christopher Croke
Calculus 115
For two events A, B they are independent if Pr (A) = Pr (A|B).
We can see {A1 , A2 , ..., An } independent if for any two disjoint
subsets {i1 , i2 , ..., ik } and {j1 , j2 , ..., jl } of {1, 2, ..., n} we have:
Pr (Ai1 ∩ Ai2 ∩ ... ∩ Aik ) = Pr (Ai1 ∩ Ai2 ∩ ... ∩ Aik |Aj1 ∩ Aj2 ∩ ... ∩ Ajl ).
For example:
Pr (A2 ∩ A6 ∩ A8 ) = Pr (A2 ∩ A6 ∩ A8 |A1 ∩ A3 ∩ A5 ∩ A7 ).
{A1 , A2 , ..., An } are conditionally independent given B if for
every subset {Ai1 , Ai2 , ..., Aik } we have
Pr (Ai1 ∩ Ai2 ∩ ... ∩ Aik |B) = Pr (Ai1 |B)Pr (Ai2 |B)...Pr (Aik |B).
Christopher Croke
Calculus 115