Download Math 141 - Lecture 1: Conditional Probability

Math 141
Lecture 1: Conditional Probability
Albyn Jones1
1 Library
304
[email protected]
www.people.reed.edu/∼jones/courses/141
Albyn Jones
Math 141
Last Time
Definitions: Sample Space, Events
Albyn Jones
Math 141
Last Time
Definitions: Sample Space, Events
Axioms or Rules of probability
what are they?
Albyn Jones
Math 141
Last Time
Definitions: Sample Space, Events
Axioms or Rules of probability
what are they?
Consequences:
A ⊂ B =⇒ P(A) ≤ P(B)
P(A) = 1 − P(Ac )
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
Albyn Jones
Math 141
This Time
Conditional Probability
The Multiplication Rule
Independent Events
Albyn Jones
Math 141
Motivation
A standard probability puzzle!
Suppose I tell you that I have two children. You see me walking
in the park with a girl, whom I introduce as my daughter. What
is the probability that my other child is also a girl?
The Real Question: How should you update your probability
evaluations to account for new information?
Albyn Jones
Math 141
Sample Space and Events
Original Sample Space:
Ω = {{BB}, {GB}, {BG}, {GG}}
Albyn Jones
Math 141
Sample Space and Events
Original Sample Space:
Ω = {{BB}, {GB}, {BG}, {GG}}
Event Probabilities: Assume that each of those four
outcomes has probability 1/4.
Albyn Jones
Math 141
Sample Space and Events
Original Sample Space:
Ω = {{BB}, {GB}, {BG}, {GG}}
Event Probabilities: Assume that each of those four
outcomes has probability 1/4.
What event have we observed?
Albyn Jones
Math 141
Sample Space and Events
Original Sample Space:
Ω = {{BB}, {GB}, {BG}, {GG}}
Event Probabilities: Assume that each of those four
outcomes has probability 1/4.
What event have we observed?
At least one child is female!
{GB} ∪ {BG} ∪ {GG}
Albyn Jones
Math 141
The New Sample Space
The Conditional Sample Space: One event is ruled out,
leaving:
ΩC = {{GB}, {BG}, {GG}}
Albyn Jones
Math 141
The New Sample Space
The Conditional Sample Space: One event is ruled out,
leaving:
ΩC = {{GB}, {BG}, {GG}}
If the three events are still equally likely, then each has
probability 1/3.
Albyn Jones
Math 141
The New Sample Space
The Conditional Sample Space: One event is ruled out,
leaving:
ΩC = {{GB}, {BG}, {GG}}
If the three events are still equally likely, then each has
probability 1/3.
Conditional probability The probability that both are girls,
given we know at least one is a girl, is
P({GG}|{GB} ∪ {BG} ∪ {GG}) = 1/3
Albyn Jones
Math 141
The New Sample Space
The Conditional Sample Space: One event is ruled out,
leaving:
ΩC = {{GB}, {BG}, {GG}}
If the three events are still equally likely, then each has
probability 1/3.
Conditional probability The probability that both are girls,
given we know at least one is a girl, is
P({GG}|{GB} ∪ {BG} ∪ {GG}) = 1/3
The original (or marginal) probability of {GG} was 1/4.
Albyn Jones
Math 141
The Formal Version
Definition: Let A and B be events. Then the conditional
probability of A, given we know B occurs is
P(A | B) =
P(A ∩ B)
P(B)
if P(B) 6= 0.
Albyn Jones
Math 141
A Venn Diagram for Conditional Probability
The definition P(A | B) = P(A ∩ B)/P(B) restricts the sample
space to B, and rescales to give P(B|B) = 1:
A
A∩B
Albyn Jones
B
Math 141
The Puzzle Revisited
Let A be the event {GG}, ie I have two daughters. Let B be the
event that I have at least one daughter:
B = {GB} ∪ {BG} ∪ {GG}.
Then A ∩ B is
{GG} ∩ ({GB} ∪ {BG} ∪ {GG}) = {GG}
Albyn Jones
Math 141
The Computation
P(A | B) =
P(A ∩ B)
P(B)
=
P({GG} ∩ ({GB} ∪ {BG} ∪ {GG}))
P({GB} ∪ {BG} ∪ {GG})
=
P({GG})
P({GB} ∪ {BG} ∪ {GG})
=
P({GG})
P({GB}) + P({BG}) + P({GG})
=
1/4
= 1/3
1/4 + 1/4 + 1/4
Albyn Jones
Math 141
A Multiplication Rule
Rearrange the definition of conditional probability:
P(A | B) =
P(A ∩ B)
P(B)
multiplying both sides by P(B) we have
P(A ∩ B) = P(A | B) · P(B)
This gives us a tool for computing probabilities of more
complicated events!
Albyn Jones
Math 141
Application: Cards
Suppose we have a well shuffled standard deck (52 cards):
every card is equally likely to be in every position, and all
possible orders of the cards are equally likely.
Albyn Jones
Math 141
Application: Cards
Suppose we have a well shuffled standard deck (52 cards):
every card is equally likely to be in every position, and all
possible orders of the cards are equally likely.
What is the probability that the first card in the deck is an
ace?
Albyn Jones
Math 141
Application: Cards
Suppose we have a well shuffled standard deck (52 cards):
every card is equally likely to be in every position, and all
possible orders of the cards are equally likely.
What is the probability that the first card in the deck is an
ace?
What is the probability that the second card is an ace?
Albyn Jones
Math 141
Application: Cards
Suppose we have a well shuffled standard deck (52 cards):
every card is equally likely to be in every position, and all
possible orders of the cards are equally likely.
What is the probability that the first card in the deck is an
ace?
What is the probability that the second card is an ace?
What is the probability that the last card is an ace?
Albyn Jones
Math 141
Application: Cards
Suppose we have a well shuffled standard deck (52 cards):
every card is equally likely to be in every position, and all
possible orders of the cards are equally likely.
What is the probability that the first card in the deck is an
ace?
What is the probability that the second card is an ace?
What is the probability that the last card is an ace?
What is the probability that the first two cards on the top of
the deck are both aces?
Albyn Jones
Math 141
The Probability of Two Aces at the Top of the Deck
Let A1 be the event that the first card is an ace, and A2 be the
event that the second card is an ace.
Albyn Jones
Math 141
The Probability of Two Aces at the Top of the Deck
Let A1 be the event that the first card is an ace, and A2 be the
event that the second card is an ace.
P(A1 ) = 4/52 = 1/13
Albyn Jones
Math 141
The Probability of Two Aces at the Top of the Deck
Let A1 be the event that the first card is an ace, and A2 be the
event that the second card is an ace.
P(A1 ) = 4/52 = 1/13
P(A2 |A1 ) = 3/51 = 1/17
Albyn Jones
Math 141
The Probability of Two Aces at the Top of the Deck
Let A1 be the event that the first card is an ace, and A2 be the
event that the second card is an ace.
P(A1 ) = 4/52 = 1/13
P(A2 |A1 ) = 3/51 = 1/17
P(A1 ∩ A2 ) = P(A2 |A1 ) · P(A1 )
or
4·3
1
1
=
=
52 · 51
13 · 17
221
Albyn Jones
Math 141
More About Cards
Suppose we have a well shuffled standard 52 card deck.
Albyn Jones
Math 141
More About Cards
Suppose we have a well shuffled standard 52 card deck.
What is the probability that the first card is an ace given
that the second card is an ace?
Albyn Jones
Math 141
More About Cards
Suppose we have a well shuffled standard 52 card deck.
What is the probability that the first card is an ace given
that the second card is an ace?
What is the probability that the last card is an ace given
that the first card is an ace?
Albyn Jones
Math 141
More About Cards
Suppose we have a well shuffled standard 52 card deck.
What is the probability that the first card is an ace given
that the second card is an ace?
What is the probability that the last card is an ace given
that the first card is an ace?
What is the probability that the first card is an ace given the
last card is an ace?
Albyn Jones
Math 141
More About Cards
Suppose we have a well shuffled standard 52 card deck.
What is the probability that the first card is an ace given
that the second card is an ace?
What is the probability that the last card is an ace given
that the first card is an ace?
What is the probability that the first card is an ace given the
last card is an ace?
What is the probability that the second card is a king given
the first is an ace?
Albyn Jones
Math 141
Symmetry!
P(A ∩ B) = P(A|B) · P(B) = P(B|A) · P(A)
Albyn Jones
Math 141
Extensions of the Multiplication Rule
P(A ∩ B) = P(A|B) · P(B)
Assuming that P(A ∩ B ∩ C) > 0,
P(A ∩ B ∩ C) = P(A|B ∩ C) · P(B ∩ C) = P(A|B ∩ C) · P(B|C) · P(C)
The multiplication rule may be extended to arbitrary collections
of sets with non-zero probability:
P(A ∩ B ∩ C ∩ D) = etc.
Question: Why do we need the condtion that the joint
probability is positive?
Albyn Jones
Math 141
Examples
Question: What is the probability that the first three cards in a
well shuffled deck are all Aces?
Question: What is the probability that the first four cards in a
well shuffled deck are all hearts?
Albyn Jones
Math 141
Summary: Conditional Probability
All probabilities are conditional probabilities! We always
condition on the set of possible outcomes for an
experiment or measurement.
Sometimes new information allows us to revise our
probabilities. The formula for conditional probability tells us
how to do so.
Sometimes we can use conditioning arguments to evaluate
probabilities of complicated events!
Albyn Jones
Math 141
Independent Events
Independent events are events that contain no information
about each other: knowing that one has occured does not help
to predict whether the other will occur.
Definition: Events A and B are independent if
P(A|B) = P(A)
or
P(A ∩ B) = P(A) · P(B)
Albyn Jones
Math 141
Example
Toss a fair coin (P(H) = 1/2 = P(T )) twice, independently.
Albyn Jones
Math 141
Example
Toss a fair coin (P(H) = 1/2 = P(T )) twice, independently.
P(H1 ∩ H2 ) = P(H1 ) · P(H2 ) =
Albyn Jones
1
2
·
1
2
Math 141
=
1
4
Example
Toss a fair coin (P(H) = 1/2 = P(T )) twice, independently.
P(H1 ∩ H2 ) = P(H1 ) · P(H2 ) =
P(H1 ∩ T2 ) = P(H1 ) · P(T2 ) =
Albyn Jones
1
2
1
4
·
1
2
Math 141
=
1
4
Example
Toss a fair coin (P(H) = 1/2 = P(T )) twice, independently.
P(H1 ∩ H2 ) = P(H1 ) · P(H2 ) =
P(H1 ∩ T2 ) = P(H1 ) · P(T2 ) =
P(T1 ∩ H2 ) = P(T1 ) · P(H2 ) =
Albyn Jones
1
2
1
4
1
4
·
1
2
Math 141
=
1
4
Example
Toss a fair coin (P(H) = 1/2 = P(T )) twice, independently.
1
2
P(H1 ∩ T2 ) = P(H1 ) · P(T2 ) = 41
P(T1 ∩ H2 ) = P(T1 ) · P(H2 ) = 41
P(T1 ∩ T2 ) = P(T1 ) · P(T2 ) = 41
P(H1 ∩ H2 ) = P(H1 ) · P(H2 ) =
Albyn Jones
·
1
2
Math 141
=
1
4
Example
Toss a fair coin (P(H) = 1/2 = P(T )) twice, independently.
1
2
P(H1 ∩ T2 ) = P(H1 ) · P(T2 ) = 41
P(T1 ∩ H2 ) = P(T1 ) · P(H2 ) = 41
P(T1 ∩ T2 ) = P(T1 ) · P(T2 ) = 41
P(H1 ∩ H2 ) = P(H1 ) · P(H2 ) =
·
1
2
P(H1 ∩ T1 ) =?
Albyn Jones
Math 141
=
1
4
Important Point!
In general, disjoint events are not independent: if P(A) 6= 0, and
P(A ∩ B) = 0, then
P(A|B) =
P(A ∩ B)
= 0 6= P(A)
P(B)
Similarly, independent events with non-zero probability are not
disjoint.
If A and B are disjoint, knowing that A has occurred tells you
that B could not occur, which is informative!
Albyn Jones
Math 141
Pairwise vs Complete Independence
For 3 or more events, pairwise independence is not the same
as complete (or mutual) independence. For complete
independence, all possible combinations must be independent,
including:
P(A|B ∩ C) = P(A)
P(B|A ∩ C) = P(B)
P(C|A ∩ B) = P(C)
P(A ∩ B ∩ C) = P(A)P(B)P(C)
In other words, no combination of events gives information
about any other combination of events.
Albyn Jones
Math 141
Example: pairwise independent but not mutually
independent events
Roll 2 fair dice independently, say one is red and the other blue.
Albyn Jones
Math 141
Example: pairwise independent but not mutually
independent events
Roll 2 fair dice independently, say one is red and the other blue.
Let R1 be the event ‘rolled 1 on the red die’.
Albyn Jones
Math 141
Example: pairwise independent but not mutually
independent events
Roll 2 fair dice independently, say one is red and the other blue.
Let R1 be the event ‘rolled 1 on the red die’.
Let B1 be the event ‘rolled 1 on the blue die’.
Albyn Jones
Math 141
Example: pairwise independent but not mutually
independent events
Roll 2 fair dice independently, say one is red and the other blue.
Let R1 be the event ‘rolled 1 on the red die’.
Let B1 be the event ‘rolled 1 on the blue die’.
Let S7 be the event ‘The sum of the two rolls is 7’.
Albyn Jones
Math 141
Example: pairwise independent but not mutually
independent events
Roll 2 fair dice independently, say one is red and the other blue.
Let R1 be the event ‘rolled 1 on the red die’.
Let B1 be the event ‘rolled 1 on the blue die’.
Let S7 be the event ‘The sum of the two rolls is 7’.
What is
P(R1 ∩ B1 |S7 )?
Albyn Jones
Math 141
Our puzzle again
Assuming that children are equally likely to be male or female,
and that the sex of successive births are independent, then
P(G1 ∩ G2 ) = P(G1 ) · P(G2 ) =
Albyn Jones
Math 141
1 1
1
· =
2 2
4
Our puzzle again
Assuming that children are equally likely to be male or female,
and that the sex of successive births are independent, then
P(G1 ∩ G2 ) = P(G1 ) · P(G2 ) =
What is P(G2 |G1 )?
Albyn Jones
Math 141
1 1
1
· =
2 2
4
Our puzzle again
Assuming that children are equally likely to be male or female,
and that the sex of successive births are independent, then
P(G1 ∩ G2 ) = P(G1 ) · P(G2 ) =
What is P(G2 |G1 )?
What is P(G1 |G2 )?
Albyn Jones
Math 141
1 1
1
· =
2 2
4
Our puzzle again
Assuming that children are equally likely to be male or female,
and that the sex of successive births are independent, then
P(G1 ∩ G2 ) = P(G1 ) · P(G2 ) =
What is P(G2 |G1 )?
What is P(G1 |G2 )?
What is P(G1 ∩ G2 |‘at least one girl’)?
Albyn Jones
Math 141
1 1
1
· =
2 2
4
Summary
Conditional Probability:
P(A|B) =
Albyn Jones
P(A ∩ B)
P(B)
Math 141
Summary
Conditional Probability:
P(A|B) =
P(A ∩ B)
P(B)
The multiplication Rule:
P(A ∩ B) = P(A|B)P(B)
Albyn Jones
Math 141
Summary
Conditional Probability:
P(A|B) =
P(A ∩ B)
P(B)
The multiplication Rule:
P(A ∩ B) = P(A|B)P(B)
Independent events:
P(A|B) = P(A)
or
Albyn Jones
P(A ∩ B) = P(A)P(B)
Math 141