Download Definition and Calculus of Probability

Outline
Conditional Probability and Independence
Definition and Calculus of Probability
Michael Akritas
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
I
In experiments with multivariate outcome variable, knowledge
of the value of one variable may help predict another.
I
For now, the word prediction will mean update the
probabilities of events regarding the other variable.
I
The updated probabilities are called conditional
probabilities.
For example,
1. Knowing a man’s height helps update the probability that he
weighs over 170lb.
2. Knowing which assembly line a product came from, helps
update the probability that it has a particular defect.
3. Knowing a person’s education level helps update the
probability of that person being in a certain income category.
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Given partial information about the outcome of the experiment,
results in a reduction of either the sample space or the number of
eligible units. For example:
Example
P(A
B )
.1
A
P(A
.3
B)
P(A
.5
U
I
U
I
If the outcome of rolling a die is known to be even, what is
the probability it is a 2?
If the selected card from a deck is known to be a figure card,
what is the probability it is a king?
Given event A = {household subscribes to paper 1}, what is
the probability of B = {household subscribes to paper 2}?
U
I
B
B)
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Definition
The conditional probability of the event A given the information
that event B has occurred is denoted by P(A|B) and equals
P(A|B) =
P(A ∩ B)
, provided P(B) > 0
P(B)
Note that P(B|B) = 1, which highlights the fact that when we are
given the information that B occurred, B becomes the new sample
space.
Proposition
The set function P(·|B) satisfies the three axioms of probability.
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Example
In the card game of bridge, the 52 cards are dealt out equally to 4
players called East, West, North and South. Given that North and
South have a total of 8 spades among them, what is the
probability that East has 3 of the remaining 5 spades? Use the
reduced sample space defined by the given information.
Solution: In the reduced sample space experiment there are 26
cards containing 5 spades and will be divided randomly between
East and West. Thus, the probability that East ends up with 3 of
the remaining 5 spaces is
5 21
3
10
26
13
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Do each of the following examples in two ways: a) by considering
the reduced sample space, and b) by applying the formula for the
conditional probability.
Example
1. Roll a die twice. Given that the first roll results in 3 find the
probability the sum is 8.
2. Roll a die twice. Given that one of the roll results in 3 find the
probability the sum is 8.
3. An urn contains r red and b blue balls. n balls (n ≤ r + b) are
selected at random and without replacement. Given that k of
the n balls are blue, what is the probability that the first ball
chosen is blue?
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
The Multiplication Rule
The definition of P(A|B) yields the formula:
P(A ∩ B) = P(A|B)P(B) or P(A ∩ B) = P(B|A)P(A)
The rule extends to more than two events. For example,
P(A ∩ B ∩ C ) = P(A)P(B|A)P(C |A ∩ B)
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
The Multiplication Rule: EXAMPLES
1. 40% of bean seeds come from supplier A and 60% come from
supplier B. Seeds from supplier A have 50% germination rate
while those from supplier B have a 75% rate. What is the
probability that a randomly selected seed came from supplier
A and will germinate?
ANSWER: P(A ∩ G ) = P(G |A)P(A) = 0.5 × 0.4 = 0.2
2. Three players are dealt a card in succession. What is the
probability that the 1st gets an ace, the 2nd gets a king, and
the 3rd gets a queen?
ANSWER:
P(A∩B ∩C ) = P(A)P(B|A)P(C |A∩B) =
Michael Akritas
4 4 4
52 51 50
= 0.000454
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Example
An urn contains 8 red, each having weight r , and 4 white balls,
each having weight w . Two are selected without replacement.
What is the probability both are red if at each draw:
1. each ball is equally likely, and
2. the probability of each ball equals its weight divided by the
sum of weights of all balls currently in the urn.
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Example (The Mens’ Hats Problem)
In the Men’s Hats Problem, find the probability that exactly k of
the N men get their own hat.
Solution: Let Fi = {man i gets his own hat},
E = {men 1, . . . , k get their own hats} = F1 ∩ · · · ∩ Fk , and
G = {men k + 1 . . . , N do not get their own hats}. Then,
P(E ∩ G ) = P(E )P(G |E ) and
P(E ) = P(F1 )P(F2 |F1 )P(F3 |F1 ∩ F2 ) · · · P(Fk |F1 ∩ · · · Fk−1 )
=
1
1 1
1
(N − k)!
···
=
N N −1N −2
N −k +1
N!
For P(G |E ) see page 23. Thus, the final answer is:
N (N−k)! PN−k
i
i=0 (−1) /i!
k
N!
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Independence of two events
The formula for the probability of A ∪ B yields
P(A ∩ B) = P(A) + P(B) − P(A ∪ B).
I
I
I
A simpler formula for P(A ∩ B) is possible if A and B are
independent.
Independent events arise (quite often but not always) in
connection with independent experiments or independent
repetitions of the same experiment. Thus there is no
mechanism through which the outcome of one experiment will
influence the outcome of the other. For example, two rolls of
a die.
For independent events,
P(A ∩ B) = P(A)P(B).
This also serves as the definition of independent events.
Michael Akritas
Definition and Calculus of Probability
Independence
The Law of Total Probability
Bayes Theorem
Outline
Conditional Probability and Independence
Example
Toss a coin twice. To find the probability of two heads (Hs), we
can argue that, since the two tosses are independent,
P(H in toss 1 ∩ H in toss 2) =
1
11
=
22
4
Alternatively, we can say that since
P(H in toss 1 ∩ H in toss 2) =
1
= P(H in toss 1)P(H in toss 1),
4
the events ”H in toss 1” and ”H in toss 2” are independent
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Example
1. Select a card at random from a deck of 52 cards. Let
E = {the card is an ace}, and F = {the card is a spade}. Are
E and F independent?
2. Roll a die twice. Let
E = {the sum of the two outcomes is 6}, and
F = {the first roll results in 4}. Are E and F independent?
3. Same as above except that
E = {the sum of two outcomes is 7}.
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Proposition
If E and F are independent, so are E and F c .
Proof:
P(E ∩ F c ) = P(E ) − P(E ∩ F ) = P(E )(1 − P(F )) = P(E )P(F c ).
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Independence of Multiple Events
Definition
The events A1 , . . . , An are mutually independent if
P(Ai1 ∩ Ai2 ∩ . . . ∩ Aik ) = P(Ai1 )P(Ai2 ) . . . P(Aik ) for any
sub-collection Ai1 , . . . , Aik of k events chosen from A1 , . . . , An
Example
Consider rolling a die and define the events A = {1, 2, 3},
B = {3, 4, 5}, C = {1, 2, 3, 4}. Verify that
P(A ∩ B ∩ C ) = P(A)P(B)P(C ), but that A, B, C are not
mutually independent.
Solution: First, A ∩ B ∩ C = {3}, so P(A ∩ B ∩ C ) = 16 =
Next, A ∩ B = {3}, so P(A ∩ B) = 16 6= 12 21 .
Michael Akritas
Definition and Calculus of Probability
114
2 2 6.
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Example
Roll a die twice. Let E = {the sum of the two outcomes is 7},
F = {the first roll results in 4}, and
G = {the second roll results in 3}. Are E , F and G independent?
ANSWER: E , F and G pairwise independent but
P(E |F ∩ G ) = 1.
So they are not independent.
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Example
1. A system of n components connected in series fails if one
component fails. If component i fails with a probability pi ,
independently of the others, what is the probability that the
system fails?
2. What if the n components of the system are connected in
parallel?
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Example
Two dice are rolled and the sum of the two outcomes is recorded.
What is the probability that 5 happens before 7?
Solution: If En =
{no 5 or 7 appear on the first n − 1 rolls and a 5 appears on the nth},
then
P(∪∞
i=1 En ) =
∞
X
P(En )
i=1
=
∞
X
i=1
Michael Akritas
(1 −
2
10 n−1 4
)
=
36
36
5
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Example
There are n types of coupons, and each new one is independently
of type i with probability pi . If each of k shoppers collects a
coupon let Ai = {at least one type i coupon has been collected}.
Find P(Ai |Aj ).
Solution: Use independence and
P(Ai ∩ Aj ) = P(Ai ) + P(Aj ) − P(Ai ∪ Aj )
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Let the events A1 , A2 . . . , Ak be disjoint and make up the entire
sample space, and let B denote an event whose probability we
want to calculate, as in the figure
A1
B
A2
A3
A4
If we know P(B|Aj ) and P(Aj ) for all j = 1, 2, . . . , k, the Law of
Total Probability gives [see (3.4), p.73 and (3.1), p.65 in Ross]
P(B) = P(A1 )P(B|A1 ) + · · · + P(Ak )P(B|Ak )
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
The Law of Total Probability: EXAMPLES
1. 40% of bean seeds come from supplier A and 60% come from
supplier B. Seeds from supplier A have 50% germination rate
while those from supplier B have a 75% rate. What is the
probability that a randomly selected seed will germinate?
ANSWER: P(G ) = P(A)P(G |A) + P(B)P(G |B)
= 0.4 × 0.5 + 0.6 × 0.75 = 0.65
2. Three players are dealt a card in succession. What is the
probability that the 2nd gets a king?
4
Why?
ANSWER:
52
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Example
Two dice are rolled and the sum of the two outcomes is recorded.
What is the probability that 5 happens before 7?
Solution: Let B be the desired event, A1 = {first roll results in 5},
A2 = {first roll results in 7},
A3 = {first roll results in neither 5 not 7}. Then
P(B) = P(B|A1 )P(A1 ) + P(B|A2 )P(A2 ) + P(B|A3 )P(A3 )
= P(A1 ) + 0 + P(B)P(A3 ).
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Example (The Monte Hall Problem)
In the game Let’s Make a Deal, the host asks a participant to
choose one of the doors A, B or C . Behind one of the doors is a
big prize. The participant selects door A. The host then opens
door B showing that the big prize is not behind it. The host asks
the participant to either
1. stick with his/her original choice, or
2. select the other of the remaining two closed doors.
Find the probability that the participant will win the big prize for
each of the strategies a) and b).
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Solution of the Monte Hall Problem Let
Ai = {prize is behind door i}, i = 1, 2, 3, and set
B1 = {player wins when choosing door 1 and does not change},
B2 = {player wins when choosing door 1 and then changes}.
Then,
P(B1 ) =
3
X
P(B1 |Ai )P(Ai ) = 1 ×
i=1
P(B2 ) =
3
X
1
1
+0+0=
3
3
P(B2 |Ai )P(Ai ) = 0 + 1 ×
i=1
Michael Akritas
1
2
1
+1× = .
3
3
3
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Consider events B and A1 , . . . , Ak as in the Law of Total
Probability. Now, however, we ask a different question:
I
Given that B has occurred, what is the probability that a
particular Aj has occurred?
The answer is provided by the Bayes theorem:
P(Aj |B) =
P(Aj ∩ B)
P(Aj )P(B|Aj )
= Pk
P(B)
j=1 P(Ai )P(B|Ai )
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
Bayes Theorem: EXAMPLES
1. 40% of bean seeds come from supplier A and 60% come from
supplier B. Seeds from supplier A have 50% germination rate
while those from supplier B have a 75% rate. Given that a
randomly selected seed germinated, what is the probability
that it came from supplier A?
ANSWER:
P(A|G ) =
0.2
P(A)P(G |A)
=
P(A)P(G |A) + P(B)P(G |B)
0.65
2. Given that the 2nd player got an ace, what is the probability
that the 1st got an ace?
3
ANSWER:
(Why?)
51
Michael Akritas
Definition and Calculus of Probability
Outline
Conditional Probability and Independence
Independence
The Law of Total Probability
Bayes Theorem
EXAMPLE: Law of Total Prob. and Bayes Theorem
Seventy percent of the light aircraft that disappear while in flight
in a certain country are subsequently discovered. Of the aircraft
that are discovered, 60% have an emergency locator, whereas 10%
of the aircraft not discovered have such a locator. Suppose a light
aircraft has disappeared.
1. What is the probability that it has an emergency locator and
it will not be discovered? Answer: 0.1 × 0.3 = 0.03
2. What is the probability that it has an emergency locator?
Answer: 0.1 × 0.3 + 0.6 × 0.7 = 0.45
3. If it has an emergency locator, what is the probability that it
will not be discovered? Answer: 0.03/0.45
Michael Akritas
Definition and Calculus of Probability