Download Math 425 Introduction to Probability Lecture 10

Multiplication Rule
Multiplication Rule for 2 events
Math 425
Introduction to Probability
Lecture 10
Lemma (Multiplication Rule)
For any events E and F (where P (F ) > 0),
Kenneth Harris
[email protected]
P (E ∩ F ) = P (E) · P(F | E)
Department of Mathematics
University of Michigan
February 9, 2009
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
February 9, 2009
1 / 32
Kenneth Harris (Math 425)
Multiplication Rule
February 9, 2009
3 / 32
Multiplication Rule
Multiplication Rule for 3 events
Example: 3 events
We can extend the Multiplication Rule to three events.
Example
Lemma
For any events E, F , G (provided P (E ∩ F ∩ G) > 0)
An urn is filled with 6 red balls, 5 blue balls, and 4 green balls. Three
balls chosen at random are removed from the urn.
What is the probability that the balls are of the same color?
P (E ∩ F ∩ G) = P (E) · P(F | E) · P(G | E ∩ F )
We are interested in the events (where i = 1, 2, 3)
Ri : ith ball drawn is red,
Proof. Use the Mutiplication Rule twice,
P (E ∩ F ∩ G)
Math 425 Introduction to Probability Lecture 10
Bi : ith ball drawn is blue,
= P (E ∩ F ) · P(G | E ∩ F )
Gi : ith ball drawn is green.
= P (E) · P(F | E) · P(G | E ∩ F )
C: three balls are the same color.
We need P (E ∩ F ∩ G) 6= 0 to ensure the conditional probabilities exist.
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
February 9, 2009
4 / 32
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
February 9, 2009
5 / 32
Multiplication Rule
Multiplication Rule
Example – continued
General multiplication Rule
Urn: 6 red, 5 blue, and 4 green. Use the Multiplication Rule,
P (R1 ∩ R2 ∩ R3 )
P (B1 ∩ B2 ∩ B3 )
P (G1 ∩ G2 ∩ G3 )
The Multiplication rule is the probabilistic version of the product rule
= P (R1 ) · P(R2 | R1 ) · P(R3 | R1 ∩ R2 )
6
5
4
4
=
·
·
=
15 14 13
91
for counting.
Theorem (Generalized Multiplication Rule)
Let E1 , E2 , . . . , En be any events such that
= P (B1 ) · P(B2 | B1 ) · P(B3 | B1 ∩ B2 )
5
4
3
2
·
·
=
=
15 14 13
91
P (E1 ∩ E2 ∩ · · · ∩ En ) > 0.
Then
= P (G1 ) · P(G2 | G1 ) · P(G3 | G1 ∩ G2 )
4
3
2
4
=
·
·
=
15 14 13
455
P (E1 ∩ E2 ∩ . . . ∩ En ) = P (E1 ) · P(E2 | E1 ) · P(E3 | E1 ∩ E2 ) · · ·
· · · P(En | E1 ∩ E2 ∩ . . . ∩ En−1 ).
Since these events are mutually exclusive,
P (C) =
Kenneth Harris (Math 425)
(See Ross p. 71.)
4
2
4
34
+
+
=
≈ 0.0747.
91 91 455
455
Math 425 Introduction to Probability Lecture 10
February 9, 2009
6 / 32
Kenneth Harris (Math 425)
Multiplication Rule
February 9, 2009
7 / 32
Multiplication Rule
Example: 6 events
Example – solution
Solution. We solved this before (Lecture 5) by counting using the
Product Rule for counting. We conditionalize here.
Example
In Pick-Six Lottery: A person purchases a ticket, and can choose 6
distinct numbers in the set {1, 2, 3, . . . , 49}.
Later a Lottery Machine picks 6 distinct numbers at random in the
set {1, 2, 3, . . . , 49}.
A winning ticket is one which matches the six numbers chosen by
the Machine (in any order of selection).
+ What are the odds of winning with one ticket?
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
Math 425 Introduction to Probability Lecture 10
February 9, 2009
8 / 32
Let E be the event that there are i matches. We want to compute
i
P (E6 ) = P (E1 ∩ E2 ∩ . . . ∩ E6 )
= P (E1 ) · P(E2 | E1 ) · · · P(E6 | E1 ∩ . . . ∩ E5 )
6 5 4 3 2 1
=
·
·
·
·
·
49 48 47 46 45 44
1
=
13, 983, 816
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
February 9, 2009
9 / 32
Independence and the Product Rule
Independence and the Product Rule
Dependence
Independence
Sometimes, changes in the conditions of an experiment change the
Sometimes, changes in the conditions of an experiment have no
probability of some outcomes.
effect on the probability of some outcomes.
Example. An urn has 7 red balls and 5 blue balls. The balls are well
mixed. A ball is drawn, its color is noted and put aside.
Compare the probability that the second ball is red (R2 ) given that
the first drawn ball is red (R1 ) versus that it is blue (R1c ).
Example. An urn has 7 red balls and 5 blue balls. The balls are well
mixed. A ball is drawn, its color is noted and returned to the urn, which
is again well mixed.
Compare the probability that the second ball is red (R2 ) given that
the first drawn ball is red (R1 ) versus that it is blue (R1c ).
P(R2 | R1 ) =
6
11
P(R2 | R1c ) =
7
11
P(R2 | R1 ) =
The probability that the second ball is red is
P (R2 )
Kenneth Harris (Math 425)
P(R2 | R1c )
= P(R2 | R1 ) · P (R1 ) +
6
7
7
5
7
=
·
+
·
=
11 12 11 12
12
7
12
P(R2 | R1c ) =
7
12
The probability that the second ball is red is
·
P (R1c )
Math 425 Introduction to Probability Lecture 10
February 9, 2009
P (R2 )
11 / 32
Independence and the Product Rule
Kenneth Harris (Math 425)
= P(R2 | R1 ) · P (R1 ) + P(R2 | R1c ) · P (R1c )
7
7
5
7
7
=
·
+
·
=
12 12 12 12
12
Math 425 Introduction to Probability Lecture 10
February 9, 2009
12 / 32
Independence and the Product Rule
Independence
Independence and the Product Rule
Suppose E and F are events with P(E | F ) = P(E | F ).
By the Partition Rule
c
P (E)
Definition (Product Rule)
Events E and F and independent if and only if
= P(E | F )P (F ) + P(E | F c )P (F c )
= x · P (F ) + P (F c ) = x
where x is P(E | F ) or P(E | F c )
P (E ∩ F ) = P (E) · P (F ).
Equivalently, E and F are independent if and only if
So, P (E) = P(E | F ) and P (E) = P(E | F c ).
By the Multiplication Rule
P(E | F ) = P (E) = P(E | F c )
P (E ∩ F ) = P(E | F ) · P (F ) = P (E) · P (F ).
Events which are not independent are said to be dependent.
So, P (E ∩ F ) = P (E) · P (F )
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
February 9, 2009
13 / 32
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
February 9, 2009
14 / 32
Independence and the Product Rule
Independence and the Product Rule
Proof of Equivalence
Example
We have already shown the
P(E | F ) = P(E | F c )
⇒
P(E | F ) = P (E) and P (E ∩ F ) = P (E) · P (F ).
Conversely, suppose P (E ∩ F ) = P (E) · P (F ).
By the Conditioning Rule
P(E | F ) =
Example
A standard 52 card deck is well shuffled. Are the following events
independent:
E: Draw a ♠,
P (E ∩ F )
P (E) · P (F )
=
= P (E).
P (F )
P (F )
F : Draw an ace?
By the Conditioning and Partition Rules,
P(E | F c ) =
=
=
Kenneth Harris (Math 425)
P (E ∩ F c )
P (F c )
P (E) − P (E ∩ F )
c
since P (E) = P (E ∩ F ) + P (E ∩ F )
P (F c )
P (E) · (1 − P (F ))
= P (E)
P (F c )
Math 425 Introduction to Probability Lecture 10
February 9, 2009
Solution. E and F are independent.
P(E | F ) =
15 / 32
Kenneth Harris (Math 425)
Independence and the Product Rule
1
4
P(E | F c ) =
12
1
=
48
4
Math 425 Introduction to Probability Lecture 10
February 9, 2009
16 / 32
Extended Product Rule
Example
Extended Product Rule
Example
Definition (Extended Product Rule)
Three dice are thrown. Are the following events independent:
The events E1 , E2 , . . . (possibly infinitely many events) are independent
if and only if
E6 : Throw a six on at least one die,
P (Ei1 ∩ Ei2 ∩ · · · ∩ Ein ) = P (Ei1 ) · P (Ei2 ) · · · P (Ein )
S16 : The sum of the dice is 16?
for any finite subset of indices i1 , i2 , . . . , in .
Solution. E and F are dependent.
P(S16 | E6 ) > 0
Challenge: verify P(S16 | E6 ) =
Kenneth Harris (Math 425)
Equivalently, the events E1 , . . . , E2 , . . . are independent if and only if
P(S16 | E6c ) = 0
P(Ein | Ei1 ∩ Ei2 ∩ · · · ∩ Ein−1 ) = P (Ein ),
6
91 .
Math 425 Introduction to Probability Lecture 10
for any finite (n ≥ 2) subset of indices i1 , i2 , . . . , in .
February 9, 2009
17 / 32
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
February 9, 2009
19 / 32
Extended Product Rule
Conditional Independence
Example
Conditional Independence
Example
It is possible that two events E and F are not independent,
A sequence of fair coins is flipped n times, and each outcome is
equiprobable. Let Ei be the event that the ith flip is heads.
Are the events E1 , E2 , . . . , En equiprobable?
but they become so on the assumption that a third event G occurs.
Definition
Events E and F are conditionally independent given G if
Solution. They are independent: Fix any k + 1 events. Then
P(Eik +1 | Ei1 ∩ · · · ∩ Eik ) =
P (Eik +1 ) =
Kenneth Harris (Math 425)
2n−k −1
2n−k
2n−1
2n
=
=
P(E ∩ F | G) = P(E | G) · P(F | G).
1
2
Equivalently,
P(E | F ∩ G) = P(E | G).
1
.
2
Math 425 Introduction to Probability Lecture 10
February 9, 2009
20 / 32
Kenneth Harris (Math 425)
Conditional Independence
22 / 32
Example
Suppose P(E | F ∩ G) = P(E | G). Then,
P (E ∩ F ∩ G)
= P(E | G) Conditioning Rule for P(E | F ∩ G)
P (F ∩ G)
P (E ∩ F ∩ G) = P(E | G) · P (F ∩ G)
= P(E | G) · P(F | G)
February 9, 2009
Conditional Independence
Proof of Equivalence
P(E ∩ F | G)
Math 425 Introduction to Probability Lecture 10
Example
Suppose you roll a red and blue die. Consider the events
divide both sides by P (G)
L2 : lower score is 2,
H5 : higher score is 5.
Suppose P(E ∩ F | G) = P(E | G) · P(F | G). Then,
P (E ∩ F ∩ G)
P (G)
=
P (E ∩ F ∩ G)
P (F ∩ G)
=
P(E | G)
P(E | F ∩ G)
=
P(E | G)
Kenneth Harris (Math 425)
P (F ∩ G)
P(E | G) ·
P (G)
Math 425 Introduction to Probability Lecture 10
D: one die is greater than 3 and one die is less than 3.
Then,
(a) L2 and H5 are not independent,
Conditioning Rule
(b) L2 and H5 are conditionally independent given D.
February 9, 2009
23 / 32
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
February 9, 2009
24 / 32
Conditional Independence
Conditional Independence
Example
Example
(a). L2 and H5 are not independent
P (L2 ∩ H5 ) =
P (L2 ) · P (H5 ) =
2
1
=
36
18
Example
Suppose you roll a red and blue die. Consider the events
R2 : a 2 on the red die,
1
9 9
·
=
36 36
16
B2 : a 2 on the blue die,
D: one die is greater than 3 and one die is less than 3.
(b). L2 and H5 are conditionally independent given D
Kenneth Harris (Math 425)
P(L2 | D) =
1
2
P(L2 | H5 ∩ D) =
1
2
Then,
(a) R2 and B2 are independent,
(b) R2 and B2 are not conditionally independent given D.
Math 425 Introduction to Probability Lecture 10
February 9, 2009
25 / 32
Kenneth Harris (Math 425)
Conditional Independence
Math 425 Introduction to Probability Lecture 10
February 9, 2009
26 / 32
Example: Baseball
Example
Example: Baseball
Compare to the Problem of points, Example 3.4j of Ross, p. 95.
(a). R2 and B2 are independent:
1
P (R2 ∩ B2 ) =
= P (R2 ) · P (B2 ).
36
(b). R2 and B2 are not conditionally independent given D:
P(R2 | D) =
P (R2 ∩ D)
=
P (D)
3
36
12
36
=
Example
1
4
The Cubs (!!) and White Sox are playing in the World Series. The
Cubs win each game with probability 0.6 (independently of the games
played). What is the probability that the Cubs win the Series. (The first
team to win four games wins the series.)
P(R2 | B2 ∩ D) = 0
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
February 9, 2009
27 / 32
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
February 9, 2009
29 / 32
Example: Baseball
Example: Baseball
Example: Baseball
Example: Baseball
Method 1. (Due to Fermat)
Since the first to four wins takes the Series, there are at most 7
games (4 Cub wins to 3 Sox wins). Fermat takes the sample space to
be sequences of length 7:
If X is an outcome in the sample space and has k W s (so, 7 − k
Ls), then
P (X ) = (0.6)k (0.4)7−k
(g1 , g2 , g3 , g4 , g5 , g6 , g7 )
where gi = W , L
Any sequence with 4 W s is a Cub win, otherwise it is a Sox win (there
are at least 4 Ls).
Not all sequences represent actual outcomes, nor are all
sequences equally likely.
Why does it not matter to extend a real Series play with phantom
games?
In this sample space, any sequence with at least 4 W s is a Cubs
Series win.
P (Cubs win) =
7 X
7
n=4
n
· (0.6)n (0.4)7−n ≈ 0.71.
The phantom games do not change the probability: If the Cubs win the
series in four games, they win every extension with phantom games as well.
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
February 9, 2009
30 / 32
Example: Baseball
Example: Baseball
Method 2. (Due to Pascal)
This method allows us to treat the sample space as sequences whose
length is at most 7, with 4 W s or 4 Ls. Let
Wn,m be the event that the Cubs win the series when they have n
wins and the Sox have m wins. (where n, m ≤ 4)
At the start of the Series, we want to compute W0,0 .
W (0, 0) can be determined from the following 3 conditions:
(a) P (W4,n ) = 1, where n ≤ 3,
(b) P (Wn,4 ) = 0 where m ≤ 3,
(c) When the teams have played n + m games, and both n, m < 4,
then
P (Wn,m ) = 0.6 · P (Wn+1,m ) + 0.4 · P (Wn,m+1 ).
For example,
The next game is either a Cubs win or Sox win.
P (W3,3 ) = p
P (W3,2 ) = 0.6 + 0.4 · 0.6 = 0.84P (W2,3 ) = 0.62 = 0.36
Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
TheKenneth
method
gives the solution
W0,0 ≈ 0.71.
February 9, 2009
32 / 32
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 10
February 9, 2009
31 / 32