Download E - Didem Kivanc

OKAN
UNIVERSITY
FACULTY OF ENGINEERING AND ARCHITECTURE
MATH 265 Probability and Random Processes
03 Conditional Probability and Independence
Fall 2011
Yrd. Doç. Dr. Didem Kivanc Tureli
[email protected]
[email protected]
4/10/2011
Lecture 3
1
Conditional Probability Question
• Question : I throw a die once. I get a three. I am going to
throw the die a second time. What is the probability that the
sum of the two throws is 8?
• Experiment: Throw 2 dice
• Sample Space:
– S = {(1,1), (1,2), (1,3), (1,4), (1,5), (1,6),
(2,1), (2,2), (2,3), (2,4), (2,5), (2,6),
(3,1), (3,2), (3,3), (3,4), (3,5), (3,6),
(4,1), (4,2), (4,3), (4,4), (4,5), (4,6),
(5,1), (5,2), (5,3), (5,4), (5,5), (5,6),
(6,1), (6,2), (6,3), (6,4), (6,5), (6,6)}
4/10/2011
Lecture 3
2
Answer
• Define events E and F
• E = Getting a sum of 8
= {(1,7), (2,6), (3,5), (4,4), (5,3), (6,2), (7,1)}
• F = Getting a 3 on the first throw
={(3,1), (3,2), (3,3), (3,4), (3,5), (3,6)}
• Event of getting a sum of 8 when the result of the first throw
is 3:
• EF={(3,5)}
• Probability of getting a sum of 8 when the result of the first
throw is 3 = Number of elements in EF  1
Number of elements in E 6
4/10/2011
Lecture 3
3
Graphical Interpretation of Conditional Probability
• You already know you are inside set E.
• What is the probability that you are inside set EF?
S
E
F
4/10/2011
Lecture 3
4
Definition
• If P(F) > 0 then
P( EF )
PE | F  
P( F )
“Probability that event E will occur given that event F has occurred”
“Probability of event E given event F”
“Probability of E given F”
4/10/2011
Lecture 3
5
Examples
• A coin is flipped twice. If we assume that all four points in the
sample space S = {(H,H), (H,T), (T,H), (T,T)}, are equally likely,
what is the conditional probability that both flips result in
heads, given that the first flip does?
4/10/2011
Lecture 3
6
Examples
• E = Event that both flips result in heads = {(H, H)}
• F = Event that first flip results in heads = {(H, H), (H, T)}
• Notice that the intersection of sets E and F gives set E, that is,
EF = E. Therefore, P(EF)=P(E)
• Conditional probability that both flips result in heads, given
that the first flip does is P(E|F)
P( EF ) P( E ) 1/ 4 1
PE | F  



P( F ) P( F ) 1/ 2 2
4/10/2011
Lecture 3
7
Examples
• You are a contestant on a quiz show. On this show, an urn
contains 10 white, 5 yellow and 10 black marbles. A marble
will be chosen at random from the urn. If you guess the color
of the marble correctly, then you will win 1 million TL.
• The marble will be chosen from the bucket behind a curtain,
in a darkened room.
• You have a friend in the production team who wants to help
you to win the money. But he can’t see very well in the dark.
All he can see is that the ball that came out was light colored,
either white or yellow but definitely not black.
• Should you guess white or yellow? What is the probability
that the ball chosen is yellow?
4/10/2011
Lecture 3
8
Examples
W1
W1
W2
W2
W1
4/10/2011
W4
W7
W3
W3
W8
W9
W6
W10
W5
W4
W2
W3
W5
W6
W7
W8
W9
W4
W5
W10
Lecture 3
9
Examples
• E = Event that chosen ball is not black
– E = {W1, W2, W3, W4, W5, W6, W7, W8, W9, W10,
Y1, Y2, Y3, Y4, Y5}
• F = Event that chosen ball is yellow = {Y1, Y2, Y3, Y4, Y5}
• As with the previous example, EF = F. Therefore, P(EF)=P(E)
• The conditional probability that the chosen ball is yellow
given that the chosen ball is not black P(F|E)
P( EF ) P( F ) 5 / 25 1
PF | E 



P( E )
P( E ) 15 / 25 3
4/10/2011
Lecture 3
10
Examples
• In the card game bridge, the 52 cards are dealt out equally to
4 players – called East, West, North and South. If North and
South have a total of 8 spades among them, what is the
probability that East has 3 of the remaining 5 spades?
4/10/2011
Lecture 3
11
• There are 52 cards, 13 of which are spades.
• All the cards are divided into 4 equal piles of 13 cards each,
named north, south, east and west.
• Let (n, s, e, w) be the event that north has n spades, south has
s spades, east has e spades and west has w spades.
• Then n + s + e + w = 13. From the previous lecture, there are
13  4  1 16  16! 14 15 16

 7  5 16  560
 13    13  
6

   13!3!
possible values for (n, s, e, w) which fit this condition.
4/10/2011
Lecture 3
12
• Let F be the event that n + s = 8, therefore e + w = 5.
• Again from the previous lecture, the number of values for (n,
s, e, w) in the event F is the number of quadruplets (n, s, e, w)
which satisfy both the above equations, that is,
 5  2  1  8  2  1  6   9 
 5    8    5    8   6  9  54

 
    
4/10/2011
Lecture 3
13
• We can also find this result by realizing that we need to
choose
– numbers (e, w) from {(0,5), (1,4), (2,3), (3,2), (4,1), (5,0)},
– numbers (n, s) from {(0,8), (1,7), (2,6), (3,5), (4,4), (5,3) ,
(6,2), (7,1), (8,0)},
• So the set F is
F ={(0,8,0,5), (0,8,1,4), (0,8,2,3), (0,8,3,2), (0,8,4,1), (0,8,5,0),
(1,7,0,5), (1,7,1,4), (1,7,2,3), (1,7,3,2), (1,7,4,1), (1,7,5,0),
(2,6,0,5), (2,6,1,4), (2,6,2,3), (2,6,3,2), (2,6,4,1), (2,6,5,0),
….
(8,0,0,5), (8,0,1,4), (8,0,2,3), (8,0,3,2), (8,0,4,1), (8,0,5,0)}
4/10/2011
Lecture 3
14
• Let E be the event that e ≥ 3. E is a very large set, so let us list
only the elements of the event EF:
• EF ={(0,8,3,2), (0,8,4,1), (0,8,5,0),
(1,7,3,2), (1,7,4,1), (1,7,5,0),
(2,6,3,2), (2,6,4,1), (2,6,5,0),
….
(8,0,3,2), (8,0,4,1), (8,0,5,0)}
• The number of such elements is
 8  2  1
9
3 

3

 3  9  27



 8 
8
4/10/2011
Lecture 3
15
• Finally then the probability of east having 3 or more spades
when it is given that north and south have 8 spades between
them is:
P( EF ) 27 / 560 1
PE | F  


P( F ) 54 / 560 2
4/10/2011
Lecture 3
16
Examples
• Celine is undecided as to whether to take a French course or a
chemistry course. She estimates that her probability of
receiving an A grade would be ½ in a French course, and 2/3
in a chemistry course. If Celine decides to base her decision
on the flip of a fair coin, what is the probability that she gets
an A in chemistry?
4/10/2011
Lecture 3
17
Examples
• The sample space for this example is
• S = {(French, A), (French, no A), (Chemistry, A), (Chemistry, no A)}
• But the events do not have equal probability.
• In general, use the formula P(AB)=P(A|B)P(B) as below:
2 1 1
P  Chemistry, A   P A | Chemistry P Chemistry   
3 2 3
1 1 1
P  Chemistry, no A   P no A | Chemistry P Chemistry   
3 2 6
1 1 1
P  French, A   P A | French P French   
2 2 4
1 1 1
P  French, no A   P no A | French P French   
2 2 4
4/10/2011
Lecture 3
18
Find the probability that Celine gets an A
• There are two ways that Celine can get an A: either she takes
French and gets an A, or she takes Chemistry and gets an A:
A   Chemistry, A    French, A 
• It is also true that:
 Chemistry, A    French, A   {}
• In general the following holds:
If F  A  B then P  F   P  A  P  B   P  A  B 
• Therefore:
P A  P Chemistry, A  P French, A  P {}
1 1
7
  0 
4 3
12
4/10/2011
Lecture 3
19
Multiplication Rule
P  E1E2
En   P  E1  P  E2 | E1  P  E3 | E1E2 
P  En | E1E2
En1 
• The probability that events E1 and E2 and E3 and … and En
occur is equal to the product of
– The probability that event E1 occurs
– The probability that event E2 occurs given that event E1
occurs
– The probability that event E3 occurs given that event E1
occurs and event E2 occurs,
–…
– The probability that event En occurs given that event E1
occurs and event E2 occurs, …, and event En occurs.
4/10/2011
Lecture 3
20
Example
• An ordinary deck of 52 playing cards is randomly divided into
4 piles of 13 cards each. Compute the probability that each
pile has exactly 1 ace
• E1 = Event that ace of hearts is in any one of the piles.
• E2 = Event that ace of spades and ace of hearts are in
different piles
• E2 = Event that ace of spades, ace of hearts and ace of clubs
are in different piles
• E4 = Event that ace of spades, ace of hearts, ace of clubs
and ace of diamonds are in different piles
• P(E1) = 1
4/10/2011
Lecture 3
21
13 Card Slots Left
Pile 1
Pile 3
12 Card Slots Left
13 Card Slots Left
Pile 2
Pile 4
13 Card Slots Left
• Ace of spades can go into any one of Piles 2,3,4.
• There are 13 cards in every one of those piles, so
13  13  13
39
P  E2 | E1  

12  13  13  13 51
4/10/2011
Lecture 3
22
13 Card Slots Left
Pile 1
Pile 3
12 Card Slots Left
12 Card Slots Left
Pile 2
Pile 4
13 Card Slots Left
• Ace of diamonds can go into any one of Piles 3,4.
• There are 13 cards in every one of those piles, so
P  E3 | E1 E2  
4/10/2011
13  13
26

12  12  13  13 50
Lecture 3
23
12 Card Slots Left
Pile 1
Pile 3
12 Card Slots Left
12 Card Slots Left
Pile 2
Pile 4
13 Card Slots Left
• Ace of diamonds can go into any one of Piles 3,4.
• There are 13 cards in every one of those piles, so
P  E4 | E1 E2 E3  
4/10/2011
13
13

12  12  12  13 49
Lecture 3
24
• Now using the multiplication rule:
P  E1 E2 E3 E4   P  E1  P  E2 | E1  P  E3 | E1E2  P  E4 | E1E2 E3 
39 26 13
 1  
 0.105
51 50 49
4/10/2011
Lecture 3
25
Bayes’ Formula (Version 1)
P  A | B 
P  AB 
P  B
S
A
B
• Your sample space is now just B.
4/10/2011
Lecture 3
26
Bayes’ Formula (Version 2)
P  Fj | E  
P  EFj 
PE

P  E | Fj  P  Fj 
n
 PE | F  PF 
i 1
i
i
• Take the sample space S and divide it into n pieces.
S
E
F1
4/10/2011
F2
F3
Lecture 3
Fn
27
Example
• An insurance company believes that people can be divided
into two classes – those that are accident prone and those
that are not. Their statistics show that an accident prone
person will have an accident at some time within a fixed 1
year period with probability 0.4, whereas this probability
decreases to 0.2 for a non accident prone person. If we
assume that 30 percent of the population is accident prone,
what is the probability that a new policyholder will have an
accident within a year of purchasing a policy?
4/10/2011
Lecture 3
28
• Event A = New policy holder is accident prone
• Event B = New policy holder has an accident within a year of
purchasing the policy
S
A
AB

P  B   P  AB   P Ac B
B

Bayes' Rule : P  AB   P  A | B  P  B   P  B | A P  A



  
P  B   P  AB   P Ac B  P  B | A  P  A   P B | Ac P Ac
4/10/2011
Lecture 3
29
Universal Set: Everything may happen
P(A)=0.3
Event A:
Policy holder is
Accident Prone
P(Ac)= 0.7
Event Ac:
Policy holder is NOT
Accident Prone
P(B|A)= 0.4
P(Bc|A)= 0.6
P(B|Ac)= 0.2
Event B|A:
Accident
prone policy
holder has
an accident
Event Bc|A:
Accident
prone policy
holder DOES
NOT have an
accident
Event B|Ac:
NON-accident
prone policy
holder has an
accident
4/10/2011
Lecture 3
P(Bc|Ac)= 0.8
Event Bc|Ac:
NON-accident
prone policy
holder DOES
NOT have an
accident
30
Example
• Suppose that a new policy holder has an accident within a
year of purchasing a policy. What is the probability that
he/she is accident prone?
• Event A = New policy holder is accident prone
• Event B = New policy holder has an accident within a year of
purchasing the policy
• P(A|B)=?
P( AB) P  B | A  P  A 
P  A | B 

P  B
P  B
P  B | A P  A

P  B | A  P  A   P B | Ac P Ac
0.4  0.3
12
6



0.4  0.3  0.2  0.7 12  14 13

4/10/2011
Lecture 3
  
31
Example
• In a criminal investigation the inspector in charge is 60
percent convinced that Ahmet is guilty. Now a new piece of
evidence shows that the criminal is left handed. Ahmet is also
left handed. Given that about 5 percent of the population is
left handed, what is the probability that Ahmet is guilty given
the new evidence?
• G=Event that Ahmet is guilty
• L=Event that Ahmet is left handed
• P(G|L)=?
P  L | G  P G 
P(GL)
P G | L  

P  L  P  L | G  P G   P L | Gc P Gc
1 0.6
6


 0.9677
1 0.6  0.05  0.4 6  0.2

4/10/2011
Lecture 3
  
32
Independent Events
• The two events E and F are said to be independent if
P  EF   P  E  P  F 
• Two events E and F that are not independent are said to be
dependent.
• The three events E, F and G are said to be independent if
P  EFG   P  E  P  F  P  G 
P  EF   P  E  P  F 
P  EG   P  E  P  G 
P  FG   P  F  P  G 
4/10/2011
Lecture 3
33
Independent Events
• Proposition: If E and F are independent, then so are E and Fc.
4/10/2011
Lecture 3
34
Example
• A card is selected at random from an ordinary deck of 52
playing cards. If E is the event that the selected card is an ace
and F is the event that it is a spade, then E and F are
independent. This follows because P(EF)=1/62, whereas
P(E)=4/52 and P(F)=13/52.
4/10/2011
Lecture 3
35
Example
• S = {1, 2, 3, …, 9 , 10 , J , Q , K,
1, 2, 3, …, 9 , 10 , J , Q , K,
1, 2, 3, …, 9 , 10 , J , Q , K,
1, 2, 3, …, 9 , 10 , J , Q , K}
• E = {1, 1, 1, 1}
• F = {1, 2, 3, …, 9 , 10 , J , Q , K}
• EF= {1}
4 
PE 
52 
13 1 
P( F ) 
  P( EF )  P( E ) P( F )
52 4 
1 
P( EF ) 
52 
4/10/2011
Lecture 3
36
• Independent trials resulting in a success with probability p
and a failure with probability 1 − p are performed.
• [You perform and infinite number of trials, just start doing
trials and never stop. ]
• What is the probability that n successes occur before m
failures?
• If we think of A and B as playing a game such that A gains 1
point when a success occurs and B gains 1 point when a
failure occurs, then the desired probability is the probability
that A would win if the game were to be continued in a
position where A needed n and B needed m more points to
win.
4/10/2011
Lecture 3
37
Pascal’s Solution
• Pn,m is the probability that n successes occur before m failures.
Pn,m  pPn1,m  1  p  Pn,m1
The last period we had n – 1
successes m failures, and then
we were successful again.
The last period we had n
successes m – 1 failures, and
then we failed again.
• Pn,0=0 (why? Because m=0 means no failures, only successes
occur. There is almost no chance that in all of your infinite
number of trials, you never fail. )
• P0,m=1 (why? This is the probability that at least 0 successes
occur before m failures. So you don’t need any successes at
all, although if there is a success that is fine too. So anything
fits this option.)
4/10/2011
Lecture 3
38
Fermat’s Solution
• If you solve Pascal’s equations, you will get the answer. But
Fermat had a better way to think about it:
• He argued that the necessary and sufficient condition for n
successes to occur before m failures is that there be at least n
successes in the first n+m − 1 trials.
• The probability of exactly k successes in n+m − 1 trials is
m  n 1 k
 m  n  1 k
p
1

p


 
k


• So the probability of n successes before m failures is
m  n 1
m  n 1 k
 m  n  1 k
Pn ,m   
p 1  p 

k

k n 
4/10/2011
Lecture 3
39