Download The Calculus of Probability

The Calculus of Probability
Let A and B be events in a sample space S.
Partition rule:
P(A) = P(A ∩ B) + P(A ∩ B {)
Example: Roll a pair of fair dice
P(Total of 10)
= P(Total of 10 and double) + P(Total of 10 and no double)
=
2
3
1
1
+
=
=
36 36 36 12
Complementation rule:
P(A{) = 1 − P(A)
Example: Often useful for events of the type “at least one”:
P(At least one even number)
9
3
= 1 − P(No even number) = 1 −
=
36 4
Containment rule
P(A) ≤ P(B) for all A ⊆ B
Example: Compare two aces with doubles,
1
6
1
= P(Two aces) ≤ P(Doubles) =
=
36
36 6
Calculus of Probability, Jan 26, 2003
-1-
The Calculus of Probability
Inclusion and exclusion formula
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
Example: Roll a pair of fair dice
P(Total of 10 or double)
= P(Total of 10) + P(Double) − P(Total of 10 and double)
=
3
6
1
8
2
+
−
=
=
36 36 36 36 9
The two events are
46, 55, 64}
Total of 10 = {
and
11,22,33,44,55,66}
Double = {
The intersection is
55}.
Total of 10 and double = {
Adding the probabilities for the two events, the probability for the
event
55 is added twice.
Calculus of Probability, Jan 26, 2003
-2-
Conditional Probability
Probability gives chances for events in sample space S.
Often: Have partial information about event of interest.
Example: Number of Deaths in the U.S. in 1996
Cause
Heart
Cancer
HIV
Accidents1
Homicide2
All causes
1
All ages
1-4 5-14 15-24
733,125
207
341
920
544,161
440 1,035 1,642
32,003
149
174
420
92,998 2,155 3,521 13,872
24,486
395
513 6,548
2,171,935 5,947 8,465 32,699
Accidents and adverse effects,
2
25-44
45-64
≥ 65
16,261 102,510
612,886
22,147 132,805
386,092
22,795
8,443
22
26,554 16,332
30,564
9,261
7,717
52
148,904 380,396 1,717,218
Homicide and legal intervention
measure probability with respect to a subset of S
Conditional probability of A given B
∩ B)
P(A|B) = P(A
P(B) ,
If
if
P(B) > 0
P(B) = 0 then P(A|B) is undefined.
Conditional probabilities for causes of death:
◦ P(accident) = 0.04282
◦
◦
◦
P(age=10) = 0.00390
P(accident|age=10) = 0.42423
P(accident|age=40) = 0.17832
Calculus of Probability, Jan 26, 2003
-3-
Conditional Probability
Example: Select two cards from 32 cards
◦ What is the probability that the second card is an ace?
P(2nd card is an ace) = 18
◦ What is the probability that the second card is an ace if the
first was an ace?
P(2nd card is an ace|1st card was an ace) = 313
Calculus of Probability, Jan 26, 2003
-4-
Multiplication rules
Example: Death Rates (per 100,000 people)
All Ages 1-4 5-14 15-24 25-44 45-64 ≥ 65
872.5 38.3 22.0 90.3 177.8 708.0 5071.4
Can we combine these rates with the table on causes of death?
◦ What is the probability to die from an accident (HIV)?
◦ What is the probability to die from an accident at age 10 (40)?
P(accident|die) = P(die from accident)/P(die)
⇒ P(die from accident) = P(accident|die)P(die)
Know
Calculate probabilities:
◦ P(die from accident) = 0.04281 · 0.00873 = 0.00037
◦
◦
◦
◦
◦
P(die from accident|age = 10) = 0.42423 · 0.00090 = 0.00038
P(die from accident|age = 40) = 0.17832 · 0.00178 = 0.00031
P(die from HIV) = 0.01473 · 0.00873 = 0.00013
P(die from HIV|age = 10) = 0.02055 · 0.00090 = 0.00002
P(die from HIV|age = 40) = 0.15308 · 0.00178 = 0.00027
General multiplication rule
P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A)
Calculus of Probability, Jan 26, 2003
-5-
Independence
Example: Roll two dice
◦ What ist the probability that the second die shows
1?
P(2nd die = 1) = 61
◦ What ist the probability that the second die shows 1 if the first
die already shows 1?
P(2nd die = 1|1st die = 1) = 61
◦ What ist the probability that the second die shows 1 if the first
does not show 1?
P(2nd die = 1|1st die 6= 1) = 61
The chances of getting 1 with the second die are the same, no
matter what the first die shows. Such events are called independent:
The event A is independent of the event B if its chances are
not affected by the occurrence of B,
P(A|B) = P(A).
Equivalently, A and B are independent if
P(A ∩ B) = P(A)P(B)
Otherwise we say A and B are dependent.
Calculus of Probability, Jan 26, 2003
-6-
Let’s Make a Deal
The Rules:
◦ Three doors - one price, two blanks
◦ Candidate selects one door
◦ Showmaster reveals one loosing door
◦ Candidate may switch doors
1
2
3
Would YOU change?
Can probability theory help you?
◦ What is the probability of winning if candidate switches doors?
◦ What is the probability of winning if candidate does not switch
doors?
Calculus of Probability, Jan 26, 2003
-7-
The Rule of Total Probability
Events of interest:
◦ A - choose winning door at the beginning
◦ W - win the price
Strategy: Switch doors (S)
PS (W |A) = 0
◦ PS (A) = 13
◦ PS (W |A{) = 1
◦ PS (A{) = 32
Probability of interest: PS (W ):
PS (W ) = PS (W ∩ A) + PS (W ∩ A{)
= PS (W |A)PS (A) + PS (W |A{)PS (A{)
Know:
◦
=0·
1
+ 1 · 32 =
3
2
3
Strategy: Do not switch doors (N )
PN (W |A) = 1
◦ PN (A) = 13
◦ PN (W |A{) = 0
◦ PN (A{) = 23
Probability of interest: PN (W ):
PN (W ) = PN (W ∩ A) + PN (W ∩ A{)
= PN (W |A)PN (A) + PN (W |A{)PN (A{)
Know:
◦
=1·
1
+ 0 · 23 =
3
Calculus of Probability, Jan 26, 2003
1
3
-8-
The Rule of Total Probability
Rule of Total Probability
If B1, . . . , Bk mutually exclusive and B1 ∪ . . . ∪ Bk = S, then
P(A) = P(A|B1)P(B1) + . . . + P(A|Bk )P(Bk )
Example:
Suppose an applicant for a job has been invited for an interview.
The chance that
◦ he is nervous is P(N ) = 0.7,
P(S|N ) = 0.2,
the interview is succussful if he is not nervous is P(S|N {) = 0.9.
◦ the interview is succussful if he is nervous is
◦
What is the probability that the interview is successful?
P(S) = P(S|N )P(N ) + P(S|N {)P(N {)
= 0.2 · 0.7 + 0.9 · 0.3
= 0.441
Calculus of Probability, Jan 26, 2003
-9-
The Rule of Total Probability
Example:
Suppose we have two unfair coins:
◦ Coin 1 comes up heads with probability 0.8
◦ Coin 2 comes up heads with probability 0.35
Choose a coin at random and flip it. What is the probability of its
being a head?
Events: H=“heads comes up”, C1=“1st coin”, C2=“2nd coin”
P(H) = P(H|C1)P(C1) + P(H|C2)P(C2)
1
= (0.8 + 0.35) = 0.575
2
Calculus of Probability, Jan 26, 2003
- 10 -
Bayes’ Theorem
Example: O.J. Simpson
“Only about
1
10
of one percent of wife-batterers actually murder their wives”
Lawyer of O.J. Simpson on TV
Fact: Simpson pleaded no contest to beating his wife in 1988.
So he murdered his wife with probability 0.001?
◦ Sample space S - married couples in U.S. in which the husband
beat his wife in 1988
◦ Event H - all couples in S in which the husband has since
murdered his wife
◦ Event M - all couples in S in which the wife has been murdered
since 1988
We have ◦
◦
◦
P(H) = 0.001
P(M |H) = 1 since H ⊆ M
P(M |H {) = 0.0001 at most in the U.S.
Then
P(H)
P(H|M ) = P(MP|H)
(M )
P(M |H)P(H)
=
P(M |H)P(H) + P(M |H {)P(H {)
=
0.001
= 0.91
0.001 + 0.0001 · 0.999
Calculus of Probability, Jan 26, 2003
- 11 -
Bayes’ Theorem
Reversal of conditioning (general multiplication rule)
P(B|A)P(A) = P(A|B)P(B)
Rewriting
P(A) using the rule of total probability we obtain
Bayes’ Theorem
P(B|A) =
P(A|B)P(B)
P(A|B)P(B) + P(A|B {)P(B {)
If B1, . . . , Bk mutually exclusive and B1 ∪ . . . ∪ Bk = S, then
i )P(Bi )
P(Bi|A) = P(A|B )P(BP(A|B
) + . . . + P(A|B )P(B )
1
1
k
k
(General form of Bayes’ Theorem)
Calculus of Probability, Jan 26, 2003
- 12 -
Bayes’ Theorem
Example: Testing for AIDS
Enzyme immunoassay test for HIV:
◦ P(T+|I+) = 0.98 (sensitivity - positive for infected)
◦
◦
P(T-|I-) = 0.995 (specificity - negative for noninfected)
P(I+) = 0.0003 (prevalence)
What is the probability that the tested person is infected if the
test was positive?
P(I+)
P(I+|T+) = P(T+|I+)PP(T+|I+)
(I+) + P(T+|I-)P(I-)
=
0.98 · 0.0003
0.98 · 0.0003 + 0.005 · 0.9997
= 0.05556
Consider different population with
P(I+) = 0.1 (greater risk)
0.98 · 0.1
= 0.956
P(I+|T+) = 0.98 · 0.1
+ 0.005 · 0.9
testing on large scale not sensible (too many false positives)
Repeat test (Bayesian updating):
◦ P(I+|T++) = 0.92 in 1st population
◦
P(I+|T++) = 0.9998 in 2nd population
Calculus of Probability, Jan 26, 2003
- 13 -