Download Probability Rules

Applying Probability
• Define problem of interest
– in terms of “random variables” and/or “composite
events”
• Use real world knowledge, symmetry
– to associate probs in [0,1] with ‘elementary events’
– all probs are conditional on real world knowledge
• Use consistent prob rules
– To associate probs with rand vars/ comp events
– Multiplication and Addition Rules
ST2004 Week 7
1
Probability
• Prob Rules
Week 7
– Basic in text, Ch2.2
– Conditional Prob Bayes Rule in Ch 6
– Fuller treatment in Ch 7 8
• Discrete Prob Dist
Week 8
– Ch 4 – see lab on Queuing
– Ch 9
• Continuous Prob Dist
– Ch 5 Normal dist
– Ch 10
ST2004 Week 7
Week 9
We give more
emphasis to ‘event
identities’. Book in Ch 7
uses more math
shortcuts (binomial
coeffs) and  notation
than we will use.
Best immediate
preparation is Q1-12 in
Ch 1. Formulate and
approach via EXCEL
before attempting
probability solution.
2
Problems
• Dice: Seek prob dist of M2,S2 ,M3,S3 ,Mk,Sk
– Later E(S2) Var(S2) etc
• Mini-league: Seek prob dist of (NA, NB, NC) when
– Pr( A beats B)=2 Pr(B beats A)
 Pr( A beats B)=?
– Pr( A beats B)=pAB; similarly pBC, pAC
– Later E(NA),Var(NA) and E(NA|NC=0),Var(NA|NC=0)
ST2004 Week 7
6
Events, Random Vars, Sample Space
and Probability Rules
Event A
Simplest Random Variable
Values of A are TRUE/FALSE
Random Variable Y
Values of Y are y1, y2..yk (sample space; exhaustive list)
Events such as (Y= y)
ST2004 2010 Week 6
7
Event Algebra
Event Identities
Not A  A
D  ( A or B )
Re-express compound events in
and/or combinations of
elementary events
D  ( A and B )
D  ( A and B ) or ( A and B ) or ( A and B )
A  ( A and B ) or ( A and B )
Coin
(H orT) Experiment Happened
  ( A and A) Disjoint/Mutually Exclusive
S    ( A or A)
Exhaustive
Cards
Ace  (A♠ orA♥or A♣ or A♦)
Redand (NOT♦)  (2♥or.. or A♥)
ST2004 2010 Week 6
8
Event Identities
Not A  A
D  ( A or B )
D  ( A and B )
D  ( A and B ) or ( A and B ) or ( A and B )
A  ( A and B ) or ( A and B )
  ( A and A) Disjoint/Mutually Exclusive
S    ( A or A)
Exhaustive
Re-express in terms of and/or combs of (..)
(elementary events and/or simple compound
events). Often there is more than one way.
“A out-right winner of league”.
Use as elementary events Outcomes of games A/B,
etc, and as relatively simple compound events, the
scores NA , etc
“At least one Queen in two cards”
“Max of 3 dice is 3” and “Max of 3 dice is  3”
“Sun of 3 is 4”
ST2004 2010 Week 6
9
Event Identities
Coins: Elementary events H1 , T1 , H 2 ...........
Define F (3)=First Head on 3rd toss
F (3)  T1 , T2 , H 3 
Define F ( 3)  First Head on 4th or Higher toss
F ( 3)   T1 , T2 , T3 , H 4  OR T1 , T2 , T3 , T4 , H 5  OR.......
Alt
F ( 3)  NOT ( F (1) OR F (2) OR F (3))
ST2004 Week 7
10
Event Identities
E =  No common birth date in class 
Define Yi  birth date of student i (1,2,...365)
 (Y1  any date, n1 )

 AND (Y =any date n except n ),

2
2
1


E
 AND Y3  any date n3 , except n1 and n2  ... 


 .............

ST2004 Week 7
13
Probability Rules
Pr( A)  Pr( AisTRUE )
Pr( A) in [0,1]
Pr()  1
Pr( Aor B )  Pr( A)  Pr( B)
Plus real world
knowledge
if mut. excl
 A or A   
Pr  A or A   Pr  A   Pr  A   1
Event Identity
Whence
Also Pr     0
Pr( A and A)  0
Pr( Aand B )  0 if A, B mut. excl
Pr( Aor B ... or Z )  Pr( A)  Pr( B)  ..  Pr( Z )
ST2004 Week 7
Addition Rule
if mut. excl
17
Coin Toss
Coins/Dice/Cards
Define H  (Heads), T  (Tails)  H
H and T  ; H or T  
1  Pr( H or T )  Pr( H )  Pr(T )
since mut. excl.
Real World Knowledge
Symmetry  Pr( H )  Pr(T )  Pr( H )  Pr(T )  12
Define
One Dice
6  (Throws 6)
One Card
Q  (Draws Queen)
Compute Pr(6) Pr(6)
Pr(Q) Pr(Q)
ST2004 Week 7
18
Applying Prob Rules
Event Identity
A  ( A and B ) or ( A and B )
Pr( A)  Pr( A and B)  Pr( A and B ) since disjoint
Sim
Pr( B)  Pr( A and B)  Pr( A and B) disjoint
Event Identity ( A or B)  ( A and B ) or ( A and B) or ( A or B )
 Pr( A or B)  Pr( A and B )  Pr ( A and B)  Pr ( A or B)
Pr( A or B)  Pr( A )  Pr( B)  Pr( Aand B)
Generalisation of
Addition Rule
Example
Define : A  Team A at least jo int winner; sim B, C
Given symmetry
Pr( A)  Pr( B)  Pr(C )

Pr( A)  ?
Pr( A or B)  ?
Pr( A or B or C )  ?
ST2004 Week 7
19
Event Identities: Password
Dup in Password 3 Chars from A,B,C,D,E,F)
A
rep
1
2
3
4
5
6
7
8
9
10
B
C
Char Index
1st 2nd 3rd
5
6
3
6
3
2
3
4
6
6
1
6
3
4
4
5
4
3
2
6
3
4
6
4
1
6
5
5
4
3
D E
1st
E
F
C
F
C
E
B
D
A
E
F
Duplicate?
Not Dup?
Char
2nd
F
C
D
A
D
D
F
F
F
D
using OR
using AND
FALSE
FALSE
FALSE
TRUE
TRUE
FALSE
FALSE
TRUE
FALSE
FALSE
TRUE
TRUE
TRUE
FALSE
FALSE
TRUE
TRUE
FALSE
TRUE
TRUE
3rd
C
B
F
F
D
C
C
D
E
C
ST2004 Week 7
Elementary events
and associated probs
Pr(Dup) via addition rules
20
Conditional Probability
Pr( A) requires Real World Knowledge
PrRWK ( A)
Pr( A | B)  Pr( A given B)
Prgiven B ( A ) Real World Knowledge
includes
B  TRUE
ST2004 Week 7
21
Probability Rules
Conditional Prob and Independence
Pr( Aand B)  Pr( A | B)  Pr( B)
Book uses AB
Pr( Aand B)
Equiv Pr( A | B) 
for A and B
Pr( B)
Multiplication Rule
Important special case
Pr( Aand B)  Pr( A)  Pr( B ) when stat independent
Dice
Define 61  6 on first roll;
Cards
Define Q1  Queen on first draw;
also 62
also Q 2
Pr(61 and 62 )  ?
Pr(Q1 and Q 2 )  ?
ST2004 Week 7
23
Decomposing with Cond Probs
Prob( 2nd card is Queen)  Pr(Q2 )
Event Identity

  First Card is anything AND Q2    Q1 OR Q1  AND Q2
Q2
 Q ANDQ  OR Q OR Q 
Pr  Q   Pr   Q ANDQ  OR  Q OR Q  

2
1
2
1
1
2
2
1

Pr  Q3 
2
 Pr  Q1 ANDQ2   Pr  Q1 ANDQ2 
 Pr  Q2 | Q1  Pr(Q1 )  Pr  Q2 | Q1  Pr(Q1 )
4 48

51 52
 48 3  4
  
 51 51  52

3 4
51 52
4

52
Pr(6on 2nd dice roll)
ST2004 Week 7
24
Applying Cond Probability Rules
Define Q2  Queen on second card; also Q1
Seek
Pr(Q1 |Q2 )
given regular deck
Use
Pr( A | B)  Pr( A and B) / Pr( B)
recall PrB ( A)
poss
rel freq
not Q,not Q
848
Q,not Q
75
not Q,Q
75
Q,Q
2
Prob
Rel Freqs Q
not Q
Q 0.002 0.075 0.077
not Q 0.075 0.848 0.923
0.077 0.923
1
Q
not Q
Q 0.005 0.072 0.077
not Q 0.072 0.851 0.923
0.077 0.923
1
ST2004 Week 7
26
Applying Cond Probability Rules
Mini-league
Define A = A outright winner; also B, C
Given symmetry Pr( A)  Pr( B)  Pr(C ) 
Write down event identities
explicitly
1
4
Justify use of + or 
explicitly
Pr( A | one team is outright winner)  ?
Pr( A | team C is outright winner)  ?
Pr( A | C scored no wins)  ?
Pr( A | no info about outright winner)  ?
ST2004 Week 7
27
Bayes Rule & Thinking Backwards
Inverting the direction of conditionality
Pr This evidence | At crime scene 
or
Pr( B | A)
Pr  At crime scene | This evidence 
Pr( B)
Pr( A)
Pr( A | B) Pr( B)

Pr( A | B ) Pr( B )  Pr( A | B) Pr( B)
 Pr( A | B)
Alt Form
Pr( B | A)
Pr( A | B)

Pr( B | A)
Pr( A | B )
Posterior Odds

ST2004 Week 7
Pr( B)
Pr( B )
Prior Odds
See text, Ch
8.2
28
Bayes Rule & Thinking Backwards
Inverting the direction of conditionality
Pr This evidence | At crime scene 
or
Pr  At crime scene | This evidence 
Murder Committed
Either X or unknown Y
In absence of evidence
Evidence E
Pr( X )  0.5  Pr(Y )
Probs
A
A
X
Y
Blood group A at crime scene
X has blood group A : Pr( A | X )  1
Y blood group not known. But know Pr( A)  .10
Pr(X Guilty)
Pr( A at crime scene| E )= Pr( E | X Guilty)
Pr( E )
Pr( E )  Pr( E | X Guilty)Pr(X Guilty)+ Pr( E | Y Guilty)Pr(Y Guilty)
ST2004 Week 7
29
Bayes Rule & Thinking Backwards
Mail Arrives:
Contains "viagra"
Probs
Spam or not?
Given past data:
in general 5% of my mail is spam
Pr("viagra"|not spam) = 0.0005
Pr("viagra"|spam)
= 0.05
Pr(spam|"viagra")=
Contains "v1agra"
Spam or not?
Past data:
Pr("v1agra"|not spam) = p
Pr("v1agra"|spam)
=q
Pr(spam|"viagra")=
ST2004 Week 7
31
Main use of
probability
Probability Distributions
and Random Variables
• Output of a simulation exercise (thought expt)
• Columns defined random variables Y
– Discrete
countable list of possible values
– Continuous values
– True/False values Random Var is ‘Event’
• Discrete random vars fully described by
– 2 lists Poss Values y of Y
Associated Probs Pr(Y=y)
ST2004 Week 7
32
Applying Probability Rules – Indep Case
Dice
Define M = max(Scores on two independent rolls)
Seek prob dist of M
Two Lists: Poss (sample space)
Probs
Define elementary events; use event identity & prob rules
ST2004 Week 7
33
Applying Probability Rules – Indep Case
Mini League
Define N B = Number of wins by B
A twice as good as B, C ; B, C evenly matched
Games independent
(Sim using TWO random numbers)
Seek prob dist of N B
Two lists: poss = sample space for N B
probs
Winner in
A/B A/C B/C
Poss
A
A
B
Outcomes A
A
C
A
C
B
A
C
C
B
A
B
B
A
C
B
C
B
B
C
C
Outright
A best by
Games won Winner
Probs
factor a
a
A B C
is
All equal A best
2
2 1 0
A
0.125 0.222 Pr(A beats B) 0.6667
2 0 1
A
0.125 0.222 Pr(A beats C) 0.6667
1 1 1 N/A
0.125 0.111 Pr(B beats C)
0.5
1 0 2
C
0.125 0.111
1 2 0
B
0.125 0.111
1 1 1 N/A
0.125 0.111
0 2 1
B
0.125 0.056
0 1 2
C
0.125 0.056 ST2004 Week 7
34
Conditional Distributions
Mini-league: A more skilled
What is prob dist for N A ?
Know N c  0; what is prob dist for N A ?
Know N c  1; what is prob dist for N A ?
Know N c  2; what is prob dist for  N A , N B  ?
Winner in
A/B A/C B/C
Poss
A
A
B
Outcomes A
A
C
A
C
B
A
C
C
B
A
B
B
A
C
B
C
B
B
C
C
Outright
A best by
Games won Winner
Probs
factor a
a
A B C
is
All equal A best
2
2 1 0
A
0.125 0.222 Pr(A beats B) 0.6667
2 0 1
A
0.125 0.222 Pr(A beats C) 0.6667
1 1 1 N/A
0.125 0.111 Pr(B beats C)
0.5
1 0 2
C
0.125 0.111
1 2 0
B
0.125 0.111
1 1 1 N/A
0.125 0.111
0 2 1
B
0.125 0.056
ST2004 Week 7
0 1 2
C
0.125 0.056
38 1!
Probabilities must sum to