Download Lecture 18

Lec 18
Nov 12
Probability – definitions and simulation
(Discrete) Sample space


Experiment: a physical act such as tossing a
coin or rolling a die.
Sample space – set of outcomes.



Coin toss Sample Space S = { head, tail}
Rolling a die Sample space S = {1, 2, 3, 4, 5, 6}
Tossing a coin twice.
Sample space S = {(h,h), (h,t), (t,h), (t,t)}
Events and probability
Event E is any subset of sample space S.
 You flip 2 coins
 Sample space S = {(h,h), (h,t), (t,h), (t,t)}
 Event: both tosses produce same result
E = {(h,h), (t,t)}
 Prob(E) = |E|/ |S|
 In the above example, p(E) = 2/4 = 0.5
Question: what is the probability of getting at
least one six in three roles of a die?

Bernoulli trial


Bernoulli trials are experiments with two
outcomes. (success with prob = p and failure
with prob = 1 – p.)
Example: rolling an unloaded die. Success is
defined as getting a role of 1.
p(success) = 1/6
Random Variable

Random variable (RV) is a function that maps
the sample space to a number.


E.g. the total number of heads X you get if you flip
100 coins
Another example:


RV
Keep tossing a coin until you get a head. The RV n
is the number of tosses.
Event = { H, TH, TTH, TTTH, … }
n(H) = 1, n(TH) = 2, n(TTH) = 3, … etc.
Common Distributions


Uniform X: U[1, N]

X takes values 1, 2, …, N

PX  i  1 N

E.g. picking balls of different colors from a box
Binomial distribution



X takes values 0, 1, …, n
n i
n i
P  X  i     p 1  p 
i
N coin tosses. What is the prob. That there are
exactly k tails?
Conditional Probability


P(A|B) is the probability of event A given that
B has occurred.
 Suppose 6 coins are tossed. Given that
there is at least one head, what is the
probability that the number of heads is 3?
Definition:
p( A  B)
p(A|B) =
p( B)
Baye’s Rule
If X and Y are events, then
p(X|Y) = p(Y|X) p(X)/p(Y)
Useful in situation where p(X), p(Y) and
p(Y|X) are easier to compute than
p(X|Y).
Independent events

Definition: X and Y are independent if
P X  x  Y  y  P X  x P Y  y 
Monty Hall Problem




You're given the choice of three doors: Behind
one door is a car; behind the others, goats.
You want to pick the car.
You pick a door, say No. 1
The host, who knows what's behind the doors,
opens another door, say No. 3, which has a
goat.
Do you want to pick door No. 2 instead?
Host reveals
Goat A
or
Host reveals
Goat B
Host must
reveal Goat B
Host must
reveal Goat A
Monty Hall Problem: Bayes Rule



Ci : the car is behind door i, i = 1, 2, 3
P  Ci   1 3
Hij : the host opens door j after you
pick door i
i j
0
0
jk

P H ij Ck  
ik
1 2
 1 i  k , j  k



Monty Hall Problem: Bayes Rule continued


WLOG, i=1, j=3
P  C1 H13  

 P H13
P  H13 C1  P  C 1 
P  H13 
1 1 1
C1  P  C1    
2 3 6
Monty Hall Problem: Bayes Rule continued

P  H13   P  H13 , C1   P  H13 , C2   P  H13 , C3 
 P  H13 C1  P  C1   P  H13 C2  P  C2 

1
1
  1
6
3
1

2
16 1
P  C1 H13  

12 3
Monty Hall Problem: Bayes Rule continued



16 1
P  C1 H13  

12 3
1 2
P  C2 H13   1    P  C1 H13 
3 3
You should switch!
Continuous Random Variables



What if X is continuous?
Probability density function (pdf)
instead of probability mass function
(pmf)
A pdf is any function f  x  that
describes the probability density in
terms of the input variable x.
Probability Density Function

Properties of pdf



f  x   0, x



f  x  1
Actual probability can be obtained
by taking the integral of pdf

E.g. the probability of X being between 0
and 1 is
P  0  X  1 

1
0
f  x dx
Cumulative Distribution Function


FX  v   P  X  v 
Discrete RVs
 FX  v  


vi
P  X  vi 
Continuous RVs
 
 FX v 


v

f  x  dx
d
FX  x   f  x 
dx
Common Distributions
Normal X

N ,

2

x


 
1
 
exp 
, x 
2
2
2





f  x 

E.g. the height of the entire population
0.4
0.35
0.3
0.25
f(x)

2
0.2
0.15
0.1
0.05
0
-5
-4
-3
-2
-1
0
x
1
2
3
4
5
Moments

Mean (Expectation):   E  X 
 Discrete RVs: E  X    vi P  X  vi 
v
i


Continuous RVs:
E X 
Variance: V  X  E  X   


Discrete RVs: V  X  

2



xf  x  dx
 vi    P  X  vi 
2
vi
Continuous RVs: V  X  


x  


2
f  x dx
Properties of Moments

Mean


E  X  Y  E  X  E  Y

E  aX  aE  X

If X and Y are independent,
Variance
E  XY  E  X   E  Y

V  aX  b   a 2V  X 

If X and Y are independent,
V  X  Y   V (X)  V (Y)
Moments of Common Distributions

Uniform X U 1, , N 


Binomial X
Bin  n, p 
2
np
np
 Mean
; variance

Normal X



Mean 1  N  2 ; variance N 2  1 12

N ,
2

Mean  ; variance  2
Simulating events by Matlab programs
Write a program in Matlab to distribute the 52
cards of a deck to 4 people, each getting
13 cards. All the choices must be equally
likely.
One way to do this is as follows: map each
card to a number 1, 2, …, 52. Generate a
random permutation of the array a[1 2 …
52], then give the cards a[1:13] to first
player, a[14:26] to second player etc.
Random permutation generation
We can use ceil(rand()*n) to generate a random
number from the set {1, 2, …, n}.
Algorithm generate a random permutation:
1. Start with array a = [ 1 2 … n]
2. For j = n: -1: 1
randomly pick a number r in [1..j]. Switch a[r] and
a[j]
3. Output a.