Download Lecture17_030708

Lecture 17: Probability distribution functions, modeling of stochastic processes
Learning Objectives:
 Review of continuous and discrete random variables
 Review probability distribution functions and conditional probability
 Modeling of a Poisson noise model to evaluate signal to noise
Read Bevington, Chapter 2, “Probability Distributions.”
I. Random variables
A. Discrete case
e.g. Rolling a die:
S  1,2,3,4,5,6
S  " The sample space of all possible outcomes"
n  number of possible outcomes
N  number of trials
How often do we roll a “6” ?
1 if die lands with a 6
x

if otherwise
0

x  a discrete random variable on the sample space, S.
n  2 possible outcomes
N  Number tim es we roll the die
x is random because we cannot say for certain how many times x=0 or x=1 for a given experiment. We
can only make predictions about general behavior based on probability arguments:
i.e. Roll the die N =10 times, x = [0, 1, 1, 0, …]
Number of outcomes of state j  N p( x j ),
where p(x j ) is the probabilit y of x having outcome j.
If we ask how often on average we will roll a “6” twice in a row, then
1
P{6 consecutively} = P{6 once}P{6 once}=  
6
2
Lecture 17: Probability distribution functions, modeling of stochastic processes
1. Summary statistics:
Mean : the mean or
average of x
x
1
N
N
x
i 1
i
1 n

1 N 
lim   xi   lim   Np( x j ) x j     x  E ( x )
N  N
 i 1  N   N j 1

Standard Deviation : the
1 N
s
( x  x )2 ,
spread of the samples, x

N i 1
1
1 N

lim   ( x  x ) 2   lim 
N  N
 i 1
 N   N
Note that :
n
 Np( x )( x
j 1
j
j
Root mean
square of the
deviations
form the mean

 x ) 2    2  Variance

B. Continuous case
1. Can take on any value within an interval
2. For example,
S  People in this room
y  The heights of the people in this room
Unless we somehow bias our class height by making a specific height-related requirement in order to
be in this room (i.e. I only allowed people under 6 feet to take the class), we can not predict the height
of any one of you if I randomly point (while wearing a blind fold, say).
- Raises the concepts of sample bias and “blinded” study.
1. Mean or average value of y:
Lecture 17: Probability distribution functions, modeling of stochastic processes
II. Probability Distribution Functions (pdf’s)
A. Uniform
B. When we start to include multiple independent outcomes, e.g. roll multiple dice, we get a
convolution process that leads to non-uniform probability distributions:
(1) Binomial pdf, applies to discrete random variables with binary outcomes:
PB ( x j , n, p) 
n
 
x 
 j
p
xj
(1  p)
n x j
Recall our example of a discrete random variable,
1 if die lands with a 6
x

if otherwise
0

x  a discrete random variable on the sample space, S.
n  Now defined as the number of trials
Lecture 17: Probability distribution functions, modeling of stochastic processes
n 1
n!
  
 x j  x j ! (n  x )!
The mean and variance of a binomially distributed discrete random variable is:
n
   x j PB ( x j , n, p )  np
j 1
n
 2   ( x   ) 2 PB ( x j , n, p )  np(1  p )
j 1
(2) Poisson pdf
Consider a discrete random variable where   np  n because p<<1.
lim PB ( x j , n, p ) 
N 
p o
1 x x
 n p (1  p )  x (1  p ) n
x!
Pp ( x j ,  ) 
x
 e 
x!
=<x>≡ The average number of events expected in some interval (could be a time interval, say,
and photon counts recorded in that time interval in an imaging system).
2=
The average and variance are equivalent.
Lecture 17: Probability distribution functions, modeling of stochastic processes
(3) Gaussian pdf
For large number of trials, n, and finite probability both Binomial and Poisson distributions
converge to the Gaussian (or “Normal”) pdf:
1
PG (  ,  ) 
e
 2
 ( x   )2
2 2
Truly continuous distribution with mean of <x> = , and variance of 2=<(x-)2>.
III. Conditional probabilities
A. What does Pr( D d ) mean ?
Venn Diagram :
Pr( D, d )
Pr( d )
Intersection of D and d, normalized to the
probability of event d.
Pr( D d ) 
Pr( D, d )  Pr( D  d )  shaded area
The " joint probabilit y" of D and d.
Several properties:
Pr(D d)  Pr(d D )

 Pr( D d ) Pr( d )  Pr( D)
n 0
Careful: Pr( D d ) might be large, say 0.95, but
Pr(d) might be very small, 0.005 for example.
Lecture 17: Probability distribution functions, modeling of stochastic processes
Bayes’s Theorem :