Download Lecture 3 Gaussian Probability Distribution Introduction

Lecture 3
Gaussian Probability Distribution
Introduction
Gaussian probability distribution is perhaps the most used distribution in all of science.
also called “bell shaped curve” or normal distribution
Unlike the binomial and Poisson distribution, the Gaussian is a continuous distribution:
P(y) =
1
e
σ 2π
−
( y−µ) 2
2σ 2
µ = mean of distribution (also at the same place as mode and median)
σ2 = variance of distribution
y is a continuous variable (-∞ ≤ y ≤ ∞)
Probability (P) of y being in the range [a, b] is given by an integral:
P(a < y < b) =
b −
1
∫e
σ 2π a
−µ ) 2
(y−
2σ 2 dy
Karl Friedrich Gauss 1777-1855
The integral for arbitrary a and b cannot be evaluated analytically
The value of the integral has to be looked up in a table (e.g. Appendixes A and B of Taylor).
P(x)
p(x) =
1 −
e
σ 2π
(x−µ )2
2
2σ
gaussian
Plot of Gaussian pdf
x
L3: Gaussian Probability Distribution
1
The total area under the curve is normalized to one.
the probability integral:
(y−µ) 2
2σ 2 dy = 1
1
∫e
σ 2π −∞
We often talk about a measurement being a certain number of standard deviations (σ) away
from the mean (µ) of the Gaussian.
We can associate a probability for a measurement to be |µ - nσ|
from the mean just by calculating the area outside of this region.
nσ Prob. of exceeding ±nσ
0.67
0.5
It is very unlikely (< 0.3%) that a
1
0.32
measurement taken at random from a
2
0.05
Gaussian pdf will be more than ± 3σ
3
0.003
from the true mean of the distribution.
4
0.00006
P(−∞ < y < ∞) =
∞ −
95% of area within 2σ
Only 5% of area outside 2σ
Relationship between Gaussian and Binomial distribution
The Gaussian distribution can be derived from the binomial (or Poisson) assuming:
p is finite
N is very large
we have a continuous variable rather than a discrete variable
L3: Gaussian Probability Distribution
2
An example illustrating the small difference between the two distributions under the above conditions:
Consider tossing a coin 10,000 time.
p(heads) = 0.5
N = 10,000
For a binomial distribution:
mean number of heads = µ = Np = 5000
standard deviation σ = [Np(1 - p)]1/2 = 50
The probability to be within ±1σ for this binomial distribution is:
5000+50
10 4 !
m
10 4 −m
0.5
P=
0.5
= 0.69
∑
4
m=5000−50 (10 − m)!m!
For a Gaussian distribution:
( y−µ) 2
+
σ
µ
−
2
1
P(µ − σ < y < µ + σ ) =
∫ e 2σ dy ≈ 0.68
σ 2π µ−σ
Both distributions give about the same probability!
Central Limit Theorem
Gaussian distribution is very applicable because of the Central Limit Theorem
A crude statement of the Central Limit Theorem:
Things that are the result of the addition of lots of small effects tend to become Gaussian.
A more exact statement:
Actually, the Y’s can
Let Y1, Y2,...Yn be an infinite sequence of independent random variables
be from different pdf’s!
each with the same probability distribution.
Suppose that the mean (µ) and variance (σ2) of this distribution are both finite.
L3: Gaussian Probability Distribution
3
For any numbers a and b:
 Y1 +Y2 + ...Yn − nµ

1 b − 12 y 2
lim Pa <
< b =
dy
∫e

n→∞ 
σ n
2π a
C.L.T. tells us that under a wide range of circumstances the probability distribution
that describes the sum of random variables tends towards a Gaussian distribution
as the number of terms in the sum →∞.
Alternatively:
 Y −µ



Y −µ
1 b − 12 y 2
lim Pa <
< b = lim Pa <
< b =
dy
∫e
n→∞ 
σ / n  n→∞ 
σm
2π a

σm is sometimes called “the error in the mean” (more on that later).
For CLT to be valid:
µ and σ of pdf must be finite.
No one term in sum should dominate the sum.
A random variable is not the same as a random number.
Devore: Probability and Statistics for Engineering and the Sciences:
A random variable is any rule that associates a number with each outcome in S
S is the set of possible outcomes.
Recall if y is described by a Gaussian pdf with µ = 0 and σ = 1 then
the probability that a < y < b is given by:
1 b − 12 y 2
P ( a < y < b) =
dy
∫e
2π a
The CLT is true even if the Y’s are from different pdf’s as long as
the means and variances are defined for each pdf!
See Appendix of Barlow for a proof of the Central Limit Theorem.
L3: Gaussian Probability Distribution
4
Example: Generate a Gaussian distribution using random numbers.
Random number generator gives numbers distributed uniformly in the interval
[0,1]
µ = 1/2 and σ2 = 1/12
Procedure:
Take 12 numbers (ri) from your computer’s random number generator
Add them together
Subtract 6
Y1 +Ythat
nµif it is from a Gaussian pdf!
Geta number
looks
2 + ...Y
n − as
Pa <
< b


σ n
12


1
r
−12
⋅
∑
i


2
i=1
= Pa <
< b
1 ⋅ 12


12


12


= P−6 < ∑ ri − 6 < 6


i=1
1 6 − 12 y 2
=
∫ e dy
2π −6
Thus the sum of 12 uniform random
numbers minus 6 is distributed as if it came
from a Gaussian pdf with µ = 0 and σ = 1.
A) 5000 random numbers
C) 5000 triplets (r1 + r2 + r3)
of random numbers
B) 5000 pairs (r1 + r2)
of random numbers
D) 5000 12-plets (r1 + r2 +…r12)
of random numbers.
E) 5000 12-plets
E
(r1 + r2 +…r12 - 6) of
random numbers.
Gaussian
µ = 0 and σ = 1
-6
0
+6
L3: Gaussian Probability Distribution 12 is close to ¥!
5
Example: A watch makes an error of at most ±1/2 minute per day.
After one year, what’s the probability that the watch is accurate to within ±25 minutes?
Assume that the daily errors are uniform in [-1/2, 1/2].
For each day, the average error is zero and the standard deviation 1/√12 minutes.
The error over the course of a year is just the addition of the daily error.
Since the daily errors come from a uniform distribution with a well defined mean and variance
Central Limit Theorem is applicable:
 Y1 +Y2 + ...Yn − nµ

1 b − 12 y 2
lim Pa <
< b =
dy
∫e

n→∞ 
σ n
2π a
The upper limit corresponds to +25 minutes:
Y +Y + ...Yn − nµ 25 − 365 × 0
b= 1 2
=
= 4.5
1 365
σ n
12
The lower limit corresponds to -25 minutes:
Y +Y + ...Yn − nµ −25 − 365 × 0
a= 1 2
=
= −4.5
1 365
σ n
12
The probability to be within ± 25 minutes:
1 4.5 − 12 y 2
P=
dy = 0.999997 = 1− 3×10−6
∫ e
2π −4.5
less than three in a million chance that the watch will be off by more than 25 minutes in a year!
L3: Gaussian Probability Distribution
6
Example: The daily income of a "card shark" has a uniform distribution in the interval [-$40,$50].
What is the probability that s/he wins more than $500 in 60 days?
Lets use the CLT to estimate this probability:
 Y1 +Y2 + ...Yn − nµ

1 b − 12 y 2
lim Pa <
< b =
dy
∫e

n→∞ 
σ n
2π a
The probability distribution of daily income is uniform, p(y) = 1.
need to be normalized in computing the average daily winning (µ) and its standard deviation (σ).
50
∫ yp(y)dy
µ=
−40
50
=
1 [50 2
2
∫ p(y)dy
− (−40)2 ]
50 − (−40)
=5
−40
50
2
∫ y p(y)dy
σ = −4050
2
∫ p(y)dy
−µ =
2
1 [50 3 − (−40) 3 ]
3
− 25 = 675
50 − (−40)
−40
The lower limit of the winning is $500:
Y +Y + ...Yn − nµ 500 − 60 × 5 200
a= 1 2
=
=
=1
σ n
675 60
201
The upper limit is the maximum that the shark could win (50$/day for 60 days):
Y +Y + ...Yn − nµ 3000 − 60 × 5 2700
b= 1 2
=
=
= 13.4
σ n
675 60
201
1 13.4 − 12 y 2
1 ∞ − 12 y 2
P=
dy ≈
dy = 0.16
∫ e
∫e
2π 1
2π 1
16% chance to win > $500 in 60 days
L3: Gaussian Probability Distribution
7