Download Notes on Uniform Distribution and Normal Distribution

Econ 325/327
Notes on Uniform Distribution and Normal Distribution
By Hiro Kasahara
Uniform Distribution
Consider a random variable X of which outcome is a point selected at random from an intervale
[a, b] for −∞ < a < b < ∞. The cumulative distribution function (cdf) and the probability density
function (pdf) of X are given by

x < a,
 0,
x−a
,
a ≤ x < b,
F (x) =
 b−a
1,
b ≤ x,
and
dF (x)
=
f (x) =
dx
1
b−a ,
0,
a ≤ x < b,
otherwise,
respectively. The random variable X has a uniform distribution. The uniform distribution is
an example of continuous probability distribution because the support of random variable X is
continuous.
The expected value of X is given by
b
Z b
x
1
1
µ = E(X) = xf (x)dx =
dx =
xdx =
[(1/2)x2 ]ba
b
−
a
b
−
a
b
−
a
a
a
1 (b − a)(b + a)
a+b
1 b2 − a2
=
=
.
=
2 b−a
2
b−a
2
Z
Z
The variance of X is
Z b
Z b
1
1
1
Var(X) =
(x − µ)2
dx =
(x2 − 2µx + µ2 )
dx =
[(1/3)x3 − µx2 + µ2 x]ba
b
−
a
b
−
a
b
−
a
a
a
1
3
3
[(1/3)(b − a ) − µ(b2 − a2 ) + µ2 (b − a)]
=
b−a
1
[(1/3)(b − a)(b2 + a2 + ab) − µ(b − a)(b + a) + µ2 (b − a)]
=
b−a
b2 + a2 − 2ab
(b − a)2
=
,
= (1/3)(b2 + a2 + ab) − (a + b)2 /2 + (a + b)2 /4 =
12
12
where µ =
a+b
2 .
Normal Distribution
The normal distribution plays a very important role in statistics. First, in empirical applications,
many variables have a “bell-shaped” frequency distribution that is approximately symmetric and
has higher frequency around the mean than at the tail parts. As a result, the normal distribution
approximates the probability distributions of a wide range of random variables. Second, distributions of sample means approach a normal distribution as the sample size gets large.
1
The probability density function and the cumulative distribution function
The probability density function (pdf) of the normal random variable X is given by
1
(x − µ)2
fx (x) = √
, −∞ < x < ∞,
exp −
2σ 2
2πσ
and its cumulative distribution function (cdf) is
Z
Z x
fx (t)dt =
Fx (x) = Pr(X ≤ x) =
(1)
x
1
(t − µ)2
√
exp −
dt.
(2)
2σ 2
2πσ
−∞
−∞
R 1
2
For brevity, we say that X is N (µ, σ 2 ). It is possible to show that √2πσ
exp − (x−µ)
dx = 1
2σ 2
(See Chapter 3.3 of Hogg, Tanis, and Zimmerman).
Mean and Variance
We now show that E(X) = µ and Var(X) = σ 2 . Using the change of variable z = x−µ
σ or x = µ+σz,
the expected value of X is given by
Z ∞
1
(x − µ)2
dx
exp −
E(X) =
x√
2σ 2
2πσ
−∞
Z ∞
1
x−µ
and dx = σdz)
=
(µ + σz) √
exp −z 2 /2 σdz
(using the change of variable z =
σ
2πσ
−∞
Z ∞
Z ∞
1
1
2
√ exp −z /2 dz + σ
=µ
z √ exp −z 2 /2 dz
2π
2π
−∞
−∞
= µ × 1 + σ × 0 = µ,
R∞
where the last two lines hold because −∞ √12π exp −z 2 /2 dz = 1 given that the pdf integrates to 1
R∞
d
√1 exp −z 2 /2
=
0−0
=
0
because
=
−
while −∞ z √12π exp −z 2 /2 dz = [− √12π exp −z 2 /2 ]∞
−∞
dz
2π
1
2
z √2π exp −z /2 .
Using the change of variable z = x−µ
σ and integration by parts, we may show that the variance
of X is
Z ∞
1
(x − µ)2
2
Var(X) =
(x − µ) √
exp −
dx
2σ 2
2πσ
−∞
Z ∞
1
x−µ
=
σ 2 z 2 √ exp −z 2 /2 dz
(using the change of variable z =
and dx = σdz)
σ
2π
−∞
Z ∞
σ2
z 2 exp −z 2 /2 dz
=√
2π −∞
Z
1
σ2
2
√ exp(z 2 /2)dz
= − √ [z exp(z 2 /2)]∞
+
σ
(by integration by parts)
−∞
2π
2π
σ2
= − √ × 0 + σ2 × 1 = σ2,
2π
where the fourth line uses
by parts while the last line follows from [z exp(z 2 /2)]∞
−∞ = 0
R integration
1
2
by symmetry and from √2π exp(z /2)dz = 1.
R
0
0
0
0
R 0Integration by parts: (g(z)h(z)) = g (z)h(z)+g(z)h (z) implies that 2g(z)h (z)dz = g(z)h(z)−
g (z)h(z)dz. In this case, we choose g(z) = −z and h(z) = exp(−z /2) so that
g(z)h(z) =
R
2
0
2
0
2
2
−z exp(−z R/2), g(z)h (z) =R − exp(z /2), and g(z)h (z) = z exp(−z
/2) so that g(z)h0 (z)dz =
R
0
2
2
2
g(z)h(z) − g (z)h(z)dz = z exp(−z /2)dz = −z exp(z /2) + exp(z 2 /2)dz.
2
Standard normal distribution and change of variables
Consider a standard normal random variable Z ∼ N (0, 1), of which pdf is given by
2
1
z
√
φ(z) =
, −∞ < z < ∞
exp −
2
2π
and its cdf is given by
Z
z
Φ(z) = Pr(Z ≤ z) =
−∞
2
1
t
√ exp −
dt.
2
2π
There is no analytical expression for Φ(z) but the value of Φ(z) across different values of z can
be computed in any statistical software and any textbook on econometrics or statistics report the
table for Φ(z) (for example, Table 1 of Newbold, Carlson, and Thorne).
2
z
−z
√
The shape of φ(z) can be analyzed by taking derivatives. Note that φ0 (z) := dφ(z)
=
dz = 2π exp − 2
−zφ(z). Because φ0 (z) = 0 if and only if z = 0, the standard normal density function is flat at z = 0,
taking the maximum value of φ(0) = √12π . We may also examine how the slope of φ(z) changes by
2
φ(z)
d 0
d
taking the second order derivatives. φ00 (z) := d dz
= dz
φ (z) = dz
(−zφ(z)) = −zφ0 (z) − φ(z) =
2
2
00
(z − 1)φ(z). Therefore, φ (z) = 0 only if z = ±1. This means that the points of inflection of the
graph of the pdf of Z occurs at z = ±1; i.e., φ(z) is a concave function for |z| < 1 while φ(z) is a
convex function for |z| > 1.
For X ∼ N (µ, σ 2 ), we may compute Fx (x) = Pr(X ≤ x) from the value of Φ((x − µ)/σ), i.e.,
we may show that
x−µ
(3)
Fx (x) = Φ
σ
as follows:
Z x
(t − µ)2
1
√
exp −
dt
Fx (x) =
2σ 2
2πσ
−∞
2
Z x−µ
σ
1
z
t−µ
√
=
exp −
σdz
(using change of variables z =
and dt = σdz)
2
σ
2πσ
−∞
x−µ
=Φ
,
σ
x−µ
σ
in the domain
of z in the
b−µ
second line. Similarly, we may show that Pr(a ≤ X ≤ b) = Fx (b) − Fx (a) = Φ σ − Φ a−µ
.
σ
where the value x in the domain of X is changed into the value
Question: If X is N (µx , σx2 ), then what is the distribution of Y = a + bX for b 6= 0? Answer:
Y is N (a + bµx , b2 σx2 ). In
view of equation (3), this can be shown by showing that the cdf of Y is
y−µ
given by P (Y ≤ y) = Φ σy y , where µy = a + bµx and bσx as follows:
!
y−a
−
µ
y−a
y − (a + bµx )
x
b
P (Y ≤ y) = P (a + bX ≤ y) = P X ≤
=Φ
=Φ
.
b
σx
bσx
Linear combination of normal random variables
Suppose that X and Y are two mutually independent normal random variables with means µx and
µy and variances σx2 and σy2 . Then, the linear function of X and Y has the normal distribution.
Specifically, let a and b are some constant, then aX + bY has the normal distribution so that
aX + bY ∼ N aµx + bµy , a2 σx2 + b2 σy2 .
3
Note that for any two random variables, even when X and Y are not normally distributed, we have
E[aX + bY ] = aµx + bµy and V ar(aX + bY ) = a2 σx2 + b2 σy2 . The unique property of normal random
variables here is that the distribution of the linear combination of two normal random variables
remains the normal distribution. For the random variables that are not normally distributed, the
distribution of the linear combination of two random variables is generally neither the same as the
original distribution nor the same as the normal distribution. For example, the linear combination
two independent Bernouilli random variables neither has Bernouilli distribution nor has the normal
distribution.
We can also consider the linear combination of n independent normal random variables, X1 ,
X2 , ..., Xn , with means µ1 , µ2 , ..., µn and variances σ12 , σ22 , ..., σn2 . Let c1 , c2 , ..., cn be some
constant. Then
!
n
n
n
X
X
X
2
2
ci Xi ∼ N
µi , ci
σi
(4)
i=1
i=1
i=1
so that the linear combination of normal random variables is normally distributed.
The special case of the above result is the sample average of n independent normally distributed
random variables. Suppose that Xi is independently
drawn from N (µ, σ 2 ) for i = 1, ..., n, and
Pn
define the sample average by X̄ = (1/n) i=1 Xi . Then, by letting ci = 1/n and setting µ = µi
and σ 2 = σi2 for i = 1, ..., n in (4), we have
X̄i ∼ N µ, σ 2 /n .
Note that this result holds even when n is small (and hence this result does not use the Central
Limit Theorem).
4