Download The Sum of Independent Normal Random Variables is Normal

Steven R. Dunbar
Department of Mathematics
203 Avery Hall
University of Nebraska-Lincoln
Lincoln, NE 68588-0130
http://www.math.unl.edu
Voice: 402-472-3731
Fax: 402-472-8466
Topics in
Probability Theory and Stochastic Processes
Steven R. Dunbar
The Sum of Independent Normal Random Variables is
Normal
Rating
Student: contains scenes of mild algebra or calculus that may require guidance.
1
Question of the Day
What is another proof that the sum of independent normal random variables
is normal? What must be known to determine the distribution of the sum if
the two normal random variables are not independent?
Key Concepts
1. We can prove that the sum of independent normal random variables
is again normally distributed by using the rotational invariance of the
bivariate normal distribution.
2. The Central Limit Theorem provides a heuristic explanation of why the
sum of independent normal random variables is normally distributed.
Vocabulary
1. The bivariate normal distribution is
f (z1 , z2 ) =
exp(
−(z12 +z22 )
)
2
,
2π
the joint density function of two independent standard random variables.
2
Mathematical Ideas
There are two usual proofs that the sum of independent random variables is
normal. One is the direct proof using the fact that the distribution of the
sum of independent random variables is the convolution of the distributions
of the two independent random variables. The computation is tedious. The
computation is also not very illuminating about why the sum of independent
normal variables is normal.
The second proof uses the fact that the moment generating function of the
sum of independent random variables is the product of the moment generating functions. After computing the mgf of a normal and taking the product,
we see that the product is again the mgf of a normal random variable. Then
the proof follows by using the uniqueness theorem for an mgf, that is, the
fact that the moment generating function is uniquely determined by the distribution.
This section is a summary, explanation, and review of the article by
Eisenberg and Sullivan, [1].
The rotation proof that sum of independent normals is
normal
We may as well take the independent random variables to have mean 0, since
a general normal random variable can be written in the form σZ + µ where
Z ∼ N (0, 1) is a standard normal random variable.
Take two independent standard normal random variables Z1 and Z2 . By
taking the product of the distributions, the joint density function of the two
random variables is
−(z 2 +z 2 )
exp( 12 2 )
.
f (z1 , z2 ) =
2π
This distribution is rotationally invariant. This means that the function
has the same value for all points equally distant from the origin. This as
can be algebraically from the form of the variables z12 + z22 , or from the
graph. If T is any rotational transformation of the plane, then f (T (z1 , z2 )) =
f (z1 , z2 ). Then it follows more generally that if A is any set in the plane, then
P [(Z1 , Z2 ) ∈ A] = P [T (Z1 , Z2 ) ∈ A] for any rotational transformation of the
plane. We will apply this observation to the region which is an arbitrary
half-plane.
3
Figure 1: The bivariate normal probability distribution
exp(
2 +z 2 )
−(z1
2 )
2
2π
.
If X1 is normal with mean 0 and variance σ12 and X2 is normal with 0
and variance σ22 . Then X1 + X2 has the same distribution as σ1 Z1 + σ2 Z2 .
Hence
P [X1 + X2 ≤ t] = P [σ1 Z1 + σ2 Z2 ≤ t] = P [(Z1 , Z2 ) ∈ A] ,
where A = {(z1 , z2 )|σ1 z1 + σ2 z2 ≤ t} where the
p boundary line of the halfplane σ1 z1 + σ2 z2 = t lies at a distance d = |t|/ σ12 + σ22 from the origin. It
follows that the half-plane A can be rotated into the set
(
)
t
T (A) = (z1 , z2 )|z1 < p 2
.
σ1 + σ22
See Figure 2 for the case when t > 0, so the half-plane contains the origin.
See Figure 3 for the case when t < 0, so the origin is not in the half-plane.
Now it is easy to calculate
exp(
Z Z
P [(Z1 , Z2 ) ∈ T (A)] =
T (A)
4
−(z12 +z22 )
)
2
2π
dz1 dz2 .
Figure 2: The half-plane σ1 z1 + σ2 z2 ≤ t, t > 0 is rotated into the half-plane
z1 < √ 2t 2 .
σ1 +σ2
Figure 3: The half-plane σ1 z1 + σ2 z2 ≤ t, t < 0 is rotated into the half-plane
z1 < √ 2t 2 .
σ1 +σ2
5
i
hp
σ12 + σ22 Z1 < t . It follows that X1 + X2 is
p
normal with mean 0 and variance σ12 + σ22 .
This proof is elementary, self-contained, conceptual, uses nice geometric
ideas and requires almost no computation.
Thus P [X1 + X2 < t] = P
A heuristic explanation
It is possible to explain heuristically why the sum of independent normal
random variables is normal, using the Central Limit Theorem as given. Recall that the Central Limit Theorem says that if X1 , X2 , . . . is a sequence
of independent, identically distributed random variables with mean 0 and
variance 1, then
X1 + · · · + X n D
√
→ P [Z ≤ t]
P
n
where Z is normally distributed with mean 0 and variance 1. Then
X1 + · · · + Xn D
√
P
→ P [Z1 ≤ t]
n
and
Xn+1 + · · · + X2n D
√
P
→ P [Z2 ≤ t]
n
where Z1 and Z2 are independent, standard normal random variables. Furthermore
X1 + · · · + X2n D
√
→ P [Z3 ≤ t]
P
2n
Since
X1 + · · · + Xn Xn+1 + · · · + X2n
X1 + · · · + X2n
√
√
√
+
=
n
n
n
√ X1 + · · · + X2n
√
= 2
2n
√
it seems reasonable that the Z1 + Z2 has the same distribution as 2Z3 , that
is Z1 + Z2 is normal with variance 2.
6
Problems to Work for Understanding
1. Cite a reference that demonstrates that the distribution of the sum of
independent random variables is the convolution of the distributions of
the two independent random variables.
2. Show by direct computation of the convolution of the distributions that
the distribution of the sum of independent normal random variables is
again normal.
3. Suppose that the joint random variables (X, Y ) are uniformly
√ dis2
tributed over the unit disk. Show that X has density fX (x) = π 1 − x2
for −1 ≤ x ≤ 1. Using the ideas q
from the rotation proof, show
that aX + bY has density fc (x) =
√
c = a2 + b 2 .
2
cπ
1−
x2
c2
for −c ≤ x ≤ c where
Reading Suggestion:
References
[1] Bennett Eisenberg and Rosemary Sullivan. Why is the sum of independent normal random variables normal?
Mathematics Magazine,
81(5):362–366, December 2008.
7
Outside Readings and Links:
1. Transformations of Multiple Random Variables, Sum of Two Random
Variables.
I check all the information on each page for correctness and typographical
errors. Nevertheless, some errors may occur and I would be grateful if you would
alert me to such errors. I make every reasonable effort to present current and
accurate information for public use, however I do not guarantee the accuracy or
timeliness of information on this website. Your use of the information from this
website is strictly voluntary and at your risk.
I have checked the links to external sites for usefulness. Links to external
websites are provided as a convenience. I do not endorse, control, monitor, or
guarantee the information contained in any external website. I don’t guarantee
that the links are active at all times. Use the links here with the same caution as
you would all information on the Internet. This website reflects the thoughts, interests and opinions of its author. They do not explicitly represent official positions
or policies of my employer.
Information on this website is subject to change without notice.
Steve Dunbar’s Home Page, http://www.math.unl.edu/~sdunbar1
Email to Steve Dunbar, sdunbar1 at unl dot edu
Last modified: Processed from LATEX source on May 11, 2010
8