Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Steven R. Dunbar Department of Mathematics 203 Avery Hall University of Nebraska-Lincoln Lincoln, NE 68588-0130 http://www.math.unl.edu Voice: 402-472-3731 Fax: 402-472-8466 Topics in Probability Theory and Stochastic Processes Steven R. Dunbar The Sum of Independent Normal Random Variables is Normal Rating Student: contains scenes of mild algebra or calculus that may require guidance. 1 Question of the Day What is another proof that the sum of independent normal random variables is normal? What must be known to determine the distribution of the sum if the two normal random variables are not independent? Key Concepts 1. We can prove that the sum of independent normal random variables is again normally distributed by using the rotational invariance of the bivariate normal distribution. 2. The Central Limit Theorem provides a heuristic explanation of why the sum of independent normal random variables is normally distributed. Vocabulary 1. The bivariate normal distribution is f (z1 , z2 ) = exp( −(z12 +z22 ) ) 2 , 2π the joint density function of two independent standard random variables. 2 Mathematical Ideas There are two usual proofs that the sum of independent random variables is normal. One is the direct proof using the fact that the distribution of the sum of independent random variables is the convolution of the distributions of the two independent random variables. The computation is tedious. The computation is also not very illuminating about why the sum of independent normal variables is normal. The second proof uses the fact that the moment generating function of the sum of independent random variables is the product of the moment generating functions. After computing the mgf of a normal and taking the product, we see that the product is again the mgf of a normal random variable. Then the proof follows by using the uniqueness theorem for an mgf, that is, the fact that the moment generating function is uniquely determined by the distribution. This section is a summary, explanation, and review of the article by Eisenberg and Sullivan, [1]. The rotation proof that sum of independent normals is normal We may as well take the independent random variables to have mean 0, since a general normal random variable can be written in the form σZ + µ where Z ∼ N (0, 1) is a standard normal random variable. Take two independent standard normal random variables Z1 and Z2 . By taking the product of the distributions, the joint density function of the two random variables is −(z 2 +z 2 ) exp( 12 2 ) . f (z1 , z2 ) = 2π This distribution is rotationally invariant. This means that the function has the same value for all points equally distant from the origin. This as can be algebraically from the form of the variables z12 + z22 , or from the graph. If T is any rotational transformation of the plane, then f (T (z1 , z2 )) = f (z1 , z2 ). Then it follows more generally that if A is any set in the plane, then P [(Z1 , Z2 ) ∈ A] = P [T (Z1 , Z2 ) ∈ A] for any rotational transformation of the plane. We will apply this observation to the region which is an arbitrary half-plane. 3 Figure 1: The bivariate normal probability distribution exp( 2 +z 2 ) −(z1 2 ) 2 2π . If X1 is normal with mean 0 and variance σ12 and X2 is normal with 0 and variance σ22 . Then X1 + X2 has the same distribution as σ1 Z1 + σ2 Z2 . Hence P [X1 + X2 ≤ t] = P [σ1 Z1 + σ2 Z2 ≤ t] = P [(Z1 , Z2 ) ∈ A] , where A = {(z1 , z2 )|σ1 z1 + σ2 z2 ≤ t} where the p boundary line of the halfplane σ1 z1 + σ2 z2 = t lies at a distance d = |t|/ σ12 + σ22 from the origin. It follows that the half-plane A can be rotated into the set ( ) t T (A) = (z1 , z2 )|z1 < p 2 . σ1 + σ22 See Figure 2 for the case when t > 0, so the half-plane contains the origin. See Figure 3 for the case when t < 0, so the origin is not in the half-plane. Now it is easy to calculate exp( Z Z P [(Z1 , Z2 ) ∈ T (A)] = T (A) 4 −(z12 +z22 ) ) 2 2π dz1 dz2 . Figure 2: The half-plane σ1 z1 + σ2 z2 ≤ t, t > 0 is rotated into the half-plane z1 < √ 2t 2 . σ1 +σ2 Figure 3: The half-plane σ1 z1 + σ2 z2 ≤ t, t < 0 is rotated into the half-plane z1 < √ 2t 2 . σ1 +σ2 5 i hp σ12 + σ22 Z1 < t . It follows that X1 + X2 is p normal with mean 0 and variance σ12 + σ22 . This proof is elementary, self-contained, conceptual, uses nice geometric ideas and requires almost no computation. Thus P [X1 + X2 < t] = P A heuristic explanation It is possible to explain heuristically why the sum of independent normal random variables is normal, using the Central Limit Theorem as given. Recall that the Central Limit Theorem says that if X1 , X2 , . . . is a sequence of independent, identically distributed random variables with mean 0 and variance 1, then X1 + · · · + X n D √ → P [Z ≤ t] P n where Z is normally distributed with mean 0 and variance 1. Then X1 + · · · + Xn D √ P → P [Z1 ≤ t] n and Xn+1 + · · · + X2n D √ P → P [Z2 ≤ t] n where Z1 and Z2 are independent, standard normal random variables. Furthermore X1 + · · · + X2n D √ → P [Z3 ≤ t] P 2n Since X1 + · · · + Xn Xn+1 + · · · + X2n X1 + · · · + X2n √ √ √ + = n n n √ X1 + · · · + X2n √ = 2 2n √ it seems reasonable that the Z1 + Z2 has the same distribution as 2Z3 , that is Z1 + Z2 is normal with variance 2. 6 Problems to Work for Understanding 1. Cite a reference that demonstrates that the distribution of the sum of independent random variables is the convolution of the distributions of the two independent random variables. 2. Show by direct computation of the convolution of the distributions that the distribution of the sum of independent normal random variables is again normal. 3. Suppose that the joint random variables (X, Y ) are uniformly √ dis2 tributed over the unit disk. Show that X has density fX (x) = π 1 − x2 for −1 ≤ x ≤ 1. Using the ideas q from the rotation proof, show that aX + bY has density fc (x) = √ c = a2 + b 2 . 2 cπ 1− x2 c2 for −c ≤ x ≤ c where Reading Suggestion: References [1] Bennett Eisenberg and Rosemary Sullivan. Why is the sum of independent normal random variables normal? Mathematics Magazine, 81(5):362–366, December 2008. 7 Outside Readings and Links: 1. Transformations of Multiple Random Variables, Sum of Two Random Variables. I check all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I make every reasonable effort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your risk. I have checked the links to external sites for usefulness. Links to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don’t guarantee that the links are active at all times. Use the links here with the same caution as you would all information on the Internet. This website reflects the thoughts, interests and opinions of its author. They do not explicitly represent official positions or policies of my employer. Information on this website is subject to change without notice. Steve Dunbar’s Home Page, http://www.math.unl.edu/~sdunbar1 Email to Steve Dunbar, sdunbar1 at unl dot edu Last modified: Processed from LATEX source on May 11, 2010 8