Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
A Proof of the Central Limit Theorem Preliminaries: Two Definitions and Two Theorems Definition One: The rth moment of a random variable X about the mean is ππ = πΈ[(π β π)π ] , π = 0,1,2,3, β¦ [Note: It follows that π0 = 1, π1 = 0, and π2 = π 2 .] Thus, we have π ππ = βππ=1(π₯π β π) π(π₯π ) = β(π₯ β π)π π(π₯) β ππ = β«ββ(π₯ β π)π π(π₯)ππ₯ (discrete variable) (continuous variable) Definition Two: The moment-generating function (MGF) of X is ππ (π‘) = πΈ(π π‘π ) Thus, we have ππ (π‘) = βππ=1 π π‘π₯π π(π₯π ) = β π π‘π₯ π(π₯) β ππ (π‘) = β«ββ π π‘π₯ π(π₯)ππ₯ (discrete case) (continuous case) [Note: The name moment-generating function comes from the fact that ππ (π‘) = πΈ(π π‘π ) = πΈ (1 + π‘π + π‘2π2 2! + π‘3π3 3! π‘2 +β―) π‘3 = 1 + πΈ(π)π‘ + πΈ(π 2 ) 2! + πΈ(π 3 ) 3! + β― which shows the moments about the origin π = 0 [πΈ(π), πΈ(π 2 ), πΈ(π 3 ), β¦] are generated as the coefficients in this expansion in π‘.] Proof of Central Limit Theorem Page Two Theorem One: The MGF for the general normal distribution is π(π‘) = π ππ‘+(π 2 π‘ 2 /2) Proof: By definition, we have β 1 π(π‘) = πΈ(π π‘π₯ ) = πβ2π β«ββ π π‘π₯ π β(π₯βπ) 2 /2π 2 ππ₯ Letting (π₯ β π) / π = π£ so that π₯ = π + ππ£ and ππ₯ = πππ£, this becomes π(π‘) = 1 β2π π ππ‘+(π 2 π‘ 2 /2) β β«ββ π β(π£βππ‘) 2 /2 ππ£ Now letting π£ β ππ‘ = π€, we have π(π‘) = π ππ‘+(π 2 π‘ 2 /2) ( β 1 β« π βπ€ β2π ββ 2 /2 ππ€) = π ππ‘+(π 2 π‘ 2 /2) Corollary: The MGF for the standard normal distribution is π(π‘) = π π‘ 2 /2 Theorem Two (Uniqueness Theorem): Two random variables have the same probability distribution if and only if their moment-generating functions are identical. We are now ready to prove the Central Limit Theorem. Proof of Central Limit Theorem (CLT) Theorem (CLT): Let π1 , π2 , β¦ , ππ be independent random variables that are identically distributed (that is, all have the same probability mass function in the discrete case or probability density function in the continuous case) and have finite mean π and variance π 2 . Then if ππ = π1 + π2 + β― + ππ (n = 1, 2, β¦), lim π(π β€ πββ ππ βππ π βπ β€ π) = 1 π β« π βπ’ β2π π 2 /2 ππ’ that is, the random variable (ππ β ππ)/(πβπ ), which is the standardized variable corresponding to ππ , is asymptotically normal. Note: Since (ππ β ππ)/(πβπ ) = (πΜ β π)/(πββπ ) = π, the more familiar form of the CLT follows as a corollary. Proof of Central Limit Theorem Page Three Proof: For π = 1, 2, β¦, we have ππ = π1 + π2 + β― + ππ . Now π1 , π2 , β¦ , ππ each have mean π and variance π 2 . Thus, πΈ(ππ ) = πΈ(π1 + π2 + β― + ππ ) = πΈ(π1 ) + πΈ(π2 ) + β― πΈ(ππ ) = ππ and, because the ππ are independent, πππ(ππ ) = πππ(π1 + π2 + β― + ππ ) = πππ(π1 ) + πππ(π2 ) + β― + πππ(ππ ) = ππ 2 It follows from this that the standardized random variable corresponding to ππ is ππ β = ππ βππ π βπ The MGF for ππ β is thus β πΈ(π π‘ππ ) = πΈ[π π‘(ππ βππ)/(πβπ) ] = E[et(X1 βΞΌ)/(Οβn) et(X2 βΞΌ)/(Οβn) β― et(XnβΞΌ)/(Οβn) ] = πΈ[et(X1 βΞΌ)/(Οβn) ]β πΈ[et(X2βΞΌ)/(Οβn) ]β― πΈ[et(XnβΞΌ)/(Οβn) ] t(X1 βΞΌ) Οβn = {πΈ[e π ]} [The last step is a result of the ππ being identically distributed and the preceding step because the ππ are independent.] Now, by a Taylor series expansion, we have t(X1 βΞΌ) Οβn πΈ [e ] = πΈ[1 + π‘(π1 βπ) π βπ = πΈ(1) + = 1+ π‘ π βπ π‘ π βπ + π‘ 2 (π1 βπ)2 2!π 2 π + β―] πΈ(π1 β π) + (0) + π‘2 2π 2 π π‘2 2π 2 π πΈ(π1 β π)2 + β― (π 2 ) + β― = 1 + π‘2 2π +β― Thus, we have β π‘2 π πΈ(π π‘ππ ) = {1 + + β― } = (1 + 2π 2 π‘ 2 β2 π + β― )π But the limit of this as π β β is π π‘ β2 , which is the MGF of the standard normal distribution. Hence, by Theorem Two above, the required result follows. Proof of Central Limit Theorem Page Four Corollary: Suppose the population from which samples are taken has some probability distribution with mean π and variance π 2 . Then the standardized variable associated with πΜ , given by π= πΜ βπ π ββ π is asymptotically normal, that is lim π(π β€ π§) = πββ 1 π§ β« π βπ€ β2π ββ 2 β2 ππ€ Note: The above proof of the CLT, slightly edited, was given by the late Murray R. Spiegel of RPI in his 1975 Schaumβs Outline Series book Probability and Statistics in Problem 4.25, which appears as such in the current Third Edition of the book, with John J. Schiller and R. Alu Srinivasan of Temple University now as coauthors. DB