Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Sampling Distributions and the Central Limit Theorem Dr. Tai-kuang Ho, National Tsing Hua University The slides draw from the textbooks of Wackerly, Mendenhall, and Schea¤er (2008) & Devore and Berk (2012) 1 7.1 Introduction We will be working with functions of the variables Y1; Y2; : : : ; Yn observed in a random sample selected from a population of interest. The random variables Y1; Y2; : : : ; Yn are independent and have the same distribution. Certain function of the random variables are used to estimate unknown population parameters. Population mean 2 A random sample of n observations y1; y2; : : : ; yn It seems reasonable to estimate with the sample mean n 1X y= yi n i=1 y depends on Y1; Y2; : : : ; Yn and the function form. y is an example of a statistic. 3 DEFINITION: statistic A statistic is a function of the observable random variables in a sample and known constants. Examples of statistic sample mean: Y sample variance: S 2 maximum: max (Y1; Y2; : : : ; Yn) 4 minimum: min (Y1; Y2; : : : ; Yn) range: R = max (Y1; Y2; : : : ; Yn) min (Y1; Y2; : : : ; Yn) Statistics are used to make inferences about unknown population parameters. Because all statistics are functions of the random variables observed in a sample, all statistics are random variables. Consequently, all statistics have probability distributions, which we will call their sampling distributions. 5 The sampling distribution of a statistic provides a theoretical model for the relative frequency histogram of the possible values of the statistic that we would observe through repeated sampling. Example 7.1 A balanced die is tossed three times. Y1; Y2; Y3 Y = 13 (Y1 + Y2 + Y3) 6 Y E Y = 13 (EY1 + EY2 + EY3) = 31 (3 3:5) = 3:5 V ar Y 0:9722 = V ar 13 (Y1 + Y2 + Y3) = 91 (3 V ar (Y1)) = 13 1 P Y = 33 = p (1; 1; 1) = 6 61 6 = 216 3 P Y = 43 = p (1; 1; 2) + p (1; 2; 1) + p (2; 1; 1) = 216 7 2:9167 = ... P Y = 18 3 = Figure 7.1 Program to simulate Figure 7.1, Eviews ’ simulate.figure.7.prg create u 1 4000 8 series ergib=na for !i=1 to 4000 scalar toss1=@floor(@runif(0,6))+1 scalar toss2=@floor(@runif(0,6))+1 scalar toss3=@floor(@runif(0,6))+1 ergib(!i)=(toss1+toss2+toss3)/3 9 next Before we start the introduction of central limit theorem, we will introduction some additional distributions. 7.2 Sampling Distributions Related to the Normal Distribution We start from the case where there is an exact expression for the sampling distribution (for the statistics). Usually, this comes from speci…c assumption about the random variables Y1; Y2; : : : ; Yn, such as normal distribution. 10 In most application, obtaining exact expression for the sampling distribution is not possible; therefore we have to resort to central limit theorem. In many applied problems, it is reasonable to assume that the observable random variables in a random sample Y1; Y2; : : : ; Yn are independent with the same normal density function. THEOREM: normal sampling distribution Let Y1; Y2; :::; Yn be a random sample of size n from a normal distribution with mean and variance 2. Then 11 n 1X Yi Y = n i=1 and variance 2 = 2=n. is normally distributed with mean Y = Y Proof n Y1 Y2 1X Y = Yi = + + n i=1 n n E (Yi) = 12 Yn + n V (Yi) = 2 E Y 0 n X 1 1 Y1 Y2 Yn YiA = E + + + n i=1 n n n Y1 Y2 1 Yn = E +E + +E = n n n n n = E@ 13 = Y1 Y2 Yn + + + n n n 1 1 = V ( Y + V (Y2) + ) 1 2 2 n n 2 1 2 n = = 2 n n = V V Y 1 + 2 V (Yn) n Notice that the variance of each of the random variable Y1; Y2; : : : ; Yn is 2 and 2 the variance of the sampling distribution of the random variable Y is n . Y N 2 ; ; Y Y 2 Y Y = ; 14 = 2=n Normalization Z has a standard normal distribution. Z= Y Y Y = Y p = n p n Y ! This theorem provides the basis for development of inference procedures about the mean of a normal distribution with known variance 2. p n(Y ) N (0; 1) 15 Example 7.2 Y1; Y2; : : : ; Y9 Yi Y N N ; 2=1 2 ; 9 = 19 P 0:3 =? Y 16 P Y 0:3 = P 0 0:3 0:3 = P@ 0 B = P@ p 17 p n Y p1 9 = P ( 0:9 Example 7.3 Y n 0:3 0:3 Y p1 9 Z 0:9) 1 0:3 A p n 1 0:3 C A p1 9 P 0:3 = 0:95 Y n =? P Y 0:3 = P 0 = P@ = P 0:3 0:3 p n p 0:3 n 18 0:3 Y Y p n Z 1 0:3 A p n p 0:3 n = 0:95 P ( 1:96 1:96) = 0:95 Z p 0:3 n = 1:96 THEOREM: 2 distribution Let Y1; Y2; :::; Yn be de…ned as in the above theorem. Then Zi = independent, standard normal random variables,i = 1; 2; :::; n, and 19 (Yi ) are n X i=1 Zi2 = n X Yi i=1 has a 2 distribution with n degrees of freedom (df). Proof Figure 7.2 P 2> 2 = 20 2 P 2 2 2 is the (1 =1 ) quantile of the 2 random variable. Example 7.4 The 2 distribution is important in the inference about population variance. Suppose we wish to make an inference about the population variance 2 based on a random sample Y1; Y2; : : : ; Yn from a normal distribution. A good estimator of 2 is the sample variance: 21 S2 = 1 n X 1 i=1 n Yi Y 2 THEOREM: use of 2 distribution Let Y1; Y2; :::; Yn be a random sample from a normal distribution with mean variance 2. Then (n 1)S 2 2 n 1 X = 2 22 i=1 (Yi Y )2 and has a 2 distribution with (n variables. We know that p n(Y ) 1) df. Also, Y and S 2 are independent random N (0; 1). When 2 is known, we can use . p n(Y ) N (0; 1) to make inference about When 2 is unknown, it can be estimated by S 2. We use the quantity p n(Y S ) to make inference about . 23 What is the distribution of ! p n(Y S ) t (n p n(Y S ) ? 1) DEFINITION: Student’s t distribution Let Z be a standard normal random variable and let W be a 2-distributed variable with v df. Then, if Z and W are independent, T =q is said to have a t distribution with v df. 24 Z W=v Some properties Y1; Y2; : : : ; Yn Z = Yp W = N ; 2 N (0; 1) n (n 1)S 2 2 2 (n 1) Z and W are independent (because Y and S 2 are independent). According to the above de…nition 25 Y T =q Z W=v =r p (n n = 1)S 2 = (n 2 p n 1) Figure 7.3 Standard normal distribution and t distribution. E (T ) = 0; V (T ) = v v 2 ; v>1 v>2 26 Y S ! t (n 1) Example 7.6 Y1; Y2; : : : ; Y6 N =?; 2 =? P 2S p n Y 27 ! =? P Y 2S p n ! = P 0 2S p n Y B = P@ 2 pS n = P( 2 P Y 28 n ! =? 1 C 2A T (df = 5) If 2 were known, 2 p 2S p n Y 2) ! P Y 2 p n ! = P 0 2 p = P@ 2 = P( 2 2 p Y n Y p Z n 2) 1 n ! 2A Suppose we want to compare the variances of two normal populations based on information contained in independent random samples from the two populations. 2 ! n ! S2 1 1 1 29 2 ! n ! S2 2 2 2 S12 ! S22 S12 = S22 = 2 1 2 2 2 1 2 2 F (n1 1; n2 1) DEFINITION: F distribution Let W1 and W2 be independent 2-distributed random variables with v1 and v2 df, respectively. Then 30 W1=v1 F = W2=v2 is said to have an F distribution with v1 numerator degrees of freedom and v2 denominator degrees of freedom. Figure 7.4 E (F ) = v v2 2 ; 2 V (F ) = v2 > 2 2v22 (v1 +v2 2) ; v1 (v2 2)2 (v2 4) v2 > 4 31 W1 = W2 = (n1 1)S12 2 1 (n2 1)S22 2 2 (n1 1)S12 = (n1 2 W =v 1 F = 1 1= (n2 1)S22 W2=v2 = (n2 2 1) S12= = 2 S2 = 1) 2 Example 7.7 32 2 1 2 2 F (n1 1; n2 1) (Y1; Y2; : : : ; Y6) ! n1 = 6 (Y1; Y2; : : : ; Y10) ! n2 = 10 2 1 = 22 P P S12 S22 S12 S22 2 2 2 1 ! b = 0:95 2 b 22 1 33 ! = 0:95 P [F (5; 9) b] = 0:95 Summary If Y1; Y2; : : : ; Yn are a random sample from a normal distribution with mean and variance 2. p n(Y (n 1)S 2 2 ) N (0; 1) 2 (n 1) 34 p n(Y S S12 = S22 = 2 1 2 2 ) t (n F (n1 1) 1; n2 1) These sampling distributions will enable us to evaluate the properties of inferential procedures in later chapters. 35 7.3 The Central Limit Theorem Y1; Y2; : : : ; Yn represents a random sample from any distribution with mean and variance 2. E Y = ; V Y 2 = n This tells us only the mean and variance of Y , but the distribution of Y is unknown. If Y is normally distributed, then Y is also normally distributed. 36 If Y is not normally distributed, what is the distribution of Y ? We will develop an approximation for the sampling distribution of Y that can be used regardless of the distribution of the population from which the sample is taken. Fortunately, Y will have a sampling distribution that is approximately normal if the sample size is large. The formal statement of this result is called the central limit theorem. An experiment 37 Generate a random sample of size n from an exponential density function with mean 10 and variance 100. f (y ) = ( y 1 e 10 10 0 Figure 7.5 Figure 7.6, n = 5 Figure 7.7, n = 25 38 ; y>0 ; elsewhere Number of simulation is 1000. Table 7.1 make sure that we have enough number of simulation. THEOREM: Central Limit Theorem Let Y1; Y2; :::; Yn be independent and identically distributed random variables with E (Yi) = and V (Yi) = 2 < 1. De…ne Pn Yi i=1 Un = p n n = Y p ; = n 39 n 1X Yi: Y = n i=1 Then the distribution function of Un converges to the standard normal distribution function as n ! 1. That is, lim P (Un n!1 u) = Z u 2 1 p e t =2dt; 1 2 f or all u: DEFINITION: normal probability distribution A random variable Y is said to have a normal probability distribution if and only if, for > 0 and 1 < < 1, the density function of Y is 40 1 f (y ) = p exp 2 " (y ) 2 2 2# ; 1<y<1 The theorem states that probability statement about Un can be approximated by corresponding probabilities for the standard random normal variable if n is large. n 30 Equivalent statements: 41 Yp = n is asymptotically normally distributed with mean 0 and variance 1. Y is asymptotically normally distributed with mean 2 and variance n . The central limit theorem can be applied to a random sample Y1; Y2; : : : ; Yn from any distribution as long as E (Yi) = and V (Yi) = 2 are both …nite and the sample size is large. Example 7.8 Y1; Y2; : : : ; Yn 42 = 60 2 = 64 n = 100 P Y 58 ? P Y 58 0 60 BY p8 100 = P@ ' P (Z 43 2:5) 58 1 60 C p8 100 A 7.4 A Proof of the Central Limit Theorem Proof Un = Y p Pn Yi = i=1 p = n n 1 X Zi; = p n i=1 n n Y Zi = i 1 =p n mPn Zi (t) = mZ1 (t) mZ2 (t) i=1 44 Pn i=1 Yi mZn (t) n ! Y = aX + b mY (t) = ebt mX (at) ! 1 mUn (t) = mPn Zi p t i=1 n ! ! 1 1 = mZ1 p t mZ2 p t n n " !#n 1 = mZ1 p t n 45 ! 1 mZn p t n 00 ( ) m Z1 mZ1 (t) = mZ1 (0) + m0Z1 (0) t + t2; 2 0< mZ1 (0) = 1 m0Z1 (0) = E (Z1) = 0 m00Z ( ) 2 1 t ; mZ1 (t) = 1 + 2 46 0< <t <t t t!p n mUn (t) = = n!1) " " !#n 1 p t n mZ1 1+ 1 t2 n2 m00Z1 ( ) 2 = 41 + #n ; m00Z ( 1 0< ) 2 t <p n !23n t 5 ; p n 0< t <p n !0 2 2 2 ! 0 ) t2 m00Z ( ) ! t2 m00Z (0) = t2 ; 1 1 47 * m00Z (0) = E Z12 = 1 1 lim bn = b ) lim n!1 n!1 1+ bn n n = eb lim mUn (t) = lim n!1 mZ (t) = exp 12 t2 n!1 " 1+ 1 t2 n2 m00Z1 ( N (0; 1) lim mUn (t) n!1 48 N (0; 1) ) #n = t2 e2 7.5 The Normal Approximation of the Binomial Distribution The central limit theorem can be used to approximate probabilities for some discrete random variables when the exact probabilities are tedious to compute. An example is binomial distribution Y P (Y b (n; p) b) Method 1: use table and exact values 49 Method 2: approximation using CLT Y = n X Xi i=1 Xi = ( 1 ; success ; 0 ; otherwise E (Xi) = p 50 i = 0; 1; 2; : : : ; n V (Xi) = p (1 Un = Y p = n Y p = p Y p(1 p) p n Y N p) p; n X =r p(1 p) n p (1 p) n 1 Y Xi = X ! N = n n i=1 51 p N (0; 1) ! p (1 p) p; n ! Y ! N (np; np (1 52 p))