Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Sample Mean and Central Limit Theorem Lecture 21-22 November 17-21 Outline • • • • • Sums of Independent Random Variables Chebyshev’s Inequality Estimating sample sizes Central Limit Theorem Binomial Approximation to the normal Sample Mean Statistics Let X1,…Xn be a random sample from a population (e.g. The Xi are independent and identically distributed). The sample mean is defined as 1 n x = ∑ xi . n i =1 What can we say about the distribution of the sample mean? Sample Mean for Normal Let X1,X2,…,Xn be a random sample from a normal (μ,σ). What is the pdf of x ? Hint use the mgf method. The mgf of a normal is: mx (t ) = e ( μt + t 2α 2 ) 2 Solution The mgf of Y=∑Xi is ⎡ m y (t ) = [mx (t )] = ⎢e ⎢⎣ n t 2α 2 ( μt + ) 2 But we want x = y/n mx (t ) = m y (t / n) = e n ⎤ ⎥ =e ⎥⎦ ( nμ t / n + nt 2α 2 ( nμ t + ) 2 n ( t / n )2 α 2 ) 2 =e ( μt + ( t )2 α 2 ) 2n Thus the mean is normal with mean μ and variance σ 2 n For any distribution with mean μ and standard deviation σ What is mean and variance of x ? ⎛ ∑ xi E(x ) = E ⎜ i ⎜ n ⎜ ⎝ ⎞ ⎟ = 1 E ( x ) = 1 nμ = μ i ⎟ n∑ n i ⎟ ⎠ ⎛ ∑ xi var( x ) = var ⎜ i ⎜ n ⎜ ⎝ ⎞ ⎟= 1 ⎟ n2 ⎟ ⎠ 1 σ2 2 ∑i var( xi ) = n2 nσ = n What if we don’t know the distribution? If we don’t want to make assumptions about what the distribution is, can we still do things? • Chebyshev’s Inequality • Central Limit Theorem Amazing property true for any distribution Chebyshev inequality: Consider any random variable with mean μ and standard deviation σ. For any k>0, P[|X- μ|≥k σ)≤1/k2 E(X)-kσ E(X) E(X)+kσ In words, The probability that X deviates from its expected value at least k standard deviations is less than 1 k2 Example Suppose that, on average, a post office handles 10000 letters a day with a variance of 2000. What can be said about the probability that this post office will handle between 8000 and 12000 letters tomorrow? Using Chebshev’s X = number of letters will handle tomorrow μ = E ( X ) = 10000 α 2 = var( X ) = 2000 Want P(8000 < X < 12000). P(8000 < X < 12000) = P(−2000 < X − 10000 < 2000) = P(| X − 10000 |< 2000) = 1 − P(| X − 10000 |> 2000) ≥ 1 − .0005 = .9995 Because by Chebyshev’s P(| X − 10000 |> 2000) = P(| X − 10 |> 2000σ ) 1 ≤ = .0005 2000 Alternative forms of Chebyshev For any k>0, P[|X- μ|≥kσ)≤1/k2 For any t>0 P[|X- μ|≥t)≤ σ 2/t2 E(X)-t E(X) by setting t= kσ E(X)+t Chebyshev’s applied to Sample Mean Recall if X , …, X 1 n are independent and identically distributed with mean μ and standard deviation σ then ⎛∑x ⎞ ⎟ = 1 E ( x ) = 1 nμ = μ E(x ) = E ⎜ ⎜ n ⎟ n∑ n i i ⎜ ⎝ ⎛ ∑ xi var( x ) = var ⎜ i ⎜ n ⎜ ⎝ ⎟ ⎠ ⎞ ⎟= 1 ⎟ n2 ⎟ ⎠ i i 1 σ2 2 ∑i var( xi ) = n2 nσ = n Chebyshev’s applied to Sample Mean Applying Chebyshev’s to the sample mean then for any ε>0 α P(| X − μ |≥ ε ) ≤ 2 ε n 2 X Example A biologist wants to estimate the life span of a type of insect. He takes a sample of size n and measure the lifetime from birth to death of each insect. Then he averages these numbers. If he believes the lifetimes of the insect are iid with variance 1.5 days. How large a sample should he choose to be at least 98% sure that his average is accurate within plus or minus 0.2 days (4.8 hours)? Solution Let Xi be the life time of the ith insect. We want to find n such that P(−0.2 < X − μ < 0.2) ≥ .98 Or equivalently P(| X − μ |> 0.2) ≤ 1 − .98 = .02 By Chebyshev’s P (| X − μ |> t ) ≤ α2 n t2 (1.5) 2 37.5 P (| X − μ |> 0.2 ) ≤ = = .02 2 n (.2) n So we need n ≥ ⎡⎢37.5 / .02 ⎤⎥ = 1875. Central Limit Theorem Idea: No matter what the population distribution may be, if n is large then the distribution of the sample mean is approximately normal with 2 σ mean μ and variance . n The larger the n, the better the approximation. Good approximation for n>30. Try applet http://www.ruf.rice.edu/~lane/stat_sim/sampling_di st/ Central Limit Theorem If x Let has mean μ and standard deviation x −μ U= σ/ n For large n, the distribution of U is approximately Normal(0,1) σ n Example A soft drink machine dispenses drinks in a cup. The amount dispensed is a R.V. with mean 200 ml and s.d 15 ml. What is the probability that the average amount dispensed in a random sample of size 36 is at least 204 ml? By CLT, x ∑ x= i 36 is approximately normal (200,15/6) Solutions ⎛ ( x − μ ) n (204 − μ ) n ⎞ P ( X > 204) = P ⎜⎜ > ⎟⎟ σ σ ⎝ ⎠ (204 − 200)6 ⎞ ⎛ ≈ P⎜Z > ⎟ = P( Z > 1.6) = .0548 15 ⎝ ⎠ Z is standard normal Sample Size Problem Note that as sample size increases, the sample standard deviation σ / n gets smaller. We can use the sample mean to estimate the true mean. By making the sample size bigger, we can make the estimate as accurate as we desire. Sample Size Problem Say we know the standard deviation is σ=3 for each item in a random sample. Say we want P(| X − μ |< .5) to be close to .95. How big should the sample size be? You try it? Solution Using CLT approximation, | X −μ| .5 < ) = .95 3/ n 3/ n .5 .95 = P(| Z |< 1.96) = P (| Z |< ) 3/ n solve for n and round up n = 139 P (| X − μ |< .5) = P( Note You can also estimate n using Chebyshev’s by the bound is not as strong P (| X − μ |< .5) = 1 − P(| X − μ |≥ .5) = .95 P (| X − μ |≥ .5) = σ2 n(.5) 2 = .05 ⎡ 32 ⎤ So need n = ⎢ = 720 2⎥ ⎢ .05(.5) ⎥ which is much larger than the estimate based on CLT n = 139 Problem We admit 1500 students to RPI and we know historically 2/3 actually attend on average. We can assume the decision of each student to attend is independent. What is the probability that more than 1050 students attend? Thoughts What is the distribution of the number of students that attend? Y=Binomial(n=1500,p=2/3) How can we compute P(Y>1050)? We learned that we can approximate binomial using a Poisson with λ=np if n is very large and p is very small so that λ is small (<=10). But this is not the case here. Normal approximation to Binomial The acceptance of each student can be modeled as a Bernoulli random variable X = 1 with probability 2/3with mean p=2/3 X = 0 with probability 1/3 and s.d. p(1 − p) = 2 / 9 ∑i X i The ratio of students attending =X n can be approximated as a normal with mean 2/3 and variance 2 σ2 i i 9(1500) by CLT = n continued For Y=the number of students admitted Y is really binomial with mean np=1000 and variance np(1-p)=1500*2/3*1/3 You can approximate it by CLT as normal With same mean and variance since Y = nX (approximately normal ) E (Y ) = np equivalently var(Y ) = np(1 − p) Y − np Y − 1000 = ∼ N (0,1) np (1 − p ) 1500 ( 2 / 3) (1/ 3) ⎛ Y − 1000 1050 − 1000 ⎞ ⎟ ≈ P( Z > 2.73) = 0.0027 P⎜ > ⎜ 1500 ( 2 / 3) (1/ 3) 1500 ( 2 / 3) (1/ 3) ⎟⎠ ⎝ Note normal is continuous and binomial is discrete so we can improve the approximation a bit P(Y=1050) =0 Approximate by applying CLT to P(1049.5≤Y ≤1050.5) So better approximation for P(Y>1050) is ⎛ Y − 1000 1050.5 − 1000 ⎞ ⎟ ≈ P( Z > 2.766) = 0.0028 P⎜ > ⎜ 1500 ( 2 / 3) (1/ 3) 1500 ( 2 / 3) (1/ 3) ⎟⎠ ⎝ Note normal is continuous and binomial is discrete so we can improve the approximation a bit Consider P(Y=1050) >0 Approximate by applying CLT to P(1049.5≤Y ≤1050.5) So better approximation for P(Y>1050) is ⎛ Y − 1000 1050.5 − 1000 ⎞ ⎟ ≈ P( Z > 2.766) = 0.0028 P⎜ > ⎜ 1500 ( 2 / 3) (1/ 3) 1500 ( 2 / 3) (1/ 3) ⎟⎠ ⎝ Normal Approximation to Binomial If Y is binomial n and p, then if we want P(a≤Y≤b) use P(a-1/2 ≤Y≤b+1/2) ⎛ y − np ⎛ y − np b + 1/ 2 − np ⎞ a − 1/ 2 − np ⎞ = P⎜ ≤ ≤ ⎟⎟ − P ⎜⎜ ⎟⎟ ⎜ np (1 − p) np (1 − p ) ⎠ np(1 − p) ⎠ ⎝ ⎝ np (1 − p ) ⎛ ⎛ b + 1/ 2 − np ⎞ a − 1/ 2 − np ⎞ = P⎜Z ≤ ⎟⎟ − P ⎜⎜ Z ≤ ⎟⎟ ⎜ np (1 − p ) ⎠ np (1 − p ) ⎠ ⎝ ⎝ Check to see if you got it If Y is binomial (n, p), then if we want P(a≤Y≤b) use approximation on P(a-1/2 ≤Y≤b+1/2) P(a<Y≤b) use approximation on P(a+1/2 ≤Y≤b+1/2) P(a<Y<b) use approximation on P(a+1/2 ≤Y≤b-1/2) P(a ≤ Y<b) use approximation on P(a-1/2 ≤Y≤b-1/2) How good is the approximation? Use the normal approximation to the binomial to determine the probability of getting 2 heads and 3 tails in 5 flips of a balanced coin. What is the actual distribution? f(x) 0 1/32 1 5/32 2 3 4 10/32 10/32 5/32 5 1/32 Normal approximation Mean is np=5/2=2.5 np (1 − p ) = 1.25 s.d. = P (2 heads ) = P (2.5heads ) − P (1.5heads ) ⎛ x − 2.5 2.5 − 2.5 ⎞ ⎛ x − 2.5 1.5 − 2.5 ⎞ = P⎜ ≤ ≤ ⎟ − P⎜ ⎟ 1.118 1.118 1.118 1.118 ⎝ ⎠ ⎝ ⎠ = P( Z ≤ 0) − P ( Z < −1.0) = .5 − (1 − .8133) = .3133 Compare to .3125