Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
BEHAVIOR OF SAMPLE AVERAGES . . . CENTRAL LIMIT THEOREM a a a a a a a a a a a a a a a a a a a a a a a a a a So what exactly is special about sample averages? Let’s play a hypothetical game. Let’s suppose that the population is genuinely normal, with mean μ = 400 and standard deviation σ = 80. We are playing this as hypothetical, so that we use the actual values 400 and 80 when setting up the population but we recognize that the person taking the sample might have no idea about μ and σ. Here is a graph that represents the density of the population: Distribution Plot Normal, Mean=400, StDev=80 0.005 Density 0.004 0.003 0.002 0.001 0.000 100 200 300 400 X 500 600 700 Suppose that we took a sample of 25 values. It might look like this: 342 365 491 546 513 439 341 362 381 359 457 324 434 302 418 373 221 531 390 349 460 441 427 417 621 The values here came as rounded to the nearest integer. This sample has x = 412.2 and s = 86.5. These are reasonably close to the true population values, so it looks like a “good” sample. Of course, the person doing the sampling has no idea whether it’s good or not, as there is no access to the true values of μ and σ. 1 a a ©gs 2011 BEHAVIOR OF SAMPLE AVERAGES . . . CENTRAL LIMIT THEOREM a a a a a a a a a a a a a a a a a a a a a a a a a a The histogram for this sample suggests that it’s well-behaved. Histogram of Sample1 7 6 Frequency 5 4 3 2 1 0 200 300 400 Sample1 500 600 Here’s another sample of 25: 452 397 383 389 369 538 415 504 322 423 349 428 396 515 489 335 349 256 343 260 349 336 385 357 287 For this sample of 25, x = 385.0 and s = 74.1. The histogram also looks reasonable: Histogram of Sample2 6 Frequency 5 4 3 2 1 0 240 280 320 360 400 Sample2 440 480 520 2 a a ©gs 2011 BEHAVIOR OF SAMPLE AVERAGES . . . CENTRAL LIMIT THEOREM a a a a a a a a a a a a a a a a a a a a a a a a a a We can do this indefinitely, drawing sample after sample. It is possible to show mathematically that E X =μ SD( X ) = (here μ is 400) σ = n σ σ = 5 25 (here σ is 80, so σ is 16) 5 This population was set up as normal, so the distribution of X is exactly normal with mean μ = 400 and standard deviation equal to 16. Examine these two logical statements: If then X1, X2, …, Xn is a sample from a population with mean μ and standard deviation σ, σ E X = μ and SD( X ) = . n X1, X2, …, Xn is a sample from a population with mean μ and standard deviation σ, and if the population distribution is normal, σ then E X = μ , SD( X ) = , and the distribution of X is itself normal. n If The second is more useful, as it has the more powerful conclusion. This document will end with yet another logical statement. Check carefully! 3 a a ©gs 2011 BEHAVIOR OF SAMPLE AVERAGES . . . CENTRAL LIMIT THEOREM a a a a a a a a a a a a a a a a a a a a a a a a a a In fact, we can play with this, since we are running simulations. Let’s generate 10,000 samples, each of size 25, and let’s compute X for each. Here is a histogram of those 10,000 sample averages (with the best-fitting normal curve superimposed): Histogram of C3 Normal Mean 400.2 StDev 15.88 N 10000 500 Frequency 400 300 200 100 0 352 368 384 400 C3 416 432 448 This is an excellent picture. The inset box from Minitab identifies the average of these 10,000 averages as 400.2, very close to the target value of 400. It also identifies the standard deviation as 15.88, close to the target value of 16. Please remember….we are playing a hypothetical game. No one would really draw 10,000 separate samples, each of size 25. That’s 250,000 values, and we would probably just assemble them into a single very large sample. 4 a a ©gs 2011 BEHAVIOR OF SAMPLE AVERAGES . . . CENTRAL LIMIT THEOREM a a a a a a a a a a a a a a a a a a a a a a a a a a Let’s now add a wrinkle to this problem. Suppose that the population is not exactly normal – and in fact is badly non-normal. Below is the probability density for a population with mean μ = 800 and standard deviation σ = 565.69. Skewed population, mean = 800, standard deviation = 565.69 0.0009 0.0008 0.0007 Density 0.0006 0.0005 0.0004 0.0003 0.0002 0.0001 0.0000 0 500 1000 1500 2000 2500 3000 3500 X Technical side note. These were actually generated as a gamma distribution with shape parameter = 2 and scale parameter = 400. The mean is 2 × 400 = 800 and the standard deviation is 2 × 400 ≈ 565.69. Let’s suppose that we took a sample of size 5 from this population, and that we let X represent the average of these 5 values. We can promise that E X = 800 and SD( X ) 565.69 = ≈ 252.98. We can make no promise about the distribution of X , however. 5 Technical side note, continued. The distribution of X is gamma with shape parameter = 10 and scale parameter = 80. 5 a a ©gs 2011 BEHAVIOR OF SAMPLE AVERAGES . . . CENTRAL LIMIT THEOREM a a a a a a a a a a a a a a a a a a a a a a a a a a Here is a picture of 10,000 tries at drawing a sample of 5. The histogram shows values of the sample averages. Averages of 5 from previous skewed distribution Mean 794.0 StDev 251.5 N 10000 400 Frequency 300 200 100 0 250 500 750 1000 1250 GammaAveOf5 1500 1750 2000 The curve is an approximating normal. It’s a bad fit to normal, but somehow the averages of 5 are not as badly behaved as averages of originals. Suppose we did samples of size 10 and let X be the average of 10. This will have 565.69 E X = 800 and SD( X ) = ≈ 178.89. Here is a plot of the averages from 10,000 10 samples of size 10: Averages of 10 from previous skewed distribution 500 Mean 798.8 StDev 178.2 N 10000 Frequency 400 300 200 100 0 360 540 720 900 1080 GammaAveOf10 1260 1440 The normal approximating curve here (for averages of samples of 10) looks a little closer to the normal approximating curve than for the previous (for averages of samples of size 5). Notice too that the horizontal axis is more compact, reflecting the lower standard deviation. 6 a a ©gs 2011 BEHAVIOR OF SAMPLE AVERAGES . . . CENTRAL LIMIT THEOREM a a a a a a a a a a a a a a a a a a a a a a a a a a So . . . what if we took larger samples? Let’s try samples of size 50. Here is a repeat of the previous. Average of 50 from previous skewed distribution Mean 800.1 StDev 79.37 N 10000 500 Frequency 400 300 200 100 0 560 640 720 800 880 GammaAveOf50 This should have mean 800 and standard deviation 960 1040 1120 565.69 ≈ 80. The fit to the 50 approximating normal curve is excellent. What we’ve seen here, in a completely empirical way, is that averages from larger samples seem to have distributions that are closer to normal. In the case shown here, averages of 50 look closer to normally distributed than averages of 10 averages of 10 look closer to normally distributed than averages of 5 This can in fact be mathematically proved. It’s called the Central Limit theorem, one of the most important results in statistics. Here is that statement: X1, X2, …, Xn is a sample from a population with mean μ and standard deviation σ, and if the sample size is large, σ then E X = μ , SD( X ) = , and the distribution of X is itself n approximately normal. If 7 a a ©gs 2011 BEHAVIOR OF SAMPLE AVERAGES . . . CENTRAL LIMIT THEOREM a a a a a a a a a a a a a a a a a a a a a a a a a a This leaves open exactly what we mean by “sample size is large.” Most practitioners will invoke the rule n ≥ 30. If the original population was only mildly non-normal (say not badly skewed), then the Central Limit theorem should work with smaller n, perhaps down to 10. For even badly non-normal populations, like the one used here, the cutoff of 30 is quite reasonable. The example here certainly had bad skewness; the values were positive and the standard deviation of 565.69 was almost as large as the mean 800. We also should mention what we want to mean by “the distribution of X is itself approximately normal.” This decision hangs on the level of precision that we want. If a true probability of 0.4248 is approximated as 0.4189, we would not be upset. However, approximating a true probability of 0.0014 by 0.0031 would be a disaster. If this were a disease probability, we’d be doubling a risk! Of course, we never get to know the true probability; we only have the approximation. The troubles come in answers that are very close to zero or to 1. Thus, we should worry about the Central Limit theorem approximation in cases where our calculations produce values like 0.01 or 0.99. It’s also hard to operationalize the “worry” notion. 8 a a ©gs 2011