Download The Central Limit Theorem (CLT)

The Central Limit Theorem (CLT)
We found a very useful statistical theorem in the past century, called the Central Limit
Theorem. It will allow us to be effective at predicting future outcomes, formally called
inferential statistics.
The CLT for proportion p's:
Refer to the picture below to help explain the nature of this version of the theorem.
We start out with a population which is a categorical variable, a binary variable, with a
parameter, p, signifying the proportion of “successes” in that population. The shape of this
binary distribution is not symmetrical, but we don't know exactly how. The CLT asserts that
if we take samples “appropriately” (random, independent, and having the np/n(1-p)
criteria or assumptions), then find the p-hat of that sample, then repeat the sample and
statistic again and again, theoretically indefinitely, then the resulting sampling
distribution of those p-hats will be approximately normally distributed (bell-shaped), with
p (1− p)
center at the parameter (p) and having standard deviation
. Stated
n
symbolically we say that the distribution of
̂p ∼ N ( p , √( pq /n )) . The “squiggle” with the dot above it symbolizes the words “is
approximately distributed as”, and the whole expression is stated as “The distribution of phats is approximately normally distributed, with mean the parameter of the population (p)
and having standard deviation √(pq/n), where q = 1 – p , and n = sample size.
√
It makes sense that as n increases, the standard deviation of the sampling distribution bellshape will get smaller (the bell will get skinnier) about the center, p. And, the
mathematical formula shows that, because as n gets bigger, the square root gets smaller. In
the theoretical limit, where n is infinity, the standard deviation of that sampling
distribution is 0, and the sampling p-hat will always be the population p. We also know
that this is true from the Law of Large Numbers, which says that as the sample size
approaches infinity, the statistic p-hat will approach the theoretical population parameter,
p.
-1-
We have 3 assumptions about taking all of the samples of size n from the original
population we need to have satisfied if we are to have our bell-shaped result on the
sampling distribution.
First, we must obtain randomly attained samples. If we cannot meet that rigid criteria,
then at least we must have some evidence or assertion that the samples we take are
“representative” of the population they come from.
Next, we must have independent samples obtained. If we cannot meet that rigid criteria
or know if we are, indeed, independent (which is usually the case), then we must have the
knowledge that our sample size, n, is less than 10% of the total population of individual
units.
Last, we must have the product that np > 10 and n(1-p) = nq > 10 . This is what I call the
shape-shifting criteria, the criteria that will morph our shape from whatever its shape is
in the population to the bell-shaped sampling distribution shape. We must adhere to this
last criteria, with no “alternatives” as in the first two assumptions.
So, we say that the CLT for proportions says “The sampling distribution of the population,
given the assumptions are adhered to while sampling, is approximately normally
distributed, with center being the population parameter, p, and the standard deviation of
the sampling distribution standard deviation = √(pq/n).”
The CLT for means
µ's:
Again, use the picture below to guide you through the explanation.
Again we start out with a population of unknown shape, although this time (with means)
the shape can be normal or near normal in shape.
-2-
If we take a large number (approaching infinity) of “appropriately obtained” samples of
size n, taking the sample mean, x-bar, from each sample obtained, we will have a sampling
distribution of x-bars which is near normally distributed (bell-shaped), having center the
parameter µ of the population, and having sampling distribution standard deviation =
s/√n . The mathematical statement of the
σ
) , which says in words “the sampling distribution
√n
of x-bars is approximately normally distributed, centered at the population mean and
having standard deviation = population standard deviation divided by square root of
n”.
sampling distribution is
̄x∼N (μ ,
The first 2 assumptions which must be adhered to when sampling from the population are
the same as those for sampling from proportion p's. The last assumption, the “shape
shifting” assumption (which morphs the shape of the population into the shape of the
sampling distribution) says that either we have to start with a rather bell-shaped population
or, if not, then we must have a “healthy size” of n, in the neighborhood of about 40 or
more.
We will see later that practically speaking we rarely know what the population standard
deviation is for the means CLT case. We mostly use the sampling distribution middle (which
we can easily get by constructing various sample x-bars)--give or take—and come up with an
estimate of the unknown population mean, using the results of the CLT when we construct
hypothesis tests and confidence intervals (we will talk about them later). If we don't
know the population mean, most likely we won't know the population standard deviation
either (especially since mathematically the standard deviation is computed from the
mean!). We will talk about how we use an estimate of the population standard deviation,
the sample standard deviation (s) along with the “t” statistic, to approximate the unknown
σ we need.
We will talk about inferential statistics, and how the CLT helps us out with this branch of
“predicting future or unknown values” statistics later on.
-3-