Download Normal Approximation to Binomial Distributions

Normal Approximation to Binomial Distributions
Charlie Vollmer
Department of Statistics
Colorado State University
Fort Collins, CO
[email protected]
September 18, 2015
Abstract
This document is a supplement to class lectures for STAT 307-003, Fall, 2015. It
details how the Normal Distribution can approximate the Binomial Distribution as the
number of trials, n, gets large.
How large does n need to be?
How well does the Normal Distribution approximate a Binomial Distribution?
Let us find out....
1
Setup: Defining some terms
1.1
Expected Value
If we go to wikipedia, the following is the very first sentence that we’ll see:
In probability theory, the expected value of a random variable is intuitively the long-run
average value of repetitions of the experiment it represents.
Great! It is simply what we expect to see most often if we did something over and over
and over again! And if we go down a few more sentences on the wikipedia page, we find
something even more useful:
The expected value is also known as the expectation, mathematical expectation, EV,
mean, or first moment.
Bam! Look at that fourth synonym: the mean! That is exactly what I would expect to
see most often if I did an experiment over and over and over lots of times!
Note: if a Random Variable is Binomially Distributed, its mean is: np.
1
1.2
Standard Error, SE
If we go to wikipedia, the following is the first two sentences that we’ll see:
The standard error (SE) is the standard deviation of the sampling distribution of a
statistic,[1] most commonly of the mean. The term may also be used to refer to an estimate
of that standard deviation, derived from a particular sample used to compute the estimate.
Ok, this is wordy but it’s actually very accurate and descriptive. It’s just saying that
the standard error, SE, is the standard deviation of our statistic.
So... if our statistic is:
Sn =
n
X
Xi
i=1
Where Xi is simply a 1 or 0, in the case of a coin toss (heads or tails), then our SE of
this statistic is its standard deviation.
Now, we know -from class- that this statistic, Sn , is a Binomially Distributed Random
Variable (it follows a Binomial Distribution). In the case of a binomial, we (humans... and
now you, too!) know that the variance of a Binomially Distributed Random Variable is
simply:
npq
Do you remember how to find the standard deviation from the variance? Well, if you
need the SE, it’s just the standard deviation.
So, now we know how to get the variance from a binomial, which means we have the
standard deviation or -in other words- the standard error, SE!
2
2
The Approximation: Toss a coin 100 times
The object of this section is to illustrate how if we plot out the histogram of the number
of heads from a coin toss will be well approximated by a Normal Distribution as the number
of tosses, n gets “large.”
For instance, say that we toss a fair coin 100 times and see how many times that we get
heads. We could do this and get 88 heads. We could also do it and get 45 heads. Let’s say
we do the entire experiment (toss the coin 100 times) 50 times. Thus, we’ll get 50 different
numbers. Let’s see what that plot looks like:
6
count
4
2
0
40
45
50
55
60
65
Number of Heads
It looks like on one experiment we got 40 heads and one time we got 41 heads. On
another experiment we got 61 heads. Yet again, on 6 experiments we got 48 heads, and 6
more experiments we got 59 heads.
You get the picture.
So, in this situation, we only did this experiment (toss a coin 100 times) 50 times. And
the plot above shows our results from those 50 experiments.
What happens if we did this experiment 100 times? Or a thousand times??
Let us see...
3
Below, we see what happens when we do this experiment 500, 1000, and 5000 times:
0.075
density
density
0.10
0.050
0.05
0.025
0.00
0.000
40
45
50
55
60
40
50
fifty
60
thousand
0.08
0.075
density
density
0.06
0.04
0.050
0.025
0.02
0.00
0.000
40
50
60
30
five_hundred
40
50
60
70
five_thousand
Whoa! We see that our histograms start to look like a bell curve!
Clearly, this is no coincidence!
This is because a Binomial Random Variable begins to look like a Normally Distributed
Random Variable as the number of trials, n, grows large!
Careful!! Take notice that we did NOT increase n yet, only the number of
times that we did the experiment!
So, now if we increase n, we would expect to see this bell-shaped-looking curve actually
start to look more and more like a Normal Distribution. As of now, you can notice that it
doesn’t quite look like a normal distribution, but rather just a similar-looking curve.
4
3
The Approximation: Toss a coin 100 times
Now, we do the same thing as above, but each experiment is tossing the coin 1000 times.
What do you think this does to the Expected Value? Ponder this question:
Is it easier to get all heads if I only toss the coin 10 times?
Would it be harder to get all heads if I tossed the coin 1000 times?
These questions have us think about the expected value and the standard error.
As we do more and more trials, do we expect the mean of our sample to get closer to the
“true mean” more often?
So, let’s do the experiment where we toss the coin 1,000 times. And let’s do this
experiment 50 times, as we did before. These are our results:
6
count
4
2
0
460
480
500
520
Number of Heads
And we see that it’s centered around 500 heads, as per our intuition of the outcome,
and goes from around 450 heads in some experiments to about 550 heads in others.
Does it look -upon quick glance- that it’s about the same as our first plot??
5
4
Examine the difference between n = 100 and n = 1000:
As per the first section of these notes, we know what the variance of a Binomially
Distributed Random variable is: npq.
So, if we look at our two different situations, we see that our variances/standard deviations are:
V ar(Sn ) = npq = 100 ×
1 1
× = 25
2 2
in our first context of n = 100, and we have:
V ar(Sn ) = npq = 1000 ×
1 1
× = 250
2 2
in our second context of n = 1000.
Careful! What we care about is our standard error, SE!
We actually have that our standard errors are:
r
p
1 1 √
√
V ar(Sn ) = npq = 100 × × = 25 = 5
2 2
in our first context of n = 100, and we have:
r
p
1 1 √
√
V ar(Sn ) = npq = 1000 × × = 250 ≈ 15
2 2
in our second context of n = 1000.
Take a second to examine this further... this is actually striking!
We know that most (95%) of our data will lie between 2 standard deviations (standard
errors) in the context of a Normal Distribution. And here, that means in our first context
that most will lie between 40 and 60 heads, while it will be between 470 and 530 in the
second.
However!...
An interval of length 20 is actually 20% of the possible values of the first context (since we
could get anywhere between 0 and 100 heads in 100 coin tosses) and an interval of length
60 is only 6% of the possible values in the second context (since we could get anywhere
between 0 and 1000 heads if we flip a coin 1000 times).
That means our distribution is MUCH tighter about the mean when we made 1000
tosses (as n got larger) than when we only made 100 tosses.
6
5
Visualize 1000 tosses:
Let’s see what it looks like when we do the 1000 toss experiment many times. Below is
for 50, 500, 1000, and 5000 experiments of 1000 tosses:
0.08
0.03
density
density
0.06
0.04
0.02
0.01
0.02
0.00
0.00
475
500
525
450
fifty
475
500
525
550
thousand
0.03
0.02
density
density
0.02
0.01
0.01
0.00
0.00
450
475
500
525
480
five_hundred
520
560
five_thousand
The important thing to look at is the five thousand experiment plot in the lower-right
corner. If we compare this to the same plot in the previous 100-toss experiment, this should
look more similar to a Normal Distribution.
Let’s see as n gets even larger...
7
As n gets larger and larger:
We see what happens when n = 10, 000 below:
1250
1000
count
750
500
250
0
498000
499000
500000
501000
502000
Number of Heads
And again for n = 100, 000:
4000
3000
count
6
2000
1000
0
497000
498000
499000
500000
501000
502000
Number of Heads
And this looks pretty Normal to me!
Note: In fact... we can check that this is extremely close to a Normal Curve.
8
7
Is n = 1000 a good Approximation?
If we perform the n = 1, 000 experiment many, many times, we can actually get a
good idea of how well it is approximated by a Normal Distribution. We plot the 1000-toss
experiment done 100,000 times below:
density
0.02
0.01
0.00
450
500
550
heads
What does this show us?
Well... if we have a random variable that follows a Binomial Distribution where the n
is at least 1,000... that we find that it is almost a Normal Distribution!
This is a very important discovery of ours!
Careful! Recall that a Normal Distribution is defined by two things: its
mean and variance.
If that’s all we need, the mean and variance... well, we’re gold! We have both of those
things!
9