Download Lecture 21

Sample Mean and Central
Limit Theorem
Lecture 21-22
November 17-21
Outline
•
•
•
•
•
Sums of Independent Random Variables
Chebyshev’s Inequality
Estimating sample sizes
Central Limit Theorem
Binomial Approximation to the normal
Sample Mean Statistics
Let X1,…Xn be a random sample from a
population (e.g. The Xi are independent
and identically distributed).
The sample mean is defined as
1 n
x = ∑ xi .
n i =1
What can we say about the distribution of
the sample mean?
Sample Mean for Normal
Let X1,X2,…,Xn be a random sample from a
normal (μ,σ). What is the pdf of x ?
Hint use the mgf method. The mgf of a
normal is:
mx (t ) = e
( μt +
t 2α 2
)
2
Solution
The mgf of Y=∑Xi is
⎡
m y (t ) = [mx (t )] = ⎢e
⎢⎣
n
t 2α 2
( μt +
)
2
But we want
x = y/n
mx (t ) = m y (t / n) = e
n
⎤
⎥ =e
⎥⎦
( nμ t / n +
nt 2α 2
( nμ t +
)
2
n ( t / n )2 α 2
)
2
=e
( μt +
( t )2 α 2
)
2n
Thus the mean is normal with mean μ and
variance σ 2
n
For any distribution with mean μ
and standard deviation σ
What is mean and variance of x ?
⎛ ∑ xi
E(x ) = E ⎜ i
⎜ n
⎜
⎝
⎞
⎟ = 1 E ( x ) = 1 nμ = μ
i
⎟ n∑
n
i
⎟
⎠
⎛ ∑ xi
var( x ) = var ⎜ i
⎜ n
⎜
⎝
⎞
⎟= 1
⎟ n2
⎟
⎠
1
σ2
2
∑i var( xi ) = n2 nσ = n
What if we don’t know the
distribution?
If we don’t want to make assumptions about
what the distribution is, can we still do
things?
• Chebyshev’s Inequality
• Central Limit Theorem
Amazing property true for any
distribution
Chebyshev inequality:
Consider any random variable with mean μ
and standard deviation σ. For any k>0,
P[|X- μ|≥k σ)≤1/k2
E(X)-kσ
E(X)
E(X)+kσ
In words,
The probability that X deviates from its
expected value at least k standard
deviations is less than 1
k2
Example
Suppose that, on average, a post office
handles 10000 letters a day with a
variance of 2000. What can be said about
the probability that this post office will
handle between 8000 and 12000 letters
tomorrow?
Using Chebshev’s
X = number of letters will handle tomorrow
μ = E ( X ) = 10000
α 2 = var( X ) = 2000
Want P(8000 < X < 12000).
P(8000 < X < 12000) = P(−2000 < X − 10000 < 2000)
= P(| X − 10000 |< 2000)
= 1 − P(| X − 10000 |> 2000) ≥ 1 − .0005 = .9995
Because by Chebyshev’s
P(| X − 10000 |> 2000) = P(| X − 10 |> 2000σ )
1
≤
= .0005
2000
Alternative forms of Chebyshev
For any k>0,
P[|X- μ|≥kσ)≤1/k2
For any t>0
P[|X- μ|≥t)≤ σ 2/t2
E(X)-t
E(X)
by setting t= kσ
E(X)+t
Chebyshev’s applied to Sample
Mean
Recall if X , …, X
1
n
are independent and identically distributed
with mean μ and standard deviation σ then
⎛∑x ⎞
⎟ = 1 E ( x ) = 1 nμ = μ
E(x ) = E ⎜
⎜ n ⎟ n∑
n
i
i
⎜
⎝
⎛ ∑ xi
var( x ) = var ⎜ i
⎜ n
⎜
⎝
⎟
⎠
⎞
⎟= 1
⎟ n2
⎟
⎠
i
i
1
σ2
2
∑i var( xi ) = n2 nσ = n
Chebyshev’s applied to Sample
Mean
Applying Chebyshev’s to the sample mean
then for any ε>0
α
P(| X − μ |≥ ε ) ≤ 2
ε n
2
X
Example
A biologist wants to estimate the life span of
a type of insect. He takes a sample of
size n and measure the lifetime from birth
to death of each insect. Then he averages
these numbers. If he believes the lifetimes
of the insect are iid with variance 1.5 days.
How large a sample should he choose to
be at least 98% sure that his average is
accurate within plus or minus 0.2 days (4.8
hours)?
Solution
Let Xi be the life time of the ith insect.
We want to find n such that
P(−0.2 < X − μ < 0.2) ≥ .98
Or equivalently
P(| X − μ |> 0.2) ≤ 1 − .98 = .02
By Chebyshev’s
P (| X − μ |> t ) ≤
α2
n t2
(1.5) 2 37.5
P (| X − μ |> 0.2 ) ≤
=
= .02
2
n (.2)
n
So we need n ≥ ⎡⎢37.5 / .02 ⎤⎥ = 1875.
Central Limit Theorem
Idea: No matter what the population distribution
may be, if n is large then the distribution of the
sample mean is approximately normal with
2
σ
mean μ and variance .
n
The larger the n, the better the approximation.
Good approximation for n>30. Try applet
http://www.ruf.rice.edu/~lane/stat_sim/sampling_di
st/
Central Limit Theorem
If x
Let
has mean μ and standard deviation
x −μ
U=
σ/ n
For large n, the distribution of U is
approximately Normal(0,1)
σ
n
Example
A soft drink machine dispenses drinks in a
cup. The amount dispensed is a R.V. with
mean 200 ml and s.d 15 ml. What is the
probability that the average amount
dispensed in a random sample of size 36
is at least 204 ml?
By CLT,
x
∑
x=
i
36
is approximately normal (200,15/6)
Solutions
⎛ ( x − μ ) n (204 − μ ) n ⎞
P ( X > 204) = P ⎜⎜
>
⎟⎟
σ
σ
⎝
⎠
(204 − 200)6 ⎞
⎛
≈ P⎜Z >
⎟ = P( Z > 1.6) = .0548
15
⎝
⎠
Z is standard normal
Sample Size Problem
Note that as sample size increases, the
sample standard deviation σ / n
gets smaller.
We can use the sample mean to estimate
the true mean. By making the sample size
bigger, we can make the estimate as
accurate as we desire.
Sample Size Problem
Say we know the standard deviation is σ=3
for each item in a random sample. Say we
want P(| X − μ |< .5) to be close to .95.
How big should the sample size be?
You try it?
Solution
Using CLT approximation,
| X −μ|
.5
<
) = .95
3/ n
3/ n
.5
.95 = P(| Z |< 1.96) = P (| Z |<
)
3/ n
solve for n and round up n = 139
P (| X − μ |< .5) = P(
Note
You can also estimate n using Chebyshev’s
by the bound is not as strong
P (| X − μ |< .5) = 1 − P(| X − μ |≥ .5) = .95
P (| X − μ |≥ .5) =
σ2
n(.5)
2
= .05
⎡ 32 ⎤
So need n = ⎢
= 720
2⎥
⎢ .05(.5) ⎥
which is much larger than the estimate based on CLT n = 139
Problem
We admit 1500 students to RPI and we
know historically 2/3 actually attend on
average. We can assume the decision of
each student to attend is independent.
What is the probability that more than
1050 students attend?
Thoughts
What is the distribution of the number of
students that attend?
Y=Binomial(n=1500,p=2/3)
How can we compute P(Y>1050)?
We learned that we can approximate
binomial using a Poisson with λ=np if n is
very large and p is very small so that λ is
small (<=10). But this is not the case
here.
Normal approximation to Binomial
The acceptance of each student can be modeled
as a Bernoulli random variable
X = 1 with probability 2/3with mean p=2/3
X = 0 with probability 1/3
and s.d. p(1 − p) = 2 / 9
∑i X i
The ratio of students attending
=X
n
can be approximated as a normal with mean 2/3
and variance
2
σ2
i
i
9(1500)
by CLT
=
n
continued
For Y=the number of students admitted
Y is really binomial with mean np=1000
and variance np(1-p)=1500*2/3*1/3
You can approximate it by CLT as normal
With same mean and variance since
Y = nX (approximately normal ) E (Y ) = np
equivalently
var(Y ) = np(1 − p)
Y − np
Y − 1000
=
∼ N (0,1)
np (1 − p )
1500 ( 2 / 3) (1/ 3)
⎛
Y − 1000
1050 − 1000 ⎞
⎟ ≈ P( Z > 2.73) = 0.0027
P⎜
>
⎜ 1500 ( 2 / 3) (1/ 3)
1500 ( 2 / 3) (1/ 3) ⎟⎠
⎝
Note normal is continuous and binomial is
discrete so we can improve the
approximation a bit
P(Y=1050) =0
Approximate by applying CLT to
P(1049.5≤Y ≤1050.5)
So better approximation for P(Y>1050) is
⎛
Y − 1000
1050.5 − 1000 ⎞
⎟ ≈ P( Z > 2.766) = 0.0028
P⎜
>
⎜ 1500 ( 2 / 3) (1/ 3)
1500 ( 2 / 3) (1/ 3) ⎟⎠
⎝
Note normal is continuous and binomial is
discrete so we can improve the
approximation a bit
Consider P(Y=1050) >0
Approximate by applying CLT to
P(1049.5≤Y ≤1050.5)
So better approximation for P(Y>1050) is
⎛
Y − 1000
1050.5 − 1000 ⎞
⎟ ≈ P( Z > 2.766) = 0.0028
P⎜
>
⎜ 1500 ( 2 / 3) (1/ 3)
1500 ( 2 / 3) (1/ 3) ⎟⎠
⎝
Normal Approximation to Binomial
If Y is binomial n and p, then if we want
P(a≤Y≤b) use
P(a-1/2 ≤Y≤b+1/2)
⎛ y − np
⎛ y − np
b + 1/ 2 − np ⎞
a − 1/ 2 − np ⎞
= P⎜
≤
≤
⎟⎟ − P ⎜⎜
⎟⎟
⎜ np (1 − p)
np (1 − p ) ⎠
np(1 − p) ⎠
⎝
⎝ np (1 − p )
⎛
⎛
b + 1/ 2 − np ⎞
a − 1/ 2 − np ⎞
= P⎜Z ≤
⎟⎟ − P ⎜⎜ Z ≤
⎟⎟
⎜
np (1 − p ) ⎠
np (1 − p ) ⎠
⎝
⎝
Check to see if you got it
If Y is binomial (n, p), then if we want
P(a≤Y≤b) use approximation on
P(a-1/2 ≤Y≤b+1/2)
P(a<Y≤b) use approximation on
P(a+1/2 ≤Y≤b+1/2)
P(a<Y<b) use approximation on
P(a+1/2 ≤Y≤b-1/2)
P(a ≤ Y<b) use approximation on
P(a-1/2 ≤Y≤b-1/2)
How good is the approximation?
Use the normal approximation to the
binomial to determine the probability of
getting 2 heads and 3 tails in 5 flips of a
balanced coin.
What is the actual distribution?
f(x)
0
1/32
1
5/32
2
3
4
10/32 10/32 5/32
5
1/32
Normal approximation
Mean is np=5/2=2.5
np (1 − p ) = 1.25
s.d. =
P (2 heads ) = P (2.5heads ) − P (1.5heads )
⎛ x − 2.5 2.5 − 2.5 ⎞
⎛ x − 2.5 1.5 − 2.5 ⎞
= P⎜
≤
≤
⎟ − P⎜
⎟
1.118
1.118
1.118
1.118
⎝
⎠
⎝
⎠
= P( Z ≤ 0) − P ( Z < −1.0)
= .5 − (1 − .8133) = .3133
Compare to .3125