Download Introduction to Statistics, Lecture 4

Overview
Course 02402/02323 Introduction to Statistics
Lecture 4: Confidence intervals
1
Example
2
Distribution of sample mean
t-Distribution
3
Confidence interval for µ
Example
4
The language of statistics and the formal framework
5
Non-normal data, Central Limit Theorem (CLT)
6
A formal interpretation of the confidence interval
7
Planning of study with requirements to the precision
Example, height data again
8
Confidence interval for variance and standard deviation
Per Bruun Brockhoff
DTU Compute
Danish Technical University
2800 Lyngby – Denmark
e-mail: [email protected]
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
1 / 47
Example
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
2 / 47
Distribution of sample mean
Example - heights:
Theorem 3.2: The distribution of the mean of normal
random variables
Sample, n = 10:
168
161
167
179
184
Sample mean and standard
deviation:
166
198
187
191
179
(Sample-) Distribution/ The (sampling) distribution for X̄
Estimate population mean and
standard deviation:
x̄ = 178
µ̂ = 178
s = 12.21
σ̂ = 12.21
Assume that X1 , . . . , Xn are independent and identically normally
distributed random variables, Xi ∼ N (µ, σ 2 ), i = 1, . . . , n, then:
n
1X
X̄ =
Xi ∼ N
n
NEW:Confidence interval, µ:
i=1
σ2
µ,
n
NEW:Confidence interval, σ:
12.21
178±2.26· √
⇔ [169.3; 186.7]
10
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
[8.4; 22.3]
Spring 2015
4 / 47
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
6 / 47
Distribution of sample mean
Distribution of sample mean
Mean and variance follow from ’rules’:
We now know the distribution of the error we make:
(When using x̄ as an estimate of µ)
The Mean of X̄
n
n
i=1
i=1
The standard deviation of X̄
1X
1
1X
E(Xi ) =
µ = nµ = µ
E(X̄) =
n
n
n
σ
σX̄ = √
n
The variance of X̄
The standard deviation of (X̄ − µ)
σ
σ(X̄−µ) = √
n
n
n
1 X
1 X 2
1
σ2
Var(X̄) = 2
Var(Xi ) = 2
σ = 2 nσ 2 =
n
n
n
n
i=1
Per Bruun Brockhoff ([email protected])
i=1
Introduction to Statistics, Lecture 4
Spring 2015
7 / 47
Per Bruun Brockhoff ([email protected])
Distribution of sample mean
Introduction to Statistics, Lecture 4
Spring 2015
8 / 47
Distribution of sample mean
Standardized version of the same thing, Corollary 3.3:
Practical problem in all this, so far:
How to transform this into a specific interval for µ?
When the populations standard deviation σ is in all the formulas?
Distribution for the standardized error we make:
Assume that X1 , . . . , Xn are independent and identically normally
distributed random variables, Xi ∼ N (µ, σ 2 ) where i = 1, . . . , n,
then:
X̄ − µ
√ ∼ N 0, 12
Z=
σ/ n
Obvious solution:
Use the estimtate s in stead of σ in formulas!
That is, the standardized sample mean Z follows a standard normal
distribution.
BUT BUT:
The given theory then breaks down!!
Luckily:
We have en extended theory to handle it for us!!
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
9 / 47
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
10 / 47
Distribution of sample mean
Distribution of sample mean
t-Distribution
t-Distribution with 9 degrees of freedom (n = 10):
0.4
Theorem 3.4: More applicable extension of the same stuff:
(kopi af Theorem 2.49)
t-Distribution
X̄ − µ
√ ∼t
S/ n
0.1
T =
Black: standard normal
0.2
Assume that X1 , . . . , Xn are independent and identically
normally
2
distributed random variables, where Xi ∼ N µ, σ and i = 1, . . . , n, then:
dt(x, 9)
0.3
t-Distribution tager højde for usikkerheden i at bruge s:
Red: t(9)
0.0
where t is the t-distribution with n − 1 degrees of freedom.
−4
−2
0
2
4
x
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Distribution of sample mean
Spring 2015
11 / 47
Per Bruun Brockhoff ([email protected])
Distribution of sample mean
t-Distribution
Spring 2015
12 / 47
t-Distribution
0.3
Black: standard normal
0.1
0.1
0.2
dt(x, 9)
Black: standard normal
0.2
0.3
0.4
t-Distribution with 9 degrees of freedom and standard
normal distribution:
0.4
t-Distribution with 9 degrees of freedom and standard
normal distribution:
dt(x, 9)
Introduction to Statistics, Lecture 4
Red: t(9)
Red: t(9)
0.0
P(Z>2)=0.023
0.0
P(T>2)=0.038
−4
−2
0
2
4
−4
x
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
−2
0
2
4
x
Spring 2015
13 / 47
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
14 / 47
Confidence interval for µ
Confidence interval for µ
Metodeboks 3.8: One-sample Confidence interval for µ
Example
Student height Example
Use the right t-distribution to make the confidence interval:
For a sample x1 , . . . , xn the 100(1 − α)% confidence interval is given by:
s
x̄ ± t1−α/2 · √
n
## The t-quantiles for n=10:
qt(0.975,9)
[1] 2.2622
and we can recognize the already given result:
where t1−α/2 is the 100(1 − α)% quantile from the t-distribution with
n − 1 degrees of freedom.
12.21
178 ± 2.26 · √
10
Most commonly using α = 0.05:
The most commonly used is the 95%-confidence interval:
which is:
178 ± 8.74 = [169.3; 186.7]
s
x̄ ± t0.975 · √
n
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Confidence interval for µ
Spring 2015
16 / 47
Example
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Confidence interval for µ
Student height example, 99% Confidence interval (CI)
Spring 2015
17 / 47
Example
There is an R-function, that can do it all (and more than
that):
x <- c(168,161,167,179,184,166,198,187,191,179)
t.test(x,conf.level=0.99)
qt(0.995,9)
[1] 3.2498
12.21
178 ± 3.25 · √
10
giving
178 ± 12.55 = [165.4; 190.6]
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
18 / 47
##
## Results of Hypothesis Test
## -------------------------##
## Null Hypothesis:
mean = 0
##
## Alternative Hypothesis:
True mean is not equal to 0
##
## Test Name:
One Sample t-test
##
## Estimated Parameter(s):
mean of x = 178
##
## Data:
x
Per
Spring 2015
19 / 47
##Bruun Brockhoff ([email protected]) Introduction to Statistics, Lecture 4
## Test Statistic:
t = 46.096
The language of statistics and the formal framework
The language of statistics and the formal framework
The formal framework for statistical inference
The formal framework for statistical inference - Example
From eNote, Chapter 1:
From eNote, Chapter 1, heights example
An observational unit is the single entity/level about which
information is sought (e.g. a person) (Observationsenhed)
We measure the heights of 10 randomly selected persons in Demark
The statistical population consists of all possible “measurements” on
each observational unit (Population)
The sample:
The 10 specific numbers: x1 , . . . , x10
The sample from a statistical population is the actual set of data
collected. (Sample)
The population:
Language and concepts:
The heights for all people in Dnemark
µ and σ are parameters describing th populationen
Observational unit:
A person
x̄ is the estimate of µ (specific realization)
X̄ is the estimator of µ (now seen as a random variable)
The word ’statistic(s)’ is used for both
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
21 / 47
The language of statistics and the formal framework
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
22 / 47
The language of statistics and the formal framework
Statistical inference = Learning from data
Random Sampling
Learning from data:
Definition 3.11:
A random sample from an (infinite) population: A set of
observations X1 , X2 , ..., Xn constitutes a random sample of size
n from the infinite population f (x) if:
Is learning about parameters of distributions that describe populations.
Important for this:
1
The sample must in a meaningful way represent some well defined
population
2
What does that mean????
1
All observations must come from the same population
2
They cannot share any information with each other (e.g. if we
sampled entire families)
How to ensure this:
F.ex. by making sure that the sample is taken completely at random
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
Each Xi is a random variable whose distribution is given by
f (x)
These n random variables are independent
23 / 47
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
24 / 47
Non-normal data, Central Limit Theorem (CLT)
Non-normal data, Central Limit Theorem (CLT)
Theorem 3.13: The Central Limit Theorem
CLT in action - mean of uniformly distributed observations
No matter what, the distribution of the mean becomes a normal
distribution:
n=1
k=1000
u=matrix(runif(k*n),ncol=n)
hist(apply(u,1,mean),col="blue",main="n=1",xlab="Means")
Let X̄ be the mean of a random sample of size n taken from a population
with mean µ and variance σ 2 , then
X̄ − µ
√
σ/ n
Frequency
is a random variable whose distribution function approaches that of the
standard normal distribution, N (0, 12 ), as n → ∞
Hence, if n is large enough, we can (approximately) assume:
X̄ − µ
√ ∼ N (0, 12 )
σ/ n
Per Bruun Brockhoff ([email protected])
100
n=1
0 40
Z=
0.0
Introduction to Statistics, Lecture 4
Spring 2015
26 / 47
Per Bruun Brockhoff ([email protected])
0.4
0.8
Introduction to Statistics, Lecture 4
Spring 2015
27 / 47
Means
Non-normal data, Central Limit Theorem (CLT)
Non-normal data, Central Limit Theorem (CLT)
CLT in action - mean of uniformly distributed observations
CLT in action - mean of uniformly distributed observations
n=2
k=1000
u=matrix(runif(k*n),ncol=n)
hist(apply(u,1,mean),col="blue",main="n=2",xlab="Means")
n=6
k=1000
u=matrix(runif(k*n),ncol=n)
hist(apply(u,1,mean),col="blue",main="n=6",xlab="Means")
0.0
Per Bruun Brockhoff ([email protected])
0.4
100
0.8
Introduction to Statistics, Lecture 4
Means
0
100
Frequency
n=6
0
Frequency
n=2
0.2
Spring 2015
28 / 47
Per Bruun Brockhoff ([email protected])
0.4
0.6
Introduction to Statistics, Lecture 4
Means
0.8
Spring 2015
29 / 47
Non-normal data, Central Limit Theorem (CLT)
Non-normal data, Central Limit Theorem (CLT)
CLT in action - mean of uniformly distributed observations
n=30
k=1000
u=matrix(runif(k*n),ncol=n)
hist(apply(u,1,mean),col="blue",main="n=30",xlab="Means", nclass=15)
Our CI-method also works for non-normal data:
We can use the confidence-interval based on the t-distribution in basically
any situation, as long as n is large enough.
140
What is ”large enough”?
Actually difficult to say exactly, BUT:
Rule of thumb: n ≥ 30
60
Even for smaller n the approach can be (almost) valid for non-normal
data.
0
Frequency
n=30
Consequence of CLT:
0.35
Per Bruun Brockhoff ([email protected])
0.50
0.65
Introduction to Statistics, Lecture 4
Spring 2015
30 / 47
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
31 / 47
Spring 2015
34 / 47
Means
A formal interpretation of the confidence interval
A formal interpretation of the confidence interval
’Repeated sampling’ interpretation
’Repeated sampling’ interpretation
In the long run we catch the true value in 95% of cases:
The confidence interval will vary in both width (s) and position (x̄) if the
study is repated.
More formally expressed (Theorem 3.4 and 2.49):
|X̄ − µ|
√ < t0.975 = 0.95
P
S/ n
X−µ
P(|
S
Which is equivalent to:
S
S
P X̄ − t0.975 √ < µ < X̄ + t0.975 √
= 0.95
n
n
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
t0.025
33 / 47
Per Bruun Brockhoff ([email protected])
n
| < t0.975)
0
t0.975
Introduction to Statistics, Lecture 4
Planning of study with requirements to the precision
Planning of study with requirements to the precision
Planning of study with requirements to the precision
Example, height data again
Sample mean og standard
deviation:
Method 3.45: The one-sample CI sample size formula:
When σ is known or guessed at some value, we can calculate the sample
size n needed to achieve a given margin of error, M E, with probability
1 − α as:
z
1−α/2 · σ 2
n=
(1)
ME
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Example, height data again
Spring 2015
36 / 47
Confidence interval for variance and standard deviation
Example
Production of tablets
In the production of tablets, an active matter is mixed with
a powder and then the mixture is formed to tablets. It is
important that the mixture is homogenous, so that each
tablet has the same strength.
We consider a mixture (of the active matter and powder)
from where a large amount of tablets is to be produced.
We seek to produce the mixtures (and the final tablets) so
that the mean content of the active matter is 1 mg/g with
the smallest variance as possible. A random sample is
collected where the amount of active matter is measured.
It is assumed that all the measurements follow a normal
Per Bruun Brockhoff ([email protected])
to Statistics, Lecture 4
Spring 2015
39 / 47
distribution with the Introduction
unit mg/g.
Estimate the population mean
and standard deviation:
x̄ = 178
µ̂ = 178
s = 12.21
σ̂ = 12.21
If we want that M E = 3cm with 95% confidence, how large should n then
be?
1.96 · 12.21 2
n=
= 63.64
3
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
37 / 47
Confidence interval for variance and standard deviation
The sampling distribution of the variance estimator
(Theorem 2.53)
Variance estimators behaves like a χ2 -distribution:
Let
n
1 X
S2 =
(Xi − X̄)2
n−1
i=1
then:
χ2 =
(n − 1)S 2
σ2
is a stochastic variable following the χ2 -distribution with v = n − 1 degrees
of freedom.
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
40 / 47
Confidence interval for variance and standard deviation
Confidence interval for variance and standard deviation
χ2 -distribution with ν = 9 degrees of freedom
Metode 3.18: Confidence interval for sample variance and
standard deviation
x <- seq(0, 20, by = 0.1)
plot(x, dchisq(x, df = 9), type = "l")
varianceen:
0.08
0.04
where the quantiles come from a χ2 -distribution with ν = n − 1 degrees of
freedom.
Spredningen:
A 100(1 − α)% confidence interval for the sample standard deviation σ̂ is:
#
"s
s
(n − 1)s2
(n − 1)s2
;
χ21−α/2
χ2α/2
0.00
dchisq(x, df = 9)
A 100(1 − α)% confidence interval for a sample variance σ̂ 2 is:
"
#
(n − 1)s2 (n − 1)s2
;
χ21−α/2
χ2α/2
0
5
10
15
20
x
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
41 / 47
Confidence interval for variance and standard deviation
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
42 / 47
Confidence interval for variance and standard deviation
Example
Example
Data:
A random sample with n = 20 tablets is taken and from this we get:
2
2
So the confidence interval for the variance σ 2 becomes:
19 · 0.72 19 · 0.72
;
= [0.002834; 0.01045]
32.85
8.907
2
µ̂ = x̄ = 1.01, σ̂ = s = 0.07
95%-Confidence interval for the variance - we need the χ2 -quantiles:
and the confidence interval for the standard deviation σ becomes:
h√
i
√
0.002834; 0.01045 = [0.053; 0.102]
χ20.025 = 8.9065, χ20.975 = 32.8523
qchisq(c(0.025, 0.975), df = 19)
[1] 8.9065 32.8523
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
43 / 47
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
Spring 2015
44 / 47
Confidence interval for variance and standard deviation
Confidence interval for variance and standard deviation
Heights example
Example - heights- recap:
Sample, n = 10:
We need th χ2 -quantiles with ν = 9 degrees of freedom:
168
161
167
179
184
166
198
187
191
179
χ20.025 = 2.700389, χ20.975 = 19.022768
Sample mean and standard
deviation:
qchisq(c(0.025, 0.975), df = 9)
[1] 2.7004 19.0228
So the confidence interval for the height standard deviation σ becomes:
"r
#
r
9 · 12.212
9 · 12.212
;
= [8.4; 22.3]
19.022768
2.700389
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
2
Distribution of sample mean
t-Distribution
3
Confidence interval for µ
Example
4
The language of statistics and the formal framework
5
Non-normal data, Central Limit Theorem (CLT)
6
A formal interpretation of the confidence interval
7
Planning of study with requirements to the precision
Example, height data again
8
Confidence interval for variance and standard deviation
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
µ̂ = 178
s = 12.21
σ̂ = 12.21
NEW:Confidence interval, σ:
12.21
178±2.26· √
⇔ [169.3; 186.7]
10
Spring 2015
45 / 47
Spring 2015
47 / 47
Overview
Example
x̄ = 178
NEW:Confidence interval, µ:
Confidence interval for variance and standard deviation
1
Estimate population mean and
standard deviation:
Per Bruun Brockhoff ([email protected])
Introduction to Statistics, Lecture 4
[8.4; 22.3]
Spring 2015
46 / 47