Download Statistics II Chapter 1: Statistical inference in one population

Chapter 1. Statistical inference in one population
Contents
I
I
Statistical inference
Point estimators
I
I
The estimation of the population mean and variance
Estimating the population mean using confidence intervals
I
I
Confidence intervals for the mean of a normal population with known
variance
Confidence intervals for the mean in large samples
I
I
I
Confidence intervals for the population proportion
Confidence intervals for the mean of a normal population with
unknown variance
Estimating the population variance using confidence intervals
I
Confidence intervals for the variance of a normal population
Chapter 1. Statistical inference in one population
Learning goals
At the end of this chapter you should know how to:
I
I
Estimate the unknown population parameters from the sample data
Construct confidence intervals for the unknown population
parameters from the sample data:
I
I
In the case of a normal distribution: confidence intervals for the
population mean and variance
In large samples: confidence intervals for the population mean and
proportion
I
Interpret the confidence interval
I
Understand the impact of the sample size, confidence level, etc on
the length of the confidence interval
I
Calculate a sample size needed to control a given interval width
Chapter 1. Statistical inference in one population
References
I
Newbold, P. ”Statistics for Business and Economics”
I
I
Chapters 7 and 8 (8.1-8.6)
Ross, S. ”Introduction to Statistics”
I
Chapter 8
Statistical inference: key words (i)
I
Population: the complete set of numerical information on a
particular quantity in which an investigator is interested.
I
I
I
Sample: an observed subset (say, of size n) of the population values.
I
I
We identify the concept of the population with that of the random
variable X .
The law or the distribution of the population is the distribution of X ,
FX .
Represented by a collection of n random variables X1 , X2 , . . . , Xn ,
typically iid (independent identically distributed) .
Parameter: a constant characterizing X or FX .
Statistical inference: key words (ii)
I
Statistical inference: the process of drawing conclusions about a
population on the basis of measurements or observations made on a
sample of individuals from the population.
I
Statistic: a random variable obtained as a function of a random
sample, X1 , X2 , . . . , Xn
I
Estimator of a parameter: a random variable obtained as a function,
say T , of a random sample, X1 , X2 , . . . , Xn , used to estimate the
unknown population parameter.
I
Estimate: a specific realization of that random variable, i.e., T
evaluated at the observed sample, x1 , x2 , . . . , xn , that provides an
approximation to that unknown parameter.
Statistical inference: example
We want to know
µX = E[X ]
X ∼F
We have n copies
of X
⇒
⇓
µX = E[X ]
Expected value of X
X1 , X2 , . . . , Xn ∼ F
Sample
We have n
observed values of
X1 , X2 , . . . , Xn
⇒
⇓
⇐
Estimator of µX (r. v.)
X̄
Sample mean
x1 , x2 , . . . , xn
Observed sample
⇓
⇐
Estimate of µX (number)
x̄
Sample mean
Point estimators: introduction
I
A point estimator of a population parameter is a function, call it T ,
of the sample information X n = (X1 , . . . , Xn ) that yields a single
number.
I
Examples of population parameters, estimators and estimates:
Population
parameter
Pop. mean µX
Pop. prop. pX
Pop. var. σX2
σX2
Pop. var.
...
In general, θX
T (X n )
sample mean
sample prop.
X1 +...+Xn
n
P
2
2
i Xi −n(X̄ )
nP
2
2
i Xi −n(X̄ )
quasi. var.
n−1
sample var.
sample
...
...
=
2
n
n−1 σ̂X
Estimator:
notation
X̄ = µ̂X
p̂X
Estimate:
notation
x̄
p̂x
σ̂X2
σ̂x2
sX2
sx2
...
θ̂x
...
θ̂X
Point estimators: properties (i)
What are desirable characteristics of the estimators?
I
Unbiasdness. This means that the bias of the estimator is zero.
What’s bias? Bias equals the expected value of the estimator minus
the target parameter
Bias[θ̂X ] = E[θ̂X ] − θX
Population
parameter
Pop. mean µX
Pop. prop. pX
Pop. var. σX2
Pop. var. σX2
In general, θX
Estimator
T (X n )
X
p̂X
σ̂X2
sX2
θ̂X
Bias
E[X̄ ] − µX = 0
E[p̂X ] − pX = 0
E[σ̂X2 ] − σX2 6= 0
E[sX2 ] − σX2 = 0
E[θ̂X ] − θX
Unbiased?
Yes
Yes
No
Yes
Often
Minimum Variance
Unbiased Estimator?
Yes, if X normal
Yes
No
Yes, if X normal
Rarely
Point estimators: properties (ii)
I
Efficiency. Measured by the estimator’s variance. Estimators with
smaller variance are more efficient.
I
Relative efficiency of two unbiased estimators θ̂X ,1 and θ̂X ,2 of a
parameter θX is
Relative efficiency(θ̂X ,1 , θ̂X ,2 ) =
Var[θ̂X ,1 ]
Var[θ̂X ,2 ]
Note:
I
I
sometimes the inverse is used as a definition
in any case, an estimator with smaller variance is more efficient
Point estimators: properties (iii)
I
A more general criterion to select estimators (among unbiased and
biased ones) is the mean squared error defined as
MSE[θ̂X ] = E[(θ̂X − θX )2 ] = Var[θ̂X ] + (Bias[θ̂X ])2
Note:
I
I
I
I
the mean squared error of an unbiased estimator equals its variance
an estimator with smaller MSE is better
the minimum variance unbiased estimator has the smallest
variance/MSE among all estimators
How do we come up with the definition of the estimator T ?
I
I
In some situations, there exists an optimal estimator called minimum
variance unbiased estimator.
If that’s not the case, there are various alternative methods that
yield reasonable estimators, for example:
I
I
Maximum likelihood estimation
Method of moments
Point estimation: example
Example: 7.1 (Newbold) Price-earnings ratios for a random sample of
ten stocks traded on the NY Stock Exchange on a particular day were
10, 16, 5, 10, 12, 8, 4, 6, 5, 4
Use an unbiased estimation procedure to find point estimates of the
following population parameters: mean, variance, proportion of values
exceeding 8.5.
x̄
=
sx2
=
p̂x
=
=
80
=8
10
782 − 10(8)2
= 15.78
10 − 1
1+1+0+1+1+0+0+0+0+0
10
0.4
Point estimation: example
2
(X1 + 2X2 + . . . + nXn ) be an estimator of the
Example: Let µ̂X = n(n+1)
population mean based on a SRS X n . Compare this estimator with the sample
mean, X̄ .
σ2
We know that X̄ is an unbiased estimator of µX , whose variance is nX .
µ̂X is also unbiased:
And its variance/MSE is:
"
E[µ̂X ]
=
"
#
2
E
n(n + 1)
V[µ̂X ]
(X1 + 2X2 + . . . + nXn )
=
V
2
=
n(n + 1)
(E[X1 ] + 2E[X2 ] + . . . + nE[Xn ])
=indep.
#
2
(X1 + 2X2 + . . . + nXn )
n(n + 1)
!2
2
2
2
(V[X1 ] + 2 V[X2 ] + . . . + n V[Xn ])
n(n + 1)
2
=id
=
n(n + 1)
2µX
n(n + 1)
(µX + 2µX + . . . + nµX )
4
=id
n(n+1)/2
z
}|
{
(1 + 2 + . . . + n) = µX
n2 (n + 1)2
2(2n + 1)
=
3n(n + 1)
2
σX
⇒ Bias[µ̂X ] = 0
MSE [µ̂X ]
Relative efficiency(X̄ , µ̂X ) =
=
V[µ̂X ] + 0
σX2 /n
2(2n+1) 2
σ
3n(n+1) X
=
n(n+1)(2n+1)/6
}|
{
z
2
2
2
2
σX (1 + 2 + . . . + n )
2
2(2n + 1)
=
3n(n + 1)
2
σX
3(n + 1)
2(2n + 1)
It’s easy to see that for n ≥ 2, this ratio is smaller than 1 so X̄ is a more
efficient estimator for µX .
From point estimation to confidence interval estimation
I
I
So far, we have consider the point estimation of an unknown
population parameter which, assuming we had a SRS sample of n
observations from X , would produce an educated guess about that
unknown parameter
Point estimates however, do not take into account the variability of
the estimation procedure due to, among other factors:
I
I
I
I
sample size - surely, larger samples should provide more accurate
information about the population parameter
variability in the population - samples from populations with smaller
variance should give more accurate estimates
whether other population parameters are known
etc
These drawbacks can be overcome by considering confidence interval
estimation, that is, a method that gives a range of values (an interval) in
which the parameter is likely to fall.
Confidence interval estimator and confidence interval
Let X n = (X1 , X2 , . . . , Xn ) be a SRS from a population X with a cdf FX
that depends on an unknown parameter θ.
I A confidence interval estimator for θ at a confidence level
(1 − α) = 100(1 − α)% is an interval (T1 (X n ), T2 (X n )) that satisfies
P (θ ∈ (T1 (X n ), T2 (X n )) = 1 − α
I
I
Interpretation: we have a probability of (1 − α) that the unknown
population parameter will be in (T1 (X n ), T2 (X n )).
A confidence interval for θ at a confidence level 1 − α is the
observed value of the confidence interval estimator,
(T1 (x n ), T2 (x n ))
I
Interpretation: we can be (1 − α) confident that the unknown
population parameter will be in (T1 (x n ), T2 (x n )).
Typical levels of confidence
α
100(1 − α)%
0.01
99%
0.05
95%
0.10
90%
Finding confidence interval estimators: procedure
1. Find a quantity involving the unknown parameter θ and the sample
X n , C (X n , θ), whose distribution is known and does not depend on
the parameter - a so-called pivotal quantity or a pivot for θ
2. Use the upper 1 − α/2 and α/2 quantiles of that distribution and the
definition of the confidence interval estimator to set up the equation
double inequality
z
}|
{
P(1 − α/2 quantile<C (X n , θ)<α/2 quantile) = 1 − α
3. To find the end points T1 (X n ) and T2 (X n ) of the confidence
interval estimator, solve the double inequality for the parameter θ
4. A 100(1 − α)% confidence interval for θ is (T1 (x n ), T2 (x n ))
Confidence interval for the population mean, normal
population with known variance
1. Let X n be a SRS of size n from X . Under the assumptions:
I
I
X follows a normal distribution with parameters µX and σX2
σX2 is known (rather unrealistic)
2. The pivotal quantity for µX is
X̄ − µX
√ ∼ N(0, 1)
σX / n
I
√
Note: the standard deviation of X̄ , σX / n, (or any other stats) is
called the standard error
Confidence interval for the population mean, normal
population with known variance
3. Hence, if z1−α/2 and zα/2 are the
(1 − α/2) and (α/2) upper
quantiles of the N(0, 1), we have
P(z1−α/2 < Z < zα/2 ) = 1 − α
Standard normal density
Recall: If Z ∼ N(0, 1) then
E[Z ] = 0, V[Z ] = 1
α
2
α
2
1−α
●
●
z1−α2 = − zα2
zα2
Z
−zα/2 z }| {
z }| { X̄ − µX
√ < zα/2 ) = 1 − α
4. Therefore P(z1−α/2 <
σX / n
Confidence interval for the population mean, normal
population with known variance
5. Solve the double inequality for µX :
−zα/2
σX
−zα/2 √
n
σX
−zα/2 √ − X̄
n
σX
zα/2 √ + X̄
n
<
X̄ −µX
√
σX / n
<
< X̄ − µX <
< −µX <
> µX >
zα/2
σX
zα/2 √
n
σX
−X̄ + zα/2 √
n
σX
X̄ − zα/2 √
n
to obtain the confidence interval estimator
T2 (X n )
T1 (X n )
z
}|
{ z
}|
{
σX
σX
(X̄ − zα/2 √ , X̄ + zα/2 √ )
n
n
6. The confidence interval is:
„
« „
«
σX
σX
σX
CI1−α (µX ) = x̄ − zα/2 √ , x̄ + zα/2 √
= x̄ ∓ zα/2 √
n
n
n
Example: finding a confidence interval for µX
Example: 8.2 (Newbold) A process produces bags of refined sugar. The
weights of the contents of these bags are normally distributed with standard
deviation 1.2 ounces. The contents of a random sample of twenty-five bags had
mean weight 19.8 ounces. Find a 95% confidence interval for the true mean
weight for all bags of sugar produced by the process.
“
”
σX
Population:
Objective: CI0.95 (µX ) = x̄ ∓ zα/2 √
n
X = ”weight of a sugar bag (in oz)”
2
2
X ∼ N(µX , σX = 1.2 )
σX = 1.2
'
n = 25
SRS: n = 25
Sample: x̄ = 19.8
Area=
0.025
●
z0.025 = 1.96
x̄ = 19.8
1 − α = 0.95
⇒
α/2 = 0.025
zα/2
=
CI0.95 (µX )
=
=
z0.025 = 1.96
„
«
1.2
√
19.8 ∓ 1.96
25
(19.8 ∓ 0.47)
=
(19.33, 20.27)
Interpretation: We can be 95%
confident that µX is in
(19.33, 20.27)
Frequency interpretation of the CI, conf. level effect
In this simulated example, 150 samples of the same size n = 50 were generated
from X ∼ N(µX = −5, σX2 = 12 ) and 150 CI1−α (µX ) were constructed with
α = 0.1 and α = 0.01.
(1 − α) = 0.9, n = 50
(1 − α) = 0.99, n = 50
150
µX in approximately 150(0.99) = 148.5 ints.
(but not in 150(0.01) = 1.5)
150
µX in approximately 150(0.9) = 135 ints.
(but not in 150(0.1) = 15)
|
|
| | |
| |
|| |
| |
|
|
| |
| |
|
|
|
|
|
|
|
|
|
|
| | |
|
| |
|
|
||
|
||
|
|
|
| |
| |
|
| ||
||
|
| ||
|
|
|
|
|
| |
|
|
|
|
|
| |
|
| |
|
|
|
|
||
|
||
|
| |
|
|
|
| |
|
|
| |
|
|| |
|
|
|
|
|
||
|
|
| ||
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
|
||
|
|
|
|
|
|
|
| | |
|
|
|
|
| |
|
−6.0
−5.5
−5.0
|
100
|
50
Index
|
0
100
0
50
Index
|
|
|
| | |
| |
|| |
| |
|
|
| |
| |
|
|
|
|
|
|
|
|
|
|
| | |
|
| |
|
|
||
|
||
|
|
|
| |
| |
|
| ||
||
|
| ||
|
|
|
|
|
| |
|
|
|
|
|
| |
|
| |
|
|
|
|
||
|
||
|
| |
|
|
|
| |
|
|
| |
|
|| |
|
|
|
|
|
||
|
|
| ||
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
|
||
|
|
|
|
|
|
|
| | |
|
|
|
|
| |
|
|
|
−4.5
−4.0
−6.0
Confidence interval
The width of the interval, w = x̄ +
−5.5
−5.0
−4.5
−4.0
Confidence interval
zα/2 σX
√
n
“
− x̄ −
zα/2 σX
√
n
”
=2
zα/2 σX
√
n
,
increases with the increasing confidence level (keeping everything else the
same). Why?
Frequency interpretation of the CI, sample size effect
Here we collect 150 samples of size n = 50 and another 150 of size n = 200
from X ∼ N(µX = −5, σX2 = 12 ) .
(1 − α) = 0.9, n = 50
(1 − α) = 0.9, n = 200
150
µX in approximately 150(0.9) = 135 ints.
(but not in 150(0.1) = 15)
150
µX in approximately 150(0.9) = 135 ints.
(but not in 150(0.1) = 15)
|
|
| | |
| |
|| |
| |
|
|
| |
| |
|
|
|
|
|
|
|
|
|
|
| | |
|
| |
|
|
||
|
||
|
|
|
| |
| |
|
| ||
||
|
| ||
|
|
|
|
|
| |
|
|
|
|
|
| |
|
| |
|
|
|
|
||
|
||
|
| |
|
|
|
| |
|
|
| |
|
|| |
|
|
|
|
|
||
|
|
| ||
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
|
||
|
|
|
|
|
|
|
| | |
|
|
|
|
| |
|
|| |
|
|
|
|
|
|
|| |
| |
|
|| |
| | |
|
|| |
||
| |
||
|| |
|
|
|
|
|
| | |
|
| |
||
| | |
| ||
||
|
|| |
|
| |
| ||
|
|
| |
|
|
|
| ||
|
||
|| |
|| |
||
|||
||
| |
|
|
|
|
||
|
|
||
|
| |
| | |
| | |
||
|
|
| | |
| | |
| |
|
|||
|
|| |
|
| ||
|
|
|| |
|
||
| ||
| |
|
−6.0
−5.5
−5.0
Confidence interval
Index
50
0
100
0
50
Index
|
100
|
|
−4.5
−4.0
−6.0
−5.5
−5.0
−4.5
−4.0
Confidence interval
The width of the interval decreases with the increasing sample size (keeping
everything else the same). Why?
Question: What is the effect of σ on the width?
Example: estimating the sample size
Example: 8.14 (Newbold) The lengths of metal rods produced by an industrial
process are normally distributed with standard deviation 1.8mm. Suppose that
a production manager requires a 99% confidence interval extending no further
than 0.5mm on each side of the sample mean. How large a sample is needed to
achieve such an interval?
Population:
Objective: n such that width ≤ 1
X = “length of a metal rod (in mm)”
zα/2 σX
X ∼ N(µX , σX2 = 1.82 )
2 √
≤ 1
n
√
SRS: n =?
2zα/2 σX ≤
n
'
85.93 = (2(2.575)(1.8))2
width
z }| {
zα/2 σX
≤ 2(0.5) = 1
CI0.99 (µX ): 2 √
n
Area=
0.005
●
z0.005 = 2.575
≤
n
To satisfy the manager’s
requirement, a sample of at least
86 observations is needed.
Confidence interval for the population mean in large
samples
1. Let X n be a SRS of size n from X . Under the assumptions:
I
I
X follows a nonnormal distribution with parameters µX and σX2
the sample size n is large (n ≥ 30)
2. The pivotal quantity for µX based on the Central Limit Theorem is
X̄ − µX
√ ∼approx. N(0, 1)
σ̂X / n
Confidence interval for the population mean in large
samples
3. Hence, if z1−α/2 and zα/2 are the
(1 − α/2) and (α/2) upper
quantiles of the N(0, 1), we have
P(z1−α/2 < Z < zα/2 ) = 1 − α
Standard normal density
α
2
α
2
1−α
●
●
z1−α2 = − zα2
zα2
Z
−zα/2 z }| {
z }| { X̄ − µX
√ < zα/2 ) = 1 − α
4. Therefore P(z1−α/2 <
σ̂X / n
Confidence interval for the population mean in large
samples
5. Solve the double inequality for µX :
−zα/2 <
X̄ − µX
√ < zα/2
σ̂X / n
to obtain the confidence interval estimator
T2 (X n )
T1 (X n )
}|
{ z
}|
{
σ̂X
σ̂X
(X̄ − zα/2 √ , X̄ + zα/2 √ )
n
n
z
6. The confidence interval is:
σ̂x
σ̂x
CI1−α (µX ) = (x̄ − zα/2 √ , x̄ + zα/2 √ )
n
n
Confidence interval for the population proportion in large
samples
Application of CIs for the population mean in large samples
Let X n , n ≥ 30 be a SRS from
p a Bernoulli distr. with parameter pX
(µX = E[X ] = pX and σX = pX (1 − pX )). The sample proportion p̂X
is a special case of the sample mean of zero-one observations, p̂X = X̄ .
Thus, from the CLT
p̂X − pX
p
pX (1 − pX )/n
|
{z
}
√
σX / n
∼approx. N(0, 1)
This result remains true if we
use an estimate for the population
standard deviation
p̂X − pX
p
|
√ ∼approx. N(0, 1)
p̂X (1 − p̂X )/ n
{z
}
√
σ̂X / n
Thus, in large samples, the confidence interval for pX is:
!
r
r
p̂x (1 − p̂x )
p̂x (1 − p̂x )
, p̂x + zα/2
CI1−α (pX ) = p̂x − zα/2
n
n
Example: finding a confidence interval for pX
Example: 8.6 (Newbold) A random sample of 344 industrial buyers were asked:
”What is your firm’s policy for purchasing personnel to follow on accepting
gifts from vendors?”. For 83 of these buyers, the policy of the firm was for the
buyer to make his/her own decision. Find a 90% confidence interval for the
population proportion of all buyers who are allowed to make their own decisions.
Population:
X = 1 if a buyer makes their own
decision and 0 otherwise
X ∼ Bernoulli(pX )
'
SRS: n = 344 large
Sample: p̂x =
83
344
= 0.241
Area=
0.05
●
z0.05 = 1.645
r
Objective: CI0.9 (pX ) =
p̂x = 0.241
p̂x ∓ zα/2
p̂x (1−p̂x )
n
!
n = 344
1 − α = 0.9
⇒
α/2 = 0.05
zα/2
=
z0.05 = 1.645
0
CI0.9 (pX )
=
@0.241 ∓ 1.645
=
(0.241 ∓ 0.038)
=
(0.203, 0.279)
s
0.241(1 − 0.241)
344
Interpretation: We can be 90%
confident that the proportion of
buyers who make their own decision,
pX , falls in (0.203, 0.279)
1
A
Confidence interval for the population mean, normal
population with unknown variance
1. Let X n be a SRS of size n from X . Under the assumptions:
I
I
X follows a normal distribution with parameters µX and σX2
σX2 is unknown (quite realistic)
2. The pivotal quantity for µX is
X̄ − µX
√ ∼ tn−1
sX / n
Confidence interval for the population mean, normal
population with unknown variance
3. Hence, if tn−1;1−α/2 and tn−1;α/2 are the
(1 − α/2) and (α/2) upper quantiles of
the t distribution with n − 1 degrees of
freedom (df), we have
P(tn−1;1−α/2
∼ tn−1
z}|{
< T < tn−1;α/2 ) = 1 − α
t (Student) density
Recall: if T ∼ tn , E[T ] = 0, V[T ] =
α
2
α
2
1−α
●
●
tn−1 ; 1−α2 = − tn−1 ; α2
tn−1 ; α2
n
n−2
T ∼ tn−1
z }| {
−tn−1;α/2
z }| {
X̄ − µX
√ < tn−1;α/2 ) = 1 − α
4. Therefore P(tn−1;1−α/2 <
sX / n
Confidence interval for the population mean, normal
population with known variance
5. Solve the double inequality for µX :
−tn−1;α/2
<
X̄ −µ
√X
sX / n
<
tn−1;α/2
to obtain the confidence interval estimator
T1 (X n )
T2 (X n )
z
}|
{ z
}|
{
sX
sX
(X̄ − tn−1;α/2 √ , X̄ + tn−1;α/2 √ )
n
n
6. The confidence interval is:
sx
sx
CI1−α (µX ) = (x̄ − tn−1;α/2 √ , x̄ + tn−1;α/2 √ )
n
n
Example: finding a confidence interval for µX
Example: 8.4 (Newbold) A random sample of six cars from a particular model
year had the following fuel consumption figures, in mpg: 18.6, 18.4, 19.2, 20.8,
19.4, 20.5. Find a 90% confidence interval for the population mean fuel
consumption, assuming that the population distribution is normal.
”
“
Population:
Objective: CI0.9 (µX ) = x̄ ∓ tn−1;α/2 √sxn
X = ”mpg of a car from the model
√
year” X ∼ N(µX , σX2 ) σX2 unknown
sx =
0.96 = 0.98
'
SRS: n = 6 small
Sample: x̄ =
116.9
6
sx2 =
n=6
= 19.4833
2282.41 − 6(19.4833)2
= 0.96
6−1
Area=
0.05
●
t5 ; 0.05 = 2.015
x̄ = 19.48
1 − α = 0.9
⇒
α/2 = 0.05
tn−1;α/2
=
CI0.9 (µX )
=
=
t5;0.05 = 2.015
„
«
0.98
19.48 ∓ 2.105 √
6
(19.48 ∓ 0.81)
=
(18.67, 20.29)
Interpretation: We can be 90%
confident that the population mean
fuel consumption for these cars, µX ,
is between 18.67 and 20.29
Example: finding a confidence interval for µX
Example: 8.4 (cont.) in Excel: Go to menu: Data, submenu: Data
Analysis, choose function: Descriptive Statistics.
Column A (data), in yellow (sample mean, half-width tn−1;α/2 √sxn , lower
end-point (cell D3-D16), upper end-point (cell D3+D16)).
t and χ2 distributions
I
Recall that T ∼ tn if T = √ Z2
χn /n
, where Z ∼ N(0, 1) and χ2n follows a
chi-square distribution with df = n, independent of Z .
I
On the other hand, χ2n is the distribution of the sum of n independent
squared N(0, 1) random variables.
I
Note that the rescaled sample quasi variance follows a chi-square
distribution with n − 1 degrees of freedom
Pn
«2
n „
2
X
(n − 1)sX2
Xi − X̄
i=1 (Xi − X̄ )
=
=
∼ χ2n−1
σ
σX2
σX2
X
i=1
Why n − 1 and not n?
If we knew µX , the number of
degrees of freedom would be n,
because we would have n iid random
X
variables Xi σ−µ
X
Since we have to estimate µX with
X̄ , the df are n − 1, because we only
have n − 1 iid random variables Xσi −X X̄
(once you know n − 1 of them, you
can figure out the remaining one)
We say that one degree of freedom is used up to estimate µX
t and χ2 distributions
0.15
χ2 densities
df=20
df=15
df=10
df=5
0.0
0.00
0.1
0.05
0.2
N(0,1)
df=10
df=5
df=3
0.10
0.3
0.4
t and N(0, 1) densities
−4
−2
0
2
4
0
10
20
30
40
Confidence interval for the population variance, normal
population
1. Let X n be a SRS of size n from X . Under the assumptions:
I
X follows a normal distribution with parameter σX2
2. The pivotal quantity for σX2 is
(n − 1)sX2
∼ χ2n−1
σX2
Confidence interval for the population variance, normal
population
3. Hence, if χ2n−1;1−α/2 and χ2n−1;α/2 are
the (1 − α/2) and (α/2) upper
quantiles of the chi-square distribution
with n − 1 degrees of freedom, we have
P(χ2n−1;1−α/2 < χ2n−1 < χ2n−1;α/2 ) = 1 − α
Chi-square density
α
2
1−α
●
●
χ2n−1 ; 1−α
2
χ2n−1 ; α
2
Recall: E[χ2n ] = n, V[χ2n ] = 2n
4. Therefore P(χ2n−1;1−α/2
α
2
χ2n−1
z }| {
(n − 1)sX2
<
< χ2n−1;α/2 ) = 1 − α
σX2
Confidence interval for the population variance, normal
population
5. Solve the double inequality for σX2 :
χ2n−1;1−α/2
1
χ2n−1;1−α/2
(n − 1)sX2
χ2n−1;1−α/2
<
(n−1)sX2
σX2
<
χ2n−1;α/2
>
σX2
(n−1)sX2
>
1
χ2n−1;α/2
> σX2 >
(n − 1)sX2
χ2n−1;α/2
to obtain the confidence interval estimator
(n − 1)sX2 (n − 1)sX2
,
χ2n−1;α/2 χ2n−1;1−α/2
!
6. The confidence interval is:
CI1−α (σX2 )
=
(n − 1)sx2 (n − 1)sx2
,
χ2n−1;α/2 χ2n−1;1−α/2
!
Example: finding a confidence interval for σX2 and σX
Example: 8.8 (Newbold) A random sample of fifteen pills for headache relief
showed a quasi standard deviation of 0.8% in the concentration of the active
ingredient. Find a 90% confidence interval for the population variance for these
pills. How would you obtain a CI for the population standard deviation?
0
Population:
X = ”concentration of an active
ingredient in a pill (in %)”
X ∼ N(µX , σX2 )
'
SRS: n = 15
Sample: sx = 0.8
Area=
0.05
●
Area=
0.05
●
χ214 ; 0.95 χ214 ; 0.05
=6.57
=23.68
2) = @
Objective: CI0.9 (σX
1
(n−1)sx2
(n−1)sx2
A
, 2
χ2
χ
n−1;α/2
n−1;1−α/2
sx2 = 0.82 = 0.64
n = 15
1 − α = 0.9
⇒
α/2 = 0.05
χ2n−1;1−α/2
=
χ214;0.95 = 6.57
χ2n−1;α/2
=
CI0.9 (σX2 )
=
=
χ214;0.05 = 23.68
«
„
14(0.64) 14(0.64)
,
23.68
6.57
(0.378, 1.364) ⇒
√
√
( 0.378, 1.364)
=
(0.61, 1.17)
=
CI0.9 (σX )
To obtain CI (σX ) we apply
end-points of CI (σX2 )
√
to the
Confidence intervals formulae
Summary for one population
I
Let X n be a simple random sample from a population X with mean µX
and variance σX2
Parameter
Mean
Assumptions
Normal data
Known variance
X̄ −µX
√ ∼ N(0, 1)
σX / n
Nonnormal data
Large sample
X̄ −µX
√ ∼approx. N(0, 1)
σ̂X / n
Bernoulli data
Large sample
Normal data
Unknown variance
Variance
Normal data
Standard dev.
Normal data
(1 − α) Conf. Interval
Pivotal quantity
q
p̂X −pX
∼approx. N(0, 1)
p̂X (1−p̂X )/n
X̄ −µX
√ ∼ tn−1
sX / n
2
(n−1)sX
∼ χ2
n−1
σ2
X
2
(n−1)sX
∼ χ2
n−1
σ2
X
„
«
σ
σ
x̄ − zα/2 √X , x̄ + zα/2 √X
n
n
„
–
σ̂
σ̂
µX ∈ x̄ − zα/2 √x , x̄ + zα/2 √x
n
n
#
r
p̂x (1−p̂x )
pX ∈ p̂x ∓ zα/2
n
µX ∈
„
s
s
x̄ − tn−1,α/2 √x , x̄ + tn−1,α/2 √x
n
n
0
1
2
2
(n−1)s
(n−1)s
2 ∈ @
x
x ,
A
σX
χ2
χ2
n−1;α/2
n−1;1−α/2
0v
1
v
u
u
(n−1)sx2
u (n−1)sx2
u
A
σX ∈ @t 2
,t 2
χ
χ
n−1;α/2
n−1;1−α/2
µX ∈
«
Confidence intervals for the population mean:
when to use what?
X ∼ distribution with mean µX and standard deviation σX
.
.
↓
σ known
&
↓
z-based (exact)
t-based (exact)
X ∼ normal
σ unknown
&
.
↓
X normal
n small
Methods beyond
Est II
&
↓
n large
z-based
(approx. CLT)