Download estimating a parameter . educated guess at the value point

Estimation
• Read Sections 7.1, 7.2, and 7.3 of Devore.
• The problem with which we are concerned is
estimating a parameter
educated guess
• That is we want to form an
of the
parameter
.
at the value
on the basis of available data.
• That is, on the basis of observations.
• An estimate of the value, in the form of a single number, (a single
“best guess”) is called a
point estimate
.
• In Stat 2593 we don’t sweat the problem of how to calculate point
estimates.
• Read Chapter 6 of Devore to find out a bit about what this problem
involves.
Estimation (Cont’d.)
point estimates
• In the context of this course,
come by.
are easy to
• You estimate the population quantity by the corresponding sample
quantity.
• E.g. you estimate the population
mean
mean
.
• Likewise you estimate the population
sample
by the sample
proportion
.
1
proportion
by the
• But point estimates are guesses and, as they said in Casablanca, a
guess is just a guess.
• What we REALLY WANT is some measure of
such a guess is.
how good
Interval Estimation
• The usual way in which the “goodness” of a point estimate is
expressed, is through a
confidence
• Such an interval is also called an
interval.
interval estimate
.
• The “goodness” being measured depends on how much variability or
“volatility”
there is in the observations and how MANY
observations there are.
• Less variability and/or more observations makes for a narrower or
“tighter” confidence interval.
• That is, for a
more precise
estimate.
Technical Definition of a Confidence Interval
• A
100(1 − α)% confidence interval for
the parameter in
which we are interested is an interval whose endpoints are random
variables A and B such that
P (A ≤ parameter ≤ B) =
1−α
• The random variables A and B are calculated from (i.e. are
functions of
) the available data.
2
The (fairly) General Form of a Confidence Interval
• In the majority of contexts (in all contexts with which we are
concerned) the forgoing formalism translates into: A confidence
interval is given by
point estimate ± (table value) ×
(st. dev. of point estimate)
“t-tables”
• The table value comes from the
• It depends on the desired confidence level i.e. upon
.
α
.
• The “usual” or “standard” values of α are
0.10, 0.05, and 0.01
with 0.05 being the most “standard of all”.
• For α = 0.05 we are talking about
95%
confidence intervals.
Degrees of Freedom
• The table value also depends on how precisely the
standard deviation
of the population, i.e.
σ
has
been
estimated.
• This dependence is specified by the so-called
degrees of freedom
.
• Large degrees of freedom mean that σ is well estimated.
• Small degrees of freedom mean that σ is not so well estimated.
3
• If σ is known — so that we don’t have to estimate it at all — then
the degrees of freedom are infinity, i.e. ∞.
perfectly
• I.e. σ is
“estimated”.
Degrees of Freedom (Cont’d.)
• When σ must actually be estimated (i.e. in real life (!), the degrees of
sample size n
freedom depend on the
.
• In the sort of problem with which we are initially concerned, namely
one sample problems, the degrees of freedom are equal to
n−1
n−k
• More generally, the degrees of freedom are equal to
where k is the number of parameters that you had to estimate
BEFORE you could start
to estimate σ
.
WARNING
“classical”
• This whole
procedure for finding confidence
intervals is valid ONLY WHEN
EITHER the distribution of the data is
large enough
OR n is
are “effectively” ∞.
normal
so that the degrees of freedom
• What is going on is that we need n large enough so that (1) the
Central Limit Theorem kicks in, and (2) the sample standard
deviation is such a good estimate of σ that we can pretend that it
IS σ
.
4
.
Confidence Intervals for Means
• Setting: We have a sample X1 , X2 , . . . , Xn from a (normal)
population with mean µ and variance σ 2 .
• We want a
100(1 − α)%
• The point estimate of µ is X̄.
confidence interval for µ.
√
σ/ n
• The standard deviation of X̄ is
.
• When σ is unknown, we √
estimate it by the sample standard deviation
S, so we actually use S/ n.
• To calculate S we must first calculate X̄ which is of course an
estimate of µ.
• That is we have to estimate 1 parameter namely µ before we can start
estimating σ.
• Hence
k=1
and so the degrees of freedom are
Using the t-Tables
n−1
• The degrees of freedom tell you what line or row of the t-tables to
look at in order to find your
table value
.
• If the degrees of freedom are larger than 30 most people (including
me) take them to be equal to ∞.
• I.e. if the degrees of freedom are larger than 30 we might as well
pretend
that σ is known.
• This is just a pretence, but it makes no practical difference to the
answer.
• The column of the t-tables to look at is the
5
α/2
column.
.
• This is because we are putting half of the “missing confidence” at
each end.
• E.g. if α = 0.05 and n = 17, the table value is
16 and column 0.025 of the t-tables.
2.120
from row
SUMMARY
normal
• If the data are
, or if n is large, a 100(1 − α)%
confidence interval for the population mean is given
S
X̄ ± (table value) × √
n
• In the unlikely event that σ is known, replace S by
foregoing.
σ
• The table value comes from the (degrees of freedom) =
row and the α/2 column of the t-tables.
in the
n−1
• If σ is known or if n is large take the degrees of freedom to be
Interpretation of Confidence Intervals
∞
.
• Roughly speaking a confidence interval is an interval such that you
true value
of the
are “pretty damned sure” that the
parameter that you are estimating lies in the interval.
• But does “sure” mean here? It SHOULD be interpreted in terms of
probability.
• Remember that “theoretically” a confidence interval [A, B] is a pair of
random variables
A and B such that
P (A ≤ parameter ≤ B) = 1 − α.
6
observed
• But when we calculate a confidence interval we have
our data and so A and B turn magically from random variables into
numbers.
• This makes the foregoing
tricky to think about.
probability statement
Confidence and Probability
• E.g. suppose your (95%) confidence interval for µ turns out to be
[42, 56]
. It doesn’t really MEAN anything to say
“P (42 ≤ µ ≤ 56) = 0.95”.
constant
• Since µ is not a random variable, but rather a
statement 42 ≤ µ ≤ 56 is either true or it isn’t. Theres’s no
probability involved.
, the
• You should be aware that there are some strange people called
“Bayesians”
who try to get around this problem by thinking
of µ as a random variable.
• But this just messes everything up and has to rest on a concept called
“personal”
probability which is just plain silly.
Thought Experiments
• One rational way of thinking about the meaning of the probability
statement is through a
“gedanken experiment”
.
• This experiment consists in considering what would happen if if a
“large number” of people were to collect data (independently) and
were each to calculate (e.g.) a
95% confidence interval
estimated.
7
for the parameter being
• The meaning of the probability statement is that in such a situation
95% of those confidence intervals would contain or
“cover” the true value of the parameter.
(about)
• And about 5% would miss it.
Believeable Values
• An informal but very useful way of thinking about confidence
plausible
intervals is to think of them as being “a range of
believeable values” for the parameter being estimated.
or
• That is you can “believe” a value for the parameter if that value
lies in
the confidence interval.
• You don’t/can’t/shouldn’t believe values that
the confidence interval.
don’t lie in
• Of course whether you “believe” a value or not depends on the
level of confidence
that you are imposing.
Example of a Confidence Interval for a Mean
• Problem 13, page 297 Devore 6th ed. page 293 5th ed.
• Summary of information: n = 69, x̄ =
0.163
1.028
, and s =
.
• Since n is large we needn’t worry about whether the data are normal.
• Also, since n is large, the degrees of freedom are
8
∞
.
• Hence a 95% confidence interval for µ is given by
s
x̄ ± (table value) × √
n
0.163
= 1.028 ± 1.96 × √
69
= 1.028 ±
0.038
Example (Cont’d.)
• Notice in the foregoing that since our sample has been observed, the
random variables
numbers x̄ and s.
X̄ and S have (Poof!) turned into the
• The foregoing calculation gives the interval
[0.990, 1.066]
.
• Thus (at the 95% confidence level) a value of (e.g.) 1 is believable for
the population mean dye layer density.
• However a value of (e.g.)
1.1
is not (quite) believable.
Another Example
• Problem 32, page 306 Devore, 6th ed., page 302 5th ed.
• Summary of info: n = 8, x̄ =
30.2
,s=
3.1
.
• Here we have a difficulty — n is small so our methods yield a valid
confidence interval only if the data are normal.
• There’s no way to tell if the data
really ARE normal
• The problem says to assume normality, so we will.
9
.
• Under the assumption of normal data, a 95% confidence interval for µ
is given by
s
x̄ ± (table value) × √
n
3.1
= 30.2 ± 2.365 × √
8
2.59
= 30.2 ±
Another Example (Cont’d)
[26.61, 32.79]
• The foregoing gives the interval
.
28
• Hence (e.g.)
is a plausible value for the population mean
“interfacial shear yield stress”.
• At least this is true if it is true that the data are
normal
.
25
is not a plausible value, subject again to the
• But (e.g.)
assumption of the normality of the data.
One Last Example
• Problem 5, page 290 Devore 6th ed., page 286 5th ed.
• We are told to assume normal data, and that the population standard
deviation is
σ = 0.75
.
4.85
• Part (a): Info — n = 20, x̄ =
.
• A 95% confidence interval for µ is given by
σ
x̄ ± (table value) × √
n
0.75
= 4.85 ± 1.96 × √
20
= 4.85 ±
10
0.328
.
• This gives the interval
[4.522, 5.178]
.
One Last Example (Cont’d.)
4.56
• Part (b): Info — n = 16, x̄ =
• A
98%
.
confidence interval for µ is given by
σ
x̄ ± (table value) × √
n
0.75
= 4.56 ± 2.326 × √
16
= 4.56 ±
• This gives the interval
0.436
[4.124, 4.996]
.
One Last Example (Cont’d.)
• Part (c): To get the
we’d need
width
0.75
1.96 × √
n
of a 95% CI to be (at most) 0.40
≤ 0.20
• This says that
54.02
n ≥ (1.96 × 0.75/0.2)2 =
• So we could get away with
n = 55
11
.
One Last Example (Cont’d.)
• Part (d): To get the half -width for a
we’d need
0.75
2.576 × √
n
99%
CI to be (at most) 0.2
≤ 0.2
• This says that
n ≥ (2.576 × 0.75/0.2)2 =
• Hence we’d need
n = 94
93.3
.
How Big a Sample Do I Need?
• The foregoing examples have illustrated how to deal with this issue.
• To answer the question, one needs to specify the answers to three
preliminary questions:
1. How
“confident”
2. How
precise
3. How
variable
do I want to be?
do I want my estimate to be?
will the observations be?
• The answer to (1) is almost always 95%.
• The answer to (2) is expressed as “to within ±d”.
• The value of
d
depends on the practicalities of the area of study.
• In a course such as Stat 2593 the required level of precision will
always be supplied as part of the question, so you don’t need to worry
about this issue.
12
Guessing at the Variability
real difficulty
• Finding an answer to (3) is the
.
• The problem is that you want to know, but you don’t know, and
never will know the value of σ.
guess
• What you have to do is find some way of making a
and then use this guess where you would like to use the true
value of σ.
• In effect, you pretend that your guess is the
you know this is not really the case.
truth
at σ,
even though
• Having answered the three preliminary questions you calculate the
sample size from
n≥
(table value)2 × (
best guess at σ 2
)
d2
Getting the Table Value — df = ∞
at least as big
“round high” .
• Note that this says that n must be
right hand side, so you always
as the
• The table value comes from the ∞ line of the t-tables.
• This is because:
• (a) You don’t know
n
so where else could you go?
• (b) It will (always?) turn out that n is so big that the degrees of
freedom might as well be ∞.
• (c) You are pretending that σ is
degrees of freedom should be ∞.
13
“known”
anyhow, so the
How to Guess at the Variability
σ
• Some ways of getting a guess at
are:
sample standard deviation
from an
• (i) Use the
existing sample (which turned out to give inadequate precision). (This
is the usual scenario for questions in introductory statistics courses.)
• (ii) Find the results of “similar” research in the literature.
• (iii) Do a pilot study.
• (iv) Guess at the maximum and at the minimum values that will be
observed. (People almost always have a better intuitive feeling for
these than they have for
σ≈
σ
.) Then use
max − min
4
=
“range”
4
Confidence Intervals for Proportions
• Suppose we are trying to estimate p which is the
of successes in a certain population.
proportion
probability
• We may also describe p as being the
of getting a
“success” when we draw a random item from the population.
• We draw a sample of size n from the population, and on that basis we
wish to form an
(interval) estimate
of p.
• The general principal applies; such a confidence interval is given by
point estimate±(table value)×
(st. dev. of point estimate)
14
The Sample Size Must be Large
• The methods we shall use for CIs for
valid when n is
LARGE
proportions
are only
.
degrees of freedom
calculating a CI for a proportion are always ∞ .
• Hence the associated
to use when
Point Estimates and St. Devs. for Proportions
• The point estimate of p is the
p̂ = X/n
• This is
the sample.
sample proportion
.
where X is the number of “successes” in
• The only problem left is to determine the standard deviation of p̂.
• But we know that
X ∼ Bin(n, p)
.
• Hence V (X) = np(1 − p).
• Hence
V (p̂) = V (X/n) = V (X)/n2
= np(1 − p)/n2 =
p(1 − p)/n
But We Don’t Know p!
• Since we don’t know p — that’s what we’re trying to estimate for
crying out loud! — we do the best we can under the circumstances.
• That is, we replace p by our
point estimate
15
of p, i.e. by p̂.
• SUMMARY: A confidence interval for a proportion is given by
p̂ ± (table value) ×
r
p̂(1 − p̂)
n
• Devore does something a little more complicated — he solves a
quadratic
to get the CI.
• See expressions (7.10), page 291 5th ed., page 295, 6th ed.
• PLEASE DON’T use (7.10)!!! Use the simpler expression given above
(equivalently expression
(7.11)
in Devore).
Degrees of Freedom; Point Estimate
• The
table value
to be used when calculating confidence
intervals for proportions comes from the
∞
row of the t-tables.
• This stuff only works if the sample size is large!
• And if the sample size is large, the
are (effectively) infinite.
degrees of freedom
• The point estimate in this setting is the sample proportion.
p̂ = X/n
• That is, it is
where X is the number of “successes”
in the sample and n is the sample size.
Example
• Problem 21, page 298 Devore 6th ed., page 294 5th ed.
• Summary of info: n = 539, x = 133. Hence
16
p̂ = 133/539
.
• The required confidence level is 99% so α = 0.01 whence α/2 = 0.005.
2.576
• The table value is therefore
freedom = ∞ row of the tables.
from the degrees of
• The interval estimate of p is therefore given by
r
p̂(1 − p̂)
p̂ ± (table value) ×
n
r
133
1
133 406
=
± 2.576 ×
×
×
= 0.2468 ±
539
539 539 539
giving the interval
0.0478
[0.1990, 0.2946]
How Big a Sample Do I Need?
• The
0.05.
half-width
of the forgoing confidence interval is about
• Suppose we wanted the estimate of p to be accurate to within ±0.03,
with 99% confidence.
• How big a sample should we take?
• The procedure is very similar to what we do in the estimating a mean
setting. We need
r
p(1 − p)
2.576 ×
n
≤ 0.03
• Now of course we don’t know p.
• So we must come up with a “reasonable guess” at
p
.
• This is analogous to the fact that in the “means” setting we needed a
“reasonable guess” at
σ
.
17
Guessing at p
“preliminary”
• In the current situation we have a
sample,
i.e. the sample which gave us an (insufficiently precise) estimate of p.
• We can use the p̂ from that preliminary sample, i.e.
our “reasonable guess” at p.
• This gives
n≥
so we’d take
2.576
0.03
2
× 0.2468 × 0.7532 =
0.2468
as
1370.6
n = 1371
• But suppose we didn’t have a preliminary sample; what then?
• We might have some sort of guess at p.
The Worst That Can Happen
“worst case scenario”
• Failing that, we could use the
• Note that proportions have an advantage over means in this regard.
• For means there is no “worst case”; things can get
arbitrarily bad
.
• For proportions the worst case is the value of p that makes p(1 − p)
largest and hence makes n largest; that value is
p = 1/2
• If we had to go with the worst case “guess” at p in the current
example, i.e. with
n≥
so we’d take
p = 1/2
2.576
0.03
2
n = 1844
we’d get
× 0.5 × 0.5 = 1843.3
.
18
.
.
Summary on Sample Size Calculations
• For both means and proportions the sample size calculations boil
down to:
n≥
best guess at σ 2
(table value)2 × (
)
d2
where we want our estimate to be “accurate” to within ±d.
• In the case of proportions, σ 2 becomes
• The table value comes from the
∞
p(1 − p)
.
line of the t-tables.
• In the case of proportions we can, if all else fails, make a guess at p by
using the “worst case scenario” i.e. p = 0.5.
• In the case of means, a good way of making a guess at σ is to guess
instead at the max and min of the observations, set range =
max − min and use
σ ≈ range/4
.
Using Minitab to Calculate Confidence Intervals
• The calculations for confidence intervals can all be done with Minitab.
• Click on Stat then on
Basic Statistics
, then on
1 sample t or
1 sample Z or
1 proportion .
• The ‘‘1 sample Z’’ is used for (artificial) problems in which σ is
known
.
• There’s a wee window in which you specify “Sigma”.
19
default
• For all three possibilities the
confidence level is 95%.
• You can specify the confidence level (to be something other than
95%) by clicking on
Options
.
Minitab and CIs for Means
• You need only concern yourself about using the ‘‘1 sample t’’ and
DATA
‘‘1 sample Z’’ utilities with
.
• I.e. use these utilities when you have the actual sample stored in a
column
of the Minitab worksheet.
• Minitab 14 will do the calculations for you on the basis of the
“summary statistics”
namely n, x̄, and s (or σ in the
“1 sample Z” case).
• But the calculations are not arduous; you might as well do them
by hand
.
Minitab and CIs for Proportions
• For proportions, the only problems you will ever get will involve
summary statistics (i.e.
sample size
and
successes ).
• Minitab will do such problems for you, if you insist.
• But I’m not going to go there.
• Minitab will also do proportions problems based on
columns of data
.
20
number
of
• I.e. it will determine the sample size (length of the column) and count
the
number of “successes”
in the column.
• And then do all the necessary calculations.
• I won’t go there either!
Prediction Intervals
• As I have previously tried to emphasize, we often DO NOT CARE
about
the mean
of the population.
• We are more interested in individual members of the population, i.e.
in future individual
observations
.
• Confidence intervals tell us nothing directly about individual
observations.
• E.g. suppose that someone takes a sample of U.N.B. students and
finds that a 95% confidence interval for the mean height of U.N.B.
students is
[168, 176]
cm.
• What does this tell you about the height of the next student to walk
through the door? NADA!
• To say something about the value of the next observation we need a
prediction interval
or PI for short.
Normality is Crucial
• The following technique for calculating prediction intervals is valid
ONLY IF
the data are normal
.
• There are various techniques for checking on whether data are normal.
21
goodness of fit tests
• These include
which we will touch
on briefly later on in the course, and normal probability plots which
we unfortunately don’t have time to cover.
• You can read about probability plots on pages 188 to 197 of Devore
6th ed., pages 185 to 195 5th ed.
• In any case you need
draw any reasonably
your data are normal.
quite a lot of data before you can
certain conclusions about whether
Calculating Prediction Intervals for Normal Populations
• Given a sample from a normal population we form a
100(1 − α)%
confidence level prediction interval for “the
next observation” as
point estimate ± (table value) ×
st. dev.
where the “point estimate” is the sample mean i.e. X̄ and where “st.
dev.” is the
appropriate
standard deviation for prediction.
• In fact “st. dev.” here is equal to the standard deviation of X − X̄
where X is
predict.
the new observation
that you are trying to
The “Appropriate” Standard Deviation
• Explicitly “st. dev.” here is equal to
S
p
1 + 1/n
where S
is the sample standard deviation of your sample.
• In the unlikely event that σ is known, you of course use this known
value instead of the estimate S.
22
from X
• The “1” inside the square root comes
comes
from X̄
and the “1/n”
,
• Actually the 1/n makes a negligible amount of difference and can be
ignored for “large” n (e.g. n = 30 or 40 or more).
• If you ignore 1/n then the prediction interval becomes
x̄ ± (table value) × S
.
Example of a Prediction Interval
• Problem 38 page 307 Devore 6th ed., problem 38, page 303 5th ed.
• Summary of info: n = 25,
s = 0.0065
x̄ = 0.0635
and
.
• Part (b) of the problem asks us to calculate a 95% prediction interval
for the amount of warpage of a single piece of laminate.
• This prediction interval is given by
p
1 + 1/n
p
= 0.0635 ± 2.064 × 0.0065 × 1 + 1/25
x̄ ± (table value) × s
= 0.0635 ±
giving the interval
0.0137
[0.0498, 0.0772]
.
Example of a Prediction Interval (Cont’d.)
• The table value comes from the
row of the t-tables.
“degrees of freedom”
• In this current scenario, the degrees of freedom are
n − 1 = 25 − 1 = 24
23
.
• Since α = 0.05 we look in the α/2 = 0.025 column.
• Compare the foregoing prediction interval with the much narrower
confidence interval for the population mean warpage, explicitly
[0.0608, 0.0662]
.
• For instance 0.0550 is “believable” as the value of an individual
observation from this population.
NOT believable
• However it is
population mean.
24
as the value of the