Download TMS-062: Lecture 5 Hypotheses Testing

Hypothesis testing problem
Construction of CR
Tests for Mean
Hypothesis testing problem
Construction of CR
Tests for Mean
Hypothesis Testing Problem
TMS-062: Lecture 5
Hypotheses Testing
Sergei Zuyev
Same basic situation as before:
Data: random i. i. d. sample X1 , . . . , Xn from a population and
we wish to draw inference about unknown population
parameter θ. But the question is different:
Suppose we know that in the past θ = θ0 but conditions have
changed. Has θ changed too?
Two cases, called hypotheses are to consider:
Null hypothesis: H0 : θ = θ0
Alternative hypothesis: HA
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Sergei Zuyev
Hypothesis testing problem
Construction of CR
Tests for Mean
TMS-062: Lecture 5 Hypotheses Testing
Test Statistic
We use three forms of alternative hypothesis HA :
(
HA : θ > θ0
One sided:
HA : θ < θ0
Sergei Zuyev
– there is a change
Hypothesis testing problem
Construction of CR
Tests for Mean
Alternative Hypotheses
Two sided:
– no change
HA : θ 6= θ0
TMS-062: Lecture 5 Hypotheses Testing
We cannot observe θ directly, but we wish to decide from
sample X = (X1 , . . . , Xn ) which of H0 and HA is true.
So we compute a the value of some statistics, say T (X),
called the test statistic and we decide H0 if T (X) ∈ S0 and
HA if T (X) ∈ SA .
Such a procedure is called Test of the hypothesis H0
against HA .
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Hypothesis testing problem
Construction of CR
Tests for Mean
Hypothesis testing problem
Construction of CR
Tests for Mean
Critical Region
Types of Decision Errors
Since our decision is based on a random value T (X) then there
may be two associated types of errors: to reject H0 when it is
true (type I error) and to accept H0 when HA holds (type II
error).
Domain of T
Sample Space
S0
T
H0
T (X)
X
SA
H0
HA
H0
OK
Type I error
HA
Type II error
OK
HA
Traditionally, SA is called the critical region (CR). How to define
it sensibly?
Sergei Zuyev
DECISION
REALITY
TMS-062: Lecture 5 Hypotheses Testing
Sergei Zuyev
Hypothesis testing problem
Construction of CR
Tests for Mean
TMS-062: Lecture 5 Hypotheses Testing
Hypothesis testing problem
Construction of CR
Tests for Mean
Test Paradigm
Usually it is considered more serious to reject H0 when, in
fact, it is true, so the probability of type I error is preset to a
particular small value α (typically, 5% or 1%).
The parameter α is called the significance level of the test.
Now a test with that level α is sought that minimises
probability of type II error:
( P T (X) ∈ SA
P T (X) ∈ S0
Sergei Zuyev
H0
=α
HA
−→ min
TMS-062: Lecture 5 Hypotheses Testing
1
Specify H0 and HA .
2
Choose a level α and a test statistic T .
3
Derive the critical region.
4
Compute the test statistic T .
5
Decide HA if T in CR, otherwise accept H0 .
6
Interpret your decision always citing the significance level
α at which you conducted the test.
7
If we decide HA , then θ 6= θ0 and we usually estimate θ and
construct a confidence interval for it.
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Hypothesis testing problem
Construction of CR
Tests for Mean
Hypothesis testing problem
Construction of CR
Tests for Mean
Construction of CR
Distribution of T
if HA true
Assume the distribution of T given θ is monotone with respect
to θ, so that HA = {θ < θ0 } implied that lesser values of T are
more likely than those under H0 . This needs not always be the
case, but it will be so in this course. Indeed many (but not all) of
the test statistics we meet have the following form:
b ) is an unbiased estimator of θ (i. e. the
if H0 = {θ = θ0 } and θ(X
b ) is θ) then
mean of θ(X
b ) − θ0
θ(X
T (X ) =
.
b
st.error(θ(X))
(1)
if H0 true
α
SA
S0
T0
only this tail supports HA
If HA is one-sided, we chose SA to be as far on that side as
possible to maximise its probability under HA .
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Hypothesis testing problem
Construction of CR
Tests for Mean
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Hypothesis testing problem
Construction of CR
Tests for Mean
Power of the Test
if H0 true
possible HA
possible HA
α/2
SA
α/2
S0
SA
T1
T2
both tails support HA
If HA is two sided, then HA –distribution of T can be on either
side. So SA has two pieces on both tails of the distribution of T
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
under H0 .
The test is constructed so as to ‘protect’ H0 against
possibility to erroneously decide HA . Since we preset
probability of this event to α (the error level of the test),
probability of erroneously reject H0 is α.
But we do not have such a protection for HA : we do not
know the prob. of acceptance of H0 if HA is true for
one-sided or two-sided tests as HA specifies no value for θ.
The function
β(θ) = P decide HA θ ,
which is the probability to accept HA if the value of the
parameter is θ, is called the power of the test.
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Hypothesis testing problem
Construction of CR
Tests for Mean
Hypothesis testing problem
Construction of CR
Tests for Mean
Z -test for Mean – σ is Known
Critical Region
Settings: X1 , . . . , Xn is an i. i. d. sample from N (µ, σ 2 )
distribution with a known σ.
To test:
(i) H0 = {µ = µ0 } vs. HA = {µ 6= µ0 }
or
(ii) H0 = {µ = µ0 } vs. HA = {µ < µ0 }
or
Given the critical level α, an obvious choice of CR is
(i) (−∞, −Zα/2 ) ∪ (Zα/2 , +∞)
(ii) (−∞, −Zα )
(iii) H0 = {µ = µ0 } vs. HA = {µ > µ0 }
Test statistic
Z =
X − µ0
√ =
σ/ n
√
(iii) (Zα , +∞) (see Figure above)
where Zα is such that P{N (0, 1) > Zα } = α.
n(X − µ0 )
.
σ
Then if H0 holds true, Z ∼ N (0, 1).
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Sergei Zuyev
Hypothesis testing problem
Construction of CR
Tests for Mean
TMS-062: Lecture 5 Hypotheses Testing
Hypothesis testing problem
Construction of CR
Tests for Mean
Example
Solution
1
Take α = 5%, for instance. From tables, Z0.05 = 1.645 and
Z0.025 = 1.96. For the first question, the zero hypothesis:
H0 = {µ = 56}, alternative hypothesis HA = {µ 6= 56}
(two-sided). Critical region
SA2 = (−∞,
√ −1.96) ∪ (1.96, +∞). The value of the test
statistic 25(50.60 − 56)/15 = −1.8 ∈ SA2 so there is no
sufficient evidence at the 5% level to suggest that the value
has changed.
2
In contrast, the claim that the mean is now less than 56
SEK is supported with 5% error level as the value −1.8 of
the test statistic does fall in (−∞, −1.645) = SA1 – the CR
for one-sided alternative hypothesis HA = {µ < 56}.
From past records, a mail order company have found that the
value of an order is normally distributed with mean 56.00 SEK
and st.dev. 15 SEK. Recent economic reverses have caused
them to reassess this view. A random sample of 25 orders is
chosen and the average is 50.60 SEK.
1
Does this mean that the mean value of an order is now
different from 56 SEK?
2
Suppose you have been asked if the mean is now less
than 56 SEK – what is your response?
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Hypothesis testing problem
Construction of CR
Tests for Mean
Hypothesis testing problem
Construction of CR
Tests for Mean
t-test for Mean – σ is Unknown
Critical Region
The settings, H0 , HA are the same, but σ is unknown.
Therefore replacing it with its estimate – sample variance, leads
to statistic
√
n(X − µ0 )
,
t(X ) =
S
where
n
1 X 2
2
2
S =
Xi − nX
n−1
i=1
CR’s are as above with Zα replaced by tα , where tα is such that
P{t > tα } = α for t having t(n − 1) distribution.
As σ is rarely known, t-test is most often used. However, for
large samples (n ≥ 30) by the CLT, t(n − 1) is close to the
standard normal distribution N (0, 1) so that Z -test is also
possible even if the data are not normally distributed.
If H0 is true, t(X ) has t(n − 1)-distribution with n − 1 degrees of
freedom.
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Sergei Zuyev
Hypothesis testing problem
Construction of CR
Tests for Mean
Hypothesis testing problem
Construction of CR
Tests for Mean
Test for Proportion’s Value
Test Statistic
Settings: total population proportion possessing a certain
feature is p, a random sample proportion of size n is p̂. We
assume here that n is large (n ≥ 50).
To test:
(i) H0 = {p = p0 } vs. HA = {p 6= p0 }
or
(ii) H0 = {p = p0 } vs. HA = {p < p0 }
or
(iii) H0 = {p = p0 } vs. HA = {p > p0 }
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
TMS-062: Lecture 5 Hypotheses Testing
Let Xi = 1 if i-th individual in the sample possesses the feature
b = Sn /n, where
and Xi = 0 otherwise. Then p
Sn = X1 + · · · + Xn . But Sn ∼ Bin(p, n), so that E Sn = np and
var Sn = np(1 − p). Therefore by the CLT, the following test
statistic
p̂ − p0
.
Z =r
p0 (1 − p0 )
n
is asymptotically N (0, 1) provided H0 is true.
So the CR is as for Z -test.
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Hypothesis testing problem
Construction of CR
Tests for Mean
Hypothesis testing problem
Construction of CR
Tests for Mean
p-value
Up to now we first decided upon the critical level α, then
constructed the corresponding CR, finally computed
T ∗ = T (X ) and checked whether its value falls into CR.
Alternatively, instead of specifying α and CR at the
beginning, we could derive the maximal value of error α at
which H0 can still be accepted given T ∗ . This value is
called
Distribution of T under H0
HA = {θ > θ0 }
p-value
p-value of T ∗ = P{obtaining a sample as extreme as T ∗ }

∗

θ = θ0 } if HA = {θ > θ0 }

 P{T ≥ T
=
P{T ≤ T ∗


2P{T ≥ T ∗
θ = θ0 }
θ = θ0 }
T∗
T
if HA = {θ < θ0 }
if HA = {θ 6= θ0 }
Note the coefficient 2 for symmetrical statistics in two-sided
case.
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Sergei Zuyev
Hypothesis testing problem
Construction of CR
Tests for Mean
TMS-062: Lecture 5 Hypotheses Testing
Hypothesis testing problem
Construction of CR
Tests for Mean
Example
Notice: T ∗ ∈ CR ⇐⇒ p-value < α. Therefore, an alternative
procedure is to compute the p-value of the observed test
statistic and
if p-value < α
decide HA
if p-value > α
decide H0 .
Statistical packages generally quote p-values when tests are
performed as they do not have pre-assigned significance levels
and let the users decide upon their own α. The danger here is
to decide after getting the value whether it is big or small.
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing
Mail orders (revisited): the average is 56 SEK and the
alternative hypothesis is µ < 56. The observed value of
Z -statistic is -1.8.
As for Z ∼ N (0, 1) we have P(Z ≤ −1.8) = 0.036, then we can
accept H0 only at the level of 3.6%. p-value is thus 0.036 here.
As it is less than our pre-set value of 5%, we reject H0 .
Sergei Zuyev
TMS-062: Lecture 5 Hypotheses Testing