Download hypothesis testing

Review of Statistics





Estimation of the Population Mean
Hypothesis Testing
Confidence Intervals
Comparing Means from Different Populations
Scatterplots and Sample Correlation
Estimation of the Population Mean


One natural way to estimate the population mean,
, is simply
to compute the sample average from a sample of n i.i.d.
observations. This can also be motivated by law of large
numbers.
But, is not the only way to estimate
. For example, Y1 can
be another estimator of .
Key Concept 3.1
Copyright © 2003 by Pearson
Education, Inc.
3-4



In general, we want an estimator that gets as close as possible to
the unknown true value, at least in some average sense.
In other words, we want the sampling distribution of an
estimator to be as tightly centered around the unknown value as
possible.
This leads to three specific desirable characteristics of an
estimator.
Three desirable characteristics of an estimator.
Let
denote some estimator of
,
 Unbiasedness:
 Consistency:
 Efficiency.
Let
be another estimator of
, and suppose both and
are unbiased. Then
is said to be more efficient than
if
Properties of


It can be shown that E( )=
and
(from law of large
numbers), is both unbiased and consistent.
But, is efficient?
Examples of alternative estimators.
Example 1: The first observation Y1?
Since E(Y1)=
, Y1 is an unbiased estimator of

if n≥2,
is more efficient than Y1.
. But,
Example 2:
where n is assumed to be an even number.
The mean of is
and its variance is
Thus is unbiased and, because Var(
consistent.
However, is more efficient than .
) → 0 as n→∞,
is
In fact, is the most efficient estimator of
among all unbiased
estimators that are weighted averages of Y1, … , Yn. (Weighted
average implies that the estimators are all unbiased.)
Nonstatistical Hypothesis Testing
A criminal trial is an example of hypothesis testing
without the statistics.
In a trial a jury must decide between two hypotheses.
The null hypothesis is
H0: The defendant is innocent
The alternative hypothesis or research hypothesis is
H1: The defendant is guilty
The jury does not know which hypothesis is true. They
must make a decision on the basis of evidence presented.
Nonstatistical Hypothesis Testing
In the language of statistics convicting the defendant is
called
rejecting the null hypothesis in favor of the
alternative hypothesis.
That is, the jury is saying that there is enough evidence
to conclude that the defendant is guilty (i.e., there is
enough evidence to support the alternative hypothesis).
Nonstatistical Hypothesis Testing
If the jury acquits it is stating that
there is not enough evidence to support the
alternative hypothesis.
Notice that the jury is not saying that the defendant is
innocent, only that there is not enough evidence to
support the alternative hypothesis. That is why we never
say that we accept the null hypothesis.
Nonstatistical Hypothesis Testing
There are two possible errors.
A Type I error occurs when we reject a true null
hypothesis. That is, a Type I error occurs when the jury
convicts an innocent person.
A Type II error occurs when we don’t reject a false null
hypothesis. That occurs when a guilty defendant is
acquitted.
Nonstatistical Hypothesis Testing
The probability of a Type I error is denoted as α (Greek
letter alpha). The probability of a type II error is β
(Greek letter beta).
The two probabilities are inversely related. Decreasing
one increases the other.
Nonstatistical Hypothesis Testing
5. Two possible errors can be made.
Type I error: Reject a true null hypothesis
Type II error: Do not reject a false null
hypothesis.
P(Type I error) = α
P(Type II error) = β
Decision Results
H0: Innocent
Jury Trial
Actual Situation
Verdict
Innocent
Guilty
Innocent
Guilty
H0 Test
Actual Situation
Decision
Correct
Error
Accept
H0
Error
Correct
Reject
H0
H0 True
H0
False
1–a
Type II
Error
(b)
Type I Power
Error (a) (1 – b)
Hypothesis Testing
The hypothesis testing problem (for the mean): make a
provisional decision, based on the evidence at hand, whether a null
hypothesis is true, or instead that some alternative hypothesis is
true. That is, test
Key Concept 3.5
Copyright © 2003 by Pearson
Education, Inc.
3-19
Terminology



Significance level; p-value; critical value
Confidence interval; acceptances region, rejection region
Size; power
Type I and Type II Error

Type I Error (Red) ，Type II Error (Blue)
虛無分配
對立假設下的
對立假設
虛無分配
α/2
臨界值
Size=2*α/2
β
Power=1-β


p-value = probability of drawing a statistic (e.g. ) at least as
adverse to the null as the value actually computed with your data,
assuming that the null hypothesis is true.
The significance level of a test is a pre-specified probability of
incorrectly rejecting the null, when the null is true.
Calculating the p-value based on
:
where
is the value of actually observed.
 To compute the p-value, you need to know the distribution of
.
 If n is large, we can use the large-n normal approximation.
where
denotes the standard deviation of the distribution of
.
Calculating the p-value with Y known
Confidence Intervals
A 95% confidence interval for
is an interval that contains the
true value of Y in 95% of repeated samples.
Digression: What is random here? the confidence interval— it will
differ from one sample to the next; the population parameter,
,
is not random.
In practice,
is unknown—it must be estimated.
Estimator of the variance of Y:
Fact: If (Y1, … , Yn ) are i.i.d. and E(Y4)< ∞ , then
 Why does the law of large numbers apply? Because
is a
sample average.
 Technical note: we assume E(Y4)< ∞ because here the average is
not of Yi , but of its square.
For the first term,
 Define
are i.i.d.
, then
, and W1, … , Wn
.


Thus W1, … , Wn are i.i.d. and Var(Wi) < ∞ , so

Therefore,

For the second term, because
Therefore,
Computing the p-value with
estimated:
The p-value and the significance level
With a prespecified significance level (e.g. 5%):
 reject if |t| > 1.96.
 equivalently: reject if p ≤ 0.05.
 The p-value is sometimes called the marginal significance
level.
What happened to the t-table and the degrees of
freedom?

Digression: the Student t distribution
If Yi, i = 1,…, n is i.i.d. N(Y, ), then the t-statistic has the
Student t-distribution with n – 1 degrees of freedom. The critical
values of the Student t-distribution is tabulated in the back of all
statistics books. Remember the recipe?
 Compute the t-statistic
 Compute the degrees of freedom, which is n – 1
 Look up the 5% critical value
 If the t-statistic exceeds (in absolute value) this critical value,
reject the null hypothesis.
Comments on this recipe and the Student tdistribution
The theory of the t-distribution was one of the early triumphs of
mathematical statistics. It is astounding, really: if Y is i.i.d. normal,
then you can know the exact, finite-sample distribution of the tstatistic – it is the Student t. So, you can construct confidence
intervals (using the Student t critical value) that have exactly the right
coverage rate, no matter what the sample size. But….
34
Comments on Student t distribution, ctd.
If the sample size is moderate (several dozen) or large (hundreds or
more), the difference between the t-distribution and N(0,1) critical
values are negligible. Here are some 5% critical values for 2-sided
tests:
degrees of freedom
(n-1)
10
20
30
60

5% t -distribution
critical value
2.23
2.09
2.04
2
1.96
35
Comments on Student t distribution, ctd.

So, the Student-t distribution is only relevant when the
sample size is very small; but in that case, for it to be
correct, you must be sure that the population
distribution of Y is normal. In economic data, the
normality assumption is rarely credible. Here are the
distributions of some economic data.


Do you think earnings are normally distributed?
Suppose you have a sample of n = 10 observations from one
of these distributions – would you feel comfortable using the
Student t distribution?
36
37
Comments on Student t distribution, ctd.
Consider the t-statistic testing the hypothesis that two means
(groups s, l) are equal:
t
Ys  Yl
ss2
ns

sl2
nl

Ys  Yl
SE (Ys  Yl )
Even if the population distribution of Y in the two groups is
normal, this statistic doesn’t have a Student t distribution!
There is a statistic testing this hypothesis that has a normal
distribution, the “pooled variance” t-statistic – see SW (Section 3.6)
– however the pooled variance t-statistic is only valid if the
variances of the normal distributions are the same in the two
groups. Would you expect this to be true, say, for men’s v.
women’s wages?
38
The Student-t distribution – summary





The assumption that Y is distributed N(Y, ) is rarely plausible in
practice (income? number of children?)
For n > 30, the t-distribution and N(0,1) are very close (as n
grows large, the tn–1 distribution converges to N(0,1))
The t-distribution is an artifact from days when sample sizes
were small and “computers” were people
For historical reasons, statistical software typically uses the tdistribution to compute p-values – but this is irrelevant when the
sample size is moderate or large.
For these reasons, in this class we will focus on the large-n
approximation given by the CLT
39
Summary on The Student t-distribution
If Y is distributed
, then the t-statistic has the Student tdistribution (tabulated in back of all stats books)
Some comments:
 For n > 30, the t-distribution and N(0,1) are very close.
 The assumption that Y is distributed
is rarely
plausible in practice (income? number of children?)
 The t-distribution is an historical artifact from days when sample
sizes were very small.
 In this class, we won’t use the t distribution - we rely solely on
the large-n approximation given by the CLT.
Confidence Intervals
A 95% confidence interval for
is an interval that contains the
true value of Y in 95% of repeated samples.
Digression: What is random here? the confidence interval— it will
differ from one sample to the next; the population parameter,
,
is not random.
A 95% confidence interval can always be constructed as the set of
values of
not rejected by a hypothesis test with a 5%
significance level.
This confidence interval relies on the large-n results that
approximately normally distributed and
is
Summary:
From the assumptions of:
(1) simple random sampling of a population, that is,
(2)
we developed, for large samples (large n):
 Theory of estimation (sampling distribution of
)
 Theory of hypothesis testing (large-n distribution of t-statistic
and computation of the p-value).
 Theory of confidence intervals (constructed by inverting test
statistic).
Are assumptions (1) & (2) plausible in practice? Yes
Tests for Difference between Two Means
Let
be the mean hourly earning in the population of women
recently graduated from college and let
be population mean for
recently graduated men. Consider the null hypothesis that earnings
for these two populations differ by certain amount d, then
Replace population variances by sample variances, we have the
standard error
and the t-statistic is
If both nm and nw are large, the t-statistic has a standard normal
distribution.
47
Summarize the relationship between variables
Scatterplots:
The population covariance and correlation can be estimated by the
sample covariance and sample correlation.
The sample covariance is
The sample correlation is
It can be shown that under the assumptions that (Xi , Yi) are i.i.d.
and that Xi and Yi have finite fourth moments,
It is easy to see that the second term converges in probability to
zero because
so
by Slutsky’s theorem.
By the definition of covariance, we have
To apply the law of large numbers on the first term, we need to
have
which is satisfied since

The second inequality follows by applying the Cauchy-Schwartz
inequality, and the last inequality follows because of the finite
fourth moments for (Xi , Yi).
The Cauchy-Schwartz inequality is

Applying the law of large numbers, we have

Also,
, therefore
Scatterplots and Sample Correlation