Download Chapter_22

Chapter 22
What Is a Test of Significance?
Chapter 22
1
Thought Question 1
The defendant in a court case is either guilty
or innocent. Which of these is assumed to be
true when the case begins? The jury looks at
the evidence presented and makes a decision
about which of these two options appears
more plausible. Depending on this decision,
what are the two types of errors that could be
made by the jury? Which is more serious?
Chapter 22
2
Thought Question 2
Suppose 60% (0.60) of the population are in favor
of new tax legislation. A random sample of 265
people results in 175, or 0.66, who are in favor.
From the Rule for Sample Proportions, we know
the potential sample proportions in this situation
follow an approximately normal distribution, with a
mean of 0.60 and a standard deviation of 0.03.
Find the standardized score for the observed value
of 0.66; then find the probability of observing a
standardized score at least that large or larger.
Chapter 22
3
Thought Question 2: Bell-Shaped Curve
of Sample Proportions (n=265)
z  0.66  0.60  2.0
0.03
mean = 0.60
S.D. = 0.03
2.27%
0.51
0.54
0.57
0.60
Chapter 22
0.63
0.66
0.69
4
Thought Question 3
Suppose that in the previous question we do not know
for sure that the proportion of the population who favor
the new tax legislation is 60%. Instead, this is just the
claim of a politician. From the data collected, we have
discovered that if the claim is true, then the sample
proportion observed falls at the 97.73 percentile (about
the 98th percentile) of possible sample proportions for
that sample size.
Should we believe the claim and conclude that we just
observed strange data, or should we reject the claim?
What if the result fell at the 85th percentile?
At the 99.99th percentile?
Chapter 22
5
Thought Question 3: Bell-Shaped Curve
of Sample Proportions (n=265)
99.99th
98th
85th
0.51
0.54
0.57
0.60
0.63
Chapter 22
0.66
0.69
6
Case Study
Parental Discipline
Brown, C. S., (1994) “To spank or not to spank.” USA
Weekend, April 22-24, pp. 4-7.
What are parents’ attitudes and
practices on discipline?
Chapter 22
7
Case Study: Survey
Parental Discipline
 Nationwide
random telephone survey of
1,250 adults.
– 474 respondents had children under 18
living at home
– results on behavior based on the smaller
sample
 reported
margin of error
– 3% for the full sample
– 5% for the smaller sample
Chapter 22
8
Case Study: Results
Parental Discipline
“The 1994 survey marks the first time a majority
of parents reported not having physically
disciplined their children in the previous year.
Figures over the past six years show a steady
decline in physical punishment, from a peak of
64 percent in 1988”
– The 1994 sample proportion who did not spank
or hit was 51% !
– Is this evidence that a majority of the population
did not spank or hit?
Chapter 22
9
The Five Steps of Hypothesis
Testing
 Determining
the Two Hypotheses
 Computing the Sampling Distribution
 Collecting and Summarizing the Data
(calculating the observed test statistic)

Determining How Unlikely the Test Statistic is
if the Null Hypothesis is True
(calculating the P-value)
 Making a Decision/Conclusion
(based on the P-value, is the result statistically significant?)
Chapter 22
10
The Null Hypothesis: H0
population parameter equals some value
 status quo
 no relationship
 no change
 no difference in two groups
 etc.


When performing a hypothesis test, we assume
that the null hypothesis is true until we have
sufficient evidence against it
Chapter 22
11
The Alternative Hypothesis: Ha
 population
parameter differs from some
value
 not status quo
 relationship exists
 a change occurred
 two groups are different
 etc.
Chapter 22
12
The Hypotheses for Proportions
 Null:
H0: p = p0
 One
sided alternatives
Ha: p > p0
Ha: p < p0
 Two sided alternative
Ha: p  p0
Chapter 22
13
Case Study: The Hypotheses
 Null:
The proportion of parents who
physically disciplined their children in the
previous year is the same as the proportion
[p] of parents who did not physically
discipline their children. [H0: p = .5]
 Alt: A majority of parents did not physically
discipline their children in the previous year.
[Ha: p > .5]
Chapter 22
14
Sampling Distribution for
Proportions
If numerous simple random samples of size n
are taken, the sample proportions ( p
ˆ ) from the
various samples will have an approximately
normal distribution with mean equal to p (the
population proportion) and standard deviation
equal to
Since we assume the
p(1  p)
n
null hypothesis is true,
we replace p with p0 to
complete the test.
Chapter 22
15
Test Statistic for Proportions
To determine if the observed proportion p̂ is
unlikely to have occurred under the assumption
that H0 is true, we must first convert the
observed value to a standardized score:
z
pˆ  p0
p0 (1  p0 )
n
Chapter 22
16
Case Study: Test Statistic
Based on the sample:
n=474 (large, so proportions follow normal distribution)
no physical discipline: 51%
–
.50(1  .50)
 0.023
– standard error of p̂ :
474
(where .50 is p0 from the null hypothesis)
standardized score (test statistic)
z = (0.51 - 0.50) / 0.023 = 0.43
Chapter 22
17
P-value

The P-value is the probability of observing data
this extreme or more so in a sample of this size,
assuming that the null hypothesis is true.

A small P-value indicates that the observed data
(or relationship) is unlikely to have occurred if
the null hypothesis were actually true
– The P-value tends to be small when there is evidence
in the data against the null hypothesis
– The P-value is NOT the probability that the null
hypothesis is true
Chapter 22
18
P-value for Testing Proportions
 H a:
p > p0
 When
the alternative hypothesis includes
a greater than “>” symbol, the P-value is
the probability of getting a value as large
or larger than the observed test statistic
(z) value.
 look up the percentile for the value of z in
the standard normal table (Table B)
 the P-value is 1 minus this probability
Chapter 22
19
P-value for Testing Proportions
 H a:
p < p0
 When
the alternative hypothesis includes
a less than “<” symbol, the P-value is the
probability of getting a value as small or
smaller than the observed test statistic
(z) value.
 look up the percentile for the value of z in
the standard normal table (Table B)
 the P-value is this probability
Chapter 22
20
P-value for Testing Proportions
 H a:
p  p0
 When
the alternative hypothesis includes
a not equal to “” symbol, the P-value is
found as follows:
 make the value of the observed test statistic (z)
positive (absolute value)
 look up the percentile for this positive value of z
in the standard normal table (Table B)
 find 1 minus this probability
 double the answer to get the P-value
Chapter 22
21
Alternative Method for P-value
Caution: Use this method only when Ha has a “>” sign and Z is positive, or
when Ha has a “<“ sign and Z is negative, or when Ha has a “” sign.
1.
Make the value of the observed test statistic
(z) negative
2.
Look up the percentile for this negative value
of z in the standard normal table (Table B)
 if the alternative hypothesis includes a greater
than “>” or less than “<“ symbol, the P-value is
this probability in step 2
 if the alternative hypothesis includes a not
equal to “” symbol, double this probability in
step 2 to get the P-value
Chapter 22
22
Case Study: P-value
Ha: p > .50
p̂ : 0.431
z:
-3
P-value
= 0.3446
0.454
0.477
-2
-1
0.51
0.500
0.523
0
1
0.546
2
0.569
3
z=0.43
From Table B, z=0.4 is the 65.54th percentile.
Chapter 22
23
* Decision *
 If
we think the P-value is too low to believe
the observed test statistic is obtained by
chance only, then we would reject chance
(reject the null hypothesis) and conclude that
a statistically significant relationship exists
(accept the alternative hypothesis).
 Otherwise,
we fail to reject chance and
do not reject the null hypothesis of no
relationship (result not statistically significant).
Chapter 22
24
Typical Cut-off for the P-value
 Commonly,
P-values less than 0.05 are
considered to be small enough to reject
chance (reject the null hypothesis).
 Some
researchers use 0.10 or 0.01 as
the cut-off instead of 0.05.
 This
“cut-off” value is typically referred
to as the significance level  of the test
Chapter 22
25
Case Study: Decision
Since the P-value (.3446) is not small, we cannot
reject chance as the reason for the difference
between the observed proportion (0.51) and the
(null) hypothesized proportion (0.50).
 We do not find the result to be statistically
significant.
 We fail to reject the null hypothesis. It is plausible
that there was not a majority (over 50%) of parents
who refrained from using physical discipline.

Chapter 22
26
Decision Errors: Type I

If we decide there is a relationship in the population
(reject null hypothesis)
– This is an incorrect decision only if the null hypothesis is
true.
– The probability of this incorrect decision is equal to the
cut-off () for the P-value.

If the null hypothesis is true and the cut-off is 0.05
– There really is no relationship and the extremity of the
test statistic is due to chance.
– About 5% of all samples from this population will lead us
to wrongly reject chance.
Chapter 22
27
Decision Errors: Type II
 If
we decide not to reject chance and
thus allow for the plausibility of the null
hypothesis
– This is an incorrect decision only if the
alternative hypothesis is true.
– The probability of this incorrect decision
depends on
 the magnitude of the true relationship,
 the sample size,
 the cut-off for the P-value.
Chapter 22
28
Power of a Test
This is the probability that the sample we
collect will lead us to reject the null hypothesis
when the alternative hypothesis is true.
 The power is larger for larger departures of
the alternative hypothesis from the null
hypothesis (magnitude of difference)
 The power may be increased by increasing
the sample size.

Chapter 22
29
Case Study: Decision Error?
 Decision:
fail to reject H0
 If in the population there truly was a
majority of parents who did not physically
discipline their children, then we have
committed a Type II error.
 Could we have committed a Type I error
with the decision that we made?
[No! Why?]
Chapter 22
30
Key Concepts (1st half of Ch. 22)
 Decisions
are often made on the basis
of incomplete information.
 Five Steps of Hypothesis Testing
 P-values and Statistical Significance
 Decision Errors
 Power of a Test
Chapter 22
31
Inference for Population Means
Hypothesis Testing
The remainder of this chapter discusses
the situation when interest is completing
hypothesis tests about population means
rather than population proportions
Chapter 22
32
Thought Question 4
A study showed that the difference in sample means for
the heights of tomato plants when using a nutrient rich
potting soil versus using ordinary top soil was 6.82 inches.
The corresponding standard error was 3.10 inches.
Suppose the means are actually equal, so that the mean
difference in heights for the populations is actually zero.
What is the standardized score (z) corresponding to the
observed difference of 6.82 inches?
How often would you expect to see a standardized score
that large or larger?
Chapter 22
33
Thought Question 4: Answer
standardized score:
[(sample mean diff.)  (population mean diff.)]
divided by
[standard error of the mean difference]
z = (6.82  0) / 3.10 = 2.2
This is the 98.61 percentile for a standard
normal curve; so the probability of seeing a
z-value this large or larger is 1.39% (.0139)
Chapter 22
34
Thought Question 5
One of the conclusions made by researchers from a
study comparing the amount of bacteria in carpeted
and uncarpeted rooms was, “The average difference
[in mean bacteria colonies per cubic foot] was 3.48
colonies (95% CI: -2.72, 9.68; P-value = 0.29).”
What are the null and alternative hypotheses being
tested here?
Is there a statistically significant difference between the
means of the two groups?
Chapter 22
35
Thought Question 5: Answer
 Null:
The mean number of bacteria for carpeted
rooms is equal to the mean number of bacteria for
uncarpeted rooms.
 Alt: The mean number of bacteria for carpeted
rooms is different from the mean number of
bacteria for uncarpeted rooms.

P-value is large (>.05), so there is not a significant
difference (fail to reject the Null hypothesis)
[Also, the confidence interval for the difference contains 0.]
Chapter 22
36
Case Study (cont. from Ch. 21)
Exercise and Pulse Rates
Hypothetical
Is the mean resting pulse rate of adult
subjects who regularly exercise different
from the mean resting pulse rate of
those who do not regularly exercise?
Use Hypothesis Testing for means
Chapter 22
37
Case Study: Results
Exercise and Pulse Rates
Nonexer.
Exercisers
n mean std. dev.
31
75
9.0
29
66
8.6
std. err.
1.6
1.6
difference in means = 75  66 = 9
 Standard error of the difference = 2.26 (given)
 Observed
Chapter 22
38
The Five Steps of Hypothesis
Testing
 Determining
the Two Hypotheses
 Computing the Sampling Distribution
 Collecting and Summarizing the Data
(calculating the observed test statistic)

Determining How Unlikely the Test Statistic is
if the Null Hypothesis is True
(calculating the P-value)
 Making a Decision/Conclusion
(based on the P-value, is the result statistically significant?)
Chapter 22
39
The Hypotheses for a
Single Mean
 Null:
H 0: m = m 0
 One
sided alternatives
H a: m > m 0
H a: m < m 0
 Two sided alternative
H a: m  m 0
Chapter 22
40
The Hypotheses for a
Difference in Two Means
 Null:
H0: mdiff = mdiff,0
(usually = 0)
 One
sided alternatives
Ha: mdiff > mdiff,0
Ha: mdiff < mdiff,0
 Two sided alternative
Ha: mdiff  mdiff,0
Chapter 22
41
Case Study: The Hypotheses
 Null:
The mean resting pulse rate of adult
subjects who regularly exercise is the same
as the mean resting pulse rate of those who
do not regularly exercise? [H0: mdiff = 0]
 Alt: The mean resting pulse rate of adult
subjects who regularly exercise is different
from the mean resting pulse rate of those
who do not regularly exercise? [Ha: mdiff  0]
Chapter 22
42
Sampling Distribution for Means
If numerous simple random samples of size n are
taken, the sample means (X ) from the various
samples will have an approximately normal
distribution with mean equal to m (the population
mean) and standard deviation equal to 
Since we assume the
null hypothesis is true,
we replace m with m0 to
complete the test.
n
Also, we again replace
the unknown value of 
with the sample value s
Chapter 22
43
Test Statistics for Means
x  m0
x  m0

z
SEM
s n
x1  x2   mdiff,0
z
SE of Difference
where μ diff,0 is usually 0.
Chapter 22
44
Case Study: Test Statistic
Based on the samples:
 large, random samples, so sample
means follow normal distribution
 observed difference in means = 9
 standard error of the difference = 2.26
 standardized score (test statistic)
z = (9  0) / 2.26 = 3.98
Chapter 22
45
P-value for Testing Means
 H a:
m > m0
 When
the alternative hypothesis includes
a greater than “>” symbol, the P-value is
the probability of getting a value as large
or larger than the observed test statistic
(z) value.
 look up the percentile for the value of z in
the standard normal table (Table B)
 the P-value is 1 minus this probability
Chapter 22
46
P-value for Testing Means
 H a:
m < m0
 When
the alternative hypothesis includes
a less than “<” symbol, the P-value is the
probability of getting a value as small or
smaller than the observed test statistic
(z) value.
 look up the percentile for the value of z in
the standard normal table (Table B)
 the P-value is this probability
Chapter 22
47
P-value for Testing Means
 H a:
m  m0
 When
the alternative hypothesis includes
a not equal to “” symbol, the P-value is
found as follows:
 make the value of the observed test statistic (z)
positive (absolute value)
 look up the percentile for this positive value of z
in the standard normal table (Table B)
 find 1 minus this probability
 double the answer to get the P-value
Chapter 22
48
Case Study: P-value
P-value < 20.0003 = 0.0006
Ha: mdiff  0
< 0.0003
diff: -9.04
-4
z:
< 0.0003
-6.78
-4.52
-3
-2
-2.26
-1
0.00
2.26
4.52
6.78
9.0
9.04
0
1
2
3
4
From Table B, 3.98 is beyond the 99.97th percentile.
Chapter 22
z=3.98
49
Case Study: Decision
 Since
the P-value (<.0006) is small, we reject
chance as the reason for the difference between
the observed means (9).
 We

find the result to be statistically significant.
We reject the null hypothesis. The data provide
evidence that the two population means (resting
pulse rates) are not the same.
Chapter 22
50
Case Study: Decision Error?
 If
in the population the mean resting pulse
rate does not differ with exercise, then we
have committed a Type I error.
 Could
we have committed a Type II error
with the decision that we made?
[No! Why?]
Chapter 22
51
Key Concepts (2nd half of Ch. 22)
 Hypothesis
Testing for a Single Mean
 Hypothesis
Testing for a Difference in
Two Means
Chapter 22
52