Download Introduction to Research

Hypothesis Testing
Quantitative Methods in HPELS
440:210
Agenda
Introduction
 Hypothesis Testing  General Process
 Errors in Hypothesis Testing
 One vs. Two Tailed Tests
 Effect Size and Power
 Instat
 Example

Introduction  Hypothesis Testing

Recall:
 Inferential
Statistics: Calculation of sample statistic to
make predictions about population parameter
 Two potential problems with samples:
 Sampling error
 Variation between samples
 Infinite # of samples  predictable pattern 
sampling distribution
 Normal
 µ = µM
 M = /√n
Introduction  Hypothesis Testing



Common statistical procedure
Allows for comparison of means
General process:
State hypotheses
2. Set criteria for decision making
3. Collect data  calculate statistic
4. Make decision
1.
Introduction  Hypothesis Testing

Remainder of presentation will use
following concepts to perform a
hypothesis test:

Z-score
 Probability
 Sampling distribution
Agenda
Introduction
 Hypothesis Testing  General Process
 Errors in Hypothesis Testing
 One vs. Two Tailed Tests
 Effect Size and Power
 Instat
 Example

General Process of HT




Step 1: State hypotheses
Step 2: Set criteria for decision making
Step 3: Collect data and calculate
statistic
Step 4: Make decision
Step 1: State Hypotheses

Two types of hypotheses:
Null Hypothesis (H0):
2. Alternative Hypothesis (H1):
1.




Directional
Non-directional
Only one can be true
Example 8.1, p 223
Assume the following about
2-year olds:
µ = 26
=4
M = /√n = 1
n = 16
Researchers want to know if
extra handling/stimulation of
babies will result in increased
body weight once the baby
reaches 2 years of age
Null Hypothesis:
Alternative Hypothesis:
H0: Sample mean = 26
H1: Sample mean ≠ 26
Reality: Only ONE sample will
be chose
Assume that this
distribution is the
“TRUE” representation
of the population
µM
Recall: If an INFINITE number of
samples are taken, the
SAMPLING DISTRIBUTION will
be NORMAL with µ = µM and will
be identical to the population
distribution
What is the probability of choosing a
sample with a mean (M) that is 1, 2, or
3 SD above or below the mean (µM)?
µM
µM
p(M > µM + 1 ) = 15.87%
p(M > µM + 2 ) = 2.28%
µM
p(M > µM + 3 ) = 0.13%
Inferential statistics is based on
the assumption that our sample
is PROBABLY representative of
the population
µM
It is much more PROBABLE that our sample mean (M) will fall closer to
the mean of the means (µM) as well as the population mean (µ)
µM
Our sample could be here, or here, or here, but we assume that it is here!
H0: Sample mean = 26
If true (no effect):
1.) It is PROBABLE that the sample
mean (M) will fall in the middle
2.) It is IMPROBABLE that the
sample mean (M) will fall in the
extreme edges
H1: Sample mean ≠ 26
If true (effect):
1.) It is PROBABLE that the sample
mean (M) will fall in the extreme
edges
2.) It is IMPROBABLE that the
sample mean (M) will fall in the
middle
µ = 26
M = 30
Assume that M = 30 lbs
Accept or reject?
(n = 16)
H0: Sample mean = 26
What criteria do you use to make
the decision?
Step 2: Set Criteria for Decision

A sampling distribution can be divided into
two sections:
 Middle:
Sample means likely to be obtained if
H0 is accepted
 Ends: Sample means not likely to be obtained
if H0 is rejected

Alpha (a) is the criteria that defines the
boundaries of each section
Step 2: Set Criteria for Decision

Alpha:
 AKA 

level of significance
Ask this question:
 What
degree of certainty do I need to reject
the H0?
90% certain: a = 0.10
 95% certain: a = 0.05
 99% certain: a = 0.01

Step 2: Set Criteria for Decision

As level of certainty increases:
a
decreases
 Middle section gets larger
 Critical regions (edges) get smaller

Bottom line: A larger test statistic is needed
to reject the H0
Step 2: Set Criteria for Decision
Directional vs. nondirectional alternative
hypotheses
 Directional:

 H1:

Non-directional:
 H1:

M > or < X
M≠X
Which is more difficult
to reject H0?
Step 2: Set Criteria for Decision


Z-scores represent boundaries that divide
sampling distribution
Non-directional:
a
a
= 0.10 defined by Z = 1.64
= 0.05 defined by Z = 1.96
 a = 0.01 defined by Z = 2.57

Directional:
a
a
= 0.10 defined by Z = 1.28
= 0.05 defined by Z = 1.64
 a = 0.01 defined by Z = 2.33
Critical Z-Scores  Non-Directional Hypotheses
Z=1.64
Z=1.64
Z=1.96
Z=1.96
Z=2.58
Z=2.58
90%
95%
99%
Critical Z-Scores  Directional Hypotheses
Z=1.28
Z=1.64
Z=2.34
90%
95%
99%
Step 2: Set Criteria for Decision

Where should you set alpha?
research  0.10
 Most common  0.05
 0.01 or lower?
 Exploratory
Step 3: Collect Data/Calculate Statistic

Z = M - µ / M where:


M = sample mean
µ = value from the null hypothesis


H0: sample = X
M = /√n

Note: Population  must be known
otherwise the Z-score is an inappropriate
statistic!!!!!
Step 3: Collect Data/Calculate Statistic

Example 8.1  Continued
M = 30
Assume the following about
2-year olds:
µ = 26
Researchers want to know if
extra handling/stimulation of
babies will result in increased
body weight once the baby
reaches 2 years of age
=4
M = /√n = 1
Z = M - µ / M
n = 16
Z = 30 – 26 / 1
Z = 4 / 1 = 4.0
Step 4: Make Decision

Process:
1. Draw a sketch with critical Z-score


2.
3.
Assume non-directional
Alpha = 0.05
Plot Z-score statistic on sketch
Make decision
Step 1: Draw sketch
Critical Z-score
Critical Z-score
Z = 1.96
Z = 1.96
µ = 26
M = 30
Step 2: Plot Z-score
Z = 4.0
Step 3: Make Decision: Z = 4.0  falls inside the critical region
If H0 is false, it is PROBABLE that the Z-score will fall in the critical region
ACCEPT OR REJECT?
Agenda
Introduction
 Hypothesis Testing  General Process
 Errors in Hypothesis Testing
 One vs. Two Tailed Tests
 Effect Size and Power
 Instat
 Example

Errors in Hypothesis Testing

Recall  Problems with samples:
 Sampling
error
 Variability of samples
Inferential statistics use sample statistics
to predict population parameters
 There is ALWAYS chance for error

Errors in Hypothesis Testing

1.
2.
There is potential for two kinds of error:
Type I error
Type II error
Type I Error
Rejection of a true H0
 Recall  alpha = certainty of rejecting H0

 Example:
Alpha = 0.05
 95% certain of correctly rejecting the H0
 Therefore 5% certain of incorrectly rejecting the H0

Alpha maybe thought of as the “probability
of making a Type I error
 Consequences:

 False
report
 Waste of time/resources
Type II Error
Acceptance of a false H0
 Consequences:

 Not
as serious as Type I error
 Researcher may repeat experiment if type II
error is suspected
Agenda
Introduction
 Hypothesis Testing  General Process
 Errors in Hypothesis Testing
 One vs. Two Tailed Tests
 Effect Size and Power
 Instat
 Example

One vs. Two-Tailed Tests

One-Tailed (Directional) Tests:
 Specify
an increase or decrease in the
alternative hypothesis
 Advantage: More powerful
 Disadvantage: Prior knowledge required
One vs. Two-Tailed Tests

Two-Tailed (Non-Directional) Tests:
 Do
not specify an increase or decrease in the
alternative hypothesis
 Advantage: No prior knowledge required
 Disadvantage: Less powerful
Agenda
Introduction
 Hypothesis Testing  General Process
 Errors in Hypothesis Testing
 One vs. Two Tailed Tests
 Effect Size and Power
 Instat
 Example

Recall  Step 4: Make a Decision
Statistical Software  p-value
 The p-value is the probability of a type I
error
 Recall alpha (a)

Recall  Step 4: Make a Decision
 If
the p-value > a  accept the H0
 Probability
of type I error is too high
 Researcher is not “comfortable” stating that
differences are real and not due to chance
 If
the p-value < a  reject the H0
 Probability
of type I error is low enough
 Researcher is “comfortable” stating that
differences are real and not due to chance
Statistical vs. Practical Significance

1.
2.
Distinction:
Statistical significance: There is an
acceptably low chance of a type I error
Practical significance: The actual
difference between the means are not
trivial in their practical applications
Practically Significant?


Knowledge and experience
Examine effect size
 The

magnitude of the effect
Examples of measures of effect size:
Eta-squared (h2)
 Cohen’s d
 R2


Interpretation of effect size:
– 0.2 = small effect
 0.21 – 0.8 = moderate effect
 > 0.8 = large effect
 0.0

Examine power of test
Statistical Power
Statistical power: The probability that you
will correctly reject a false H0
 Power = 1 – b where

b

= probability of type II error
Example: Statistical power = 0.80
therefore:
 80%
chance of correctly rejecting a false H0
 20% of accepting a false H0 (type II error)
Researcher
Conclusion
Accept H0
Reality No real difference
About exists
Test
Real difference
exists
Reject H0
Correct
Type I
Conclusion error
Type II
error
Correct
Conclusion
Statistical Power

1.
What influences power?
Sample size: As n increases, power increases
- Under researcher’s control
2.
Alpha: As a increases, b decreases therefore
power increases
- Under researcher’s control (to an extent)
3.
Effect size: As ES increases, power increases
- Not under researcher’s control
Statistical Power
How much power is desirable?
 General rule: Set b as 4*a
 Example:

a
= 0.05, therfore b = 4*0.05 = 0.20
 Power = 1 – b = 1 – 0.20 = 0.80
Statistical Power

What if you don’t have enough power?
 More

subjects
What if you can’t recruit more subjects and you
want to prevent not having enough power?
 Estimate
optimal sample size a priori
 See statistician with following information:



Alpha
Desired power
Knowledge about effect size  what constitutes a small,
moderate or large effect size relative to your dependent
variable
Statistical Power

1.
2.
Examples:
Novice athlete improves vertical jump
height by 2 inches after 8 weeks of
training
Elite athlete improves vertical jump
height by 2 inches after 8 weeks of
training
Agenda
Introduction
 Hypothesis Testing  General Process
 Errors in Hypothesis Testing
 One vs. Two Tailed Tests
 Instat
 Example

Instat

Type data from sample into a column.

Label column appropriately.




Choose “Statistics”

Choose “Simple Models”


Choose “Manage”
Choose “Column Properties”
Choose “Name”
Choose “Normal, One Sample”
Layout Menu:

Choose “Single Data Column”
Instat

Data Column Menu:


Choose variable of interest.
Parameter Menu
Choose “Mean, Known Variance (z-interval)”
 Enter known SD or variance value.


Confidence Level:

90% = alpha 0.10
 95% = alpha 0.05
Instat

Check “Significance Test” box:
Check “Two-Sided” if using non-directional
hypothesis.
 Enter value from null hypothesis.




What population value are you basing
your sample comparison?
Click OK.
Interpret the p-value!!!
Agenda
Introduction
 Hypothesis Testing  General Process
 Errors in Hypothesis Testing
 One vs. Two Tailed Tests
 Instat
 Example

Example (p 246)

Researchers want to investigate the effect of
prenatal alcohol on birth weight in rats
 Independent variable?
 Dependent variable?

Assume:
µ
= 18 g
=4
 n = 16
 M = /√n = 4/4 = 1
 M = 15 g
Step 1: State hypotheses (directional or non-directional)
H0: µalcohol = 18 g
H1: µalcohol ≠ 18 g
Step 2: Set criteria for decision making
Alpha (a) = 0.05
Step 3: Sample data and calculate statistic
Z = M - µ / M
Z = 15 – 18 / 1 = -3.0
Step 4: Make decision
Does Z-score fall inside or
outside of the critical region?
Accept or reject?
Statistical Software:
p-value = 0.02  Accept or reject?
p-value = 0.15  Accept or reject?
Homework

Problems: 3, 5, 6, 7, 8, 11, 21