Download BHS 307 – Statistics for the Behavioral Sciences

PSY 307 – Statistics for the
Behavioral Sciences
Chapter 10-11 – Introduction to
Hypothesis Testing
Why Hypothesis Tests?



The first step in any study is to test
against chance.
We cannot draw any conclusions
about our results without making
sure our results are not accidental.
We never know for sure what the
true situation is, but we try to
minimize possibility of error.
Kinds of Hypothesis Testing






Testing against 0 (zero)
Testing against chance
Testing the null hypothesis versus
an alternative hypothesis
Testing two hypotheses that predict
different or incompatible outcomes
Modeling
Exploratory data analysis
Hypotheses are about Underlying
Populations
Underlying Population
Sample
What can we know about the
population by measuring the
sample?
Guessing about the Population
Alternative
Population(s)
Null Population
?
?
Which population is
the true underlying
population for our
sample?
We cannot know for
sure.
Sample
We assume the null
population is true,
then see how likely
the sample would be
to occur under those
circumstances.
Common vs Rare Outcomes

We always start with the
assumption that the null population
is true. In that case we ask:



How likely would our current result be
if the null population were the true
underlying population?
A result
curve is
A result
curve is
in the center of the normal
highly likely (common).
in the tails of the normal
much less likely (rare).
Common vs Rare
All possible
sample results
RARE
RARE
COMMON
-1.96
1.96
We use +/- 1.96 (2 SD) as the cutoff between common and rare.
Hypothesis Testing



A hypothesis is a prediction about
the results of a study.
Null hypothesis (H0) – a prediction
that the null population is the true
underlying population.
The null is a hypothetical sampling
distribution – what would exist if
nothing special were happening.

Results are compared against it.
Manipulations

A comparison is being made:




A sample against a known population.
Two groups are compared with each
other (e.g., males vs females).
Two groups are compared after a
manipulation (e.g., treatment vs
control.
The expected differences are stated
as the alternative hypothesis (H1).
Null Hypothesis
RARE
RARE
COMMON
-1.96
Null hypothesis is true
(always assumes no difference)
1.96
Alternative Hypothesis
RARE
RARE
COMMON
-1.96
1.96
Null hypothesis is NOT true
(a difference most likely exists)
Boundaries for Outcomes



Common outcome – small
difference from the hypothesized
population mean m.
Rare outcome – too large a
difference from the hypothesized
mean to be probable.
A set of boundaries (critical values)
can be found to decide whether an
outcome is rare or common.
Different Critical Values can be
Used
p < .05
RARE
RARE
COMMON
-1.96
1.96
Null hypothesis is NOT true
(a difference most likely exists)
Critical Values can be Stricter
p < .01
RARE
RARE
COMMON
-2.58
2.58
Null hypothesis is NOT true
(a difference most likely exists)
Or More Lenient
p < .10
RARE
RARE
COMMON
-1.65
1.65
Null hypothesis is NOT true
(a difference most likely exists)
Decision Rule




The decision rule specifies precisely
when the null hypothesis can be
rejected (assumed to be untrue).
Critical score – the boundary for a
rare outcome.
Most studies in psychology use a
critical value of p < .05 unless they
have a good reason not to.
Level of significance (a) – rarity.
Decision and Interpretation

Reject the null hypothesis when the
observed mean results in a test
statistic beyond the critical value.


If your level of significance is p<.05
then a z-score above 1.96 or below
-1.96 is rare.
With a rare result, conclude that the
null hypothesis is untrue (reject it).

With a common result, retain it.
One-Tailed & Two-Tailed Tests

Two-tailed tests (non-directional)
divide the probability of error (a)
between the two tails.


Expressed using equality signs (=, /=).
One-tailed tests (directional) place
all of the probability of error in a
single tail in the direction of
interest.

Expressed using inequality signs (<,
>=)
One-tailed vs Two-tailed Tests

A two-tailed test only predicts a
difference:



H0: m = 25, m1 = m2
H1: m ≠= 25, m1 ≠ m2
A one-tailed test predicts a
difference in a specific direction:


H0: m ≤ 25, m1 ≤ m2
H1: m > 25, m1 > m2
One-Tailed Test (Upper Tail
Critical)
p < .05
RARE
COMMON
1.65
Null hypothesis is NOT true
(a difference most likely exists)
One-Tailed Test (Lower Tail
Critical)
p < .05
RARE
COMMON
-1.65
Null hypothesis is NOT true
(a difference most likely exists)
Two-tailed Test
a = .05
.025
.025
COMMON
-1.96
1.96
One-tailed Test
a = .05
.05
COMMON
1.65
One-tailed Test (other direction)
a = .05
.05
COMMON
-1.65
Two-tailed Test
a = .05
.025
.025
COMMON
-1.96
1.96
Hypothesis Test Outcomes

A hypothesis test has four possible
outcomes:




H0 is true and H0 is retained – a correct
decision.
H0 is true but H0 is rejected – a Type I
error (false alarm).
H1 is true and H0 is rejected (H1 is
retained) – a correct decision.
H1 is true but H1 is rejected and H0 is
retained – a Type II error (miss).
Possible Decisions
Reality –
Null is True
Reality –
Null is False
We decide that the
Null Hypothesis is
True
Correctly retain
null
Retain null but
make a
Type II error
(miss)
We decide that the
Null Hypothesis is
False
Reject null and
make a
Type I error (false
alarm)
Correctly reject
null
Strong and Weak Decisions

Retaining the null hypothesis (H0) is
a weak decision because it is
ambiguous and uninformative.


H0 could be true or there may be a
difference but the study couldn’t
demonstrate it, due to poor methods.
Rejecting the null hypothesis is a
strong decision because it implies
that H0 is probably false.
Decisions are Usually Correct




We never actually know what is
true, H0 or H1.
Our test procedure produces a
result that is usually correct when
H0 is either true or seriously false.
Type I error – rejecting a true null
hypothesis.
Type II error – retaining a false null
hypothesis.
Probabilities of Error

Probability of a Type I error is a.



Most of the time a = .05
A correct decision exists .95 of the time
(1 - .05 = .95).
Probability of a Type II error is b.


When there is a large effect, b is very
small.
When there is a small effect, b can be
large, making a Type II error likely.
Power Curves

Power equals the probability of
detecting an effect.


Power = 1 – b or
(1 – probability of Type II error)
Power curves show how power
varies with sample size.


They specify what sample size will
show an effect if it is present.
A way around arbitrary sample sizes.
Deciding Which to Use

To identify the type of test in an
exam question or report:



If the null hypothesis is expressed
using inequalities, it is directional.
Look for directional words in text.
To choose a test-type in research:

Use a non-directional (two-tailed) test
unless you have a strong reason to do
otherwise.