Download Social Science Reasoning Using Statistics

Statistics for the Social Sciences
Psychology 340
Spring 2005
Hypothesis testing
Outline (for week)
Statistics for the
Social Sciences
• Review of:
– Basic probability
– Normal distribution
– Hypothesis testing framework
• Stating hypotheses
• General test statistic and test statistic distributions
• When to reject or fail to reject
Hypothesis testing
Statistics for the
Social Sciences
• Example: Testing the effectiveness of a new memory
treatment for patients with memory problems
– Our pharmaceutical company develops a new drug
treatment that is designed to help patients with impaired
memories.
– Before we market the drug we want to see if it works.
– The drug is designed to work on all memory patients, but
we can’t test them all (the population).
– So we decide to use a sample and conduct the following
experiment.
– Based on the results from the sample we will make
conclusions about the population.
Hypothesis testing
Statistics for the
Social Sciences
• Example: Testing the effectiveness of a new memory
treatment for patients with memory problems
Memory
patients
Memory
treatment
Memory 55
Test
errors
No Memory
treatment
Memory 60
errors
Test
• Is the 5 error difference:
– A “real” difference due to the effect of the treatment
– Or is it just sampling error?
5 error
diff
Testing Hypotheses
Statistics for the
Social Sciences
• Hypothesis testing
– Procedure for deciding whether the outcome of a study
(results for a sample) support a particular theory (which
is thought to apply to a population)
– Core logic of hypothesis testing
• Considers the probability that the result of a study could have
come about if the experimental procedure had no effect
• If this probability is low, scenario of no effect is rejected and
the theory behind the experimental procedure is supported
Basics of Probability
Statistics for the
Social Sciences
Possible successful outcomes
Probability 
All possible outcomes
• Probability
– Expected relative frequency of a particular outcome
• Outcome
– The result of an experiment
Flipping a coin example
Statistics for the
Social Sciences
What are the odds of getting a “heads”?
n = 1 flip
Possible successful outcomes
Probability 
All possible outcomes
One outcome classified as heads
Total of two outcomes
=
1
2
= 0.5
Flipping a coin example
Statistics for the
Social Sciences
n=2
Number of heads
2
1
1
What are the odds of
getting two “heads”?
One 2 “heads”
outcome
Four total
outcomes
= 0.25
0
This situation is known as the binomial
# of outcomes = 2n
Flipping a coin example
Statistics for the
Social Sciences
n=2
Number of heads
2
1
1
0
What are the odds of
getting “at least one
heads”?
Three “at least one
heads” outcome
Four total
outcomes
= 0.75
Flipping a coin example
Statistics for the
Social Sciences
n=3
3=
n
=
2
2
8 total outcomes
HHH
Number of heads
3
HHT
2
HTH
2
HTT
1
THH
2
THT
1
TTH
1
TTT
0
Flipping a coin example
Statistics for the
Social Sciences
Number of heads
3
Distribution of possible outcomes
probability
(n = 3 flips)
.4
.3
.2
.1 .125
.375 .375 .125
0 1 2 3
Number of heads
2
X
f
p
3
1
.125
2
2
1
3
3
.375
.375
1
0
1
.125
1
2
1
0
Flipping a coin example
Statistics for the
Social Sciences
Distribution of possible outcomes
probability
(n = 3 flips)
.4
.3
.2
.1 .125
.375 .375 .125
0 1 2 3
Number of heads
Can make predictions about
likelihood of outcomes based on
this distribution.
What’s the probability of
flipping three heads in a
row?
p = 0.125
Flipping a coin example
Statistics for the
Social Sciences
Distribution of possible outcomes
probability
(n = 3 flips)
.4
.3
.2
.1 .125
.375 .375 .125
0 1 2 3
Number of heads
Can make predictions about
likelihood of outcomes based on
this distribution.
What’s the probability of
flipping at least two heads
in three tosses?
p = 0.375 + 0.125 = 0.50
Flipping a coin example
Statistics for the
Social Sciences
Distribution of possible outcomes
probability
(n = 3 flips)
.4
.3
.2
.1 .125
.375 .375 .125
0 1 2 3
Number of heads
Can make predictions about
likelihood of outcomes based on
this distribution.
What’s the probability of
flipping all heads or all tails
in three tosses?
p = 0.125 + 0.125 = 0.25
Hypothesis testing
Statistics for the
Social Sciences
Distribution of possible outcomes
(of a particular sample size, n)
Can make predictions about
likelihood of outcomes based on
this distribution.
• In hypothesis testing, we
compare our observed samples
with the distribution of possible
samples (transformed into
standardized distributions)
• This distribution of possible
outcomes is often Normally
Distributed
The Normal Distribution
Statistics for the
Social Sciences
• The distribution of days before and after due date (bin
width = 4 days).
-14
0
14
Days before and after due date
The Normal Distribution
Statistics for the
Social Sciences
• Normal distribution
The Normal Distribution
Statistics for the
Social Sciences
• Normal distribution is a commonly found
distribution that is symmetrical and unimodal.
– Not all unimodal, symmetrical curves are Normal, so be careful
with your descriptions
• It is defined by the following equation:
1
2 2

-2
-1
0
1
2
e (X  )
2
/ 2 2
The Unit Normal Table
Statistics for the
Social Sciences
z
.00
.01
-3.4
-3.3
:
:
0
:
:
1.0
:
:
3.3
3.4
0.0003
0.0005
:
:
0.5000
:
:
0.8413
:
:
0.9995
0.9997
0.0003
0.0005
:
:
0.5040
:
:
0.8438
:
:
0.9995
0.9997
• The normal distribution is often
transformed into z-scores.
• Gives the precise proportion of scores (in
z-scores) between the mean (Z score of
0) and any other Z score in a Normal
distribution
– Contains the proportions in the tail to the
left of corresponding z-scores of a
Normal distribution
• This means that the table lists only
positive Z scores
Using the Unit Normal Table
Statistics for the
Social Sciences
z
.00
.01
-3.4
-3.3
:
:
0
:
:
1.0
:
:
3.3
3.4
0.0003
0.0005
:
:
0.5000
:
:
0.8413
:
:
0.9995
0.9997
0.0003
0.0005
:
:
0.5040
:
:
0.8438
:
:
0.9995
0.9997
50%-34%-14% rule
Similar to the 68%-95%-99% rule
34.13%
13.59%
-2
-1
0
1
2.28%
2
At z = +1: 15.87% (13.59% and 2.28%)
of the scores are to the right of the score
100%-15.87% = 84.13% to the left
Using the Unit Normal Table
Statistics for the
Social Sciences
z
.00
.01
-3.4
-3.3
:
:
0
:
:
1.0
:
:
3.3
3.4
0.0003
0.0005
:
:
0.5000
:
:
0.8413
:
:
0.9995
0.9997
0.0003
0.0005
:
:
0.5040
:
:
0.8438
:
:
0.9995
0.9997
• Steps for figuring the
percentage above of below a
particular raw or Z score:
1. Convert raw score to Z score
(if necessary)
2. Draw normal curve, where the
Z score falls on it, shade in the
area for which you are finding
the percentage
3. Make rough estimate of
shaded area’s percentage
(using 50%-34%-14% rule)
Using the Unit Normal Table
Statistics for the
Social Sciences
z
.00
.01
-3.4
-3.3
:
:
0
:
:
1.0
:
:
3.3
3.4
0.0003
0.0005
:
:
0.5000
:
:
0.8413
:
:
0.9995
0.9997
0.0003
0.0005
:
:
0.5040
:
:
0.8438
:
:
0.9995
0.9997
• Steps for figuring the
percentage above of below a
particular raw or Z score:
4. Find exact percentage using
unit normal table
5. If needed, add or subtract 50%
from this percentage
6. Check the exact percentage is
within the range of the estimate
from Step 3
SAT Example problems
Statistics for the
Social Sciences
• The population parameters for the SAT are:
 = 500,  = 100, and it is Normally distributed
Suppose that you got a 630 on the SAT. What percent of
the people who take the SAT get your score or worse?
z
X 


630  500
From the table:
1.3
100
z(1.3) =.0968
So 90.32% got your
score or worse
-2
-1
That’s 9.68%
above this score

1
2
The Normal Distribution
Statistics for the
Social Sciences
• You can go in the other direction too
– Steps for figuring Z scores and raw scores from
percentages:
1. Draw normal curve, shade in approximate area for the
percentage (using the 50%-34%-14% rule)
2. Make rough estimate of the Z score where the shaded area
starts
3. Find the exact Z score using the unit normal table
4. Check that your Z score is similar to the rough estimate from
Step 2
5. If you want to find a raw score, change it from the Z score
Inferential statistics
Statistics for the
Social Sciences
• Hypothesis testing
– Core logic of hypothesis testing
• Considers the probability that the result of a study could have
come about if the experimental procedure had no effect
• If this probability is low, scenario of no effect is rejected and
the theory behind the experimental procedure is supported
– A five step program
•
•
•
•
•
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data
Step 4: Compute your test statistics
Step 5: Make a decision about your null hypothesis
Hypothesis testing
Statistics for the
Social Sciences
• Hypothesis testing: a five step program
– Step 1: State your hypotheses: as a research hypothesis and a
null hypothesis about the populations
• Null hypothesis (H0)
This is the one that you test
• There are no differences between conditions (no effect of treatment)
• Research hypothesis (HA)
• Generally, not all groups are equal
– You aren’t out to prove the alternative hypothesis
• If you reject the null hypothesis, then you’re left with
support for the alternative(s) (NOT proof!)
Testing Hypotheses
Statistics for the
Social Sciences
• Hypothesis testing: a five step program
– Step 1: State your hypotheses
In our memory example experiment:
One -tailed
– Our theory is that the
treatment should improve
memory (fewer errors).
H0: Treatment > No Treatment
HA:Treatment < No Treatment
Testing Hypotheses
Statistics for the
Social Sciences
• Hypothesis testing: a five step program
– Step 1: State your hypotheses
In our memory example experiment:
direction
One -tailed
specified
– Our theory is that the
treatment should improve
memory (fewer errors).
no direction
specified
Two -tailed
– Our theory is that the
treatment has an effect on
memory.
H0: Treatment > No Treatment
H0: Treatment = No Treatment
HA:Treatment < No Treatment
HA:Treatment ≠ No Treatment
One-Tailed and Two-Tailed Hypothesis Tests
Statistics for the
Social Sciences
• Directional
hypotheses
– One-tailed test
• Nondirectional
hypotheses
– Two-tailed test
Testing Hypotheses
Statistics for the
Social Sciences
• Hypothesis testing: a five step program
– Step 1: State your hypotheses
– Step 2: Set your decision criteria
• Your alpha () level will be your guide for when to reject or fail
to reject the null hypothesis.
– Based on the probability of making making an certain type of error
Testing Hypotheses
Statistics for the
Social Sciences
• Hypothesis testing: a five step program
– Step 1: State your hypotheses
– Step 2: Set your decision criteria
– Step 3: Collect your data
Testing Hypotheses
Statistics for the
Social Sciences
• Hypothesis testing: a five step program
–
–
–
–
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data
Step 4: Compute your test statistics
• Descriptive statistics (means, standard deviations, etc.)
• Inferential statistics (z-test, t-tests, ANOVAs, etc.)
Testing Hypotheses
Statistics for the
Social Sciences
• Hypothesis testing: a five step program
–
–
–
–
–
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data
Step 4: Compute your test statistics
Step 5: Make a decision about your null hypothesis
• Based on the outcomes of the statistical tests researchers will either:
– Reject the null hypothesis
– Fail to reject the null hypothesis
• This could be correct conclusion or the incorrect conclusion
Error types
Statistics for the
Social Sciences
• Type I error (): concluding that there is a
difference between groups (“an effect”) when
there really isn’t.
– Sometimes called “significance level” or “alpha level”
– We try to minimize this (keep it low)
• Type II error (): concluding that there isn’t an
effect, when there really is.
– Related to the Statistical Power of a test (1-)
Error types
Statistics for the
Social Sciences
There really
isn’t an effect
Reject
H0
Experimenter’s
conclusions
Fail to
Reject
H0
Real world (‘truth’)
H0 is
correct
H0 is
wrong
There
really is
an effect
Error types
Statistics for the
Social Sciences
Real world (‘truth’)
I conclude that
there is an
effect
H0 is
correct
Reject
H0
Experimenter’s
conclusions
Fail to
Reject
H0
I can’t detect
an effect
H0 is
wrong
Error types
Statistics for the
Social Sciences
Real world (‘truth’)
H0 is
correct
Reject
H0
Experimenter’s
conclusions
Fail to
Reject
H0
H0 is
wrong
Type I
error

Type II
error

Performing your statistical test
Statistics for the
Social Sciences
• What are we doing when we test the hypotheses?
Real world (‘truth’)
H0: is true (no treatment effect)
H0: is false (is a treatment effect)
One
population
Two
populations
XA
the memory treatment sample are the
same as those in the population of
memory patients.
XA
they aren’t the same as those in the
population of memory patients
Performing your statistical test
Statistics for the
Social Sciences
• What are we doing when we test the hypotheses?
– Computing a test statistic: Generic test
Could be difference between a sample and a
population, or between different samples
observed difference
test statistic 
difference expected by chance
Based on standard error or an
estimate of the standard error
“Generic” statistical test
Statistics for the
Social Sciences
• The generic test statistic distribution (think of this as the distribution
of sample means)
– To reject the H0, you want a computed test statistics that is large
– What’s large enough?
• The alpha level gives us the decision criterion
Distribution of the test statistic
-level determines where
these boundaries go
“Generic” statistical test
Statistics for the
Social Sciences
• The generic test statistic distribution (think of this as the distribution
of sample means)
– To reject the H0, you want a computed test statistics that is large
– What’s large enough?
• The alpha level gives us the decision criterion
Distribution of the test statistic
If test statistic is
here Reject H0
If test statistic is here
Fail to reject H0
“Generic” statistical test
Statistics for the
Social Sciences
• The alpha level gives us the decision criterion
Two -tailed
One -tailed
 = 0.05
Reject H0
Reject H0
0.025
split up
into the
two tails
0.025
Fail to reject H0
Reject H0
Fail to reject H0
Fail to reject H0
“Generic” statistical test
Statistics for the
Social Sciences
• The alpha level gives us the decision criterion
Two -tailed
One -tailed
 = 0.05
all of it in
one tail
Reject H0
Reject H0
0.05
Fail to reject H0
Reject H0
Fail to reject H0
Fail to reject H0
“Generic” statistical test
Statistics for the
Social Sciences
• The alpha level gives us the decision criterion
Two -tailed
One -tailed
 = 0.05
Reject H0
all of it in
one tail
Reject H0
0.05
Fail to reject H0
Reject H0
Fail to reject H0
Fail to reject H0
“Generic” statistical test
Statistics for the
Social Sciences
An example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a
memory improvement treatment.
• After the treatment they have an
average score of X = 55 memory errors.
• How do they compare to the general
population of memory patients who have
 of memory errors that is
a distribution
Normal,  = 60,  = 8?
•
Step 1: State your hypotheses
H0: the memory treatment
sample are the same as
those in the population of
memory patients.
Treatment = pop = 60
HA: they aren’t the same as
those in the population of
memory patients
Treatment ≠ pop ≠ 60
“Generic” statistical test
Statistics for the
Social Sciences
An example: One sample z-test
H0: Treatment = pop = 60
HA: Treatment ≠ pop ≠ 60
Memory example experiment:
• We give a n = 16 memory patients a
memory improvement treatment.
• After the treatment they have an
average score of X = 55 memory errors.
• How do they compare to the general
population of memory patients who have
 of memory errors that is
a distribution
Normal,  = 60,  = 8?
•
Step 2: Set your decision
criteria
 = 0.05
One -tailed
“Generic” statistical test
Statistics for the
Social Sciences
An example: One sample z-test
H0: Treatment = pop = 60
HA: Treatment ≠ pop ≠ 60
Memory example experiment:
• We give a n = 16 memory patients a
memory improvement treatment.
• After the treatment they have an
average score of X = 55 memory errors.
• How do they compare to the general
population of memory patients who have
 of memory errors that is
a distribution
Normal,  = 60,  = 8?
One -tailed
•
 = 0.05
Step 3: Collect your data
“Generic” statistical test
Statistics for the
Social Sciences
An example: One sample z-test
H0: Treatment = pop = 60
HA: Treatment ≠ pop ≠ 60
Memory example experiment:
• We give a n = 16 memory patients a
memory improvement treatment.
• After the treatment they have an
average score of X = 55 memory errors.
• How do they compare to the general
population of memory patients who have
 of memory errors that is
a distribution
Normal,  = 60,  = 8?

 = 0.05
One -tailed
•
Step 4: Compute your test
statistics
zX 
X  X
X
= -2.5


55  60
 8



 16 
“Generic” statistical test
Statistics for the
Social Sciences
An example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a
memory improvement treatment.
H0: Treatment = pop = 60
HA: Treatment ≠ pop ≠ 60
 = 0.05
One -tailed
zX  2.5
• Step 5: Make a decision
• After the treatment they have an
about your null hypothesis
average score of X = 55 memory errors.
• How do they compare to the general
population of memory patients who have 5%
 of memory errors that is
a distribution
Normal,  = 60,  = 8?
-2
-1

Reject H0
1
2
“Generic” statistical test
Statistics for the
Social Sciences
An example: One sample z-test
H0: Treatment = pop = 60
HA: Treatment ≠ pop ≠ 60
Memory example experiment:
• We give a n = 16 memory patients a
memory improvement treatment.
• After the treatment they have an
average score of X = 55 memory errors.
• How do they compare to the general
population of memory patients who have
 of memory errors that is
a distribution
Normal,  = 60,  = 8?
One -tailed
 = 0.05
zX  2.5
•
Step 5: Make a decision
about your null hypothesis
- Reject H0
- Support for our HA, the
evidence suggests that the
treatment decreases the
number of memory errors