Download November 12

Statistical Inference
The larger the sample size (n)
the more confident you can be
that your sample mean is a good
representation of the population
mean. In other words, the "n"
justifies the means.
~ Ancient Kung Foole Proverb
Hypothesis Testing
One- and Two Tailed Probabilities
• One-tailed
– The probability that an observation will occur
at one end of the sampling distribution.
• Two-tailed
– The probability that an observation will occur
at either extreme of the sampling distribution.
Statistical Significance Testing
• Indirect Proof of a Hypothesis
• Null Hypothesis
– A statement that specifies no relationship or
difference on a population parameter.
• Alternative Hypothesis
• Conceptual (Research) Hypothesis
– A general statement about the relationship
between the independent and dependent
variables
• Statistical Hypothesis
– A statement that can be shown to be
supported or not supported by the data.
Examples of the Null and
Alternative Hypotheses
Nondirectional Test
Directional Test
Ho: µ = 5
Ha: µ ≠ 5
Ho: µ ≥ 5 or µ ≤ 5
Ha: µ < 5 or µ > 5
– A statement that specifies some value other than the
null hypothesis is true.
1
Rejecting the Null
• Alpha Level
• Type I
– The level of significant set by the
experimenter. It is the confidence with which
the researcher can decide to reject the null
hypothesis.
• Significance Level
– The probability value used to conclude that
the null hypothesis is an incorrect statement.
Common significance levels are .05, .01 and
.001.
Type 1 Error & Type 2 Error
Scientist’s Decision
Reject null hypothesis
Fail to reject null hypothesis
Null hypothesis
is true
Null hypothesis
is false
Type 1 Error
Two Types of Error
Type 1 Error
probability = α
Correct Decision
Probability = 1- α
Correct decision
probability = 1 - β
Type 2 Error
probability = β
=α
Cases in which you reject
null hypothesis when it is
really true
Type 2 Error
– When a researcher rejects the null hypothesis
when in fact it is true. The probability of a type
I error is α.
• Type II
– An error that occurs when a researcher fails
to reject a null hypothesis that should be
rejected. The probability of a Type II error is β.
The OJ Trial
For a nice tutorial go to:
http://www.socialresearchmethods.net/OJtrial/ojhome.htm
=β
Cases in which you fail to
reject null hypothesis when
it is false
The Problems with SST
Statistical Significance Testing
• We misunderstand what it does tell us.
• It does not tell us what we want to know.
• We often overemphasize SST.
2
Four Important Questions
1. Is there a real relationship in the
population?
Statistical Significance
SST is all about . . .
• Sampling Error
– The difference between what I see in my
sample and what exists in the target
population.
– Simply because I sampled, I could be wrong.
– This is a threat to Internal Validity
2. How large is the relationship?
Effect Size or Magnitude
3. Is it a relationship that has important,
powerful, useful, meaningful implications?
Practical Significance
4. Why is the relationship there?
??????
How it works:
How it works (cont’d):
1. Assume sampling error occurred; there is
no relationship in the population.
2. Build a statistical scenario based on this
null hypothesis
3. How likely is it I got the sample value I
got when the null hypothesis is true?
(This is the fabled p-value.)
• How unlikely does my result have to be to
rule out sampling error? alpha (α).
• If p < α, then our result is statistically rare,
is unlikely to occur when there isn’
isn’t a
relationship in the population.
What it does tell us
What it does not tell us
• What is the probability that we would see a
relationship in our sample when there is
no relationship in the population?
• Can we rule out sampling error as a
competing hypothesis for our finding?
•
•
•
•
•
•
Whether the null hypothesis is true.
Whether our results will replicate.
Whether our research hypothesis is true.
How big the effect or relationship is.
How important the results are.
Why there is a relationship.
3
Causal-Comparative Research
Chapter Sixteen
What is Causal-Comparative
Research?
• Investigators attempt to determine the cause of differences that
already exist between or among groups of individuals.
• This is viewed as a form of Associative Research since both
describe conditions that already exist (a.k.a. ex post facto).
• The group difference variable is either a variable that cannot be
manipulated or one that might have been manipulated but for one
reason or another, has not been.
• Studies in medicine and sociology are causal-comparative in
nature, as are studies of differences between men and women.
Similarities and Differences Between
Causal-Comparative and
Correlational Research
• Similarities
– Associative research
– Attempt to explain
phenomena of interest
– Seek to identify variables
that are worthy of later
exploration through
experimental research
– Neither permits the
manipulation of variables
– Attempt to explore
causation
• Differences
– Causal studies compare
two or more groups of
subjects
– Causal studies involve at
least one categorical
variable
– Causal studies often
compare averages or use
crossbreak tables instead
of scatterplots and
correlations coefficients
Steps Involved in CausalComparative Research
Similarities and Differences Between
Causal-Comparative and Experimental
Research
• Similarities
Independent
variable
Dependent
variable
I
C
(Group possesses
characteristic)
O
(Measurement)
II
–C
(Group does
not possess
characteristic)
O
(Measurement)
I
C1
(Group possesses
characteristic 1)
O
(Measurement)
II
C2
(Group possesses
characteristic 2)
O
(Measurement)
Group
(a)
• Sample
• Selection of the sample of individuals to be studied by carefully
identifying the characteristics of select groups
• Instrumentation
• There are no limits on the types of instruments that are used in
Causal-comparative studies
• Design
• The basic design involves selecting two or more groups that differ on
a particular variable of interest and comparing them on another
variable(s) without manipulation (see Figure 16.1)
– In experimental research,
the independent variable is
manipulated
– Causal studies are likely to
provide much weaker
evidence for causation
– In experimental studies,
researchers can assign
subjects to treatment
groups
– The researcher has greater
flexibility in formulating the
structure of the design in
experimental research
The Basic Causal-Comparative Designs
• Problem Formulation
• The first step is to identify and define the particular phenomena of
interest and consider possible causes
• Differences
– Require at least one
categorical variable
– Both compare group
performances to determine
relationships
– Both compare separate
groups of subjects
(b)
4
Examples of the Basic CausalComparative Design
Threats to Internal Validity in
Causal-Comparative Research
• Subject Characteristics
• The possibility exists that the groups are not equivalent on
one or more important variables
• One way to control for an extraneous variable is to match
subjects from the comparison groups on that variable
• Creating or finding homogeneous subgroups would be
another way to control for an extraneous variable
• The third way to control for an extraneous variable is to use
the technique of statistical matching
Does a Threat to Internal Validity Exist? (Figure 16.3)
Other Threats
•
•
•
•
•
Loss of subjects
Location
Instrumentation
History
Maturation
•
•
•
•
•
Data collector bias
Instrument decay
Attitude
Regression
Pre-test/treatment
interaction effect
Evaluating Threats to Internal Validity in
Causal-Comparative Studies
• Involves three sets of steps as shown below:
– Step 1: What specific factors are known to affect the variable
on which groups are being compared or may be logically be
expected to affect this variable?
– Step 2: What is the likelihood of the comparison groups
differing on each of these factors?
– Step 3: Evaluate the threats on the basis of how likely they
are to have an effect and plan to control for them.
5
Data Analysis
• In a Causal-Comparative Study, the first step is to
construct frequency polygons.
• Means and SD are usually calculated if the variables
involved are quantitative.
• The most commonly used inference test is a t-test for
differences between means.
• Results should always be interpreted with caution since
they do not provide strong evidence of cause and effect.
• It is common to find researchers who treat quantitative
variables conceptually as if they were categorical, but
nothing is gained by this procedure and it should be
avoided.
6