Download Non-Experimental Design

Non-Experimental Design
Where are the beakers??
What kind of research is considered
the “gold standard” by the Institute of
Education Sciences?
A.
B.
C.
D.
Descriptive
Causal-Comparative
Correlational
Experimental
Why?
Why does most educational
research use non-experimental
designs?
There are ethical and logistical
considerations that often impede the use
of experimental studies.
What is the purpose of
non-experimental designs?
Describe current existing characteristics
such as achievement, attitudes,
relationships, etc.
There is no manipulation of an
independent variable
First…
Some Thought Questions
Causal-Comparative Design
A study in which the researcher attempts to
determine the cause, or reason, for preexisting differences in groups of individuals
At least two different groups are compared on
a dependent variable or measure of
performance (called the “effect”) because the
independent variable (called the “cause”) has
already occurred or cannot be manipulated
Causal-Comparative Design
A “kissing cousin” to correlational
research design.
Ex-post facto
– Causes studied after they have exerted
their effect on another variable.
Causal-Comparative Design
Drawbacks
– Difficult to establish causality based on
collected data.
– Unmeasured variables (confounding
variables) are always a source of potential
alternative causal explanations.
Causal-Comparative Example
Green & Jaquess (1987)
– Interested in the effect of high school
students’ part-time employment on their
academic achievement.
– Sample: 477 high school juniors who were
unemployed or employed > 10 hours/wk.
Correlational Design
Determines whether and to what degree
a relationship exists between two or
more quantifiable variables.
Example of Correlation
Correlational Design
The degree of the relationship is
expressed as a coefficient of correlation
Examples
– Relationship between math achievement
and math attitude
– Relationship between degree of a school’s
racial diversity and student use of
stereotypical language
– Your topics?
Correlation coefficient…
-1.00
strong negative
0.00
+1.00
strong positive
no
relationship
Advantages of Correlational Design
Analysis of relationships among a large
number of variables in a single study
Information about the degree of the
relationship between the variables being
studied
Cautions
A relationship between two variables
does not mean one causes the other
(Think about the reading achievement
and body weight correlations on p. 189)
Possibility of low reliability of the
instruments makes it difficult to identify
relationships
Cautions
Lack of variability in scores (e.g.
everyone scoring very, very low;
everyone scoring very, very high; etc.)
makes it difficult to identify relationships
Large sample sizes and/or using many
variables can identify significant
relationships for statistical reasons and
not because the relationships really exist
(Avoid shotgun approach)
Cautions
Need to identify your sample to know
what is actually being compared.
If using predictor variables, time interval
between collecting the predictor and
criterion variable data is important.
Correlational Designs
Guidelines for interpreting the size of
correlation coefficients
– Much larger correlations are needed for
predictions with individuals than with groups
Crude group predictions can be made with
correlations as low as .40 to .60
Predictions for individuals require
correlations above .75
Correlational Designs
Guidelines for interpreting the size of
correlation coefficients
– Exploratory studies
Correlations of .25 to .40 indicate the need
for further research
Much higher correlations are needed to
confirm or test hypotheses
Correlational Designs
Criteria for evaluating correlational studies
– Causation should not be inferred from
correlational studies
– Practical significance should not be confused
with statistical significance
Correlational Designs
Criteria for evaluating correlational studies
– The size of the correlation should be
sufficient for the use of the results
(individuals vs groups)
– Prediction studies should report the accuracy
of predictions for new subjects
– Procedures for collecting data should be
clearly indicated
Think…
If you were going to take your action
research topic, and create a causalcomparative study, what would it look
like?
--OR-If you were going to take your action
research project, and create a
correlational study, what would it look
like?