Download Mid-Term Review #2

Research Methods:
Midterm Review Part 2
Dr. Dodge
February 28, 2006
Variables
Variables



Variables: building blocks of hypotheses
that are held together by the “glue” of the
relationship we are studying.
Wide range of definitions and categories of
variables.
Characteristics are not fixed but are able
to vary (take on more than one value)
Variables Functions
Independent variable: “…is the factor that is
manipulated or controlled by the researcher”
 Variable that is “independent of the outcome
being measured. What causes or influences the
outcome”
_________________________________________




Dependent variable: “is a measure of the effect
(if any) of the independent variable
Is influenced by the independent variable
Factor that is observed or measured to determine
the effect of the independent variable
Variables: Measurement Scales

Two different scales for
measurement of variables:
1. Continuous or categorical
2. Nominal, ordinal, interval, or ratio
Variables: Measurement Scales
1.
Continuous or Categorical
• Continuous variables: have an ordered
set of values within a certain range.
Values between two points (e.g., 4 and
5) on the range actually mean
something.
• Categorical variables (i.e., discrete
variables): measured in categories. An
observation is either in a category or it
isn't. There is no meaningful “in
between” option.
• When planning data collection, always
try to collect data in a continuous
format
Variables: Measurement Scales
2.
Nominal, Ordinal, Interval, or Ratio
• Nominal: Names, classes, or symbols
designating unique characteristics - simple
classification, no order.
• Ordinal: Assignment of numbers of
symbols indicates order of relationship.
Order only is indicated; there is no
indication of amount. Ex: rank order data.
• Interval: has the same ordering properties
as ordinal data and it also has equal,
meaningful intervals and an arbitrary zero
point.
• Ratio: has the same properties as interval
data and also has an absolute zero point.
Variable Levels and Factors




The most basic experimental design has two
variables
• Independent Variable
• Dependent Variable
The independent variable has two Levels
• Experimental Group (Usually receives
treatment)
• Control Group (Usually does not receive
treatment)
A grouping variable is called a “factor”
The number of groups are called “levels”
Levels and Factors
(4 Level Factor)
Treatment 1
Treatment 2
Treatment 3
Control
Research Questions
Research Questions
Questions that guide your research
 Should be debatable and of interest
to both you and your potential
readers
 Should also be based on a narrow
topic
 Should guide your research
 You can have more than one
research question in a study

Defining Research Questions

To help define questions:
• People, patients or population - who
are you asking the question about?
• Intervention - what intervention are
you interested in?
• Control or comparison - what are you
comparing the intervention to?
• Outcome - what outcome are you
interested in measuring?
Hypotheses
Hypotheses
Hypotheses: predictions about
the relationship among two or
more variables or groups based
on a theory or previous research
 Assumptions or theories that a
researcher makes and tests

Hypotheses

Hypotheses are important because they:
• Direct our observation: identifies the
variables examined and data to be
collected
• Describe a relationship among
variables: state that as one variable
increases, the other will decrease; as
one variables increases, the other will
increase …
• Refer to populations: help researchers
infer that results of a sample will
translate to a population!
Hypotheses


Hypotheses have four functions:
• Estimate population characteristics
• Correlate variables
• Display differences among two or more
populations
• Show possible cause and effect
Two types of hypotheses:
• Research hypotheses
• Statistical hypotheses
Research Hypotheses



Research Hypothesis: statement of the
relationship among two or more variables
or groups
Acceptance or non-acceptance is based on
resolving a logical alternative with a null
hypothesis.
Example: Students who attend school
regularly will score higher on their FCAT
exams than students who do not.
Research Hypotheses


Can be “directional” or “nondirectional.”
Directional hypotheses: predict specific
relationship among two or more
variables or groups
• Show possible cause and effect
• Ex: IQ scores will correlate in a
positive manner with shoe size. (And
why would that be?  )
Research Hypotheses

Non-Directional Hypotheses: predict
differences among two or more
groups, but do not specify the
direction of the differences
• Ex: Men and women will differ on
measures of sexual arousal when
exposed to explicit auditory sexual
stimuli.
Statistical Hypotheses




Statistical hypotheses: mathematical, or
logical, statements that help researchers
interpret the results of research
Statistical hypotheses consist of the Null
Hypothesis (H0), the hypothesis of no
difference, and the Alternative Hypothesis
(H1 or HA) which is similar in form to the
research hypothesis. Both can be
expressed in alphanumerical formulae.
Null: (H0: µ1 - µ2 = 0 )
Alternative: (H1: µ1-µ2 ≠ 0)
Statistical Hypotheses

In other words …
• Null: There will be no difference on
measures of aggression between
students who have completed the
research methods midterm review and
students who have not completed the
review.
• Alternative: There will be a difference on
measures of aggression between
students who have completed the
research methods midterm review and
students who have not completed the
review.
N.B.!
The null hypothesis always implies
that there is no relation or statistical
difference between variables or
groups
 The alternative hypothesis always
implies that there is a meaningful
relationship among variables or
groups

Testing Hypotheses



We only test the null hypothesis
We do not test the research hypothesis
We use a variety of statistical procedures
to test null hypotheses. Depends on a
variety of factors including the research
hypothesis, the data, the sampling
strategy, and what we want to be able to
say as a result of our testing.
Types of Tests



Statistical procedures: correlation, analysis
of variance (ANOVA), analysis of
covariance (ANCOVA), regression,
multivariate analysis of variance
(MANOVA), t-tests, and Chi-Square.
Each procedures has an associated test
statistic used to determine significance.
Ex: ANOVA, ANCOVA, and regression use F
statistics and their associated p-values.
Important: All test statistics are eventually
related to a probability distribution and a
p-value. The p-values mean the same
thing across test statistics.
Error Types



Type I and Type II Errors – inherent in
hypothesis testing. Errors are mistakes
that we can make when judging the null
hypothesis.
Type I Error: the tested hypothesis is
falsely rejected. (You say you found
something, but that something is really an
error.) A type I error is a false positive.
Type II Error: when a false tested
hypothesis is not rejected (You do not find
something that is, in fact, there.) A type II
error is a false negative.
Alpha and Beta



Alpha: level of probability (pre-set by the
researcher) that the tested hypothesis will be
falsely rejected. Alpha is the pre-set risk of a
Type I error. The degree of risk that you accept,
in advance of conducting the study, that what
you find will be an error.
Beta: level of probability that a false null
hypothesis will not be rejected. The probability
that you won’t find what you are looking for if, in
fact, it is really there.
Probability (p) Value: Probability that observed
relationships or differences are due to chance.
Alpha is also known as significance level or
rejection region.
Error Types Chart
Reject H0
Decision
Fail to
Reject
(decide in
favor of H0)
H0 is True
H1 is
True
Type I
α
Correct
1- β
Correct
1- α
Type II
β
Power, Effect Size, and
Measurement


•
•
•
•
Statistical power: probability of rejecting a
null hypothesis that is, in fact, false. The
probability of finding relationships or
differences that in fact exist
Statistical power is related to:
Sample size
Effect size
Statistical design
Significance criteria
Power, Effect Size, and
Measurement

Effect size (ES): amount of variance
between the independent variable(s)
(IV) and the dependent variable(s)
(DV). Degree to which changes in
the IV(s) result in changes in the
DV(s).
Power, Effect Size, and
Measurement



Relationships of measurement, research
design, and statistical power means that
large treatment effects can actually be
observed as small effects.
Even if an intervention is very effective,
measurement and design complications
may make the effect appear small and
thus require high statistical power for
detection.
We will discuss these issues in the second
half of the semester.
Test Statistics, Probability, and
Significance


WHETHER YOU ARE LOOKING AT OBTAINED
VALUES OF TEST STATISTICS IN RELATION TO
CRITICAL VALUES OR YOU ARE LOOKING AT
ACTUAL PROBABILITY LEVELS, IT IS
IMPORTANT TO NOTE THAT TEST STATISTICS
AND THEIR ASSOCIATED PROBABILITIES ONLY
TELL US THE PROBABILITY THAT A
DIFFERENCE OR RELATIONSHIP OCCURRED BY
CHANCE.
THESE STATISTICS DO NOT TELL US THE SIZE
OF GROUP DIFFERENCES OR THE STRENGTH
OF RELATIONSHIPS
Steps in Hypothesis Testing for
Quantitative Research Designs

•
•
•
•
Hypothesis testing is a 4 phase
procedure:
Phase I: Research Hypotheses,
Design, and Variables
Phase II: Statistical Hypotheses
Phase III: Hypotheses Testing
Phase IV: Decision/Interpretation
Phase I: Research Hypotheses,
Design, and Variables
1.
2.
3.
State your research hypotheses.
Decide on a research design based
on your research problem, your
hypotheses, and what you really
want to be able to say about your
results (Ex: if you want to say that
A caused B, you will need an
experimental design).
Operationally define your variables.
Recall that one variable can have
more than one operational
definition.
Phase II: Statistical
Hypotheses
1.
2.
Consider your chosen statistical
procedures.
Write one statistical null hypotheses
for each operational definition of
each variable that reflects that
statistical operations to be
performed.
Phase III: Hypotheses
Testing
1.
2.
3.
4.

Select a significance level (alpha).
2. Compute the value of the test statistic (e.g., F,
r, t).
3. Compare the obtained value of the test
statistics with the critical value associated with
the selected significance level or compare the
obtained p-value with the pre-selected alpha
value.
4. If the obtained value of the test statistic is
greater than the critical value (or if the obtained
p-value is less than the pre-selected alpha
value), reject the null hypothesis. If the obtained
value is less than the critical value of the test
hypothesis, fail to reject the null hypothesis.
In other words: If p is less than or equal to
alpha, reject the null hypothesis.
Phase IV:
Decision/Interpretation
1.
2.
3.
4.
For each research hypothesis, consider
the decisions regarding the statistical
null hypotheses.
For each research hypothesis, consider
qualitative contextual information
relating potential plausibility.
Cautiously explain your findings with
respect to the research hypotheses.
List and discuss the limitation.
Questions?