Download File

Research Methods
Research Methods Principles
• The goal is research literacy.
– Most of our us will not be career
researchers.
– BUT….we are all consumers of research.
– How do you tell the good from the bad?
• No research is perfect. Everything is open to
critique—a great platform for critical thinking.
• Vitamin C benefit
Methods Is Like Whac-A-Mole…
a fun game for which there is no perfect solution
Whac-A-Mole
• If I’m conducting my own research, the goal is
to minimize the moles.
• If I’m considering the research of others, the
goal is to identify as many moles as I can.
Topic 1: Moles
BIAS
• Hindsight bias
• 20/20 vision
• Confirmation bias
• Buying a jeep
• Overconfidence
• Someone who cannot sing at all but who believes she has a great voice
and decides to try out for American Idol. When she submits her audition
tape, she could end up being laughed at or ridiculed for her terrible voice
because of her overconfidence.
• Hawthorne Effect
• Placebo Effect
Hawthorne Effect
• By simply knowing
you are part of study
can affect the results
• Hawthorne Works
factory Illinois 1920’s
• Told workers about
the change in lighting
to increase
productivity
• What do you think
happened?
Placebo Effect
•
•
•
•
•
Effect solely due to participants belief
Inert components of an experiment
Sugar pills (drug study)
Non-electric stimulation (Parkinsons)
Athletic performance (Phelps)
98% Certainty
in your notebook answer the following
1. The area of the US in square miles?
2. The population of Australia 2007?
3. American battle deaths in Spanish-American
War?
4. Female psychiatrists in the US in 2005?
5. Operating nuclear plants worldwide in 2007?
98% Certainty
1.
2.
3.
4.
5.
Area of US:
Australian pop.:
Battle deaths:
Female psychiatrists:
Nuclear plants:
3.6 million sq. miles
20.4 million
385
13,079
435
Topic 2: Descriptive Techniques
• Case studies
• Study one or a group in depth and/or over long period of time
• Surveys
• Questionnaires
• Naturalistic observation.
• Correlational Study
• More on this later 
Case Studies: Phineas Gage
Phineas Gage
1823-1860
Brad Wray’s Ballad of Phineas Gage
Disadvantage: Cannot generalize to whole population
Topic 3: Experimentation
• The purpose of an experiment is to establish a
cause-and-effect relationship.
• Experiments are the only research method that
can establish cause-and-effect.
IV, DV, CONTROL
• Independent variable (IV): Variable the experimenter
manipulates (i.e. changes) – assumed to have a direct
effect on the dependent variable.
• Dependent variable (DV): Variable the experimenter
measures, after making changes to the IV that are
assumed to affect the DV.
• Control: Experimental controls are mechanisms in
science that eliminate extraneous factors that might
otherwise affect the results of an experiment
• i.e. one group is given an experimental medicine and
another group is given an inactive placebo that is
identical in appearance.
Example Experiment
General hypothesis: Food affects learning.
Specific hypothesis with operational variables:
Students who eat an (OD)oatmeal raisin cookie
before class each day will have higher average
scores on the (OD) semester final than students
who don’t eat a cookie.
Eating cookies before class each day will lead to
higher average scores.
Variables:
Independent (IV)
Controlled by experimenter
The “cause” variable
Dependent (DV)
Predicted by experimenter
The “effect” variable
Eating cookies before class each day will
lead to higher average scores.
What if kids get cookies and A’s?
Groups (conditions): to establish different levels of
the IV
Experimental group
Exposed to IV
Get cookie
Control group
Not exposed to IV
No cookie
Eating cookies before class each day will
lead to higher average scores.
Confounding
Variables
IV
DV
Expt. Gp.
Cookie
95%
Environmental
Cntrl. Gp.
No Cookie
82%
Expectations
Individual
differences
Double Blind procedure
• Both the experimenter and the participants
do not know who will receive the
drug/treatment/experimentation
• Great way to reduce confounding variables
Random sampling vs. random assignment
Random Sampling
• To select participants from population
• Allows you generalize results
Random Assignment
• To divide participants into groups
• Controls individual difference confounding
variables
Note: I do not provide this slide to my students.
Eating cookies before class each day will
lead to higher average scores.
IV
DV
Expt. Gp.
Cookie
95%
Cntrl. Gp.
No Cookie
82%
85%
93%
Confounding Variables
• Anything that will affect the outcome of a
study
• We have to reduce as many as possible
• What are some confounding variables that
you can think of for the cookie and grades
experiment?
• How can reduce confounding Variables? OD!!!
Importance of Operational Definitions
Students are more likely to smile for their senior
pictures if they have a friendly photographer.
IV?
Photographer friendliness
DV? Smiling
Operational definitions are needed for both of these
variables. To illustrate the importance of this, have
students determine how many of the students on the
following slide are smiling.
Experimental Design Terms
•
•
•
•
•
•
•
•
Hypothesis
Operational definitions
Participant selection
IV & DV
Experimental & control groups
Confounding variables
Random assignment
Placebo control
•
•
•
•
Today
Ethics
Correlation
Statistical significance (p value)
Ethics
Describe how ethical and legal guidelines (e.g.,
those provided by the American Psychological
Association, federal regulations, local
institutional review boards) protect research
participants and promote sound ethical
practice.
Was John Watsons study ethical?
Stanley Milgram
• https://www.youtube.com/watch?v=yr5cjyok
VUs
APA guidelines
• American Psychological Association
• Set up precise guidelines for an ethical study
• Every study must go through an IRB (institutional
Review Board) and Must include:
• Informed Consent
• Confidentiality
• Protection from harm
• Option to leave the study
• Debriefing
Validity vs Reliability
• Validity- Does the study measure what its trying to measure
• If Mr. Andrews gave you a math test in english class it would not be a valid
test
• Reliability- same results after multiple trials and with different samples or
populations
• A good study MUST BE VALID AND RELIABLE
• ACT SAT, Valid and Reliable? why
• Buzzfeed quiz to determine which Hogwarts house you belong
• How do we determine if a study is both?
Statistics
The Joys of Stats!
https://www.youtube.com/watch?v=
jbkSRLYSojo
Randomness
Randomness
• Random Sampling- Choosing a
representation of a larger population
• i.e. this class is a sample of students from
Summit- who do we need to add to ensure
its an equal representation?
• Random Assignment- assigning groups to
participant in a study
• Number one’s and two’s
Guiding Principles
• Our focus should be conceptual, not
computational.
• Statistics are necessary to understand the
meaning of a set of numbers.
• We need to demonstrate the importance of
statistics throughout the entire course, not
just in the methods unit.
Topic 1: Frequency Distributions
Putting scores in order adds meaning
Bar graphs (histograms) are visual
representations of frequency
distributions.
A 40 4
39 7
38 10
37 8
36 15
B 35 8
34 8
33 8
32 7
C 31 4
30 5
29 7
28
D 27
26 2
25 1
24 2
F <24 1
45%
32%
16%
5%
1%
Topic 2: What’s the center of the distribution?
Measures of Central Tendency
Mode
--Most common = 4
Mean
--Arithmetic avg = 20/5 = 4
Median
--Middle score = 4
Quiz Scores
4
3
5
4
4
Central Tendency: Mean vs. Median
1968 TOPPS Baseball Cards
Nolan Ryan
Billy Williams
Luis Aparicio
Harmon Killebrew
Orlando Cepeda
Maury Wills
Jim Bunning
Tony Conigliaro
Tony Oliva
Lou Pinella
Mickey Lolich
$1500
$8
$5
$5
$3.50
$3.50
$3
$3
$3
$3
$2.50
Elston Howard
Jim Bouton
Rocky Colavito
Boog Powell
Luis Tiant
Tim McCarver
Tug McGraw
Joe Torre
Rusty Staub
Curt Flood
With Ryan:
Median=$2.50
Mean=$74.14
$2.25
$2
$2
$2
$2
$1.75
$1.75
$1.5
$1.25
$1
Without Ryan:
Median=$2.38
Mean=$2.85
The median is a better measure of central tendency than
the mean when there are extreme scores.
Topic 3: How spread out are the data?
Measures of variation
Range
• The spread
between the
highest number &
the lowest number.
• Only considers two
numbers
Standard deviation
Calculation Example for Standard Deviation
Punt
Distance
36
38
41
45
Deviation
from Mean
-4
-2
+1
+5
Deviation
Squared
16
4
1
25
std. dev. =
Variance =
11.5 = 3.4 yds
Mean =
160/4 =
40 yds
46
46/4 = 11.5 =
variance
Topic 4: Properties of the Normal Curve
In a large, randomly distributed data set
• 68% of scores will be within 1 SD of the mean.
• 95% of scores will be within 2 SDs of the mean.
• 99.7% of scores will be withing 3 SDs of the mean.
Topic 4: Properties of the Normal Curve
Marilyn Vos Savant: claimed IQ of 228.
Is it more meaningful to express her IQ as points
above average or as standard deviations above
average?
Topic 5: Correlation
• A measure of the strength of the relationship
between two variables.
• Can be positive or negative.
• Useful for making predictions.
• You can fairly easily calculate correlations with
Excel or Google Docs.
Topic 5: Correlation
What does a correlation looks like?
Scatterplots
Positive Correlation
Negative Correlation
Topic 5: Correlation
No Correlation
Topic 5: Correlation
How do you express a correlation numerically?
The Correlation Coefficient
R= +/- 1
Topic 5: Correlation
A strong correlation is not enough to establish a
cause and effect relationship.
Example: There is a correlation between TV
watching and grades.
Do you think it’s positive, or negative?
From this, what do we know about cause-andeffect.
Topic 5: Correlation
Even correlations that are clearly not cause-andeffect relationships can be used for prediction.
Ex: College entrance exams and freshman GPA.
Ex: Shoe size and vocabulary size in elementary
school children.
Ex: Ice cream sales and the rate of violent crimes.
Topic 5: Correlation
Weird correlations:
http://www.ebaumsworld.com/pictures/view/84284804/?autoplay=true
Topic 5: Correlation
Weird correlations:
http://www.ebaumsworld.com/pictures/view/84284804/?autoplay=true
Topic 6: Statistical Significance
• A measure of the likelihood that a result is
caused by chance.
• In an experiment, we want that likelihood
to be low so we can conclude a causeand-effect relationship exists between the
IV and the DV.
Topic 6: Statistical Significance
• Several statistics (e.g., chi square, t-test)
can be used to calculate statistical
significance, but we don’t need to know
these 
• They do need to know how to interpret the
results of these tests—the p value.
Topic 6: Statistical Significance
• P value is an estimate of the probability
that a result was caused by chance.
• In an experiment, it’s the likelihood that the
difference between the experimental and
control conditions as measured by the DV
was caused by chance.
• We want this difference to be caused by
our manipulation—the IV—not by chance.
Statistical Significance
p value
likelihood a result is caused by chance
can be no greater than 5%
p ≤ .05
Topic 6: Statistical Significance
• To say that the results of an experiment
are statistically significant means that
there is a small likelihood that the results
were caused by chance; that is, a high
likelihood they were caused by the IV.
• The threshold for statistical significance is
no more than a 5% likelihood the results
were caused by chance.
• We express this: p ≤ .05