Download Lecture18 - University of Idaho

PSYC512: Research Methods
Lecture 18
Brian P. Dyre
University of Idaho
PSYC512: Research Methods
Lecture 18 Outline


Research Proposal Information
 Writing Research Proposal
 The Art of Peer Review
 Addressing Reviewer’s Comments
Inferential Statistics
 Testing for differences



Determining if a sample represents a population
Determining if two or more samples differ
Testing for relationships


Form of relationship
Strength of relationship
PSYC512: Research Methods
Writing a Research Proposal

Purpose of Research Proposal

Present a literature review defining concepts and
intervening variables pertinent to research question

Present a specific research question with explicit
hypotheses to be tested

Develop a plan for addressing that research question
empirically, including descriptions of

Target population and subject sampling

The research design, including explicit definition of
independent and dependent variables and how the
stimuli and procedures implement these variables

Aspects of stimuli and procedures that control for
extraneous and confounding variables

Types of analyses to be used, predicted results, and
how these predictions relate to the hypotheses
PSYC512: Research Methods
Research Proposals: Global
Concerns


Scientific Writing Style

Precision more important than entertainment, but…

Research is part science and part advertising – not
only do you need to develop good ideas but you
must be able to sell your ideas

Proposal is often the basis of first impression for the
quality of the research project
Proposal must be clear on ALL levels of analysis

Words

Sentences

Paragraphs
OUTLINE!

Sections
PSYC512: Research Methods
Research Proposals: Global
Concerns

APA guidelines

Orderly expression of ideas - organization

Smoothness of expression - transitions

Economy of expression - concise language

Precision and clarity – use scientific
vocabulary “jargon” correctly and insure that
all terms are defined the first time they are
used
PSYC512: Research Methods
Organization of a Research
Proposal: APA STYLE


Title Page – title should specifically describe what the
paper is about so that it is useful information for other
researchers’ literature searches
Abstract

essentially a mini-paper for lit. searches

be extremely CONCISE! (< 150 words)

introduce specific topic

discuss variables, etc.

present major results (no statistics!)

discuss important conclusions
PSYC512: Research Methods
Organization of a Research
Proposal : APA STYLE

Introduction

Purpose

Demonstrate knowledge of relevant research

Define intervening variables and their relation to
manipulations and measurements used in previous
research

Present and justify research question and hypotheses

Present and justify the general method to be used

Organization

Start broad then narrow to your general purpose

Discuss only relevant research in a logical flow

Near the end provide an explicit statement of
hypotheses and an overview of the general research
design
PSYC512: Research Methods
Organization of a Research
Proposal: APA STYLE


Method

explicitly state how variables are manipulated

define in separate sub-sections

Participants (subjects)

Design

Stimuli/Apparatus/Materials

Procedures
Results

Describe the Scales of IVs and DVs

Describe Transformations of DVs

Summarized planned statistical analyses

Summarize predictions
PSYC512: Research Methods
Important Elements of Style
(Brian’s pet peeves)




Define scientific terms (“jargon”) and abbreviations at
first use, then use terms consistently
Do not use informal or colloquial language

e.g., you don’t “run” subjects, you test subjects
Ambiguous pronouns: if you use the word “it” make
sure the surrounding context makes the meaning of the
word “it” obvious, otherwise avoid using it (as in “it”)
Sexist pronouns (he vs. she) – word sentences to avoid
having to use these if at all possible, or use “he or she”
PSYC512: Research Methods
Important Elements of Style
(Brian’s pet peeves)

Verb tense

Introduction Section:

Discussion of specific previous research  past
tense,

Discussion of ideas or concepts that are general
for all time and not linked to specific moment in
the past present tense

Method and results

Proposal: future tense

research report: past tense
PSYC512: Research Methods
Important Elements of Style
(Brian’s pet peeves)





Plural/singular mismatches
Passive voice – word sentences in active voice
Superfluous, imprecise language (e.g., avoid vague
adverbs—search for all words ending in “ly” and
consider eliminating them)
Style issues like “that” vs. “which”
For more information on writing style I recommend:

Strunk, W., jr., and White, E.B. (2000) The Elements
of Style, 4th Ed. Boston: Allyn and Bacon.
PSYC512: Research Methods
The Art of Peer Review



Example review available online in lecture schedule
Goal: Assist author in improving the clarity and impact
of the paper (or proposal) by offering specific
constructive criticism (no name calling!) and
complements (where appropriate)
Structure
 Summary
 Major (or general) criticisms (typically <=3)
 Minor (specific) criticisms (any number, specifically
listed by page and line number)
PSYC512: Research Methods
Addressing Reviewer’s
Comments


Never blame a reviewer for a negative review
 Consider all criticism as constructive—the reviewer is
trying to help you!
 Assume misunderstandings are your fault, not the
reviewers. All misunderstandings occur because you
did not write clearly enough
The cover letter (required with submission of revised
proposal)
 Specifically responds to the reviewers comments by
 describing specific changes made in the paper to
address criticisms
 Presenting a rationale for why a reviewer’s
criticism was not addressed
 Must be worded very diplomatically!
PSYC512: Research Methods
Presenting Research












Time-allotted: 20 minutes  strictly enforced!
Use same general format as the written report
introduction (roughly 10 minutes)
method (roughly 8 minutes)
Summary (2 minutes or so)
Given time allotted you cannot go into the same
level of detail as your written report
Materials
Talk from an outline of points you wish to make
Visual Aids  power point
Elements of Style
PRACTICE YOUR PRESENTATION!
Anticipate questions and how you will answer them
PSYC512: Research Methods
Using Inferential Statistics


Which Statistic?
 The statistical decision tree Howell Figure 1.1
Testing for relationships vs. differences (a false distinction)
 Relationships: assessing the strength of relationship
between measured (dependent) variables
 Differences: comparing different groups or treatments
on some measurement
 But what causes those differences? The relationship
between the independent variable defining the
groups or treatment and the dependent variable
 Hence, testing for differences is really testing the
relationship between the IV and DV
PSYC512: Research Methods
Analyzing Frequencies
(Howell, Chapter 5)


Bernoulli Trials: series of independent trials that result in one of two
mutually exclusive outcomes
 E.g. coin flips, gender of babies born, increase of decrease in a
measure after application of a treatment
The Binomial Distribution
p ( X )  C XN p X q ( N  X ) where,
C XN  The number of combinations of N things taken X at a time 
N!
p X q ( N  X ) where
X !( N  X )!
p ( X )  The probabilit y of X successes
N  The number of trials
p  The probabilit y of " success" on any one trial
q  (1  p )  The probabilit y of " failure" on any one trial
p( X ) 
PSYC512: Research Methods
N!
, hence
X !( N  X )!
Analyzing Frequencies
(Howell, Chapter 5)


N!
Using the binomial distribution
p( X ) 
p X q(N  X )
X !( N  X )!
 Mean number of successes = Np
 Variance in number of successes = Npq
Testing Hypotheses using the binomial distribution: The Sign Test
 Ho is typically p= q = .50 (50-50 chance of success of failure), but
that doesn’t have to be the case
 H1 is typically p ≠q
 Plug in values for N, X, p, and q and p(X) directly provides the
probability that the pattern of data could result given the null
hypothesis is true
 Sum the probabilities p(X) for all number >= X to get the total
probability of finding p(>=X)
 Important: The sign test takes into account direction of differences
but not magnitude
PSYC512: Research Methods
Analyzing Frequencies
(Howell, Chapter 5)

What about multiple (more than 2) possible outcomes?
 Multinomial distribution
N!
p( X 1 , X 2 ,... X k ) 
p X 1 p X 2 ... p X k where,
X 1! X 2 !... X k !
where
p( X 1 , X 2 ,... X k )  The probabilit y of frequency X in each category, k
N  The number of trials
p X k  The probabilit y of observation X being in category k on any one trial
PSYC512: Research Methods
Analyzing Frequencies
(Howell, Chapter 5)
p( X 1 , X 2 ,... X k ) 


Using the multinomial distribution
 Mean Xk = NpXk
 Variance in Xk = NpXk (1-pXk)
N!
p X1 p X 2 ... p X k
X 1! X 2!...X k !
Testing Hypotheses using the multinomial distribution:
 Ho is typically pX1= pX2 … = pXk = 1/k (each outcome has the
same chance), but that doesn’t have to be the case
 H1 is typically pX1 ≠ pX2 …≠ pXk
 Plug in values for N, X, and pX, and p(X1, X2…Xk) directly provides
the probability that this particular pattern of data could result
given the null hypothesis is true
 Must sum the probabilities for all patterns that deviate equal to or
more to get the total probability – time consuming!
PSYC512: Research Methods
Analyzing Frequencies
(Howell, Chapter 6)



Easier Alternative to
Multinomial distribution:
Chi-square (c2) test
Compare computed value
of c2 to value of c2
distribution with df=k-1
Expected frequencies for
the null hypothesis
typically = N/k, where N
is the total number of
observations
c
(Oi  Ei )

Ei
i 1
k
2
k 1
k is the number of
2
categories in the variable
O is the observed frequency
for each category
E is the expected frequency
for each category
i is the category index
PSYC512: Research Methods
Analyzing Frequencies
(Howell, Chapter 6)
R




c2 with
Using
multiple
dimensions: contingency
tables—frequencies of one
dimension are contingent on
the other dimension
Eij = RiCj/N
N is the total number of
observations
Compare computed value of c2
to value of c2 distribution with
df=(R-1)(C-1)
C
c (2R 1)(C 1)  
i 1 i 1
(Oij  Eij ) 2
Eij
R is the number of categories in
the dimension defined by the
rows of the table
C is the number of categories in
the dimension defined by the
columns of the table
O is the observed frequency for
each category
E is the expected frequency for
each category
i and j are category indices
PSYC512: Research Methods
Analyzing Frequencies
(Howell, Chapter 6)

Assumptions of the c2 test
 Each observation is independent
 Inclusion of non-occurrences
PSYC512: Research Methods