Download class 8

SWK 707 Research for Social Work
Practice
Nechama Sammet Moring
CLASS 8

Tonight’s Plan

6:00-6:10 check in, logistics

6:10-7:00 qualitative coding lecture and activity

7:00-7:10 review descriptive statistics

7:10-7:30 inferential statistics part 1

7:30-7:40 break

7:40-8:00 inferential statistics part 2

8:00-8:10 research question and study design lecture

8:10-8:55 assignment 3 group work

8:55 evals
Strengths
Qualitative Research
 Used in exploratory research to examine a question
or construct about which little is known
 Useful in formative stages of research to gain a
better understanding of cultural variations,
appropriate language, and meaningful categories
 Open-ended questions can provide context and
depth to questions of interest
 Documents individuals’ responses in own words,
thoughts, phrases
Process of QL research
A question arises
Re-consideration of theory,
literature from findings
Grounding in
theory, literature –
provide
justification
Hone question, aims
- design, sample, few
questions chosen
with care
Ongoing Data collection,
analysis – redevelopment of questions
- simultaneous
Qualities Important for Qualitative Research

Good conceptual understanding of the goals of research

Ability to develop rapport with participants

Ability to listen and absorb information

Empathy

Intuitive sense to build conversation

Ability to analyze, synthesize and utilize information immediately (ie
your interviewee says something unexpected)

Reflexivity-sense of yourself, your own opinions, experiences,
background and life experiences, willingness to acknowledge your
opinion as simply one of many, formed as a result of your life
experience (i.e. my disability research is different than someone else’s
because I am a family member, a woman, able-bodied etc)
Qualitative Research Techniques
 Interviews
 Informal
 Unstructured
 Semi-structured/guided
 Formal/structured
 Focus groups
 http://www.huffingtonpost.com/2014/09/22/pets-are-
people-too_n_5842444.html
 Observation:
 Participant observation
 Unobtrusive observation
Semi-structured Interviews
 Based on a set of questions derived from theory,
previous research, and experience
 Interview guides have specific topics (constructs) to
cover, using specific questions as prompts
 Provides minimally directive framework that
enables both researcher and participant to define
questions and generate new ideas partnership
Interview tips
 Do your homework beforehand! Get a sense of your
interviewee’s priorities and be prepared to address these
topics
 Begin with general questions to establish rapport and move
to more specific/more controversial questions
 List constructs (concepts of importance) and develop
questions that address these constructs. Be prepared to
address the construct in more than one way.

Construct: Impact of dog ownership on quality of life
Questions: How does dog ownership impact your quality of
life? What are some of the pros of having a dog? What are
some of the ways that having a dog impacts you negatively?
How has having a dog changed your quality of life?
More interview tips
 Practice beforehand-interview someone else using your
guide and ask for honest feedback re: clarity, flow etc.
 Open-ended (as opposed to close-ended) questions allow
participants to express and explore their ideas. Close-ended
questions have more potential to influence responses.
(“What are some of the impacts of poverty on young
mothers?” Answer: talk, talk, talk vs. “does poverty impact
young mothers?” Answer: Yes.)
 How, why, what, describe….
Redirects and prompts in interviews
 The fine art of re-directs:


I see, so you (their thing). Was that where/when you first
(your thing)?
I want to be conscious of your time, and I have just a few
more questions about (your thing)
 The fine art of prompts:




Tell me more.
Ummhmmm
How so?
Can you expand on that? Can you give me an example?
Approach often determines data analysis
 Inductive-start with themes or conceptual model
already identified (usually from lit search or similar)
and use data to explore the model
 Content analysis
 Deductive-build theory from the data itself
 Grounded, coded theory
Theme identification in qualitative
research
 Importance of distinguishing between qualitative
and quantitative
 Theme identification vs. concept measurement
 Beyond counting…
 Letting the themes “rise up” from the data
 Letting the collective voice of study respondents
define what the themes are in answer to questions
 Can be confusing – demographic data sometimes
included as qualitative
12
Sample coding sheet
Code
Code 1
Code 2
Code 3
Demographic
data
Interview
1
Interview 2
Interview 3
Interview 4
Coding process
•
Codes: Identifying anchors - key points of the data
gathered
•
Themes: Collections of codes of similar content –
allows data to be grouped
•
Second level themes: Broad groups of similar
concepts used to generate a theory or relate back
On inductive, deductive research
Deductive
Inductive
Inductive vs deductive coding
 Deductive-starts with theory, code book already
built from the literature, your theory; content
analysis
 Inductive:-starts with observation, code book and
theory is built from the transcripts, grounded
theory
Interpreting qualitative data
1.
Become saturated
2.
Look for patterns – constant comparison
3.
Corroborate/legitimate themes
4.
Represent the accounts accurately via checking
back
1.
2.
3.
Self
Team
Member checking
Pure qualitative data analysis
•
Become grounded in the
data
•
“Open coding”
•
•
•
•
Express
data/phenomenon in
concept form
Categorize concepts that
are relevant to question
Allow understanding to
emerge from close study of
the data
•
“Axial coding” for subcategories or larger
themeing
Discover patterns, themes
•
“In-vivo” coding
How to begin the coding process
1.
Read memos, interviews, other (no notes)
2.
Read memos, interviews, other (make notes)
3.
Re-read (look for themes within transcript)


4.
Re-read (look for themes across transcripts)
In grounded theory: Sensitizing concepts
Review your thoughts with your team



Do you agree on themes in transcripts?
Do you agree on themes across transcripts
How do these themes connect?
Let’s try it!
 Content analysis
 Finding themes, commonalities in the first
encounter “bucket”
 Code book pre-built from my literature search I
totally did
 Code alone, then compare and discuss with a
partner
Discuss content analysis coding
 What did we learn?
 What did we miss?
 What about outliers?
Let’s try it again: grounded coding
 This time, you have a complete interview, and we don’t
have a code book pre-developed
 Read your transcript once through
 On your second read, make notes about codes
 Working with a partner, group each of your codes into
themes to create a code book
 What is important?
 Ok to have a “wild card” theme
Discuss grounded theory
 What did we learn?
 What did we miss?
 Outliers?
Making qualitative data more rigorous
(free of bias; scientific)
 Inter-coder reliability and ways to check this (group coding, 2+
people code all transcripts and discuss; 1 person codes most with a
second person to “spot check”)
 Good decisions in purposive sampling
 Triangulation of data (i.e. multiple data sources and data collection
methods yield the same findings)
 Respondent verification (people you interviewed read and review
your work)
 Saturation (the point at which new themes stop emerging) and
coding a few past saturation
 Positionality and reflexivity (self-awareness; knowing your “blind
spots” and designing your research to check your own blind spots)
Use of software programs
 Nvivo, Atlas-ti
 Document loading, linkage
 Open coding, axial coding, In vivo coding
 What is different with the software?
 Assign categories to transcripts – comparisons
 Word count, auto-code features (slippery slope)
 Pull documents together by theme/filter
 Drop-down codebook
 Code by recording location
 Mapping of code relationships
4 basic types of statistical tests:
Description
•Mean, standard deviation
•Median, Mode
•Percentage, frequency
Correlation
•Pearson’s correlation
Comparison
•Student’s t tests
•Chi-square tests
•ANOVA
•Odds ratios
Prediction
•OLS regression
•Logit regression
Descriptive statistics review: Why?
 Need to present data  aggregate rather than describing each
data point to make sense of large amounts of data
 Find and describe patterns
 Shows what is typical and what outliers exist-extent and range
 Does not speak to relationships between variables, just
describes individual variable, rather than their relationship to
each other
 Demographic information
Cheat sheet: Descriptive statistics
Type:
Used for:
Frequency
Counting
Mean/average,
median or mode
Measure of
central
tendency
Variable
structure:
Any
Continuous
Review of kinds of variables
 Dichotomous-one of 2 options (i.e. yes/no)
 Nominal-mutually exclusive categories (zombie, human,
sheep, elephant)
 Ordinal-mutually exclusive categories that go in order
(1st, 2nd, 3rd)
 Continuous (interval)-rank ordered, mutually exclusive
and there is the same amount of difference between
each variable, like height, 5, 5’1, 5’2 etc)
 Ratio-continuous but with a fixed 0 (number of kids, 0-
19)
Dependent and Independent Variables
 Quant research is about demonstrating
relationships between variables
 Independent variable just exists. (I am either
undead or not)
 Dependent variable is influenced by the
independent variable (whether or not I’m undead
has a relationship to if I eat brains or not)
 A change in the independent variable leads to a
change in the dependent variable
What is a frequency measure and what do
you do with it?
 Counts
 How often something occurs
 Numbers of
people/places/things with:


a certain characteristic
a certain set or
combination of
characteristics
 How do you report these
results in a way that is
accessible to the reader?


Percentages
Mean (SD), median, mode
Measures of central tendency
 Mean=mathematical average (add up all scores, divide
by number of scores)
 median = the score in the exact middle of all the scores,
the midpoint
 Mode=the score that occurs the most; often used with
nominal variables (i.e. more Catholics than Protestants
in my sample, because you can’t really come up with the
average religion)
 Outlier-something out in left field that can really skew
the mean (but not so much the median or the mode)
Median and mode
 Sometimes the mean is skewed
 If so, using one of these makes more sense:
 Median
 the value below which 50% of the scores fall, or the middle
score
 Median age at which people first had a counseling
experience
 Mode
 the most frequent score
 Most common age at which people first had a counseling
experience
Discussion
 What kind of information do we get from each
measure of central tendency?
 When might each be important?
 What kind of research designs lead to each?
Ways to measure dispersion of data
 Range (from what to what), the overall spread of
data from lowest to highest
 Variance- a statistical measure (we won’t be doing
the math) that gives a “score” based on how far
apart the data is from each other.
 Standard deviation-math calculated more precise
way of finding out the “average” difference from
the mean score, calculated with the square root of
the variance
Table 1: Demographics of Elders with IDD/SA
Sociobehavioral
model
Predisposing
Characteristics
Enabling
resources
Need factors
***p<.001 *p<.05
Variable
IDD/SA
(N=350)
NoIDD/SA
(N=48,014)
Test
Gender (male)
238 (68%)
25,064 (52%)
OR=1.9***
Mean age (SD)
70 (5)
73 (7)
t=4.5*
Race (white)
260 (74%)
28,741 (59%)
OR=1.9***
(SSI/SSDI)
199 (57%)
29,131 (61%)
NS
Dually eligible
303 (87%)
43,226 (90%)
OR=0.7*
FFS coverage
262 (75%)
34,283 (71%)
NS
Low state SA
coverage
141 (40%)
16,899 (35%)
OR= OR= 1.2*
Urban location
213 (61%)
30,027 (63%)
NS
SMI diagnosis
151 (43%)
7,540 (16%)
OR= 4.1***
Long-term SA
diagnosis
19 (5%)
5,549 (12%)
OR= 0.4***
Standard deviation:
 Most commonly used measure of dispersion around a
mean
 how “spread out” are the values?
 Always reported together – otherwise considered to be
biased
 Can’t be done with nominal variables
Standard deviation in a normally
distributed sample
•Dark blue < 1 “standard deviation” from the mean
•Accounts for 68.3%
Variance
 Variance- a statistical measure (we won’t be doing the math)
that gives a “score” based on how far apart the data is from
each other.
 Small/low variance score means that all data points are close
together and don’t vary much. Scores are clumped around the
mean (leptokurtosis-narrow and pointy)
 Large/high variance score means that data points have a lot of
variation from each other. Scores differ from the mean
(platykurtosis-flat & wide)
Rodriquez & Murphy
Rodriguez & Murphy:
Frequencies? Ranges?
Break
4 basic types of statistical tests:
Description
•Mean, standard deviation
•Median, Mode
•Percentage, frequency
Correlation
•Pearson’s correlation
Comparison
•Student’s t tests
•Chi-square tests
•ANOVA
•Odds ratios
Prediction
•OLS regression
•Logit regression
Review: Variable structure
 Continuous (a.k.a. numeric)
 Examples?
 Nominal (a.k.a. dummy variable, categorical
variables)
 Examples?
Parametric statistics
 Kind of bivariate analysis (finding the relationship
between 2 variables-the independent variable and
the dependent variable)
 There are rules (parameters) that must be met,
including a large enough sample size
 Correlation: Pearson’s r, also called Pearson’s
correlation, or just correlation
 Comparison: t-tests, ANOVA
 Prediction: regression (we’ll talk about this next
week)
Measures of association
 Ways to measure the correlation (relationship)
between 2 variables (i.e untreated chronic disease
and mental illness)
 How change in the dependent variable is related to
change in the independent variable (so if I increase
the independent variable by X%, how much will the
dependent variable change?)
Correlation tests
 Measures association/relationship between 2
continuous variables
 Does not measure causation
 Distinguish vernacular usage of the term from
statistical usage
 Requires a logic model/theory/research-based idea
4 kinds of correlation
 No correlation-there is no relationship (i.e your program is
ineffective)
 Positive correlation-the dependent variable increases when the
independent variable increases (the older I get (I), the more gray
hair I get(D))
 Negative correlation-the dependent variable decreases when the
independent variable decreases (the older I get (I), the less I can
remember (D). Also called inverse correlation
 Curvilinear correlation-the dependent variable curves in response
to changes in the independent variable-there is a cut off point (my
comfort in New England starts off low in January, increases through
June and gets worse in August, if temp is the dependent variable
Correlation test:
A measure of association
Years of
employment
variable
Burnout
score
variable
0-5 years
6-10 years
11-20 years
The longer you work at the agency….
30 (Low)
60 (Mid)
90 (High)
..the more likely you are to experience “burnout”
This is a positive correlation,
when one variable increases, so does the other
Correlation test:
A measure of association
Caseload
variable
0-18
19-22
23-30
The lower your caseload…
Burnout
score
variable
30 (Low)
60 (Mid)
90 (High)
..the less likely you are to experience “burnout”
This is a positive correlation too,
when one variable decreases, so does the other
Correlation test:
A measure of association
 With higher scores on job satisfaction….and lower
scores on burnout
This is an inverse/negative correlation,
when one variable increases, the other decreases
Correlation tests
 Number between -1 and 1
 0= no correlation
 Arrived at through math
 Negative numbers = a negative correlation, which can be high
(-0.9) or low (-0.1)
 Called the R score
 Numerical way of saying how related 2 things are,
mathematically
Correlation tests: Interpretation
 Magnitude/stregnth
 .9 to 1 very high correlation
.7 to .9 high correlation
.5 to .7 moderate correlation
.3 to .5 low correlation
.0 to .3 little if any correlation
Correlation tests: Interpretation
Value of r
Interpretation
r= 0
The two variables do not vary together at
all
Positive
The two variables tend to increase or
decrease together
r = 1.0
Perfect correlation – something is wrong
Negative/Inverse
One variable increases as the other
decreases
r = -1.0
Perfect negative or inverse correlation
P values
 Mathematical way of saying how probable it was
that the relationship is statistically significant (not
due to chance alone)
 0.05 is good. If p = 0.05 or less, the results are
statistically significant
 In other words, if p = more than 0.05, your results
could just be a fluke, because life is full of
uncertainty and coincidence.
Confidence intervals
 A range that tells us how confident we can be that
our sample is representative of the population we
sampled from
 Should be 95% or higher
 Always expressed as a range, i.e. (95% CI 12, 16)
 Bigger sample sizes make the numbers within the
95% confidence interval smaller (i.e. 12-16 vs 2-20)
Look for the “r” and “p” values
 Statistically
significant
association
between
variables
 p-level should
be between .05.001
 p<.05*
 p<.01**
 p<.001
Correlation: You tell me
 Is there a relationship between…
 Age and number of MD visits per year among 0-3 year
olds?
 Gender and number of MD visits per year?
 High vs. low levels of burnout scores and working at
DMH for over 20 years?
 Burnout scores and number of years with DOC?
4 basic types of statistical tests:
Description
•Mean, standard deviation
•Median, Mode
•Percentage, frequency
Correlation
•Pearson’s correlation
Comparison
•Student’s t tests
•Chi-square tests
•ANOVA
•Odds ratios
Prediction
•OLS regression
•Logit regression
t-tests
 AKA:
 Independent samples t-test
 Paired samples t-test
 Students’ t-test
 When these tests can’t be conducted due to small N,
similar tests can be used:
 (Independent) Mann-Whitney U test
 (Paired) Binomial test or Wilcoxon signed-rank test
Students’ t-test: Why use it?
 Assesses whether the means of 2 groups are
statistically different from each other
 Appropriate whenever you want to compare the
means of 2 groups
 After doing some math, you get a t-score and
significance score (p value). The t-score tells you how
different the mean of each group is from the other
group. Higher t-scores are higher differences.
 The significance score is how likely this is to be due to
chance alone; should be over 0.05
“Student’s t” test: Choices
Independent samples
 Groups are independent of
Paired samples
 Groups are paired
each other
 Each group member has a
 Individuals randomly
assigned into two groups
unique relationship with
a particular member of
the other sample
“Student’s t” test:
What are you looking for?
“Student’s t” test:
What are you looking for?
ANOVA tests (analysis of variance)

AKA:




Fisher’s test of variance
Fisher’s ANOVA
Fisher’s analysis of variance
One-way ANOVA

Like t-tests, but they compare 2
or more groups (usually used
with 3 or more)

Compare groups on a
continuous variable

Instead of a t-statistic, you get
an f-statistic and a significance
score (p-value).

F score should be higher than 2
if the means are difference,
significance score should be
over 0.05
Do all three social work units have the
same average caseload?
 Unit A
 Unit B
 Unit C
Odds ratios:
 A way of comparing whether the probability of a
certain event is the same for two groups
 Requires two groups
 Comparison of groups on a nominal variable
 Intuitive: Easy to interpret
 Easy for your audience to interpret!
Odds ratios:
What does it tell you?
 What are the odds that one group is more likely
than another to experience one condition
 Male vs. Female ex-offenders on post-incarceration
employment retention for a year or more (Y/N)?
 People with and without disabilities: Who is more
likely to access substance abuse treatment (Y/N)?
Odds ratios:
What you are looking for
 OR = 1

Condition equally likely in both groups
 OR > 1


Looks like this: OR=2.34*
Condition is more likely in the first group
 OR < 1


Looks like this: OR=0.34*
Condition is less likely in the first group
Odds ratios:
How to interpret them
OR=1.5***
1.5 times more likely…
OR=12.5***
Almost 13 times more
likely…
OR=0.50***
Fifty percent less likely…
OR=O.89***
11 percent less likely…
Non-parametric statistics
 Don’t meet the rules (parameters) to be parametric
statistics
 Still inferential
 Smaller sample size
 Chi square
Χ2 or Chi-Square Tests
 AKA:
 Chi-square goodness-of-fit test, commonly referred




to as the chi-square test
Pearson’s chi-square test
Yates’ chi-square test, also known as Yates'
correction for continuity
Mantel-Haenszel chi-square test
Linear-by-linear association chi-square test
Χ2 or Chi-Square Tests
 Compare 2 or more groups (2 is easiest)
 Commonly used with small sample sizes
 Compare groups on a nominal variable only
 Way to tell if there is a difference between groups of
observations
Assignment 3
78
The bottom line:
research is about…
 Asking questions
 Deciding how to get answers
 Thinking about what those answers mean
 Sharing all of the above with the relevant
parties – in a way “real people” can understand
Purpose
 Synthesize course concepts
 Demonstrate literature review, study design, program
evaluation skills
 Apply these skills to your area of interest/work
experience
Assumptions
 Your agency just got a grant to serve your specific
population of interest
 Based on your clinical experience and knowledge of the
population, they have asked you to recommend a SPECIFIC
intervention (i.e. wrap around services, peer support, foster
care, guardianship etc)
 In order for the grant to get reviewed, the funder is requiring
a program evaluation of the intervention (NOT the agency as
a whole). You will describe how the intervention should be
evaluated in part 4
Logistics
 Unless you hate yourself, the intervention you recommend
to the agency should be the intervention you wrote about
in assignment 2
 This assignment should incorporate my feedback from
assignments 1 and 2
 We will talk about the literature review section next week,
and do an activity about it-don’t worry about it now
Introduction
 Very condensed, focused version of assignment 1
 Highlight the fit between your agency and your
intervention-why is your agency qualified to offer this
intervention?
 Social problem the intervention is designed to address
 Scope and impact of this problem
 Research question-what is the best evidence to support
(intervention) with (population) at (agency)
Introduction
 Remember, respectful language, avoid generalization
 Incorporate my feedback from assignment 1
 Social problem (people are never the problem)
Literature review
 Take me through the literature on your intervention
and help me see why your intervention is needed
 We’ll talk next week
Recommendation
 Describe your intervention and how it should be tailored to your
population
 For this assignment, assume that the agency does not yet offer
your intervention. In real life, it might offer the intervention
already, but ignore it. I want you to focus on how YOU think it
should be implemented, given what you know about your
population
 In other words, this is where you demonstrate your ability to use
evidence based practice-how will you integrate research evidence,
your clinical experience/knowledge and client preferences and
needs?
86
EBPs in social work
Program evaluation
 As part of the process, you will design a way of evaluating
your intervention with your population
 Note that you will not actually conduct this study (though
save yourself some work next semester and think about a
study you might want to do in the fall & spring courses!)
 Note that you are evaluating your recommended program
at your agency, not proving whether or not an
intervention in general works (scope)
 Design a feasible, practical study
Types of research projects
Research
Qualitative
Program evaluation
Process/formative
Quantitative
Outcome/summative
How to pick a study design
 Study design is guided by your research question:
what is the best way to answer this particular
question with the least amount of bias?
 Funding considerations: what is practical and
feasible? Do you have the resources you need for
“gold standard” designs? If not, what is the most
feasible, highest quality Plan B?
Funnel process
Area of interest
Existing knowledge/
theory
Problem area
Research question
Specific aims, hypotheses
Methods
43
Group work
 Describe your population and intervention
 Identify the social problem(s) that your intervention
addresses
 Brainstorm evaluation research questions
 Share your question with me
 Brainstorm means of answering your question(s)
(study design)
Next week
 You’ll get feedback on assignment 2
 Literature review exercise
 Continue study design work
 Multivariate statistics-lots of candy
 Wheelan, C. (2015) Chapter 11: Regression Analysis
– The Miracle Elixir. Naked Statistics: Stripping the
Dread from the Data. New York: W. W. Norton.