Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
SWK 707 Research for Social Work Practice Nechama Sammet Moring CLASS 8 Tonight’s Plan 6:00-6:10 check in, logistics 6:10-7:00 qualitative coding lecture and activity 7:00-7:10 review descriptive statistics 7:10-7:30 inferential statistics part 1 7:30-7:40 break 7:40-8:00 inferential statistics part 2 8:00-8:10 research question and study design lecture 8:10-8:55 assignment 3 group work 8:55 evals Strengths Qualitative Research Used in exploratory research to examine a question or construct about which little is known Useful in formative stages of research to gain a better understanding of cultural variations, appropriate language, and meaningful categories Open-ended questions can provide context and depth to questions of interest Documents individuals’ responses in own words, thoughts, phrases Process of QL research A question arises Re-consideration of theory, literature from findings Grounding in theory, literature – provide justification Hone question, aims - design, sample, few questions chosen with care Ongoing Data collection, analysis – redevelopment of questions - simultaneous Qualities Important for Qualitative Research Good conceptual understanding of the goals of research Ability to develop rapport with participants Ability to listen and absorb information Empathy Intuitive sense to build conversation Ability to analyze, synthesize and utilize information immediately (ie your interviewee says something unexpected) Reflexivity-sense of yourself, your own opinions, experiences, background and life experiences, willingness to acknowledge your opinion as simply one of many, formed as a result of your life experience (i.e. my disability research is different than someone else’s because I am a family member, a woman, able-bodied etc) Qualitative Research Techniques Interviews Informal Unstructured Semi-structured/guided Formal/structured Focus groups http://www.huffingtonpost.com/2014/09/22/pets-are- people-too_n_5842444.html Observation: Participant observation Unobtrusive observation Semi-structured Interviews Based on a set of questions derived from theory, previous research, and experience Interview guides have specific topics (constructs) to cover, using specific questions as prompts Provides minimally directive framework that enables both researcher and participant to define questions and generate new ideas partnership Interview tips Do your homework beforehand! Get a sense of your interviewee’s priorities and be prepared to address these topics Begin with general questions to establish rapport and move to more specific/more controversial questions List constructs (concepts of importance) and develop questions that address these constructs. Be prepared to address the construct in more than one way. Construct: Impact of dog ownership on quality of life Questions: How does dog ownership impact your quality of life? What are some of the pros of having a dog? What are some of the ways that having a dog impacts you negatively? How has having a dog changed your quality of life? More interview tips Practice beforehand-interview someone else using your guide and ask for honest feedback re: clarity, flow etc. Open-ended (as opposed to close-ended) questions allow participants to express and explore their ideas. Close-ended questions have more potential to influence responses. (“What are some of the impacts of poverty on young mothers?” Answer: talk, talk, talk vs. “does poverty impact young mothers?” Answer: Yes.) How, why, what, describe…. Redirects and prompts in interviews The fine art of re-directs: I see, so you (their thing). Was that where/when you first (your thing)? I want to be conscious of your time, and I have just a few more questions about (your thing) The fine art of prompts: Tell me more. Ummhmmm How so? Can you expand on that? Can you give me an example? Approach often determines data analysis Inductive-start with themes or conceptual model already identified (usually from lit search or similar) and use data to explore the model Content analysis Deductive-build theory from the data itself Grounded, coded theory Theme identification in qualitative research Importance of distinguishing between qualitative and quantitative Theme identification vs. concept measurement Beyond counting… Letting the themes “rise up” from the data Letting the collective voice of study respondents define what the themes are in answer to questions Can be confusing – demographic data sometimes included as qualitative 12 Sample coding sheet Code Code 1 Code 2 Code 3 Demographic data Interview 1 Interview 2 Interview 3 Interview 4 Coding process • Codes: Identifying anchors - key points of the data gathered • Themes: Collections of codes of similar content – allows data to be grouped • Second level themes: Broad groups of similar concepts used to generate a theory or relate back On inductive, deductive research Deductive Inductive Inductive vs deductive coding Deductive-starts with theory, code book already built from the literature, your theory; content analysis Inductive:-starts with observation, code book and theory is built from the transcripts, grounded theory Interpreting qualitative data 1. Become saturated 2. Look for patterns – constant comparison 3. Corroborate/legitimate themes 4. Represent the accounts accurately via checking back 1. 2. 3. Self Team Member checking Pure qualitative data analysis • Become grounded in the data • “Open coding” • • • • Express data/phenomenon in concept form Categorize concepts that are relevant to question Allow understanding to emerge from close study of the data • “Axial coding” for subcategories or larger themeing Discover patterns, themes • “In-vivo” coding How to begin the coding process 1. Read memos, interviews, other (no notes) 2. Read memos, interviews, other (make notes) 3. Re-read (look for themes within transcript) 4. Re-read (look for themes across transcripts) In grounded theory: Sensitizing concepts Review your thoughts with your team Do you agree on themes in transcripts? Do you agree on themes across transcripts How do these themes connect? Let’s try it! Content analysis Finding themes, commonalities in the first encounter “bucket” Code book pre-built from my literature search I totally did Code alone, then compare and discuss with a partner Discuss content analysis coding What did we learn? What did we miss? What about outliers? Let’s try it again: grounded coding This time, you have a complete interview, and we don’t have a code book pre-developed Read your transcript once through On your second read, make notes about codes Working with a partner, group each of your codes into themes to create a code book What is important? Ok to have a “wild card” theme Discuss grounded theory What did we learn? What did we miss? Outliers? Making qualitative data more rigorous (free of bias; scientific) Inter-coder reliability and ways to check this (group coding, 2+ people code all transcripts and discuss; 1 person codes most with a second person to “spot check”) Good decisions in purposive sampling Triangulation of data (i.e. multiple data sources and data collection methods yield the same findings) Respondent verification (people you interviewed read and review your work) Saturation (the point at which new themes stop emerging) and coding a few past saturation Positionality and reflexivity (self-awareness; knowing your “blind spots” and designing your research to check your own blind spots) Use of software programs Nvivo, Atlas-ti Document loading, linkage Open coding, axial coding, In vivo coding What is different with the software? Assign categories to transcripts – comparisons Word count, auto-code features (slippery slope) Pull documents together by theme/filter Drop-down codebook Code by recording location Mapping of code relationships 4 basic types of statistical tests: Description •Mean, standard deviation •Median, Mode •Percentage, frequency Correlation •Pearson’s correlation Comparison •Student’s t tests •Chi-square tests •ANOVA •Odds ratios Prediction •OLS regression •Logit regression Descriptive statistics review: Why? Need to present data aggregate rather than describing each data point to make sense of large amounts of data Find and describe patterns Shows what is typical and what outliers exist-extent and range Does not speak to relationships between variables, just describes individual variable, rather than their relationship to each other Demographic information Cheat sheet: Descriptive statistics Type: Used for: Frequency Counting Mean/average, median or mode Measure of central tendency Variable structure: Any Continuous Review of kinds of variables Dichotomous-one of 2 options (i.e. yes/no) Nominal-mutually exclusive categories (zombie, human, sheep, elephant) Ordinal-mutually exclusive categories that go in order (1st, 2nd, 3rd) Continuous (interval)-rank ordered, mutually exclusive and there is the same amount of difference between each variable, like height, 5, 5’1, 5’2 etc) Ratio-continuous but with a fixed 0 (number of kids, 0- 19) Dependent and Independent Variables Quant research is about demonstrating relationships between variables Independent variable just exists. (I am either undead or not) Dependent variable is influenced by the independent variable (whether or not I’m undead has a relationship to if I eat brains or not) A change in the independent variable leads to a change in the dependent variable What is a frequency measure and what do you do with it? Counts How often something occurs Numbers of people/places/things with: a certain characteristic a certain set or combination of characteristics How do you report these results in a way that is accessible to the reader? Percentages Mean (SD), median, mode Measures of central tendency Mean=mathematical average (add up all scores, divide by number of scores) median = the score in the exact middle of all the scores, the midpoint Mode=the score that occurs the most; often used with nominal variables (i.e. more Catholics than Protestants in my sample, because you can’t really come up with the average religion) Outlier-something out in left field that can really skew the mean (but not so much the median or the mode) Median and mode Sometimes the mean is skewed If so, using one of these makes more sense: Median the value below which 50% of the scores fall, or the middle score Median age at which people first had a counseling experience Mode the most frequent score Most common age at which people first had a counseling experience Discussion What kind of information do we get from each measure of central tendency? When might each be important? What kind of research designs lead to each? Ways to measure dispersion of data Range (from what to what), the overall spread of data from lowest to highest Variance- a statistical measure (we won’t be doing the math) that gives a “score” based on how far apart the data is from each other. Standard deviation-math calculated more precise way of finding out the “average” difference from the mean score, calculated with the square root of the variance Table 1: Demographics of Elders with IDD/SA Sociobehavioral model Predisposing Characteristics Enabling resources Need factors ***p<.001 *p<.05 Variable IDD/SA (N=350) NoIDD/SA (N=48,014) Test Gender (male) 238 (68%) 25,064 (52%) OR=1.9*** Mean age (SD) 70 (5) 73 (7) t=4.5* Race (white) 260 (74%) 28,741 (59%) OR=1.9*** (SSI/SSDI) 199 (57%) 29,131 (61%) NS Dually eligible 303 (87%) 43,226 (90%) OR=0.7* FFS coverage 262 (75%) 34,283 (71%) NS Low state SA coverage 141 (40%) 16,899 (35%) OR= OR= 1.2* Urban location 213 (61%) 30,027 (63%) NS SMI diagnosis 151 (43%) 7,540 (16%) OR= 4.1*** Long-term SA diagnosis 19 (5%) 5,549 (12%) OR= 0.4*** Standard deviation: Most commonly used measure of dispersion around a mean how “spread out” are the values? Always reported together – otherwise considered to be biased Can’t be done with nominal variables Standard deviation in a normally distributed sample •Dark blue < 1 “standard deviation” from the mean •Accounts for 68.3% Variance Variance- a statistical measure (we won’t be doing the math) that gives a “score” based on how far apart the data is from each other. Small/low variance score means that all data points are close together and don’t vary much. Scores are clumped around the mean (leptokurtosis-narrow and pointy) Large/high variance score means that data points have a lot of variation from each other. Scores differ from the mean (platykurtosis-flat & wide) Rodriquez & Murphy Rodriguez & Murphy: Frequencies? Ranges? Break 4 basic types of statistical tests: Description •Mean, standard deviation •Median, Mode •Percentage, frequency Correlation •Pearson’s correlation Comparison •Student’s t tests •Chi-square tests •ANOVA •Odds ratios Prediction •OLS regression •Logit regression Review: Variable structure Continuous (a.k.a. numeric) Examples? Nominal (a.k.a. dummy variable, categorical variables) Examples? Parametric statistics Kind of bivariate analysis (finding the relationship between 2 variables-the independent variable and the dependent variable) There are rules (parameters) that must be met, including a large enough sample size Correlation: Pearson’s r, also called Pearson’s correlation, or just correlation Comparison: t-tests, ANOVA Prediction: regression (we’ll talk about this next week) Measures of association Ways to measure the correlation (relationship) between 2 variables (i.e untreated chronic disease and mental illness) How change in the dependent variable is related to change in the independent variable (so if I increase the independent variable by X%, how much will the dependent variable change?) Correlation tests Measures association/relationship between 2 continuous variables Does not measure causation Distinguish vernacular usage of the term from statistical usage Requires a logic model/theory/research-based idea 4 kinds of correlation No correlation-there is no relationship (i.e your program is ineffective) Positive correlation-the dependent variable increases when the independent variable increases (the older I get (I), the more gray hair I get(D)) Negative correlation-the dependent variable decreases when the independent variable decreases (the older I get (I), the less I can remember (D). Also called inverse correlation Curvilinear correlation-the dependent variable curves in response to changes in the independent variable-there is a cut off point (my comfort in New England starts off low in January, increases through June and gets worse in August, if temp is the dependent variable Correlation test: A measure of association Years of employment variable Burnout score variable 0-5 years 6-10 years 11-20 years The longer you work at the agency…. 30 (Low) 60 (Mid) 90 (High) ..the more likely you are to experience “burnout” This is a positive correlation, when one variable increases, so does the other Correlation test: A measure of association Caseload variable 0-18 19-22 23-30 The lower your caseload… Burnout score variable 30 (Low) 60 (Mid) 90 (High) ..the less likely you are to experience “burnout” This is a positive correlation too, when one variable decreases, so does the other Correlation test: A measure of association With higher scores on job satisfaction….and lower scores on burnout This is an inverse/negative correlation, when one variable increases, the other decreases Correlation tests Number between -1 and 1 0= no correlation Arrived at through math Negative numbers = a negative correlation, which can be high (-0.9) or low (-0.1) Called the R score Numerical way of saying how related 2 things are, mathematically Correlation tests: Interpretation Magnitude/stregnth .9 to 1 very high correlation .7 to .9 high correlation .5 to .7 moderate correlation .3 to .5 low correlation .0 to .3 little if any correlation Correlation tests: Interpretation Value of r Interpretation r= 0 The two variables do not vary together at all Positive The two variables tend to increase or decrease together r = 1.0 Perfect correlation – something is wrong Negative/Inverse One variable increases as the other decreases r = -1.0 Perfect negative or inverse correlation P values Mathematical way of saying how probable it was that the relationship is statistically significant (not due to chance alone) 0.05 is good. If p = 0.05 or less, the results are statistically significant In other words, if p = more than 0.05, your results could just be a fluke, because life is full of uncertainty and coincidence. Confidence intervals A range that tells us how confident we can be that our sample is representative of the population we sampled from Should be 95% or higher Always expressed as a range, i.e. (95% CI 12, 16) Bigger sample sizes make the numbers within the 95% confidence interval smaller (i.e. 12-16 vs 2-20) Look for the “r” and “p” values Statistically significant association between variables p-level should be between .05.001 p<.05* p<.01** p<.001 Correlation: You tell me Is there a relationship between… Age and number of MD visits per year among 0-3 year olds? Gender and number of MD visits per year? High vs. low levels of burnout scores and working at DMH for over 20 years? Burnout scores and number of years with DOC? 4 basic types of statistical tests: Description •Mean, standard deviation •Median, Mode •Percentage, frequency Correlation •Pearson’s correlation Comparison •Student’s t tests •Chi-square tests •ANOVA •Odds ratios Prediction •OLS regression •Logit regression t-tests AKA: Independent samples t-test Paired samples t-test Students’ t-test When these tests can’t be conducted due to small N, similar tests can be used: (Independent) Mann-Whitney U test (Paired) Binomial test or Wilcoxon signed-rank test Students’ t-test: Why use it? Assesses whether the means of 2 groups are statistically different from each other Appropriate whenever you want to compare the means of 2 groups After doing some math, you get a t-score and significance score (p value). The t-score tells you how different the mean of each group is from the other group. Higher t-scores are higher differences. The significance score is how likely this is to be due to chance alone; should be over 0.05 “Student’s t” test: Choices Independent samples Groups are independent of Paired samples Groups are paired each other Each group member has a Individuals randomly assigned into two groups unique relationship with a particular member of the other sample “Student’s t” test: What are you looking for? “Student’s t” test: What are you looking for? ANOVA tests (analysis of variance) AKA: Fisher’s test of variance Fisher’s ANOVA Fisher’s analysis of variance One-way ANOVA Like t-tests, but they compare 2 or more groups (usually used with 3 or more) Compare groups on a continuous variable Instead of a t-statistic, you get an f-statistic and a significance score (p-value). F score should be higher than 2 if the means are difference, significance score should be over 0.05 Do all three social work units have the same average caseload? Unit A Unit B Unit C Odds ratios: A way of comparing whether the probability of a certain event is the same for two groups Requires two groups Comparison of groups on a nominal variable Intuitive: Easy to interpret Easy for your audience to interpret! Odds ratios: What does it tell you? What are the odds that one group is more likely than another to experience one condition Male vs. Female ex-offenders on post-incarceration employment retention for a year or more (Y/N)? People with and without disabilities: Who is more likely to access substance abuse treatment (Y/N)? Odds ratios: What you are looking for OR = 1 Condition equally likely in both groups OR > 1 Looks like this: OR=2.34* Condition is more likely in the first group OR < 1 Looks like this: OR=0.34* Condition is less likely in the first group Odds ratios: How to interpret them OR=1.5*** 1.5 times more likely… OR=12.5*** Almost 13 times more likely… OR=0.50*** Fifty percent less likely… OR=O.89*** 11 percent less likely… Non-parametric statistics Don’t meet the rules (parameters) to be parametric statistics Still inferential Smaller sample size Chi square Χ2 or Chi-Square Tests AKA: Chi-square goodness-of-fit test, commonly referred to as the chi-square test Pearson’s chi-square test Yates’ chi-square test, also known as Yates' correction for continuity Mantel-Haenszel chi-square test Linear-by-linear association chi-square test Χ2 or Chi-Square Tests Compare 2 or more groups (2 is easiest) Commonly used with small sample sizes Compare groups on a nominal variable only Way to tell if there is a difference between groups of observations Assignment 3 78 The bottom line: research is about… Asking questions Deciding how to get answers Thinking about what those answers mean Sharing all of the above with the relevant parties – in a way “real people” can understand Purpose Synthesize course concepts Demonstrate literature review, study design, program evaluation skills Apply these skills to your area of interest/work experience Assumptions Your agency just got a grant to serve your specific population of interest Based on your clinical experience and knowledge of the population, they have asked you to recommend a SPECIFIC intervention (i.e. wrap around services, peer support, foster care, guardianship etc) In order for the grant to get reviewed, the funder is requiring a program evaluation of the intervention (NOT the agency as a whole). You will describe how the intervention should be evaluated in part 4 Logistics Unless you hate yourself, the intervention you recommend to the agency should be the intervention you wrote about in assignment 2 This assignment should incorporate my feedback from assignments 1 and 2 We will talk about the literature review section next week, and do an activity about it-don’t worry about it now Introduction Very condensed, focused version of assignment 1 Highlight the fit between your agency and your intervention-why is your agency qualified to offer this intervention? Social problem the intervention is designed to address Scope and impact of this problem Research question-what is the best evidence to support (intervention) with (population) at (agency) Introduction Remember, respectful language, avoid generalization Incorporate my feedback from assignment 1 Social problem (people are never the problem) Literature review Take me through the literature on your intervention and help me see why your intervention is needed We’ll talk next week Recommendation Describe your intervention and how it should be tailored to your population For this assignment, assume that the agency does not yet offer your intervention. In real life, it might offer the intervention already, but ignore it. I want you to focus on how YOU think it should be implemented, given what you know about your population In other words, this is where you demonstrate your ability to use evidence based practice-how will you integrate research evidence, your clinical experience/knowledge and client preferences and needs? 86 EBPs in social work Program evaluation As part of the process, you will design a way of evaluating your intervention with your population Note that you will not actually conduct this study (though save yourself some work next semester and think about a study you might want to do in the fall & spring courses!) Note that you are evaluating your recommended program at your agency, not proving whether or not an intervention in general works (scope) Design a feasible, practical study Types of research projects Research Qualitative Program evaluation Process/formative Quantitative Outcome/summative How to pick a study design Study design is guided by your research question: what is the best way to answer this particular question with the least amount of bias? Funding considerations: what is practical and feasible? Do you have the resources you need for “gold standard” designs? If not, what is the most feasible, highest quality Plan B? Funnel process Area of interest Existing knowledge/ theory Problem area Research question Specific aims, hypotheses Methods 43 Group work Describe your population and intervention Identify the social problem(s) that your intervention addresses Brainstorm evaluation research questions Share your question with me Brainstorm means of answering your question(s) (study design) Next week You’ll get feedback on assignment 2 Literature review exercise Continue study design work Multivariate statistics-lots of candy Wheelan, C. (2015) Chapter 11: Regression Analysis – The Miracle Elixir. Naked Statistics: Stripping the Dread from the Data. New York: W. W. Norton.