Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Research Design and Validity Threats
Document related concepts
Transcript
Research Design and Validity Threats Evaluation Designs • Quantitative versus Qualitative • Combination Categories of Research Designs • Nonexperimental or Pre-experimental designs • One group, little validity control • Quasi-experimental • Experimental and comparison group, but no random assignment or selection • Experimental • Random assignment of experimental and control groups Terminology • Internal validity • Extent to which an observed outcome can be attributed to a planned intervention • External validity • Extent to which an observed outcome can be attributed to a replicable intervention and generalized to other settings and populations Internal validity threats • History • An event that occurs during the intervention that could have an impact on the results • Maturation • Bias from biological, natural, or social events that can bias results Internal validity threats • Testing • Testing might cue a person in to change behavior, regardless of the program • Instrumentation • Bias in data collection instruments Internal validity threats • Statistical Regression • Bias from selecting a group with unusually high or low scores on something • Selection • Comparison groups are unequal Internal validity threats • Attrition/subject mortality • Dropouts of subjects; if there is more than one group, then unequal dropouts between groups • Interactive effects • Combinations of the above Other internal validity issues • Diffusion • Contamination of comparison condition or intervention condition • Demoralization • Subjects upset they are not receiving the other condition External Validity • Threats • • • • • Social desirability Expectancy effect Hawthorne effect Placebo effect Novelty effect Research Designs • Key to abbreviations: O = data collection X = treatment/intervention R = random assignment Solid line separating groups – equal groups Dashed line separating groups – unequal groups Pre-experimental designs • One group, pretest, post-test O X O • Good for pilot testing • Does not control for IV threats • Be certain to use valid and reliable instruments Pre-experimental designs • One shot case study X O • No control for validity threats, no pretest measures Quasi-experimental designs • Nonequivalent comparison group O X O ----O O • A comparison group is added, but they are not equal Quasi-experimental designs • Time series O O O O X O O O O • Several measures to assess if there is a trend • No comparison group Quasi-experimental designs • Multiple Time series O O O O X O O O O -----------------O O O O O O O O • Added a comparison group Experimental Designs • Pre-test, post-test, control group design R O X O R O O • Randomly assigned to groups Experimental Designs • post-test only, control group design R X O R O • Randomly assigned to groups Experimental Designs • Solomon Four Group Design R O X O R O O R X O R O Why do you care? • Having this knowledge will help you determine the quality of research studies, which will impact your conclusions regarding the results.
