Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Research Methods Review
Document related concepts
Transcript
AP Psychology Research Methods Review Research Design! Types of Research Design Correlational The variables are only measured Some relationships can only be investigated using correlational designs due to practicality or ethical concerns Cannot imply causality Examples include: Naturalistic Observation, Surveys, Case Studies, and Archival Analysis Experimental The independent variable is manipulated in the experiment Subjects/Participants are randomly assigned to conditions Experimental Group (receives manipulation) Control Group (no manipulation) Experimental or Correlational? Experimental – you must be able to MANIPULATE the independent variable within ethical and practical boundaries. Experiment consists of comparing 2 or more groups, at least one of which receives the manipulation and one standard control. Use statistics that compare the groups on the outcome variable Assignment to either group is RANDOM You can manipulate more than one variable/dimension if you want! T-test ANOVA Correlational – you only measure the two (or more) variables in question Participants are measured on these domains either all at once or over time (longitudinally) Uses statistics that measure the direction, intensity, and strength of the relationship between variables Correlation = relationship -1<r<1 Regression = predictive a + bx = y Experimental Variables Independent Variable: variable that is manipulated by the experimenter. Dependent Variable: variable that is measured – the outcome variable. The variables must have clear operational definitions How do you define aggression? Intelligence? Operational Definitions help researchers REPLICATE experiments Increases RELIABILITY Casual Relationships Causal Relationships: Causal relationships can only be determined in controlled experiments Requirements to imply causation: The cause (independent variable) must precede the effect (dependent variable) in time The cause and effect covarry (they have a predictable relationship) The relationship between the variables cannot be explained by another third or confounding variable Three Core Requirements for Research Reliability ◦ Validity ◦ The consistency of data resulting from psychological testing or experimental research The information produced by the research accurately reflects the variables that were intended to manipulate or measure Standardization ◦ a set of uniform procedures for treating each participant in a test, interview, or experiment. ◦ Standard Operating Procedures (SOP’s) need to be followed to ensure each participant/client is treated the same ◦ If standardization is not followed, confounding variables may influence and invalidate the results. Reliability Test-retest Reliability If you test the same person a second time, do they get the same score? Internal Consistency The degree to which items on a test are correlated with each other Some items should be positively correlated, some negatively correlated If you split the test into two equal parts, do you get the same result for each part? Inter-rater Reliability Do multiple raters score the same # of behaviors? Operational Definition Boxing (Pacquiao v. Bradley) Validity (of the experiment) Internal Validity Is the effect due to the experimental manipulation? Within the experiment and those participants involved, can the effect be said to ONLY be the result of the Independent Variable External Validity Population External Validity Can the effect be generalized to individuals not involved in the experiment? What similarities might be important for the population to share with the original participants in order for the effect to be the same? Ecological Validity To what extent does the experiment translate “in the wild”? Validity: Efficacy vs. Effectiveness vs. Efficiency Efficacy: under controlled circumstances, does the intervention produced the desired effect? Effectiveness: under real-world circumstances, does the intervention produced the desired effect? Efficiency: is the treatment worth it’s cost to the individual or society? Validity (of measures) Face Validity Does it look like it measures what it’s supposed to? Content Validity Does the test cover all aspects of the construct? Concurrent Validity An IQ test that only tests math is not a valid IQ test Does the measure correlate with other known measures of the same construct? Predictive Validity How well does the measure predict other measures of the same construct over time? An IQ test should predict future intellectual success Standardization Standardization: a set of uniform procedures for treating each participant in a test, interview, or experiment. Standard Operating Procedures (SOP’s) need to be followed to ensure each participant/client is treated the same If standardization is not followed, confounding variables may influence and invalidate the results. Threats to Experimental Integrity Threats to Internal Validity Mortality Regression toward the mean Selection Maturation Instrumentation Testing History Compensatory rivalry/Resentful demoralization Experimenter Bias Threats to Experimental Integrity Threats to External Validity Placebo Effect Situational Constraints Sample Homogeneity Artificiality Demand Characteristics The “good” participant The “bad” participant The “faithful” participant The “apprehensive” participant Random Stuff Random Sampling Having access to the entire population and randomly picking individuals from that population Allows for generalization of results to the entire population If the sample is large enough, it should be representative of the entire population Random Assignment Randomly assigning the sample of subjects into experimental/control conditions Ensures the groups are roughly the same at the begining Blind & Double-Blind Blind Design: Participants do not know what condition they are in “Farce” or “sham” treatments are used Reduces demand characteristics Double-Blind Design: Neither the participants nor the research assistants (who have contact with the participants) know which condition the participant is in. Eliminates the self-fulfilling prophecy Sometimes not feasible Quasi-Experimental Quasi-Experimental Designs Similar to a traditional experiment, but without random assignment of participants to the experimental/control groups The independent variable is still manipulated, but the groups were purposefully chosen by the experimenter Vulnerable to confounding variables Practical or ethical restrictions prohibit the use of a “true experiment” Education / at risk youth Drug Trials Quantitative or Qualitative? Quantitative: Numerical in nature Lends easily to statistical analysis Feels more “science-y” Qualitative: Observations that cannot easily be given a numerical value Creates a narrative/story Provides insight into the research question, generates hypotheses For future quantitative research Uncovers trends Typically can only be done with a few participants Psychological Research Methods Between-Subjects design Separate groups of people are assigned different treatments Within-Subjects Design The same people experience the different treatments and are re-tested Control Group vs. Experimental Group Before vs. After Matched-Pairs Design Pairs of subjects (or parts of subjects) are separated and given different treatments Twin Studies Change over Time? Longitudinal Investigates the same individuals over time Vulnerable to a variety of threats to internal validity Can imply developmental trajectories Can be long (obviously) and expensive Cross-Sectional Investigates different cohorts of individuals varying in age Cohort Effects Cannot directly imply developmental trajectory Generally more practical, time- and cost-effective Sequential Mixes elements of Longitudinal & Cross-Sectional Designs Has benefits and vulnerabilities of both Ethics! Research with living things Humans = Participants Animals = Subjects Human research is governed at the university level by the Institutional Review Board (IRB) Animal research is governed at the university level by the Institutional Animal Care and Use Committee (IACUC) Animal research is also governed by various federal entities (USDA, FDA, NSF) Principles of Research with Human Participants “Psychologists strive to benefit those with whom they work and take care to do no harm…Psychologists respect the dignity and worth of all people, and the rights of individuals to privacy, confidentiality, and self-determination.” (Ethical Principles of Psychologists and Code of Conduct, APA) Principles of Research with Human Participants Risk/Benefit analysis: comparison of the potential benefit (both to the individual and the population) and the risk of harm to the participant Only if the benefit outweighs the risk can a study be considered ethical The allowable risk associated for a study investigating a cure for cancer is higher than a study looking at productivity in the workplace If more than ‘minimal risk’ is involved in a study, increased scrutiny is placed on the investigator to justify their method Principles of Research with Human Participants • Informed consent: the participant must be aware of what they will be doing, what risk may be involved, and their rights as a participant. ▫ The participant must be 18 years or older and be able to give consent. ▫ Informed Assent is given to minors or special populations, along with the consent of their legal guardian Some level of deception is required in most studies, but that’s OK Principles of Research with Human Participants Freedom to withdrawal at any time Without penalty Confidentiality All participant information must be kept confidential Exceptions: Harm to self, Harm to others, Neglect, Potential victimization Debriefing and protection from harm Since most experiments require from deception at the beginning, participants must be told the true purpose of the study after they have completed the experiment Principles of Research with Animal Subjects • Risk/Benefit analysis is also carried out with animal subjects – but researchers aren’t as concerned about the benefit to the individual animal • Ethical research with animals strives to: ▫ Reduce: use as few animals as possible to get the desired information ▫ Replace: replace sentient animal models with alternatives, such as tissue analysis and computer models ▫ Refine: the use of methods to minimize pain, suffering, and distress. Also includes the use of enrichment to enhance animal welfare Principles of Research with Animal Subjects Issues arise with the degree to which information from animal models can be generalized to humans. However, many studies could ONLY be ethically conducted with animals Prenatal Studies Drug Trials Basic physiological and behavioral processes are indistinguishable across species Statistics! Statistics Supplement Descriptive Statistics: Only focus on describing the participants which have been observed Measures of central tendency Mean Median Mode Variability Range Standard Deviation Statistics Supplement Inferential statistics Determine whether a sample of data is due to chance responding or due to a meaningful trend Can compare two or more groups Can compare a group to a known norm Can compare longitudinal data from the same group The Normal Distribution Statistics Supplement Statistics Supplement Many things are normally distributed: Intelligence Scores [(Mental Age) / (Chronological Age)] x 100 = IQ Mean IQ = 100 Std. dev = 15 IQ over 130 is exceptionally smart IQ under 70 is one criteria for Intellectual Disability The Normal Distribution Z-scores are used to find out where a person stands in reference to the population Percentiles: uses the normal distribution and the z-score to identify where a particular score is compared with the population. If someone is at the 50th percentile, they are exactly in the middle If someone is at the 30th percentile, they have 30% below them and 70% above them If someone is at the 90th percentile, they have scored above 90% of the population Statistics Supplement SAT v. ACT SAT Mean = 1026 SD = 209 ACT Mean = 20.8 SD = 4.8 Which is better, getting a 1277 on the SAT or a 28 on the ACT? Common Statistical Tests T-test Used when comparing 1 group mean to a known [population] value Used when comparing 2 group means against each other One-way ANOVA Used when comparing the means 3 or more groups than vary on 1 dimension Two-way ANOVA Used when comparing the means of groups that vary on 2 dimensions simultaneously Used to investigate the interaction between two factors Chi-Square Test Used when comparing the categorical outcomes between two or more groups Correlation, Pearson’s R Used to look at the relationship between two numerical variables Significant Results(?) Null Hypothesis Significance Testing (NHST) H0: There is no effect of the treatment Ha: There is an effect of the treatment Assumes that the null hypothesis is true Given this assumption the sample distribution would have predictable characteristics NHST tests to see how unlikely it would be to obtain a particular sample, given this assumed distribution An extreme score would represent a statistically unlikely event Typically, p-values less than .05 indicate a significant difference between groups A more stringent or relaxed level of significance may be appropriate Significant Results(?) Null Hypothesis Significance Testing will tell you how improbable getting a particular sample is, under the assumption that there are no differences between groups NHST does not tell you the Effect Size, or whether or not these differences actually translate into something meaningful NHST is sensitive to large sample sizes – the larger the sample size the more likely it is to detect a difference (Power) However these differences may not translate into something meaningful. Power Power is the ability of a given study or statistical test to detect a relationship that exists in the population. i.e. the ability to correctly reject the null hypothesis Power is influenced by a variety of factors: Sample size (bigger n = more power) Actual Effect Size (effect size in the population) Reliability and Validity of measures You want measures that are accurate and precise Using a between subjects or a within-subjects design Statistics Supplement Statistical Significance Statistical Significance is achieved when the probability of getting a specific set of data by chance is extremely slim Typically, this probability is less than .05 or 5% IF the groups were the same, THEN the probability of getting that sample by chance would be very unlikely. Therefore, the researcher concludes that the groups are (probably) not the same. Because of the way statistics tests work, a researcher can never “prove” anything. They can only demonstrate how probable or improbable a certain event is. If someone claims they have “proven” anything, don’t trust them. Statistics Supplement Type I Error: False Positive The researcher incorrectly determines that there is an effect when in reality none exists The probability of a Type I error is α α is determined by the researcher based on how bad implications for a false positive would be Type II Error: False Negative The researcher incorrectly determines that there is not effect when in reality there is an effect The probability of a Type II error is β β can be reduced by increasing Power Statistics Supplement Correlation & Regression Correlation Coefficient R describes the strength and direction of the relationship between two observed variables -1≤ r ≤ 1 -1 being a perfect negative correlation +1 being a perfect positive correlation 0 represents no relationship Linear Regression equations draw an imaginary line through the data cluster; the slope and intercept of the line is used to predict future values based on previous data Expressed as y = mx + b Statistics Supplement Correlation & Regression Graphs! Graphs are an excellent way to convey a lot of information efficiently Bar graphs summarizing group means or proportions Separate bars for each group Used for comparing groups in an Experiment Scatter plots Dots for each case/individual with 2 measures (x, y) Used for Correlational Designs Line graphs demonstrating longitudinal performance Separate lines for each group, connecting dots at each time point For Longitudinal Designs
