Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
PSYC512: Research Methods Lecture 7 Brian P. Dyre University of Idaho PSYC512: Research Methods Lecture 7 Outline Questions about material covered in Lecture 6 Measures: scales and sensitivity More on Measurement Reliability, Precision, and Validity Hypothesis testing and Variables Variables and Research Design Defining Variables PSYC512: Research Methods Features of Measures: Reliability The ability of a measure to produce consistent results when repeated measurements are taken under identical conditions Types: precision: physical measurement (1/noise) margin of error: sampling in surveys interrater reliability: observers viewing the same behavior Test-retest, parallel forms and split-half reliabilities: psychological tests PSYC512: Research Methods Other Features of Measures Accuracy does a measure produce results that agree with a known standard? Accuracy vs. Precision Validity Measurement validity: the extent to which your measure indeed measures what it is intended to measure Types: Face validity, Content validity, Criterionrelated validity (concurrent vs. predictive), Construct validity Relationship between reliability and validity PSYC512: Research Methods Hypothesis Testing: Variables Hypothesis testing is the process by which hypothetical relationships between variables (something that varies in quantity or quality) are assessed (the relationships are deduced from one or more theories) Types of variables Dependent variable measure Independent variable manipulation Extraneous variable not pertinent to hypotheses Confounding variable extraneous variable that covaries with your manipulated variable (typically we try to control these to eliminate the covariance) Intervening variable theoretical construct of interest that is not directly observable (e.g., group cohesiveness, mental workload) PSYC512: Research Methods Variables and Research Designs Relationships can be hypothesized between Multiple dependent measures correlational research design: presence or absence of a relation between the variables can be tested, but not causality Manipulated (independent) variables and some measure experimental design, with proper control of confounding variables (e.g., random assignment to experimental treatment groups) causality may be inferred PSYC512: Research Methods Defining Variables: Operationism Operationism: psychological concepts are equivalent to the operations (manipulations or measures) used to define those concepts Hunger: the state produced by food deprivation Only observable operations are included in theoretical or hypothetical statements You cannot separate the concept from its operations—cannot generalize, concept has no external validity PSYC512: Research Methods Defining Variables: Converging Operations or Network Specification Multiple operations or a set of operations can be used to define a concept, not just one Operations can converge to scientifically isolate intervening variables through a process of converging operations (Garner, Hake, & Eriksen, 1956) selective influence – experimental manipulations affect particular intervening variables but not others convergence – different operations can be used to manipulate or measure a common intervening variable or psychological construct PSYC512: Research Methods Converging Operations Example: The phenomenon of “Perceptual” Defense (Garner, Hake, & Eriksen, 1956) Two Possibilities perceptual discrimination of vulgar words takes longer responding with a vulgar word takes longer Operationist: perception is the discrimination response, therefore, we can’t tell which Converging operations: add a second, orthogonal operation—exchange the vulgar and neutral response mappings PSYC512: Research Methods Network Specification of Meaning Psychological Concepts are defined by their relations with other concepts rather than a unitary operational definition Introduction and Discussion sections of papers describe the relationships of our variables to all other relevant variables and concepts—what G, H, & E call assumed operations Method and results sections describe the specific converging operations we use PSYC512: Research Methods Construct Validity The soundness of our operations, do they manipulate or measure the intervening variable that they are intended to manipulate or measure? Types (Campbell & Fiske, 1959) Discriminant validation: operation should not affect or correlate with operations on other intervening variables Convergent validation: operation should affect or correlate with other operations on the same intervening variable PSYC512: Research Methods Testing Hypotheses Hypothesis testing is the process by which hypothetical relationships between intervening variables are assessed Hypotheses are always tested relative to one-another or to a “null” hypothesis Examples Comparing Groups Assessing Performance Interventions Assessing Relationships between variables Problem: Measurement Noise PSYC512: Research Methods Hypothesis Testing: Probability and Statistics Why are probability and statistics important? Used to assess variability in a measure Effect (treatment) Variance Variability due to relationship between variables or effect of different levels of independent variable (treatments) “Good” variance that we want to maximize Error Variance Variability in measure due to factors other than the treatment “Bad” variance that we want to minimize Probability and Statistics are simply tools used to assess (descriptive statistics) and compare (inferential statistics) these sources of variability PSYC512: Research Methods Visualizing Variability: Distributions of Frequency and the Histogram Histograms: used to represent frequencies of data in different classes or categories Bin 6 Frequency 4 2 0 0 1 2 3 4 5 6 7 8 9 10 Grade PSYC512: Research Methods Frequency 0 0 1 0 2 0 3 0 4 3 5 1 6 6 7 4 8 2 9 6 10 1 Displaying Histograms: Stem and Leaf Plots Stem and Leaf plots are used to display histograms graphically (on their side) using only typed characters Stem Leaf (hypothetical histogram for IQ) 6 7 8 9 10 11 12 13 78 35668 012234445555667777889 00011233333334445566667889999 01112233334444445566677777888899 0001122233444566777899 0012569 02 PSYC512: Research Methods Distributions of Probability Density Similar to frequency histogram except y-axis now represents probability density (mass) rather than frequency Probability density = Frequency/N 0.5 Probability Density 0.4 0.3 0.2 0.1 0.0 0 1 2 3 4 5 6 Grade PSYC512: Research Methods 7 8 9 10 Some Types of Distributions Gamma 0.14 0.18 0.12 0.16 Probability Density Probability Density Normal 0.10 0.08 0.06 0.04 0.14 0.12 0.10 0.08 0.06 0.04 0.02 0.02 0.00 0.00 0 5 10 15 20 25 Data 0 5 10 Data PSYC512: Research Methods 15 20 25 Measures of the Center of a Distribution Measures of center represent the general magnitude of scores in a distribution Mode: most frequent score Median: the middle score of an ordered distribution Mean (average): where X is the data and N is the total number of observations X N PSYC512: Research Methods Measures of the Spread of a Distribution Measures of spread are used to assess the consistency of scores in a distribution Range = max score – min score Interquartile range = score(Q3) – score(Q1) Variance (s2) and standard deviation (s) where X is the data, 2 m is the mean of the data, X and N is the total number s2 N of observations PSYC512: Research Methods More on Variance Standard Deviation (s) = sqrt(variance) where X is the data, 2 X m is the mean of the data, s and N is the total number N of observations Why N instead of N-1? Populations vs. Samples Remembering how to compute variance “the mean of the squares – square of the means” s 2 X N 2 X N 2 PSYC512: Research Methods Describing Distributions Parametrically: Statistical Moments Any distribution based on interval or ratio data can be summarized by its statistical moments First Moment: Mean—location of distribution on x-axis Second Moment: Variance—dispersion of distribution Third Moment: Skewness—symmetry of distribution Fourth Moment: Kurtosis—degree of “peakedness” PSYC512: Research Methods Estimators Sample statistics estimate population parameters Mean: M or X vs. M Variance: s2 vs. s2 Properties of Estimators Sufficiency: uses all information in sample (mean and variance are sufficient, mode and range are not) Unbiasedness: expected value approaches real value with increased sampling Efficiency: tightness of cluster of sample statistics relative to the population parameter Resistance: influence of outliers on sample statistic PSYC512: Research Methods Next Time… Topic: Research Designs and Inferential Statistics Be sure to: Read the assigned readings (Howell chapters 6-7) Continue searching and reading the scientific literature for your proposal PSYC512: Research Methods