Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
QUESTIONS of the MOMENT... "Why are reviewers complaining about the use of moderated multiple regression in my paper?" (The APA citation for this paper is Ping, R.A. (2009). " Why are reviewers complaining about the use of moderated multiple regression in my paper?" [on-line paper]. http://home.att.net/~rpingjr/MR.doc) Multiple regression, and moderated multiple regression, assumes each independent variable is measured without error (i.e., the observed score is exactly the true score). Unfortunately, it is well known that the extent and direction of all regression coefficient is biased by even a single variable that contains (known or unknown) measurement error (e.g., Aiken and West 1991, Bohrnsted and Carter 1971, Cohen and Cohen 1983, Kenny 1979). Even though this assumption was well known, it was routinely ignored in theoretical model (hypothesis) testing until Jöreskog's proposal that, among other things, allowed modeling of measurement error (Jöreskog 1970, 1971) (i.e., structural equation analysis). As a result, reviewers may reject substantive papers that rely on regression because 1) its regression's assumption of variables with no measurement error is now believed to be violated even in demographic variables such as age and income (both are typically misreported by some respondent groups, and each is typically measured in "round numbers"). 2) reviewers are (re)aware of how regression estimates can be biased (i.e., untrustworthy) in theoretical model tests when one or more variable contains measurement error (unless they are uncorrelated with any of the other independent variables, which is unlikely in real-world data). And, 3) regression usually produces Least Squares estimates--Maximum Likelihood estimates are now preferred for theoretical model testing. As a result, some reviewers now believe that regression is an insufficient test of a theoretical model if there is measurement error in even one model variable (i.e., all the resulting coefficients used to test the hypotheses are untrustworthy). Many suggested procedures for moderated multiple regression (e.g., Barron and Kenny 1986) are now considered inappropriate for theory testing because for example, the analysis procedures (e.g., stepping variables in, etc.) also are insufficient tests of the hypotheses. Alternatives to ordinary least squares regression that account for measurement error include Fuller (1991) and Ping (1996), but each has drawbacks. Fuller's proposals are inaccessible to many substantive researchers. Ping's proposal relies on measurement parameter estimates from structural equation analysis, and begs the question, why not just use structural equation analysis? The "problems" with utilizing the now preferred structural equation analysis, appear to be several: it is not taught in all terminal degree programs. And, despite texts apparently aimed at "self teaching" it (e.g., Byrne 1990), and (powerful) graphical user interfaces now available in most structural equation analysis software packages, anecdotally, structural equation analysis still seems to be inaccessible to many substantive researchers when compared to regression. For untenured researchers who may be "on a clock," this can slow productivity. For others, this can require "finding" someone who does structural equation analysis, then "managing" their involvement in the resulting paper. Structural equation analysis also can appear to "take over" a theoretical piece, producing a perhaps unwelcome intrusion on its theoretical matters. "Solutions" to the structural equation analysis "problems" all have drawbacks. First, if structural equation analysis is not required (e.g., for a dissertation), to conserve time don't use it. However, for the reasons stated above, this may be a temporary solution. Next, consider allowing about a month to do three things: first, finding someone to help with learning structural equation analysis, then learning only enough structural equation analysis to "get by" reviewers. Then, consider quickly creating/revising a paper with a simple model (or a simple submodel of your current model) that uses (replaces regression with) structural equation analysis, and submitting it to a good conference. Rather than acceptance, the objective would be to learn structural equation analysis in a realistic setting. Any reviewer feedback would also suggest what/where more structural equation analysis work is needed. Click here for more about structural equation analysis as "regression using factor scores instead of averaged items," and how to learn the basics in a reasonable amount of time. References Aiken, L. S. and S. G. West (1991), Multiple Regression: Testing and Interpreting Interactions, Newbury Park, CA: Sage. Barron, R. M. and D. A. Kenny (1986), "The Moderator-Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations," Journal of Personality and Social Psychology, 51(6): 1173-1182. Bohrnstedt, G. W. and T. M. Carter (1971), "Robustness in regression analysis," in H.L. Costner (Ed.), Sociological Methodology (pp. 118-146), San Francisco: Jossey-Bass. Byrne, B.M. (1990). A Primer of LISREL: Basic Applications and Programming for Confirmatory Factor Analytic Models. New York: Springer-Verlag Inc. Cohen, Jacob and Patricia Cohen (1983), Applied Multiple Regression/Correlation Analyses for the Behavioral Sciences, Hillsdale, NJ: Lawrence Erlbaum. Fuller, Wayne A. (1991), "Regression Estimation in the Presence of Measurement Error, " in Measurement Errors in Surveys, B. P. Beimer, R. M. Groves, L. E. Lyberg, N. A. Mathiowetz and S. Sudman, eds., NY: Wiley. Jöreskog, Karl G. (1970), "A General Method for Analysis of Covariance Structures," Biometrika, 57, 239-251. ________ (1971) "Simultaneous Factor Analysis in Several Populations," Psychometrika, 57, 409-426. Kenny, David (1979), Correlation and Causality, New York: Wiley. Ping, R.A. (1996c), "Latent Variable Regression: A Technique for Estimating Interaction and Quadratic Coefficients," Multivariate Behavioral Research, 31 (1), 95-120.