Download here - Wright State University

QUESTIONS of the MOMENT...
"Why are reviewers complaining about the use of moderated multiple regression in my
paper?"
(The APA citation for this paper is Ping, R.A. (2009). " Why are reviewers complaining
about the use of moderated multiple regression in my paper?" [on-line paper].
http://home.att.net/~rpingjr/MR.doc)
Multiple regression, and moderated multiple regression, assumes each independent
variable is measured without error (i.e., the observed score is exactly the true score).
Unfortunately, it is well known that the extent and direction of all regression coefficient
is biased by even a single variable that contains (known or unknown) measurement error
(e.g., Aiken and West 1991, Bohrnsted and Carter 1971, Cohen and Cohen 1983, Kenny
1979).
Even though this assumption was well known, it was routinely ignored in theoretical
model (hypothesis) testing until Jöreskog's proposal that, among other things, allowed
modeling of measurement error (Jöreskog 1970, 1971) (i.e., structural equation analysis).
As a result, reviewers may reject substantive papers that rely on regression because 1) its
regression's assumption of variables with no measurement error is now believed to be
violated even in demographic variables such as age and income (both are typically
misreported by some respondent groups, and each is typically measured in "round
numbers"). 2) reviewers are (re)aware of how regression estimates can be biased (i.e.,
untrustworthy) in theoretical model tests when one or more variable contains
measurement error (unless they are uncorrelated with any of the other independent
variables, which is unlikely in real-world data). And, 3) regression usually produces
Least Squares estimates--Maximum Likelihood estimates are now preferred for
theoretical model testing.
As a result, some reviewers now believe that regression is an insufficient test of a
theoretical model if there is measurement error in even one model variable (i.e., all the
resulting coefficients used to test the hypotheses are untrustworthy).
Many suggested procedures for moderated multiple regression (e.g., Barron and Kenny
1986) are now considered inappropriate for theory testing because for example, the
analysis procedures (e.g., stepping variables in, etc.) also are insufficient tests of the
hypotheses.
Alternatives to ordinary least squares regression that account for measurement error
include Fuller (1991) and Ping (1996), but each has drawbacks. Fuller's proposals are
inaccessible to many substantive researchers. Ping's proposal relies on measurement
parameter estimates from structural equation analysis, and begs the question, why not just
use structural equation analysis?
The "problems" with utilizing the now preferred structural equation analysis, appear to be
several: it is not taught in all terminal degree programs. And, despite texts apparently
aimed at "self teaching" it (e.g., Byrne 1990), and (powerful) graphical user interfaces
now available in most structural equation analysis software packages, anecdotally,
structural equation analysis still seems to be inaccessible to many substantive researchers
when compared to regression. For untenured researchers who may be "on a clock," this
can slow productivity. For others, this can require "finding" someone who does structural
equation analysis, then "managing" their involvement in the resulting paper. Structural
equation analysis also can appear to "take over" a theoretical piece, producing a perhaps
unwelcome intrusion on its theoretical matters.
"Solutions" to the structural equation analysis "problems" all have drawbacks. First, if
structural equation analysis is not required (e.g., for a dissertation), to conserve time don't
use it. However, for the reasons stated above, this may be a temporary solution.
Next, consider allowing about a month to do three things: first, finding someone to help
with learning structural equation analysis, then learning only enough structural equation
analysis to "get by" reviewers. Then, consider quickly creating/revising a paper with a
simple model (or a simple submodel of your current model) that uses (replaces regression
with) structural equation analysis, and submitting it to a good conference. Rather than
acceptance, the objective would be to learn structural equation analysis in a realistic
setting. Any reviewer feedback would also suggest what/where more structural equation
analysis work is needed.
Click here for more about structural equation analysis as "regression using factor scores
instead of averaged items," and how to learn the basics in a reasonable amount of time.
References
Aiken, L. S. and S. G. West (1991), Multiple Regression: Testing and Interpreting
Interactions, Newbury Park, CA: Sage.
Barron, R. M. and D. A. Kenny (1986), "The Moderator-Mediator Variable Distinction in
Social Psychological Research: Conceptual, Strategic, and Statistical Considerations,"
Journal of Personality and Social Psychology, 51(6): 1173-1182.
Bohrnstedt, G. W. and T. M. Carter (1971), "Robustness in regression analysis," in H.L.
Costner (Ed.), Sociological Methodology (pp. 118-146), San Francisco: Jossey-Bass.
Byrne, B.M. (1990). A Primer of LISREL: Basic Applications and Programming for
Confirmatory Factor Analytic Models. New York: Springer-Verlag Inc.
Cohen, Jacob and Patricia Cohen (1983), Applied Multiple Regression/Correlation
Analyses for the Behavioral Sciences, Hillsdale, NJ: Lawrence Erlbaum.
Fuller, Wayne A. (1991), "Regression Estimation in the Presence of Measurement Error,
" in Measurement Errors in Surveys, B. P. Beimer, R. M. Groves, L. E. Lyberg, N. A.
Mathiowetz and S. Sudman, eds., NY: Wiley.
Jöreskog, Karl G. (1970), "A General Method for Analysis of Covariance Structures,"
Biometrika, 57, 239-251.
________ (1971) "Simultaneous Factor Analysis in Several Populations," Psychometrika,
57, 409-426.
Kenny, David (1979), Correlation and Causality, New York: Wiley.
Ping, R.A. (1996c), "Latent Variable Regression: A Technique for Estimating Interaction
and Quadratic Coefficients," Multivariate Behavioral Research, 31 (1), 95-120.