Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Eur. J. Soc. Psychol. RESEARCH ARTICLE Mediation analysis with structural equation models: Combining theory, design, and statistics Daniel Danner*, Dirk Hagemann† & Klaus Fiedler† * GESIS – Leibniz Institute for the Social Sciences, Mannheim, Germany † Institute of Psychology, Heidelberg University, Heidelberg, Germany Correspondence Daniel Danner, GESIS – Leibniz Institute for the Social Sciences, Mannheim P.O. Box 122155, D-68072, Germany. E-mail: [email protected] Received: 24 October 2013 Accepted: 24 November 2014 doi: 10.1002/ejsp.2106 Abstract Statistical tests of indirect effects can hardly distinguish between genuine and spurious mediation effects. The present research demonstrates, however, that mediation analysis can be improved by combining a significance test of the indirect effect with assessing the fit of causal models. Testing only the indirect effect can be misleading, because significant results may also be obtained when the underlying causal model is different from the mediation model. We use simulated data to demonstrate that additionally assessing the fit of causal models with structural equation models can be used to exclude subsets of models that are incompatible with the observed data. The results suggest that combining structural equation modeling with appropriate research design and theoretically stringent mediation analysis can improve scientific insights. Finally, we discuss limitations of the structural equation modeling approach, and we emphasize the importance of non-statistical methods for scientific discovery. Experimental designs are commonly considered the major method for making causal inferences (e.g., Shadish, Cook, & Campbell, 2002). If an independent variable X is manipulated between randomized experimental conditions, variation in a dependent variable Y can be attributed to variation in the independent variable. For example, if persons are randomly assigned to one of the two conditions, either receiving social support or not, resulting differences in persons’ well-being can be assumed to reflect the impact of social support. However, this basic experimental approach is limited because it does not explain how the independent variable affects the dependent variable. In pursuing a hypothetical answer to this question, a researcher may want to investigate whether the effect of social support on well-being comes about through changes in the persons’ attribution style (e.g., the tendency to attribute problems to uncontrollable factors). Testing such an explanatory hypothesis is a case for mediation analysis (e.g., Baron & Kenny, 1986; Hayes, 2013; MacKinnon, 2008). This method allows researchers to investigate whether the empirical evidence is consistent with a mediation model X→Z→Y, which states that the impact of an independent variable X on a dependent variable Y is (at least in part) causally mediated by a proposed mediator Z. Traditionally, statistical mediation analysis tests whether there is a significant indirect effect of the independent variable X via the proposed mediator Z on the dependent variable Y. Over several decades, a variety of statistical procedures have been developed for testing an indirect effect. One common procedure involves a series of models that regress the dependent variable on the independent variable, with and without the mediator as a predictor (e.g., Baron & Kenny, 1986; Judd & Kenny, 1981). The indirect effect of these regression models—that is, the effect of X on Y mediated by Z—can be tested using bootstrap procedures (e.g., Frazier, Tix, & Barron, 2004; Hayes & Scharkow, 2013; Preacher & Hayes, 2004) or parametric significance tests (e.g., Baron & Kenny, 1986; Hayes, 2013; Sobel, 1982). However, applying these statistical approaches can be misleading because a significant test result for a mediator Z does not logically imply that Z is the true mediator. Any statistical test of the model X→Z→Y presupposes as European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models an a priori premise that this is indeed the causal model of theoretical interest. There is no logical reason to exclude that other mediators Z′ or Z″ may provide a better explanation or that the three focal variables (X, Z, and Y) may be related in different ways. In other words, researchers engaging in a test of X→Z→Y rely on the a priori premise that Z deserves to be considered the true mediator, and they merely test whether this selected assumption can account for a significant part of the variance. As long as they do not test the fit of alternative models, they must not infer from a significant mediation test that they have found the true mediator statistically (e.g., James, Mulaik, & Brett, 2006; MacCallum, Wegener, Uchino, & Fabrigar, 1993; MacKinnon, 2008). As a consequence, a significant mediation test result provides necessary but not sufficient evidence for a hypothetical, selectively tested mediation model. A significant test of the mediation model X→Z→Y may be obtained even when the underlying covariance structure is different (e.g., Fiedler, Schott, & Meiser, 2011; MacKinnon, Krull, & Lockwood, 2000). Fiedler et al. (2011) demonstrated in Monte Carlo simulations that significant Sobel tests are regularly obtained when the third variable Z is not a true mediator but, for example, merely a correlate of the dependent variable Y (i.e., when Z is not generated to reflect an influence of X but of Y). In our example, a person’s attribution style may not mediate social support but actually reflect a by-product of well-being. Such demonstrations of “mediation mimicry” imply that statistical tests can lead to wrong conclusions, because they do not discriminate between alternative causal structures that may also give rise to the observed covariance structure of X, Y, and Z. Hence, a more comprehensive analysis is needed to choose between alternative models. The aim of the present research is to demonstrate systematically that combining theoretical considerations, study design, and statistical testing affords a means of overcoming this fundamental weakness of mediation analysis. First, we show that the number of possible three-variate structures (involving X, Z, and Y) in an experimental design can be limited by a few sensible constraints. In particular, given the controlled manipulation of independent variable X in an experimental design, so that the independent variable cannot be affected by the dependent variable or the proposed mediator (e.g., MacCallum et al., 1993; MacKinnon, 2008; Stone-Romero & Rosopa, 2011), there are only 12 possible causal structures. Then, we simulate empirical data for each causal structure and illustrate that a significant indirect effect may reflect not only a mediation structure but also several alternative structures (Fiedler et al., 2011; MacKinnon et al., 2000). Using structural equation modeling, we then demonstrate that there is a class of causal model that fits with the empirical data and that there is a class of causal models that do not fit with the data. Our paper adds to the existing literature by showing which of the 12 causal structures can or cannot be discriminated from each other. It will be seen that a specifiable subset of causal models can be excluded as incompatible with the given data, thus reducing the number of viable candidates substantially. The foundation for this approach has been laid by several authors (e.g., Cole & Maxwell, 2003; Lee & Hershberger, 1990; MacCallum et al., 1993; MacKinnon, 2008), who suggest specifying alternative models by omitting or changing paths in a structural equation model. Imposing Constraints on Viable Causal Structures To be sure, the problem that three variables may be causally related in several ways is a thorny one. There are a large number of different causal models that may all potentially explain an observed covariation pattern between the independent and dependent variables and the proposed mediator. As already noted, one method to reduce the number of possible models is to manipulate the independent variable X experimentally, which effectively rules out the possibility that X can be affected by Y or Z. However, even in this genuinely experimental case, there still remains a variety of structural models, in particular with regard to the relation of the proposed mediator to the dependent variable. Figure 1 provides an overview of all possible models with reference to the example of attribution as a potential mediator of the impact of social support on well-being. (i) Independence model: The variables do not affect each other. For example, persons are randomly assigned to receive social support or no support, but this does not affect well-being or attribution style. (ii) Single effect (X→Y): Social support affects wellbeing, but this effect is not mediated via the persons’ attribution style. (iii) Single effect (X→Z): Social support affects attribution style but not well-being. (iv) Single effect (Z→Y): The persons’ attribution style affects their well-being. Social support does not have an effect even though manipulated experimentally. (v) Single effect (Y→Z): The persons’ well-being causes their attribution style. The manipulation of social support does not have an effect. (vi) Complete mediation (X→Z→Y): Social support affects the persons’ attribution style, which in turn affects the persons’ well-being. European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models Fig. 1: Illustration of possible causal structures in an experimental setting (vii) Common cause (X→Z, X→Y): Social support affects the persons’ attribution style as well as their wellbeing. (viii) Common effect on Y (X→Y, Z→Y): Social support as well as the persons’ attribution style influences the persons’ well-being. However, the treatment does not affect the persons’ attribution style. (ix) Reflection model (X→Y→Z): Social support affects the persons’ well-being, which changes the persons’ attribution style. (x) Common effect on Z (X→Z, Y→Z): The persons’ attribution style is affected by social support as well as by the persons’ well-being. However, social support does not affect the persons’ well-being. (xi) Partial mediation (X→Z→Y, X→Y): Social support affects the persons’ attribution style, which in turn affects their well-being. In addition, social support affects the persons’ well-being directly. (xii) Inverse mediation (X→Z, X→Y→Z): Social support affects the persons’ well-being, which affects their attribution style. In addition, social support affects the persons’ attribution style directly. Structural Equation Modeling Several researchers recommended structural equation modeling as the preferred method for mediation analysis (e.g., Baron & Kenny, 1986; Frazier et al., 2004; Hoyle & Smith, 1994). One important reason is that the unreliability of the mediator and the dependent variable will attenuate systematic relationships in multiple regression, whereas the mediator and the dependent variable may be separated from their measurement errors in structural equation modeling. Another reason is that structural equation modeling is much more flexible than regression (e.g., it is quite easy to include multiple mediators or dependent variables). This flexibility becomes a crucial point when a decision between different causal models is the aim of the analysis. In particular, in multiple regression analysis, a causal model is translated into a series of regression equations, and each coefficient has to be estimated and tested separately (as it is carried out with the series of three regression equations in the mediation analysis of Baron & Kenny, 1986). In contrast, structural equation modeling allows (i) estimating and testing the entire causal model and (ii) comparing different causal models using sophisticated goodness-of-fit statistics (see details as follows). Given a limited number of possible causal models by the study design, structural equation modeling can be used to investigate the fit of alternative causal models and thereby help to reduce the number of viable models. In particular, structural equation models test whether the constraints of specific models fit with the observed data (e.g., Bollen, 1989; Hoyle, 1995; Kline, 2011). This allows researchers to reject those models that are incompatible with the given data and to identify those models that have to be examined more closely as European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models rivals of a hypothesized mediation model. For instance, if the observed data (generated in accordance with a simulated model) fit not only with a complete mediation structure (X→Z→Y) but also with a reflection model (X→Y→Z), further research must be designed to distinguish between these two models. Let us briefly elaborate on this example to further explain the logic of our approach. The complete mediation model (X→Z→Y) states that X is an exogenous variable and Z is a function of X and a random variable υ of the form Z = a*X + υ. In addition, this model states that Y is a function of Z and a random variable ω, Y = b*Z + ω. If the variables X, υ, and ω are not correlated, this implies that the variance σ2 of Z is a function of the variances of X and υ, σ2Z = a2*σ2X + σ2υ, and that the variance of Y is a function of the variances of Z and ω, σ2Y = b2*σ2Z + σ2ω. Accordingly, the covariance ρ between X and Y is ρX,Y = a*b*σ2X, the covariance between X and Z is ρX,Z = a*σ2X, and the covariance between Y and Z is ρY,Z = a2*b*σ2X + b*σ2υ. By comparison, the reflection model (X→Y→Z) states that X is an exogenous variable and Y is a function of X and a random variable ω of the form Y = c*X + ω. In addition, this model states that Z is a function of Y and a random variable υ, Z = b*Y + υ. Again, if the variables X, υ, and ω are not correlated, this implies that the variance σ2 of Y is a function of the variances of X and ω, σ2Y = c2*σ2X + σ2ω, and that the variance of Z is a function of the variances of Y and υ, σ2Z = b2*σ2Y + σ2υ. Accordingly, the covariance between X and Y is ρX,Y = c*σ2X, the covariance between X and Z is ρX,Z = b*c*σ2X, and the covariance between Y and Z is ρY,Z = b*c2*σ2X + b*σ2ω. Hence, the two models imply a different structure of the variances and covariances between X, Y, and Z. In sum, structural equation modeling allows comparing rival causal models for given data (e.g., Cole & Maxwell, 2003; Lee & Hershberger, 1990; MacCallum et al., 1993; MacKinnon, 2008). It can thus elucidate which causal models can be distinguished by the given empirical data and which models cannot (e.g., James et al., 2006; MacCallum et al., 1993; Stelzl, 1986). Of course, structural equation modeling is a statistical approach. It therefore cannot afford a final proof of a true causal model (e.g., MacKinnon, 2008; Shadish et al., 2002; Stone-Romero & Rosopa, 2010). It is always possible that other variables that have not been observed in the experiment can account for the covariance of X, Y, and Z. Moreover, it is always possible that replacing Z by statistically related but psychologically different variables Z′, Z″, and so on may afford better solutions. However, we use simulated data to illustrate that the present approach allows us to reduce the theoretical uncertainty about the causal structure within the trivariate theoretical space spanned by X, Y, and Z. Aim of the Present Simulation The present simulation aims at an empirical setting where the independent variable X was experimentally manipulated and both the dependent variable Y and the hypothesized mediator Z have been measured. As argued previously, there are 12 possible causal structures that may have generated the observed variances and covariances of the three variables, and only two of these causal structures are mediation (complete and partial, respectively). Using simulation techniques, we will illustrate that a significance test of the indirect effect alone will yield false decisions. This part of our simulation aims to replicate and extend our knowledge about the fallibility of this method (as already demonstrated by Fiedler et al., 2011; MacCallum et al., 1993). In a second step, we will illustrate that a combination of testing the indirect effect and structural equation modeling (i.e., deciding in favor for mediation only if the test of the indirect effect is significant and if the mediation model can be accepted) can increase the precision of the decision. This latter analysis will go beyond what is presently known, and it is hoped that this will be a substantial improvement of the methodology. METHOD Simulating Data To illustrate the present approach, we simulated 12 classes of data sets, one class for each causal structure in Figure 1. Each class included 1000 data sets, and each data set included N = 200 observations. In each data set, we first generated the latent construct variables: the independent variable X, the dependent variable Y, and the proposed mediator Z. We generated X as a binary variable (1 = experimental condition, 0 = control condition). Depending on the causal structure, we generated Y and Z as randomly distributed normal variables (M = 0, standard deviation [SD] = 1) or, if dependent on each other, as linear combinations of each other plus randomly distributed normal variables (M = 0, SD = 1). For example, in the complete mediation data sets, we generated the independent variable X as a binary variable. We generated the mediator variable Z as a combination of the independent variable X and a randomly distributed normal variable υ (M = 0, SD = 1), Z = X + υ. Then, we generated the dependent variable as a combination of the mediator variable Z and a randomly distributed normal variable ω (M = 0, SD = 1), Y = Z + ω. The algebraic specifications for all causal models are shown in Table 1. European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models Table 1. Causal structure in simulated data sets Class 1 2 3 4 5 6 7 8 9 10 11 12 Description Independence model Simple effect (X→Y) Simple effect (X→Z) Simple effect (Z→Y) Simple effect (Y→Z) Complete mediation (X→Z→Y) Common cause (X→Z, X→Y) Common effect on Y (X→Y, Z→Y) Reflection model (X→Y→Z) Common effect on Z (X→Z, Y→Z) Partial mediation (X→Z→Y, X→Y) Inverse mediation (X→Z, X→Y→Z) X= Statistical Analyses Y= Z= 1 * υi 1 * X + 1 * υi 1 * υi 1 * Z + 1 * υi 1 * υi 1 * Z + 1 * υi 1 * ωi 1 * ωi 1 * X + 1 * ωi 1 * ωi 1 * Y + 1 * ωi 1 * X + 1 * ωi [0,1] 1 * X + 1 * υi 1 * X + 1 * ωi [0,1] 1 * X + 1 * Z + 1 * υi [0,1] 1 * X + 1 * υi 1 * ωi [0,1] 1 * υi 1 * X+1 * Y+ 1 * ωi 1 * X + 1 * ωi [0,1] [0,1] [0,1] [0,1] [0,1] [0,1] [0,1] 1 * X + 1 * Z + 1 * υi [0,1] 1 * X + 1 * υi 1 * Y + 1 * ωi 1 * X+1 * Y+ 1 * ωi Note: υi /ωi = normally distributed random variables (M = 0.00, standard deviation = 1.00) and i = data set. There were 1000 data sets per class. Second, we included three indicator variables for the dependent variable Y and three indicator variables for the proposed mediator variable Z. Each indicator variable was computed as a linear combination of the latent construct variable and a normally distributed random error variable ε (M = 0, SD = 1). For example, the three indicators for the dependent variable Y were generated as Y1 = Y + ε1, Y2 = Y + ε2, and Y3 = Y + ε3. The algebraic specifications for all indicator variables in all causal structures are shown in Appendix A. For the structural equation model analyses, we used the independent variable X, the three indicators for the proposed mediator variables Z1, Z2, and Z3, and the three indicators for the dependent variables Y1, Y2, and Y3.1 All parameters in the simulation were fixed to be either zero or one. In doing so, we followed Cohen (1990) who suggested using a simple-isbetter principle in multiple regression analyses. In particular, he supposed to use unit weights in prediction equations with +1 when the predictors are positively related and with 0 when they are poorly related. The rationale is that empirically estimated beta weights always depend on the particular sample that was used for their estimation. He argues that the precision of the prediction will be more likely to become worse when empirically estimated betas are used instead of unit weights (see Cohen, 1990, p. 1306 for an analytical demonstration of this rule). In the absence of other constraints, we decided to apply this simple-is-better principle in the present simulations in hope of a great generalizability of our results to a wide range of empirical settings. In the simulated data sets, the effect sizes for the direct effects between the manifest variables (mean of the indicators) were r = .40. According to Cohen (1990), these associations reflect medium to large effects and hence can be seen as realistic for many experimental settings. 1 Testing the Indirect Effect We tested the indirect effect with the Sobel test, bootstrapped confidence intervals (CI), and structural equation models in each data set. The Sobel test is a regression approach designed for analyzing three manifest variables (e.g., Baron & Kenny, 1986; Sobel, 1982). First, the regression coefficient a of the proposed mediator Z on the independent variable X and its standard error sa are estimated. Second, the regression coefficient b of the dependent variable Y on the proposed mediator Z and its standard error sb are estimated. Third, the indirect effect a*b and its standard error sa*b are computed (see Goodman, 1960; MacKinnon et al., 2000; Preacher & Hayes, 2004; Sobel, 1982 for a discussion of different formulas). Sobel’s Z is calculated by the formula Z ¼ sa*b , a* b whereby |Z| > 1.96 indicates a significant indirect effect. The analyses were based on the independent variable X, the mean score of the three indicators of the dependent variable Y (Cronbach’s α = .65–.92), and the mean score of the three indicators of the proposed mediator Z (Cronbach’s α = .65–.93). For each data set, we estimated the indirect effect a*b and its Z-value. Because the Sobel test has statistical limitations (e.g., Hayes & Scharkow, 2013; Preacher & Hayes, 2004; Shrout & Bolger, 2002), bootstrapping can be considered superior. Bootstrapping estimates the upper limit and the lower limit of the CI of an indirect effect. A CI above zero indicates a significant positive indirect effect. We computed bootstrapped 95% CI (2000 bootstrap samples). For the present analysis, we used a nonparametric bootstrap procedure with the SAS (SAS Institute, Cary, NC, USA) script provided by Preacher and Hayes (2004). In addition, we tested the indirect effect with structural equation models (see details as follows). Within the structural equation modeling framework, the magnitude of an indirect effect was estimated with an iterative algorithm (e.g., maximum likelihood), and the significance was tested by dividing the magnitude of the indirect effect by its standard error, producing a standard normally distributed variable Z (e.g., Iacobucci, Saldanha, & Deng, 2007; Kline, 2011; MacKinnon, 2008). Testing indirect effects with structural equation models has been applied in different contexts (e.g., Ahearne, Mathieu, & Rapp, 2005; Hilliard et al., 2013; King, King, Fairbank, Keane, & Adams, 1998; Quilty, Godfrey, Kennedy, & Bagby, 2010). European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models Testing the Model Fit with Structural Equation Models Structural equation modeling can be used to test whether a predefined structural model fits an observed data set. In a first step, a covariance matrix of the observed data is estimated without any model assumptions. In a second step, a covariance matrix of the observed data is estimated given the predefined model structure (e.g., a complete mediation model X→Z→Y) by searching for model parameters that minimize the discrepancy between the model-free estimated covariance matrix and the model-dependent covariance matrix of the observed variables. In a third step, the model fit is computed by comparing both covariance matrices. A good model fit indicates that there is no substantial difference between the matrices. We used two criteria to evaluate the fit of the models: the root mean square error of approximation (RMSEA) and the significance test of the estimated model parameters. The RMSEA is a parsimonious fit index that takes the discrepancy between the observed and the modelimplied covariance matrices into account as well as the model’s complexity. A smaller RMSEA indicates a better model fit. Models with an RMSEA > 0.06 should be rejected (Hu & Bentler, 1999). Models containing nonsignificant parameter estimates (including the indirect effect from X to Y) were rejected because all models are nested in either the partial mediation model or the inverse mediation model. In other words, a model containing a zero parameter is equivalent to another more parsimonious model. For example, a partial mediation model with a zero path coefficient from the latent independent variable X to the latent dependent variable Y is equivalent to the complete mediation model. In addition, we used the χ 2 value and the Bayesian information criterion (BIC) for comparing models with each other. The χ 2 difference test can be used to test whether two nested models2 differ significantly. A smaller χ 2 value indicates a better fit. Models that differ in one degree of freedom differ significantly if they differ at least by Δχ 2 = 3.84, models that differ in two degrees of freedom differ significantly if they differ at least by Δχ 2 = 5.99, and models that differ in three degrees of freedom differ significantly if they differ by Δχ 2 = 7.81. If two models differ significantly, the better fitting model is accepted. If two models do not differ significantly, the more parsimonious model (with more degrees of freedom) is accepted. The BIC value is a descriptive fit 2 Two models are nested if one model can be formulated as a special case of the other model. For example, the reflection model can be formulated as a special case of the inverse mediation model where the path between the independent variable and the proposed mediator is set to zero. Fig. 2: Example of a complete mediation model with three indicators for the proposed mediator Z and three indicators for the dependent variable Y index, which takes the parsimoniousness of the models into account and can also be used to compare nonnested models. A ΔBIC > 10 indicates a meaningful difference (Raftey, 1995). We estimated the fit of all 12 causal models to all data sets. The structural models were specified as shown in Figure 1. We modeled each construct as a latent variable with three indicators, thus yielding sufficient degrees of freedom for the parameter estimation. The measurement models for the latent dependent variable and the latent mediator variable were specified as τ-congeneric models.3 A graphical illustration of the complete mediation model is shown in Figure 2. The model parameters were estimated using the maximum likelihood function implemented in the CALIS procedure in SAS 9.3 (SAS Institute). RESULTS AND DISCUSSION Tests of Indirect Effects (Bootstrap, Sobel, and Structural Equation Models) We estimated bootstrapped CI of the indirect effect in all data sets. There were 1000 data sets in each class and 200 observations in each data set. We used the independent variable X, the mean score of the three indicators of the dependent variable Y, and the mean score of the three indicators of the proposed mediator Z for the analyses. The results are summarized in Table 2. 3 A τ-congeneric measurement model specifies all indicators Yi of a scale as a weighted linear combination of a latent construct variable τ and a residual variable εi, Yi = λi*τ + εi. European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. Independence model Simple effect (X→Y) Simple effect (X→Z) Simple effect (Z→Y) Simple effect (Y→Z) Complete mediation (X→Z→Y) Common cause (X→Z, X→Y) Common effect on Y (X→Y, Z→Y) Reflection model (X→Y→Z) Common effect on Z (X→Z, Y→Z) Partial mediation (X→Z, Z→Y, X→Y) Inverse mediation (X→Z, X→Y, Y→Z) 1 2 3 4 5 6 7 8 9 10 11 12 1 0 3 2 2 100 3 3 99 99 100 100 Percentage of how often indirect effect was significanta 0.00 (0.01) 0.00 (0.01) 0.01 (0.07) 0.00 (0.12) 0.00 (0.09) 0.75 (0.15) 0.00 (0.07) 0.00 (0.12) 0.43 (0.10) 0.43 (0.10) 0.75 (0.15) 0.86 (0.13) Indirect effect a*b Note: SEM, structural equation model. |Z| > 1.96 indicates a significant effect. a Indirect effect based on bootstrapped confidence intervals. b No indirect effect estimated because neither the complete mediation model nor the partial mediation model fits the data. c Indirect effect estimated based on the complete mediation model. d Indirect effect estimated based on the partial mediation model. Model used to generate data Class Upper estimate 0.04 (0.02) 0.04 (0.02) 0.14 (0.08) 0.24 (0.13) 0.18 (0.10) 1.05 (0.17) 0.15 (0.08) 0.24 (0.13) 0.64 (0.12) 0.64 (0.12) 1.05 (0.17) 1.12 (0.15) Lower estimate 0.04 (0.02) 0.04 (0.02) 0.14 (0.08) 0.24 (0.13) 0.19 (0.10) 0.48 (0.13) 0.14 (0.08) 0.24 (0.13) 0.23 (0.09) 0.23 (0.09) 0.48 (0.13) 0.62 (0.11) 0.01 (0.42) 0.01 (0.44) 0.00 (0.95) 0.01 (0.97) 0.02 (0.98) 5.09 (0.65) 0.00 (0.95) 0.01 (0.97) 4.09 (0.75) 4.09 (0.75) 5.09 (0.65) 6.61 (0.54) Z-value (Sobel) .b .b .b 0.02 (0.99)c 0.02 (0.99)c 5.48 (0.76)c 0.00 (0.98)d .b 3.98 (0.69)d 3.69 (0.70)d 4.94 (0.59)d 6.88 (0.62)c Z-value (SEM) Table 2. Percentage of how often the test of indirect effect was significant (%), indirect effect, 95% confidence intervals of bootstrapped indirect effects (2000 samples), and Sobel’s Z and significance test via structural equation modeling (mean score of 1000 data sets in each class, standard deviation in brackets) D. Danner et al. Mediation analysis with structural equation models D. Danner et al. Mediation analysis with structural equation models As can be seen, the bootstrapped CI correctly indicated a mediation effect (a CI above zero) in 100% of the data sets that were actually generated according to the complete mediation structure (CI = [0.48; 1.05]) and in 100% of the data of the partial mediation structure (CI = [0.48; 1.05]). However, bootstrapping also supported a mediation effect in 99% of the data sets that were generated according to the reflection structure (CI = [0.23; 0.64]) and the common-effect-on-Z structure (CI = [0.23; 0.64]) and 100% of the data sets of the inverse mediation structure (CI = [0.62; 1.12]). As evident from Table 2, the Sobel test and significance tests via structural equation models yield the same pattern of results (cf. Hayes, 2013). Across all models and analyses, we would falsely accept a (partial) mediation in 31% of all cases when relying on the bootstrap test of the indirect effect alone. These findings demonstrate that observing a significant indirect effect—no matter whether it was tested with a bootstrap procedure, a Sobel test, or a structural equation model—does not unequivocally prove that the apparent mediation effect was actually used to generate the data. Even in an experimental setting, a significant indirect effect can be caused by different structures. Table 3. Percentage of how often a causal model (columns) was accepted in the different data sets (rows) based on root mean square error of approximation (RMSEA) Percentage model was accepted 1 2 3 4 5 6 7 8 9 10 11 12 Model used to generate data 1 2 3 4 5 6 7 8 9 10 11 12 Independence model Simple effect (X→Y) Simple effect (X→Z) Simple effect (Z→Y) Simple effect (Y→Z) Complete mediation (X→Z, Z→Y) Common cause (X→Z, X→Y) Common effect on Y (X→Y, Z→Y) Reflection model (X→Y, Y→Z) Common effect on Z (X→Z, Y→Z) Partial mediation (X→Z, Z→Y, X→Y) Inverse mediation (X→Z, X→Y, Y→Z) 96 2 1 0 0 0 0 0 0 0 0 0 5 95 0 0 0 0 2 0 0 0 0 0 5 0 96 0 0 0 1 0 0 0 0 0 5 0 0 95 94 0 0 3 1 3 0 0 5 0 0 95 94 0 0 3 1 3 0 0 0 0 2 2 2 94 5 1 43 43 15 94 0 4 5 0 0 0 95 0 0 0 0 0 0 7 0 4 5 0 0 94 14 0 1 0 0 6 0 5 5 42 4 43 94 1 94 16 0 0 5 5 5 13 0 0 1 95 0 1 0 0 0 0 0 4 2 2 88 0 93 5 0 0 0 0 0 91 5 0 5 5 5 93 Note: There were 1000 data sets per class and N = 200 observations per data set. Models were accepted if RMSEA ≤ 0.06 and model parameters were significant. Bold figures indicate how often the correct model fit the data. Table 4. Percentage of how often a causal model (columns) was accepted in the different data sets (rows) based on Bayesian information criterion (BIC) Percentage model was accepted Model used to generate data 1 2 3 4 5 6 7 8 9 10 11 12 Independence model Simple effect (X→Y) Simple effect (X→Z) Simple effect (Z→Y) Simple effect (Y→Z) Complete mediation (X→Z, Z→Y) Common cause (X→Z, X→Y) Common effect on Y (X→Y, Z→Y) Reflection model (X→Y, Y→Z) Common effect on Z (X→Z, Y→Z) Partial mediation (X→Z, Z→Y, X→Y) Inverse mediation (X→Z, X→Y, Y→Z) 1 100 5 5 5 5 0 0 1 1 0 0 0 2 3 4 5 6 7 8 9 10 11 12 2 100 0 0 0 0 5 7 6 0 0 0 1 0 100 0 0 5 5 0 0 5 0 0 0 0 0 99 99 5 0 4 5 6 0 0 0 0 0 99 99 5 0 5 5 5 0 0 0 0 0 1 1 98 0 1 33 14 5 95 0 1 2 0 0 3 100 0 3 0 5 5 0 0 0 2 2 1 0 99 33 0 4 0 0 0 0 1 1 34 0 14 98 1 93 6 0 0 0 2 2 34 0 0 1 100 0 5 0 0 0 0 0 11 0 1 99 0 98 5 0 0 0 0 0 98 0 0 13 2 5 98 Note: There were 1000 data sets per class and N = 200 observations per data set. Models were accepted if ΔBIC ≤ 10 and model parameters were significant. Bold figures indicate how often the correct model fit the data. European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models Model Fit and Model Comparison Considering the fit between the observed data and the various causal models allows us to discard distinct models as incompatible with the given covariance structure. Table 3 shows how often models were accepted in the different data sets based on their RMSEA. A model was accepted when RMSEA ≤ 0.06 (Hu & Bentler, 1999). In addition, Table 4 shows how often models were accepted based on the χ 2 value. A model was accepted if there was no nested model that fitted the data significantly better (Bollen, 1989). Table 5 shows how often models were accepted based on their BIC. A model was accepted if no other model had a BIC that was 10 points smaller (Raftey, 1995). As can be seen, no matter which fit index is used, the number of viable causal models can substantially be reduced. For example, the RMSEA suggests that in the data sets that were generated according to the complete mediation model, a complete mediation model (94%), an inverse mediation model (91%), or a reflection model (42%) fit the data best. Similarly, the χ 2 values suggest that a complete mediation model (91%) or an inverse mediation model (68%) fits the data. The BIC suggests that a complete mediation model (98%), an inverse mediation model (98%), or a reflection or common-effect-on-Z model (34%) fit best. Likewise, the RMSEA suggests that in the data sets that were generated according to the common cause model, a common cause model (99%) fits the data best. Likewise, the BIC suggests that the common cause model explains these data sets best (100%). The χ 2 difference test suggests that not only the common cause model fits the data best (94%) but also the complete mediation model (91%) or the reflection model (91%) fits the data. This is due to the fact that the χ 2 difference test can only distinguish between nested models, and the common cause model, complete mediation model, and the reflection model are not nested in each other. As can be seen in Tables 3, 4, and 5, this pattern of results was the same for all generated data sets. For each class of data sets, a class of causal models can be rejected, whereas another class of viable causal models remains. Therefore, it is not possible to use the structural equation modeling approach to completely eliminate the ambiguity that arises from testing the indirect effect only. However, these findings also show that structural equation modeling can reduce the number of possible causal explanations to a considerable degree, typically from 12 possible explanatory models down to two or three viable models that are consistent with the data (Tables 3, 4, and 5). For example, combining a test of the indirect effect with assessing the fit of the underlying causal models (via the RMSEA) decreases the false alarm rate (for a mediation) from 31% to 19% across all data sets (cf. Tables 2 and 3). In addition, structural equation modeling reveals which alternative causal models can or cannot explain the observed data. It should be noted that well-fitting models always imply covariance matrices that are very similar to the empirical covariance of the manifest variables. Thus, similarity of the covariance matrices is, not surprisingly, the key 2 Table 5. Percentage of how often a causal model (columns) was accepted in the different data sets (rows) based on χ difference test Percentage model was accepted Model used to generate data 1 2 3 4 5 6 7 8 9 10 11 12 Independence model Simple effect (X→Y) Simple effect (X→Z) Simple effect (Z→Y) Simple effect (Y→Z) Complete mediation (X→Z, Z→Y) Common cause (X→Z, X→Y) Common effect on Y (X→Y, Z→Y) Reflection model (X→Y, Y→Z) Common effect on Z (X→Z, Y→Z) Partial mediation (X→Z, Z→Y, X→Y) Inverse mediation (X→Z, X→Y, Y→Z) 1 2 3 4 5 6 7 8 9 10 11 12 100 0 0 0 0 0 0 0 0 0 0 0 0 100 7 5 5 0 4 6 6 0 0 0 0 5 95 0 0 4 5 0 0 4 0 0 0 0 0 95 95 5 0 4 5 6 0 0 0 0 0 95 95 5 0 5 4 5 0 0 0 0 0 0 0 91 91 34 11 89 5 96 0 0 0 0 0 1 94 3 0 0 5 5 0 0 0 0 0 2 3 90 12 0 5 0 0 0 0 0 0 12 91 87 89 34 94 6 0 0 0 0 0 11 3 0 2 90 0 5 0 0 0 0 0 0 0 0 67 0 86 0 0 0 0 0 0 68 0 0 1 0 0 85 2 Note: There were 1000 data sets per class and N = 200 observations per data set. If two nested models differed significantly in their χ values, the model with 2 the smaller χ value was accepted. If two nested models did not differ significantly, the more parsimonious model was accepted. Bold figures indicate how often the correct model fit the data. European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models to understanding the clusters of non-discriminable causal models.4 Appendix B additionally shows the average fit indices for the different data sets. These numbers suggest that the number of eligible causal models can be further reduced by comparing the non-rejected models with each other. For example, for the data that were generated according to the reflection model, the RMSEA suggests that a reflection model, a partial mediation model, a complete mediation model, or an inverse mediation model fit with the data. The χ 2 values and the BIC further suggest that the complete mediation model fit the data substantially worse than a reflection model, a partial mediation model, or an inverse mediation model. Hence, by combining the RMSEA with the χ 2 and the BIC, the number of eligible models may further be reduced. (ii) (iii) (iv) GENERAL DISCUSSION A significant indirect effect does not prove that the data have been generated by a causal mediation mechanism. Even in experimental settings, other causal structures than a mediation structure can also give rise to a significant test result of the indirect effect. However, as the present article demonstrates, combining a significance test of the indirect effect with evaluating the fit of alternative causal model can reduce the uncertainty within a given trivariate theory space. Our simulation results illustrate that structural equation modeling allows researchers to reject specific classes of alternative causal models and to concentrate on a clearly reduced set of viable models. Therefore, combining theoretical considerations, a significance test of the indirect effect, and structural equation modeling can be very useful to reach a better understanding of the mechanisms that may explain a given array of empirical evidence. We recommend a six-step approach to investigating mediation: (i) Before data acquisition, limit the number of possible causal models by design: An experimental design allows excluding all models in which the (v) 4 For the present analysis, we used a sample size of N = 200 observation per simulated data set because this is the lower bound recommendation for structural equation models (e.g., Hoyle, 1995) and, especially in experimental designs, it may be expensive to assess larger samples. As in some experimental settings, it may not be possible to assess 200 participants, while in other settings, it may be possible to assess even larger samples; we additionally run the analysis with N = 100 observations per data set and N = 500 observations per data set, which revealed similar results. Likewise, we changed the cutoff value to RMSEA ≤ 0.10 and alternatively used the cutoff value comparative fit index > 0.95, which also reveal similar results. (vi) independent variable is affected by the proposed mediator or the dependent variable. Identify the causal models for your experimental design. In a standard experimental design with three constructs, there are 12 different causal models (Figure 1). Specify all remaining causal models as structural equation models: Specify the proposed mediator and the dependent variable as latent variables. Specify the relation between the latent variables according to the remaining causal models that you have identified (an example of a complete mediation model is shown in Figure 2). Additional models can be constructed by changing the path between the latent construct variables. Structural equation models also allow testing whether an indirect effect is significant. Investigate the fit of each causal model and compare the fit between models. By this means, the number of eligible causal models can substantially be reduced. The RMSEA allows deciding whether a model sufficiently fits observed data or whether a model should be rejected. Models with an RMSEA > 0.06 can be rejected because of insufficient fit (Hu & Bentler, 1999). Models containing nonsignificant parameters can be rejected because a model containing a zero parameter is equivalent to another more parsimonious model. The χ 2 difference test further allows deciding whether two nested models fit the data equally well or whether one model fits the data significantly better. The BIC allows deciding whether non-nested models fit the data equally well or whether one model fits the data substantially better, where a ΔBIC > 10 suggests a meaningful difference. Using the χ 2 difference test and the BIC allows affirming that the models that were rejected based on their RMSEA do fit worse than the non-rejected models and also further reducing the number of eligible models by showing that some of the non-rejected model fits the data better than other non-rejected models. Identify the remaining causal models that cannot be rejected: The present study suggests that for each causal structure, there are typically two or three models between which cannot be discriminated by structural equation modeling. Engage in deliberate attempts to reduce the number of remaining models: This can be carried out by either a follow-up experiment where the proposed mediator variable is manipulated (e.g., Shrout & Bolger, 2002; Spencer, Zanna, & Fong, 2005; Stone-Romero & Rosopa, 2011), a European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. longitudinal study (e.g., Cole & Maxwell, 2003), or a theory-driven discourse. Limitations The Approach is Designed for an Experimental Setting The basis for the present analysis approach was rejecting several causal models by design. The experimental manipulation of the independent variable allowed us to reject all causal models in which the dependent variable or the proposed mediator affected the independent variable. In a non-experimental design, one would not be able to reject these models on a priori grounds. The set of possible causal models would then grow from 12 classes of models up to at least 27 different classes of models if the assumption is given up that one independent variable cannot be affected by the other variables. This would increase the number of models that fits with an observed data set, leaving various causal explanations for observed data. Likewise, there would be several models that show an identical fit with the observed data (James et al., 2006; MacCallum et al., 1993; Stelzl, 1986). To be sure, an experimental manipulation is not the only reason for excluding specific models. Given such variables as biological sex or age, it is possible to exclude all models that assign them the role of a dependent variable or mediator (e.g., MacCallum et al., 1993). For example, if somebody wants to investigate whether gender role self-concept mediates the relation between biological sex and dominant behavior, all causal models could be excluded in which sex is affected by behavior or gender role self-concept. Longitudinal design may also sometimes allow one to discard certain models on a priori ground. This highlights the usefulness of combining theory, design, and statistical methods and not blindly applying multivariate statistics. The Measurement of the Proposed Mediator and Dependent Variable Must be Valid The results of the present analysis are only valid if the indicators of the dependent variable and the indicators of the proposed mediator are valid. For one thing, the results of structural equation modeling will be biased if the discriminant validity of the indicators of the proposed mediator and the dependent variable cannot be established. Let us illustrate this complication with reference to the well-being example. Suppose that we want to investigate whether the relation between social support and well-being is mediated by attribution style. One of the items that we use to measure well-being is Mediation analysis with structural equation models “I am a great person.” One person may feel well and therefore agree to this item. However, another person who does not feel very well may still be convinced to be a great person because he or she often achieves success. Such an item would be a blend of well-being and attribution style, containing both well-being variance and attribution style variance. Hence, this item will covary with the attribution style items and with the other well-being items regardless of whether the attribution style and well-being are related. This must bias the structural equation modeling results because it artificially increases the association between the latent well-being variable (the dependent variable) and the latent attribution style variable (the proposed mediator). Therefore, it is necessary to ensure discriminant validity of the manifest indicators. Another problem arises if all indicators of the proposed mediator are correlated with another nonobserved construct. This problem was also discussed as the spurious mediation problem (e.g., Fiedler et al., 2011; MacKinnon et al., 2000). In particular, the validity of statistical testing of indirect effects has been criticized because the measurement of the proposed mediator Z could alternatively be interpreted as a correlate of another potential mediator Z′. For one more illustration of this fundamental problem, suppose that the persons’ attribution style is measured with items like “Success depends on good relations,” “You have to be lucky to be successful,” and “I cannot influence much of what is happening to me.” These items may indeed measure attribution style, but they may as well measure other latent variables, such as optimism. Whenever item overlap reduces the discriminant validity of latent constructs, no statistical test can decide whether the effect of social support is mediated by one or the other construct. This argument applies to regression analysis with manifest variables as well as to structural equation modeling. Still, regarding the validity of single indicators, the structural equation modeling approach is more robust than the traditional regression approach.5 This is because the latent mediator variable is specified as the common variance of all indicator variables. A serious validity problem only arises if all or most indicators are correlated with the same non-observed construct. No validity problem exists if only one indicator is correlated with another construct or if each indicator overlaps with a different non-observed construct, because the 5 However, Ledgerwood and Shrout (2011) note that the standard errors of indirect effects based on latent variables can be greater than the standard errors based on manifest variables, especially with heterogeneous items. European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models “unwanted” variance portions of these indicators would be treated as specific variance components in the structural equation model and thus as part of the residual error variance. For example, assuming that we measured attribution style with three items, one being Sartre’s quote “I am the architect of my own self, my own character and destiny.” One could criticize that this particular item does not only measure attribution style but also measure the person’s liking of Jean-Paul Sartre. Using a traditional regression approach, the Sartre item may affect the mean score of the indicators and hence, bias the results of the mediation analysis. However, using the structural equation modeling approach, the Sartre-specific variance proportion would be treated as measurement error in the model and hence, not bias the results of the mediation analysis (see also StoneRomero & Rosopa, 2010; Shadish et al., 2002). Structural Equation Modeling Cannot Exclude all Alternative Causal Structures The present results corroborate the contention that in most cases, more than one causal model fits a data set. For example, the common-effect-on-Z data set could be fitted not only by the common-effect-on-Z model but also by the inverse mediation model. Likewise, there are models that are statistically equivalent. For example, model 4 (Z→Y) implies the same covariance structure than model 5 (Y→Z) and thus cannot be distinguished by statistical procedures. Structural equation modeling only allows reducing the number of possible explanations for the indirect effect within the trivariate framework of given variables. However, as a matter of fact, researchers have to admit that no statistical method can rule out all alternative models involving other variables not included in the trivariate framework. Therefore, as a matter of principle, researchers must always go beyond statistics to complete the picture. The Present Simulations Illustrate a Method But Do Not Demonstrate General Rules When we use our method, there remains a class of models that cannot be further distinguished and that all may explain the data equally well. Because the results of a simulation depend on the choice of the parameters, it is entirely possible that the class of remaining models might have different members if we had chosen different parameters. However, we are confident that the main findings of our simulation may replicate for other parameters, that is, the structural equation modeling approach helps to eliminate some although not all of the alternative models. We therefore consider our simulation study not as a demonstration of general rules but as an illustration that our approach helps to decide between concurrent models for which we could not decide if we rely only on the Sobel test or on the bootstrap evaluation of the alleged mediation effect. Beyond Statistical Testing The approach we have described so far is a purely statistical one. Of course, there are ancillary and in several settings preferable approaches for an analysis of mediation. For example, Spencer, Zanna, and Fong (2005) suggested investigating mediation by manipulating the mediator variable experimentally. This approach has the great advantage that the direction of the relation between the proposed mediator and the dependent variable can be controlled. For example, if we could experimentally control how persons attribute their successes and failures, we could exclude all causal models that state that well-being affects attribution style. However, it may not always be possible to control constructs such as attribution style (especially over a longer period of time). In addition, it is not certain that the effect of the experimental manipulation of the proposed mediator is the same as the indirect effect of the mediator triggered by the independent variable (Kenny, 2008). StoneRomero and Rosopa (2010) and Mathieu and Taylor (2006) discuss further approaches to investigating mediation analysis with different designs, as the authors conclude the following: “ideally, the results of studies using all such alternatives should converge (p. 700).” It has also been suggested to measure the mediator and the dependent variable at several measurement occasions or in a prescribed timely order (e.g., MacCallum et al., 1993; MacKinnon, 2008; Cole & Maxwell, 2003). The rationale behind this approach is that the variable that is measured second cannot affect the variable that was measured first. However, this may not be true in all settings because the variables may have affected each other before the first measurement took place (see also MacKinnon, 2008). For example, social support may affect a person’s well-being after a first treatment. From that moment on, persons may start to attribute their successes to their capability. At the beginning of the second treatment session, the researcher assesses the persons’ attribution style and registers that the style has changed. At the beginning of the third session, the researcher assesses the person’s well-being and registers that their well-being has improved. Therefore, the selected time interval is critical, and even if the variables were measured in a prescribed timely order, the direction of the relation may not be clear (e.g., Cole & Maxwell, 2003; MacKinnon, 2008). European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. CONCLUSION Combining a significance test of the indirect effect with evaluating the fit of alternative causal models with structural equation models can improve mediation analyses. However, structural equation modeling is not a silver bullet and has its limitations and weaknesses as does every method. The strength of the present approach is that it can be combined with complementary approaches. For example, bootstrapping or a Sobel test may indicate that an indirect effect is significant, granted the mediation occurred. Structural equation modeling can demonstrate that several alternative causal explanations for the significant indirect effect can be rejected. Experimentally, manipulating the mediator variable or measuring the proposed mediator before measuring the dependent variable may further support this hypothesis. However, the most important methodological tool for a scientific explanation consists of cleverly designed follow-up experiments informed by the insights gained from structural equation modeling. We hope that such a combined approach can improve the use of mediation analysis in future research in social psychology and thereby help scientists to gain a deeper understanding of empirical reality. ACKNOWLEDGEMENTS This research was supported by grants awarded by the Deutsche Forschungsgemeinschaft to the second (HA3044/7-1) and third authors (Fi 294/23-1). We gratefully thank Matthias Blümke, Jemaine Clement, Sebastian Nagengast, Oliver Schilling, and three anonymous reviewers for helpful comments on an earlier draft of this paper. REFERENCES Ahearne, M., Mathieu, J., & Rapp, A. (2005). To empower or not to empower your sales force? An empirical examination of the influence of leadership empowerment behavior on customer satisfaction and performance. Journal of Applied Psychology, 90(5), 945–955. http://dx. doi.org/10.1037/0021-9010.90.5.945 Arbuckle, J. L. (2013). Amos (version 22.0) [computer program]. Chicago: SPSS. Baron, R. M., & Kenny, D. A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182. http://dx. doi.org/10.1037/0022-3514.51.6.1173 Bollen, K. A. (1989). Structural equations with latent variables. Oxford England: John Wiley & Sons. Mediation analysis with structural equation models Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304–1312. http://dx.doi.org/10.1037/ 0003-066X.45.12.1304 Cole, D. A., & Maxwell, S. E. (2003). Testing mediational models with longitudinal data: Questions and tips in the use of structural equation modeling. Journal of Abnormal Psychology, 112(4), 558–577. http://dx.doi.org/10.1037/ 0021-843X.112.4.558 Fiedler, K., Schott, M., & Meiser, T. (2011). What mediation analysis can (not) do. Journal of Experimental Social Psychology, 47(6), 1231–1236. http://dx.doi.org/10.1016/j. jesp.2011.05.007 Frazier, P. A., Tix, A. P., & Barron, K. E. (2004). Testing moderator and mediator effects in counseling psychology research. Journal of Counseling Psychology, 51(1), 115–134. http://dx.doi.org/10.1037/0022-0167.51.1.115 Goodman, L. A. (1960). On the exact variance of products. Journal of the American Statistical Association, 55,708–713. http://dx.doi.org/10.1080/01621459.1960.10483369 Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York: Guilford Press. Hayes, A. F., & Scharkow, M. (2013). The relative trustworthiness of inferential tests of the indirect effect in statistical mediation analysis: Does method really matter? Psychological Science, 24(10), 1918–1927. http://dx.doi.org/10.1177/ 0956797613480187 Hilliard, M. E., Holmes, C. S., Chen, R., Maher, K., Robinson, E., & Streisand, R. (2013). Disentangling the roles of parental monitoring and family conflict in adolescents’ management of type 1 diabetes. Health Psychology, 32(4), 388–396. http://dx.doi.org/10.1037/a0027811 Hoyle, R. H. (1995). Structural equation modeling: Concepts, issues, and applications. Thousand Oaks, CA US: Sage Publications, Inc. Hoyle, R. H., & Smith, G. T. (1994). Formulating clinical research hypotheses as structural equation models: A conceptual overview. Journal of Consulting and Clinical Psychology, 62, 429–440. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi: 10.1080/10705519909540118 Iacobucci, D., Saldanha, N., & Deng, X. (2007). A meditation on mediation: Evidence that structural equations models perform better than regressions. Journal of Consumer Psychology, 17(2), 139–153. http://dx.doi.org/10.1016/S10577408(07)70020-7 James, L. R., Mulaik, S. A., & Brett, J. M. (2006). A tale of two methods. Organizational Research Methods, 9(2), 233–244. http://dx.doi.org/10.1177/1094428105285144 Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating mediation in treatment evaluations. Evaluation Review, 5(5), 602–619. http://dx.doi.org/10.1177/0193841X8100500502 European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models Kenny, D. A. (2008). Reflections on mediation. Organizational Research Methods, 11(2), 353–358. http://dx.doi.org/ 10.1177/1094428107308978 King, L. A., King, D. W., Fairbank, J. A., Keane, T. M., & Adams, G. A. (1998). Resilience–recovery factors in post-traumatic stress disorder among female and male Vietnam veterans: Hardiness, postwar social support, and additional stressful life events. Journal of Personality and Social Psychology, 74(2), 420–434. http://dx.doi.org/ 10.1037/0022-3514.74.2.420 Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York, NY US: Guilford Press. Ledgerwood, A., & Shrout, P. E. (2011). The trade-off between accuracy and precision in latent variable models of mediation processes. Journal of Personality and Social Psychology, 101(6), 1174–1188. http://dx.doi.org/10.1037/ a0024776. Lee, S., & Hershberger, S. (1990). A simple rule for generating equivalent models in covariance structure modeling. Multivariate Behavioral Research, 25(3), 313–334. http://dx.doi. org/10.1207/s15327906mbr2503_4 MacCallum, R. C., Wegener, D. T., Uchino, B. N., & Fabrigar, L. R. (1993). The problem of equivalent models in applications of covariance structure analysis. Psychological Bulletin, 114(1), 185–199. http://dx.doi.org/10.1037/ 0033-2909.114.1.185 MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York, NY: Taylor & Francis Group/ Lawrence Erlbaum Associates. MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000). Equivalence of the mediation, confounding and suppression effect. Prevention Science, 1(4), 173–181. http://dx.doi. org/10.1023/a:1026595011371 Mathieu, J. E. & Taylor, S. R. (2006). Clarifying conditions and decision points for mediational type inferences in organizational behavior. Journal of Organizational Behavior, 27, 1031–1056. http://dx.doi.org/ 10.1002/job.406 Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, & Computers, 36(4), 717–731. http://dx.doi.org/10.3758/ bf03206553 Quilty, L. C., Godfrey, K. M., Kennedy, S. H., & Bagby, R. M. (2010). Harm avoidance as a mediator of treatment response to antidepressant treatment of patients with major depression. Psychotherapy and Psychosomatics, 79(2), 116–122. http://dx.doi.org/10.1159/ 000276372 Raftey, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA US: Houghton, Mifflin and Company. Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods, 7(4), 422–445. http:// dx.doi.org/10.1037/1082-989X.7.4.422 Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equations models. In S. Leinhart (Ed.), Sociological methodology (pp. 290–312). San Francisco: Jossey-Bass. Spencer, S. J., Zanna, M. P., & Fong, G. T. (2005). Establishing a causal chain: Why experiments are often more effective than mediational analyses in examining psychological processes. Journal of Personality and Social Psychology, 89(6), 845–851. http://dx.doi.org/10.1037/00223514.89.6.845 Stelzl, I. (1986). Changing a causal hypothesis without changing the fit: Some rules for generating equivalent path models. Multivariate Behavioral Research, 21(3), 309–331. http://dx.doi.org/10.1207/s15327906mbr2103_3 Stone-Romero, E. F., & Rosopa, P. J. (2010). Research design options for testing mediation models and their implications for facets of validity. Journal of Managerial Psychology, 25(7), 697–712. http://dx.doi.org/10.1108/ 02683941011075256 Stone-Romero, E. F., & Rosopa, P. J. (2011). Experimental tests of mediation models: Prospects, problems, and some solutions. Organizational Research Methods, 14(4), 631–646. http://dx.doi.org/10.1177/1094428110372673 APPENDIX A: TABLE A1 Equations used for generating the variables in the simulated data sets Model class Independence model Simple effect (X→Y) Simple effect (X→Z) Simple effect (Z→Y) Variable Xij Y1ij Y2ij Y3ij Z1ij Z2ij Z3ij Xij Y1ij Y2ij Y3ij Z1ij Z2ij Z3ij Xij Y1ij Y2ij Y3ij Z1ij Z2ij Z3ij Xij Y1ij Equation [0;1] 1 * υij + 1 * ε1ij 1 * υij + 1 * ε2ij 1 * υij + 1 * ε3ij 1 * ωij + 1 * ε4ij 1 * ωij + 1 * ε5ij 1 * ωij + 1 * ε6ij [0;1] 1 * Xij + 1 * υij + 1 * ε1ij 1 * Xij + 1 * υij + 1 * ε2ij 1 * Xij + 1 * υij + 1 * ε3ij 1 * ωij + 1 * ε4ij 1 * ωij + 1 * ε5ij 1 * ωij + 1 * ε6ij [0;1] 1 * υij + 1 * ε1ij 1 * υij + 1 * ε2ij 1 * υij + 1 * ε3ij 1 * Xij + 1 * ωij + 1 * ε4ij 1 * Xij + 1 * ωij + 1 * ε5ij 1 * Xij + 1 * ωij + 1 * ε6ij [0;1] 1 * ωij + 1 * υij + 1 * ε1ij (Continues) European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models TABLE A1 (Continued) Table 1 (Continued) Model class Simple effect (Y→Z) Complete mediation TABLE A1 (Continued) Table 1 (Continued) Variable Z1ij Z2ij Z3ij Xij 1 * ωij + 1 * υij + 1 * ε2ij 1 * ωij + 1 * υij + 1 * ε3ij 1 * ωij + 1 * ε4ij 1 * ωij + 1 * ε5ij 1 * ωij + 1 * ε6ij [0;1] 1 * υij + 1 * ε1ij 1 * υij + 1 * ε2ij 1 * υij + 1 * ε3ij 1 * υij + 1 * ωij + 1 * ε4ij 1 * υij + 1 * ωij + 1 * ε5ij 1 * υij + 1 * ωij + 1 * ε6ij [0;1] 1 * Xij + 1 * ωij + 1 * υij + 1 * ε1ij 1 * Xij + 1 * ωij + 1 * υij + 1 * ε2ij 1 * Xij + 1 * ωij + 1 * υij + 1 * ε3ij 1 * Xij + 1 * ωij + 1 * ε4ij 1 * Xij + 1 * ωij + 1 * ε5ij 1 * Xij + 1 * ωij + 1 * ε6ij [0;1] Y1ij Y2ij Y3ij Z1ij Z2ij Z3ij Xij 1 * Xij + 1 * υij + 1 * ε1ij 1 * Xij + 1 * υij + 1 * ε2ij 1 * Xij + 1 * υij + 1 * ε3ij 1 * Xij + 1 * ωij + 1 * ε4ij 1 * Xij + 1 * ωij + 1 * ε5ij 1 * Xij + 1 * ωij + 1 * ε6ij [0;1] Y1ij 1 * Xij + 1 * ωij + 1 * υij + 1 * ε1ij 1 * Xij + 1 * ωij + 1 * υij + 1 * ε2ij 1 * Xij + 1 * ωij + 1 * υij + 1 * ε3ij 1 * ωij + 1 * ε4ij 1 * ωij + 1 * ε5ij 1 * ωij + 1 * ε6ij Y2ij Y3ij Z1ij Z2ij Z3ij Xij Y1ij Y2ij Y3ij Z1ij Z2ij Z3ij Xij Y1ij Y2ij Y3ij Common cause (X→Z, X→Y) Common effect on Y (X→Y, Z→Y) Equation Y2ij Y3ij Z1ij Z2ij Z3ij (Continues) Model class Reflection model (X→Y→Z) Common effect on Z (X→Z, Y →Z) Partial mediation (X→Z→Y, X→Y) Variable Xij [0;1] Y1ij Y2ij Y3ij Z1ij Z2ij Z3ij Xij 1 * Xij + 1 * υij + 1 * ε1ij 1 * Xij + 1 * υij + 1 * ε2ij 1 * Xij + 1 * υij + 1 * ε3ij 1 * Xij + 1 * υij + 1 * ωij + 1 * ε4ij 1 * Xij + 1 * υij + 1 * ωij + 1 * ε5ij 1 * Xij + 1 * υij + 1 * ωij + 1 * ε6ij [0;1] Y1ij Y2ij Y3ij Z1ij Z2ij Z3ij Xij 1 * υij + 1 * ε1ij 1 * υij + 1 * ε2ij 1 * υij + 1 * ε3ij 1 * Xij + 1 * υij + 1 * ωij + 1 * ε4ij 1 * Xij + 1 * υij + 1 * ωij + 1 * ε5ij 1 * Xij + 1 * υij + 1 * ωij + 1 * ε6ij [0;1] Y1ij 1 * Xij + 1 * Xij + 1 * ωij + 1 * υij + 1 * ε1ij 1 * Xij + 1 * Xij + 1 * ωij + 1 * υij + 1 * ε2ij 1 * Xij + 1 * Xij + 1 * ωij + 1 * υij + 1 * ε3ij 1 * Xij + 1 * ωij + 1 * ε4ij 1 * Xij + 1 * ωij + 1 * ε5ij 1 * Xij + 1 * ωij + 1 * ε6ij [0;1] Y2ij Y3ij Inverse mediation (X→Y→Z, X→Z) Equation Z1ij Z2ij Z3ij Xij Y1ij Y2ij Y3ij Z1ij Z2ij Z3ij 1 * Xij + 1 * υij + 1 * ε1ij 1 * Xij + 1 * υij + 1 * ε2ij 1 * Xij + 1 * υij + 1 * ε3ij 1 * Xij + 1 * υij + 1 * Xij + 1 * ωij + 1 * ε4ij 1 * Xij + 1 * υij + 1 * Xij + 1 * ωij + 1 * ε5ij 1 * Xij + 1 * υij + 1 * Xij + 1 * ωij + 1 * ε6ij Note: There were 1000 data sets in each class i and 200 observation in each data set j. The terms υ, ω, and ε16 were normally distributed random variables with M = 0 and standard deviation = 1. European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models APPENDIX B: TABLE B1 Average root mean square error of approximation (RMSEA) for the different causal models (columns) in the different data sets (rows) and standard deviation in brackets RMSEA for model Model used to generate data 1 1 Independence model 2 Simple effect (X→Y) 3 Simple effect (X→Z) 4 Simple effect (Z→Y) 5 Simple effect (Y→Z) 6 Complete mediation (X→Z, Z→Y) Common cause (X→Z, X→Y) Common effect on Y (X→Y, Z→Y) Reflection model (X→Y, Y→Z) Common effect on Z (X→Z, Y→Z) Partial mediation (X→Z, Z→Y, X→Y) Inverse mediation (X→Z, X→Y, Y→Z) 7 8 9 10 11 12 0.02 (0.02) 0.11 (0.02) 0.11 (0.02) 0.16 (0.02) 0.16 (0.02) 0.21 (0.02) 0.15 (0.02) 0.18 (0.02) 0.21 (0.02) 0.18 (0.02) 0.25 (0.02) 0.25 (0.02) 2 3 4 5 6 7 8 9 10 11 12 0.02 (0.02) 0.02 (0.02) . 0.01 (0.02) 0.11 (0.02) 0.02 (0.02) 0.17 (0.02) 0.17 (0.02) 0.19 (0.02) 0.11 (0.02) . 0.02 (0.02) 0.11 (0.01) . 0.02 (0.02) 0.11 (0.02) . . . . . . . . . 0.02 (0.02) . 0.01 (0.02) . . . . . 0.02 (0.02) 0.02 (0.02) 0.11 (0.02) 0.15 (0.02) 0.10 (0.02) 0.11 (0.02) 0.10 (0.02) 0.16 (0.02) 0.16 (0.02) . . . 0.18 (0.02) 0.17 (0.02) 0.02 (0.02) . 0.02 (0.02) 0.11 (0.02) 0.12 (0.02) 0.02 (0.02) 0.09 (0.02) . 0.02 (0.02) 0.02 (0.02) 0.06 (0.03) 0.10 (0.02) 0.06 (0.03) 0.02 (0.02) 0.10 (0.02) 0.02 (0.02) 0.08 (0.03) 0.02 (0.02) 0.02 (0.02) 0.02 (0.02) 0.09 (0.02) 0.11 (0.02) . . 0.02 (0.02) 0.02 (0.02) 0.11 (0.02) 0.15 (0.02) 0.10 (0.02) 0.11 (0.02) 0.10 (0.02) 0.16 (0.02) 0.16 (0.02) 0.02 (0.02) 0.02 (0.02) . . . . 0.02 (0.02) 0.02 (0.02) . 0.17 (0.02) 0.17 (0.02) 0.20 (0.02) 0.11 (0.02) 0.17 (0.02) 0.19 (0.02) 0.20 (0.02) 0.20 (0.02) 0.23 (0.02) 0.20 (0.02) 0.17 (0.02) 0.23 (0.02) 0.20 (0.02) 0.02 (0.02) 0.02 (0.02) 0.02 (0.02) 0.02 (0.02) 0.10 (0.02) 0.17 (0.02) 0.06 (0.03) 0.06 (0.03) 0.08 (0.02) 0.02 (0.02) 0.17 (0.02) 0.18 (0.02) 0.17 (0.02) 0.17 (0.02) 0.11 (0.02) 0.16 (0.02) 0.11 (0.02) 0.02 (0.02) 0.16 (0.02) 0.11 (0.02) . . 0.02 (0.02) . 0.02 (0.02) 0.02 (0.02) 0.02 (0.03) 0.02 (0.02) 0.01 (0.02) 0.02 (0.02) Note: There were 1000 data sets per class and N = 200 observations per data set. The average RMSEA is reported if the model parameters are significant in at least 5% of the simulated data sets. TABLE B2 2 Average χ value for the different causal models (columns) in the different data sets (rows) and standard deviation in brackets 2 Model used to generate data 1 Independence model 2 Simple effect (X→Y) 3 Simple effect (X→Z) 4 Simple effect (Z→Y) 5 Simple effect (Y→Z) 6 Complete mediation (X→Z, Z→Y) 7 Common cause (X→Z, X→Y) 8 Common effect on Y (X→Y, Z→Y) χ value for model 1 2 3 4 5 6 7 8 9 14.00 (4.73) 14.20 (5.56) . 13.59 (4.48) 48.35 (11.23) 14.05 (5.27) 94.65 (15.80) 93.95 (16.03) 117.27 (18.08) 14.17 (4.71) 46.81 (11.49) . 14.17 (4.71) 46.81 (11.49) . . . . . 12.38 (5.03) . 14.29 (5.38) 14.13 (5.45) 50.12 (12.20) 13.93 (5.77) 12.92 (4.69) . 12.66 (5.23) . 14.29 (5.38) 14.13 (5.45) 50.12 (12.20) 47.47 (9.93) 13.30 (5.72) 12.40 (3.73) 14.38 (5.33) 13.26 (5.27) . 93.22 (17.17) 90.18 (16.50) 14.72 (4.99) 47.51 (11.76) 13.16 (6.49) 13.72 (5.19) 24.87 (8.45) 78.51 (14.42) 43.17 (11.39) 38.54 (14.42) 43.17 (11.39) 38.54 (11.26) 40.42 (11.38) 13.07 (5.15) . 50.43 (13.36) 13.31 (5.26) 38.95 (10.92) 24.67 (8.10) 15.09 (5.43) 49.45 (12.50) 49.32 (12.02) 92.25 (16.74) 93.10 (17.02) 146.54 (22.86) 90.99 (14.76) 96.37 (16.36) 125.52 (19.11) 83.08 (16.17) 112.31 (18.25) 48.35 (12.00) 91.22 (16.63) 47.80 (11.95) . 10 . 13.06 (4.82) 13.41 (4.62) 13.74 (5.97) 33.32 (10.15) 47.85 (10.58) . 11 12 . . 14.64 (13.39) . . . . . . . . 12.25 (5.09) . 12.08 (4.70) . . (Continues) European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Table Mediation analysis with structural equation models 1 (Continued) TABLE B2 (Continued) 2 χ value for model Model used to generate data 9 Reflection model (X→Y, Y→Z) 10 Common effect on Z (X→Z, Y→Z) 11 Partial mediation (X→Z, Z→Y, X→Y) 12 Inverse mediation (X→Z, X→Y, Y→Z) Model df 2 3 4 5 6 7 8 9 10 11 12 147.47 (22.87) 113.19 (18.22) 197.25 (25.82) 1 112.22 (18.13) 125.29 (17.88) 125.55 (19.09) 126.38 (19.45) 92.02 (16.97) 161.98 (20.18) 49.97 (12.48) 43.27 (11.30) 85.79 (15.41) 49.97 (12.48) 43.27 (11.30) 85.78 (15.41) 24.61 (8.76) 24.70 (8.43) 32.58 (10.34) 91.06 (16.84) 94.22 (16.21) 90.28 (16.47) 33.24 (10.15) . 46.59 (11.97) 13.06 (5.22) 84.22 (15.02) 12.04 (5.02) . 47.59 (11.92) 13.17 (5.37) 39.63 (10.68) 13.24 (5.25) 12.32 (5.11) 13.30 (6.35) 12.21 (4.24) 11.34 (4.30) 198.26 (25.58) 163.26 (20.03) 126.31 (19.44) 85.80 (15.29) 85.82 (15.29) 13.08 (5.30) 91.06 (16.85) 84.01 (14.73) 32.45 (10.67) 47.30 (12.28) 12.33 (4.16) 12.03 (5.08) 15 14 14 14 14 13 13 13 13 13 12 12 2 Note: There were 1000 data sets per class and N = 200 observations per data set. The average χ value is reported if the model parameters are significant in at least 5% of the simulated data sets. TABLE B3 Average Bayesian information criterion (BIC) value for the different causal models (columns) in the different data sets (rows) and standard deviation in brackets Model used to generate data 1 2 3 4 5 6 7 8 9 10 11 Independence model Simple effect (X→Y) Simple effect (X→Z) Simple effect (Z→Y) Simple effect (Y→Z) Complete mediation (X→Z, Z→Y) Common cause (X→Z, X→Y) Common effect on Y (X→Y, Z→Y) Reflection model (X→Y, Y→Z) Common effect on Z (X→Z, Y→Z) Partial mediation (X→Z, Z→Y, X→Y) BIC value for model 1 2 3 4 5 6 7 8 9 10 11 12 88.17 (4.73) 88.37 (5.56) . 87.76 (4.48) 122.53 (11.23) 88.22 (5.27) 168.83 (15.80) 168.13 (16.03) 185.44 (18.08) 88.34 (4.71) 120.99 (11.49) . 88.34 (4.71) 120.99 (11.49) . . . . . . . . . . . . . 88.46 (5.38) 88.31 (5.45) 124.29 (12.20) 92.53 (4.82) 92.89 (4.62) 93.22 (5.97) 112.79 (10.15) . . . . . 97.02 (5.09) 151.96 (16.17) 83.97 (5.43) 118.32 (12.50) 118.20 (12.02) 161.13 (16.74) 161.98 (17.02) 215.42 (22.86) 165.17 (14.76) 170.55 (16.36) 199.69 (19.11) 92.14 (5.23) . 88.46 (5.38) 88.31 (5.45) 124.29 (12.20) 92.77 (5.27) 91.87 (3.73) 93.85 (5.33) 92.74 (5.27) 93.40 (5.77) 92.39 (4.69) . 172.70 (17.17) 169.65 (16.50) 94.19 (4.99) 126.99 (11.76) 92.63 (6.49) 93.19 (5.19) 104.35 (8.45) 122.53 121.98 152.69 (12.00) (11.95) (14.42) 152.69 (14.42) 118.02 (11.26) 92.55 (5.15) 129.90 (13.36) 118.43 (10.92) 127.32 (10.58) . 96.85 (4.70) 181.19 (18.25) 165.39 (16.63) 117.34 (11.39) 117.34 (11.39) 119.89 (11.38) . 92.79 (5.26) 104.15 (8.10) . . . 216.35 (22.87) 186.40 200.55 124.14 (18.13) (19.45) (12.48) 124.14 (12.48) 104.09 (8.76) 170.53 (16.84) 112.72 (10.03) 92.64 (5.37) 126.06 (11.97) 96.81 (5.02) 98.07 (6.35) 182.07 (18.22) 199.46 166.20 117.45 (17.88) (16.97) (11.30) 117.45 (11.30) 104.17 (8.43) 173.70 (16.21) . 119.11 (10.68) 92.54 (5.22) . 96.98 (4.24) 266.13 (25.82) 199.73 236.15 159.97 (19.09) (20.18) (15.41) 159.97 (15.41) 112.06 (10.43) 169.75 (16.47) 127.06 (11.92) 92.72 (5.25) 163.69 (15.02) 97.10 (5.11) 96.12 (4.30) . . . 91.85 (5.03) . (Continues) European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models Table 1 (Continued) TABLE B2 (Continued) BIC value for model Model used to generate data 12 Inverse mediation (X→Z, X→Y, Y→Z) 1 267.13 (25.58) 2 3 4 237.18 200.49 160.00 (20.03) (19.44) (15.29) 5 6 7 8 9 10 11 12 160.00 (15.29) 92.56 (5.30) 170.54 (16.85) 163.48 (14.73) 111.93 (10.67) 126.77 (12.28) 97.11 (4.16) 96.81 (5.08) Note: There were 1000 data sets per class and N = 200 observations per data set. The average BIC is reported if the model parameters are significant in at least 5% of the simulated data sets. APPENDIX C: GUIDELINE FOR APPLYING THE PRESENT APPROACH WITH AMOS AMOS (Arbuckle, 2013) offers a graphical user interface where the user can specify a model by drawing a path diagram as shown in Figure 2: (i) Specify a manifest variable for the independent variable X. (ii) Specify a latent variable for the proposed mediator Z with three manifest variables. (iii) Specify a latent variable for the dependent variable Y with three manifest variables. (iv) Specify the relation between the latent variables according to the causal model shown in Figure 1. There will be one structural equation model for each causal model. These models differ in the number and directing of the path between the latent variables. Add a residual variable for each endogenous variable in the model. Endogenous variables are variables that are explained by other variables in the model (e.g., the dependent variable in the simple effect on Y model or the proposed mediator Z and the dependent variable Y in the complete mediation model) (v) Estimate the model parameters and the model fit for each causal model. (vi) Investigate the fit of each causal model and compare the fit between models. Models with an RMSEA > 0.06 can be rejected because of insufficient fit (Hu & Bentler, 1999). Models containing nonsignificant parameters can be rejected because a model containing a zero parameter is 2 equivalent to another more parsimonious model. Nested models can be compared with the χ difference test, where a significant difference suggests rejecting the worse fitting model. Non-nested models can be compared based on their BIC value where a ΔBIC > 10 suggests a meaningful difference. (vii) Engage in further attempts to reject the remaining models by theoretical considerations or a follow-up study. APPENDIX D: GUIDELINE FOR APPLYING THE PRESENT APPROACH WITH SAS The syntax requires SAS 9.3 and a SAS data file containing an independent variable, three indicators for the dependent variable, and three indicators for the dependent variable. First, the location and the name of the SAS data file and the name of the variables must be specified. This can easily be carried out by modifying the last line of the syntax. In the present example, “C:\” is the location of the SAS file, “data” is the name of the SAS file, “v1” is the name of the independent variable, “v2”–“v4” are the names of the indicators for the dependent variables, and “v5”–“v7” are the names of the indicators of the proposed mediator variable: %semmedðlocation ¼ C : ∖; file ¼ data; x ¼ v1; y1 ¼ v2; y2 ¼ v3; y3 ¼ v4; z1 ¼ v5; z2 ¼ v6; z3 ¼ v7Þ 2 The syntax produces a table containing the RMSEA, χ value, the df, and the BIC for each model. In addition, the script provides estimates of the indirect effect based on the complete mediation model and the partial mediation model. After evaluating the results, engage in further attempts to reject the remaining models by theoretical considerations, specific model comparisons, or a follow-up study. The SAS syntax and an exemplary output are shown as follows. The syntax can also be downloaded at http://www.gesis.org/fileadmin/upload/dienstleistung/methoden/spezielle_dienste/zis_ehes/semmed.sas %macro semmed(location,file,x,y1,y2,y3,z1,z2,z3); /*Creating the data file*/ libname library "&location." data data; set library. x = &x.; y1 = &y1.; y2 = &y2.; y3 = &y3.; European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models z1 = &z1.; z2 = &z2.; z3 = &z3.; run; %macro sem(model, y, z); /*Specifying the structural equation model*/ ods exclude all; proc calis data=data covariance alpharms=.05 outfit=fit outram=parameter; var x y1-y3 z1-z3; lineqs y1 = f_y + e1, y2 = l2 f_y + e2, y3 = l3 f_y + e3, z1 = f_z + e4, z2 = l5 f_z + e5, z3 = l6 f_z + e6,M f_x = x + e0, f_z = &y., f_y = &z.; std e0 e1 e2 e3 e4 e5 e6 e7 e8 = 0 e1_var e2_var e3_var e4_var e5_var e6_var e7_var e8_var; effpart f_x -> f_y; ods output EffectsOf=indirect_&model.; run; /*Significance of the structural paths*/ data parameter; set parameter; if _name_ = "a" or _name_ = "b" or _name_ = "c"; keep _name_ p; p = 1-probnorm(_estim_/_stderr_); run; proc transpose data=parameter out=parameter; run; /*RMSEA*/ data RMSEA; set fit; if fitindex = "RMSEA Estimate"; keep fitvalue; run; data RMSEA; format model $7.; merge RMSEA parameter indirect_&model.; if a<.05 and b<.05 and c<.05 and pindirect<.05 then RMSEA=fitvalue; model = "&model."; keep model RMSEA; run; /*Chi2*/ data Chi2; set fit; if fitindex = "Chi-Square"; European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models keep fitvalue; run; data Chi2; merge Chi2 parameter indirect_&model.; if a<.05 and b<.05 and c<.05 and pindirect<.05 then Chi2=fitvalue; model = "&model."; keep model Chi2; run; /*degrees of fredom*/ data df; set fit; if fitindex = "Chi-Square DF"; df = fitvalue; model = "&model."; keep model df; run; /*BIC*/ data BIC; set fit; if fitindex = "Schwarz Bayesian Criterion"; keep fitvalue; run; data BIC; merge BIC parameter indirect_&model.; if a<.05 and b<.05 and c<.05 and pindirect<.05 then BIC=fitvalue; model = "&model."; keep model BIC; run; data fit_&model.; format model $37. RMSEA 3.2 Chi2 6.2 df 3.0 BIC 6.2; merge RMSEA Chi2 df BIC; by model; run; %mend; /*Specifying the different causal models*/ %sem(model1, e7, e8); %sem(model2, e7, c f_x + e8); %sem(model3, a f_x + e7, e8); %sem(model4, e7, b f_z + e8); %sem(model5, b f_y + e7, e8); %sem(model6, a f_x + e7, b f_z + e8); %sem(model7, a f_x + e7, c f_x + e8); %sem(model8, e7, c f_x + b f_z + e8); %sem(model9, b f_y + e7, c f_x + e8); %sem(model10, a f_x + b f_y + e7, e8); %sem(model11, a f_x + e7, b f_z + c f_x + e8); European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models %sem(model12, a f_x + b f_y + e7, c f_x + e8); data fit; set fit_model1-fit_model12; label model = "Model"; if model="model1" then model="Independence model"; if model="model2" then model="Simple effect (X->Y)"; if model="model3" then model="Simple effect (X->Z)"; if model="model4" then model="Simple effect (Z->Y)"; if model="model5" then model="Simple effect (Y->Z)"; if model="model6" then model="Complete mediation (X->Z, Z->Y)"; if model="model7" then model="Common cause (X->Z, X->Y)"; if model="model8" then model="Common effect on Y (X->Y, Z->Y)"; if model="model9" then model="Reflection model (X->Y, Y->Z)"; if model="model10" then model="Common effect on Z (X->Z, Y->Z)"; if model="model11" then model="Partial mediation (X->Z, Z->Y, X->Y)"; if model="model12" then model="Inverse mediation (X->Z, X->Y, Y->Z)"; run; /*Indirect effect based on complete mediation model*/ data indirect_complete; format sindirect 3.2 tindirect 3.2 pindirect 4.3; label sindirect = "Std. indirect effect"; label tindirect = "t value"; label pindirect = "p value"; set indirect_model6; keep sindirect tindirect pindirect; run; /*Indirect effect based on partial mediation model*/ data indirect_partial; format sindirect 3.2 tindirect 3.2 pindirect 4.3; label sindirect = "Std. indirect effect"; label tindirect = "t value"; label pindirect = "p value"; set indirect_model11; keep sindirect tindirect pindirect; run; /*Creating output*/ ods exclude none; ods options formdlim='-' nodate; proc print data=fit noobs label; title 'Model fit for causal models'; run; proc print data=indirect_complete noobs label; title 'Indirect effect based on complete mediation model’; run; proc print data=indirect_partial noobs label; title 'Indirect effect based on partial mediation model’; run; European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd. D. Danner et al. Mediation analysis with structural equation models %mend; /*Last line of code where location, name of file, and name of variables can be specified*/ %semmed(location=C:\, file=data, x=v1, y1=v2, y2=v3, y3=v4, z1=v5, z2=v6, z3=v7); Model fit for causal models Model RMSEA Chi2 df BIC Independence model Simple effect (X->Y) Simple effect (X->Z) Simple effect (Z->Y) Simple effect (Y->Z) Complete mediation (X->Z, Z->Y) Common cause (X->Z, X->Y) Common effect on Y (X->Y, Z->Y) Reflection model (X->Y, Y->Z) Common effect on Z (X->Z, Y->Z) Partial mediation (X->Z, Z->Y, X->Y) Inverse mediation (X->Z, X->Y, Y->Z) .18 .17 16 .10 .10 04 .15 . .07 .08 . .05 106.46 93.22 84.42 40.84 40.84 17.04 71.17 . 25.72 29.97 . 16.90 15 14 14 14 14 13 13 13 13 13 12 12 175.34 167.39 158.59 115.02 115.02 96.51 150.65 . 105.20 109.44 . 101.67 Indirect effect based on complete mediation model Std. indirect effect .18 t value 4.4 p value .000 Indirect effect based on partial mediation model Std. indirect effec .19 t value 4.0 p value .000 European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.