Download Mediation analysis with structural equation models: Combining

Eur. J. Soc. Psychol.
RESEARCH ARTICLE
Mediation analysis with structural equation models: Combining
theory, design, and statistics
Daniel Danner*, Dirk Hagemann† & Klaus Fiedler†
* GESIS – Leibniz Institute for the Social Sciences, Mannheim, Germany
† Institute of Psychology, Heidelberg University, Heidelberg, Germany
Correspondence
Daniel Danner, GESIS – Leibniz Institute for the
Social Sciences, Mannheim P.O. Box 122155,
D-68072, Germany.
E-mail: [email protected]
Received: 24 October 2013
Accepted: 24 November 2014
doi: 10.1002/ejsp.2106
Abstract
Statistical tests of indirect effects can hardly distinguish between genuine and
spurious mediation effects. The present research demonstrates, however,
that mediation analysis can be improved by combining a significance test of
the indirect effect with assessing the fit of causal models. Testing only the indirect effect can be misleading, because significant results may also be obtained when the underlying causal model is different from the mediation
model. We use simulated data to demonstrate that additionally assessing
the fit of causal models with structural equation models can be used to exclude subsets of models that are incompatible with the observed data. The results suggest that combining structural equation modeling with appropriate
research design and theoretically stringent mediation analysis can improve
scientific insights. Finally, we discuss limitations of the structural equation
modeling approach, and we emphasize the importance of non-statistical
methods for scientific discovery.
Experimental designs are commonly considered
the major method for making causal inferences
(e.g., Shadish, Cook, & Campbell, 2002). If an independent variable X is manipulated between randomized
experimental conditions, variation in a dependent variable Y can be attributed to variation in the independent
variable. For example, if persons are randomly assigned
to one of the two conditions, either receiving social support or not, resulting differences in persons’ well-being
can be assumed to reflect the impact of social support.
However, this basic experimental approach is limited
because it does not explain how the independent variable affects the dependent variable. In pursuing a hypothetical answer to this question, a researcher may want
to investigate whether the effect of social support on
well-being comes about through changes in the
persons’ attribution style (e.g., the tendency to attribute
problems to uncontrollable factors). Testing such an
explanatory hypothesis is a case for mediation analysis
(e.g., Baron & Kenny, 1986; Hayes, 2013; MacKinnon,
2008). This method allows researchers to investigate
whether the empirical evidence is consistent with a
mediation model X→Z→Y, which states that the impact
of an independent variable X on a dependent variable
Y is (at least in part) causally mediated by a proposed
mediator Z.
Traditionally, statistical mediation analysis tests
whether there is a significant indirect effect of the independent variable X via the proposed mediator Z on
the dependent variable Y. Over several decades, a variety of statistical procedures have been developed for
testing an indirect effect. One common procedure involves a series of models that regress the dependent
variable on the independent variable, with and without the mediator as a predictor (e.g., Baron & Kenny,
1986; Judd & Kenny, 1981). The indirect effect of
these regression models—that is, the effect of X on
Y mediated by Z—can be tested using bootstrap procedures (e.g., Frazier, Tix, & Barron, 2004; Hayes &
Scharkow, 2013; Preacher & Hayes, 2004) or parametric significance tests (e.g., Baron & Kenny, 1986;
Hayes, 2013; Sobel, 1982).
However, applying these statistical approaches can be
misleading because a significant test result for a mediator Z does not logically imply that Z is the true mediator.
Any statistical test of the model X→Z→Y presupposes as
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
an a priori premise that this is indeed the causal model of
theoretical interest. There is no logical reason to exclude
that other mediators Z′ or Z″ may provide a better explanation or that the three focal variables (X, Z, and Y) may
be related in different ways. In other words, researchers
engaging in a test of X→Z→Y rely on the a priori premise
that Z deserves to be considered the true mediator, and
they merely test whether this selected assumption can
account for a significant part of the variance. As long
as they do not test the fit of alternative models, they must
not infer from a significant mediation test that they have
found the true mediator statistically (e.g., James, Mulaik,
& Brett, 2006; MacCallum, Wegener, Uchino, & Fabrigar,
1993; MacKinnon, 2008). As a consequence, a significant mediation test result provides necessary but not sufficient evidence for a hypothetical, selectively tested
mediation model. A significant test of the mediation
model X→Z→Y may be obtained even when the
underlying covariance structure is different (e.g., Fiedler,
Schott, & Meiser, 2011; MacKinnon, Krull, & Lockwood,
2000). Fiedler et al. (2011) demonstrated in Monte Carlo
simulations that significant Sobel tests are regularly obtained when the third variable Z is not a true mediator
but, for example, merely a correlate of the dependent
variable Y (i.e., when Z is not generated to reflect an influence of X but of Y). In our example, a person’s attribution
style may not mediate social support but actually reflect
a by-product of well-being. Such demonstrations of
“mediation mimicry” imply that statistical tests can lead
to wrong conclusions, because they do not discriminate
between alternative causal structures that may also give
rise to the observed covariance structure of X, Y, and Z.
Hence, a more comprehensive analysis is needed to
choose between alternative models.
The aim of the present research is to demonstrate systematically that combining theoretical considerations,
study design, and statistical testing affords a means of
overcoming this fundamental weakness of mediation
analysis. First, we show that the number of possible
three-variate structures (involving X, Z, and Y) in an
experimental design can be limited by a few sensible
constraints. In particular, given the controlled manipulation of independent variable X in an experimental
design, so that the independent variable cannot be
affected by the dependent variable or the proposed mediator (e.g., MacCallum et al., 1993; MacKinnon, 2008;
Stone-Romero & Rosopa, 2011), there are only 12 possible causal structures. Then, we simulate empirical data
for each causal structure and illustrate that a significant
indirect effect may reflect not only a mediation structure but also several alternative structures (Fiedler
et al., 2011; MacKinnon et al., 2000). Using structural
equation modeling, we then demonstrate that there is
a class of causal model that fits with the empirical data
and that there is a class of causal models that do not fit
with the data.
Our paper adds to the existing literature by showing
which of the 12 causal structures can or cannot be discriminated from each other. It will be seen that a specifiable subset of causal models can be excluded as
incompatible with the given data, thus reducing the
number of viable candidates substantially. The foundation for this approach has been laid by several authors
(e.g., Cole & Maxwell, 2003; Lee & Hershberger, 1990;
MacCallum et al., 1993; MacKinnon, 2008), who suggest specifying alternative models by omitting or changing paths in a structural equation model.
Imposing Constraints on Viable Causal Structures
To be sure, the problem that three variables may be
causally related in several ways is a thorny one. There
are a large number of different causal models that may
all potentially explain an observed covariation pattern
between the independent and dependent variables
and the proposed mediator. As already noted, one
method to reduce the number of possible models is to
manipulate the independent variable X experimentally,
which effectively rules out the possibility that X can be affected by Y or Z. However, even in this genuinely experimental case, there still remains a variety of structural
models, in particular with regard to the relation of the
proposed mediator to the dependent variable. Figure 1
provides an overview of all possible models with reference to the example of attribution as a potential mediator
of the impact of social support on well-being.
(i) Independence model: The variables do not affect
each other. For example, persons are randomly
assigned to receive social support or no support,
but this does not affect well-being or attribution
style.
(ii) Single effect (X→Y): Social support affects wellbeing, but this effect is not mediated via the persons’ attribution style.
(iii) Single effect (X→Z): Social support affects attribution style but not well-being.
(iv) Single effect (Z→Y): The persons’ attribution style
affects their well-being. Social support does not
have an effect even though manipulated
experimentally.
(v) Single effect (Y→Z): The persons’ well-being causes
their attribution style. The manipulation of social
support does not have an effect.
(vi) Complete mediation (X→Z→Y): Social support affects the persons’ attribution style, which in turn
affects the persons’ well-being.
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
Fig. 1: Illustration of possible causal structures in an experimental setting
(vii) Common cause (X→Z, X→Y): Social support affects
the persons’ attribution style as well as their wellbeing.
(viii) Common effect on Y (X→Y, Z→Y): Social support as
well as the persons’ attribution style influences
the persons’ well-being. However, the treatment
does not affect the persons’ attribution style.
(ix) Reflection model (X→Y→Z): Social support affects
the persons’ well-being, which changes the persons’ attribution style.
(x) Common effect on Z (X→Z, Y→Z): The persons’ attribution style is affected by social support as well
as by the persons’ well-being. However, social
support does not affect the persons’ well-being.
(xi) Partial mediation (X→Z→Y, X→Y): Social support
affects the persons’ attribution style, which in turn
affects their well-being. In addition, social support
affects the persons’ well-being directly.
(xii) Inverse mediation (X→Z, X→Y→Z): Social support
affects the persons’ well-being, which affects their
attribution style. In addition, social support affects
the persons’ attribution style directly.
Structural Equation Modeling
Several researchers recommended structural equation
modeling as the preferred method for mediation analysis (e.g., Baron & Kenny, 1986; Frazier et al., 2004;
Hoyle & Smith, 1994). One important reason is that
the unreliability of the mediator and the dependent
variable will attenuate systematic relationships in multiple regression, whereas the mediator and the dependent
variable may be separated from their measurement errors in structural equation modeling. Another reason
is that structural equation modeling is much more flexible than regression (e.g., it is quite easy to include multiple mediators or dependent variables). This flexibility
becomes a crucial point when a decision between different causal models is the aim of the analysis. In particular, in multiple regression analysis, a causal model is
translated into a series of regression equations, and each
coefficient has to be estimated and tested separately (as
it is carried out with the series of three regression equations in the mediation analysis of Baron & Kenny,
1986). In contrast, structural equation modeling allows
(i) estimating and testing the entire causal model and
(ii) comparing different causal models using sophisticated goodness-of-fit statistics (see details as follows).
Given a limited number of possible causal models by
the study design, structural equation modeling can be
used to investigate the fit of alternative causal models
and thereby help to reduce the number of viable
models. In particular, structural equation models test
whether the constraints of specific models fit with the
observed data (e.g., Bollen, 1989; Hoyle, 1995; Kline,
2011). This allows researchers to reject those models
that are incompatible with the given data and to identify
those models that have to be examined more closely as
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
rivals of a hypothesized mediation model. For instance,
if the observed data (generated in accordance with a
simulated model) fit not only with a complete mediation structure (X→Z→Y) but also with a reflection
model (X→Y→Z), further research must be designed
to distinguish between these two models.
Let us briefly elaborate on this example to further explain the logic of our approach. The complete mediation
model (X→Z→Y) states that X is an exogenous variable
and Z is a function of X and a random variable υ of the form
Z = a*X + υ. In addition, this model states that Y is a function
of Z and a random variable ω, Y = b*Z + ω. If the variables
X, υ, and ω are not correlated, this implies that the variance σ2 of Z is a function of the variances of X and υ,
σ2Z = a2*σ2X + σ2υ, and that the variance of Y is a function
of the variances of Z and ω, σ2Y = b2*σ2Z + σ2ω. Accordingly, the covariance ρ between X and Y is ρX,Y = a*b*σ2X,
the covariance between X and Z is ρX,Z = a*σ2X, and the
covariance between Y and Z is ρY,Z = a2*b*σ2X + b*σ2υ.
By comparison, the reflection model (X→Y→Z) states
that X is an exogenous variable and Y is a function of X
and a random variable ω of the form Y = c*X + ω. In addition, this model states that Z is a function of Y and a random variable υ, Z = b*Y + υ. Again, if the variables X, υ,
and ω are not correlated, this implies that the variance
σ2 of Y is a function of the variances of X and ω,
σ2Y = c2*σ2X + σ2ω, and that the variance of Z is a function
of the variances of Y and υ, σ2Z = b2*σ2Y + σ2υ. Accordingly,
the covariance between X and Y is ρX,Y = c*σ2X, the covariance between X and Z is ρX,Z = b*c*σ2X, and the covariance
between Y and Z is ρY,Z = b*c2*σ2X + b*σ2ω. Hence, the two
models imply a different structure of the variances and
covariances between X, Y, and Z.
In sum, structural equation modeling allows comparing rival causal models for given data (e.g., Cole &
Maxwell, 2003; Lee & Hershberger, 1990; MacCallum
et al., 1993; MacKinnon, 2008). It can thus elucidate
which causal models can be distinguished by the given
empirical data and which models cannot (e.g., James
et al., 2006; MacCallum et al., 1993; Stelzl, 1986). Of
course, structural equation modeling is a statistical approach. It therefore cannot afford a final proof of a true
causal model (e.g., MacKinnon, 2008; Shadish et al.,
2002; Stone-Romero & Rosopa, 2010). It is always possible that other variables that have not been observed in
the experiment can account for the covariance of X, Y,
and Z. Moreover, it is always possible that replacing Z
by statistically related but psychologically different variables Z′, Z″, and so on may afford better solutions. However, we use simulated data to illustrate that the present
approach allows us to reduce the theoretical uncertainty
about the causal structure within the trivariate theoretical space spanned by X, Y, and Z.
Aim of the Present Simulation
The present simulation aims at an empirical setting
where the independent variable X was experimentally
manipulated and both the dependent variable Y and
the hypothesized mediator Z have been measured. As
argued previously, there are 12 possible causal structures that may have generated the observed variances
and covariances of the three variables, and only two of
these causal structures are mediation (complete and
partial, respectively). Using simulation techniques, we
will illustrate that a significance test of the indirect
effect alone will yield false decisions. This part of our
simulation aims to replicate and extend our knowledge about the fallibility of this method (as already
demonstrated by Fiedler et al., 2011; MacCallum
et al., 1993). In a second step, we will illustrate that
a combination of testing the indirect effect and structural equation modeling (i.e., deciding in favor for
mediation only if the test of the indirect effect is significant and if the mediation model can be accepted) can
increase the precision of the decision. This latter analysis will go beyond what is presently known, and it is
hoped that this will be a substantial improvement of
the methodology.
METHOD
Simulating Data
To illustrate the present approach, we simulated 12
classes of data sets, one class for each causal structure
in Figure 1. Each class included 1000 data sets, and
each data set included N = 200 observations. In each
data set, we first generated the latent construct
variables: the independent variable X, the dependent
variable Y, and the proposed mediator Z. We generated X as a binary variable (1 = experimental condition,
0 = control condition). Depending on the causal structure, we generated Y and Z as randomly distributed normal variables (M = 0, standard deviation [SD] = 1) or, if
dependent on each other, as linear combinations of each
other plus randomly distributed normal variables (M = 0,
SD = 1). For example, in the complete mediation data
sets, we generated the independent variable X as a binary
variable. We generated the mediator variable Z as a combination of the independent variable X and a randomly
distributed normal variable υ (M = 0, SD = 1), Z = X + υ.
Then, we generated the dependent variable as a combination of the mediator variable Z and a randomly distributed normal variable ω (M = 0, SD = 1), Y = Z + ω. The
algebraic specifications for all causal models are shown
in Table 1.
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
Table 1. Causal structure in simulated data sets
Class
1
2
3
4
5
6
7
8
9
10
11
12
Description
Independence model
Simple effect (X→Y)
Simple effect (X→Z)
Simple effect (Z→Y)
Simple effect (Y→Z)
Complete mediation
(X→Z→Y)
Common cause
(X→Z, X→Y)
Common effect on Y
(X→Y, Z→Y)
Reflection model
(X→Y→Z)
Common effect on Z
(X→Z, Y→Z)
Partial mediation
(X→Z→Y, X→Y)
Inverse mediation
(X→Z, X→Y→Z)
X=
Statistical Analyses
Y=
Z=
1 * υi
1 * X + 1 * υi
1 * υi
1 * Z + 1 * υi
1 * υi
1 * Z + 1 * υi
1 * ωi
1 * ωi
1 * X + 1 * ωi
1 * ωi
1 * Y + 1 * ωi
1 * X + 1 * ωi
[0,1] 1 * X + 1 * υi
1 * X + 1 * ωi
[0,1] 1 * X + 1 * Z +
1 * υi
[0,1] 1 * X + 1 * υi
1 * ωi
[0,1] 1 * υi
1 * X+1 * Y+
1 * ωi
1 * X + 1 * ωi
[0,1]
[0,1]
[0,1]
[0,1]
[0,1]
[0,1]
[0,1] 1 * X + 1 * Z +
1 * υi
[0,1] 1 * X + 1 * υi
1 * Y + 1 * ωi
1 * X+1 * Y+
1 * ωi
Note: υi /ωi = normally distributed random variables (M = 0.00, standard
deviation = 1.00) and i = data set. There were 1000 data sets per class.
Second, we included three indicator variables for the
dependent variable Y and three indicator variables for
the proposed mediator variable Z. Each indicator variable was computed as a linear combination of the latent
construct variable and a normally distributed random
error variable ε (M = 0, SD = 1). For example, the three
indicators for the dependent variable Y were generated
as Y1 = Y + ε1, Y2 = Y + ε2, and Y3 = Y + ε3. The algebraic
specifications for all indicator variables in all causal
structures are shown in Appendix A. For the structural
equation model analyses, we used the independent variable X, the three indicators for the proposed mediator
variables Z1, Z2, and Z3, and the three indicators for the
dependent variables Y1, Y2, and Y3.1
All parameters in the simulation were fixed to be either zero or one. In
doing so, we followed Cohen (1990) who suggested using a simple-isbetter principle in multiple regression analyses. In particular, he supposed to use unit weights in prediction equations with +1 when the
predictors are positively related and with 0 when they are poorly related. The rationale is that empirically estimated beta weights always
depend on the particular sample that was used for their estimation.
He argues that the precision of the prediction will be more likely to become worse when empirically estimated betas are used instead of unit
weights (see Cohen, 1990, p. 1306 for an analytical demonstration of
this rule). In the absence of other constraints, we decided to apply this
simple-is-better principle in the present simulations in hope of a great
generalizability of our results to a wide range of empirical settings. In
the simulated data sets, the effect sizes for the direct effects between
the manifest variables (mean of the indicators) were r = .40. According
to Cohen (1990), these associations reflect medium to large effects and
hence can be seen as realistic for many experimental settings.
1
Testing the Indirect Effect
We tested the indirect effect with the Sobel test,
bootstrapped confidence intervals (CI), and structural
equation models in each data set. The Sobel test is a
regression approach designed for analyzing three
manifest variables (e.g., Baron & Kenny, 1986; Sobel,
1982). First, the regression coefficient a of the proposed mediator Z on the independent variable X
and its standard error sa are estimated. Second, the
regression coefficient b of the dependent variable Y
on the proposed mediator Z and its standard error sb
are estimated. Third, the indirect effect a*b and its
standard error sa*b are computed (see Goodman,
1960; MacKinnon et al., 2000; Preacher & Hayes,
2004; Sobel, 1982 for a discussion of different formulas). Sobel’s Z is calculated by the formula Z ¼ sa*b ,
a* b
whereby |Z| > 1.96 indicates a significant indirect effect. The analyses were based on the independent
variable X, the mean score of the three indicators of
the dependent variable Y (Cronbach’s α = .65–.92),
and the mean score of the three indicators of the
proposed mediator Z (Cronbach’s α = .65–.93). For
each data set, we estimated the indirect effect a*b
and its Z-value.
Because the Sobel test has statistical limitations
(e.g., Hayes & Scharkow, 2013; Preacher & Hayes,
2004; Shrout & Bolger, 2002), bootstrapping can be
considered superior. Bootstrapping estimates the upper
limit and the lower limit of the CI of an indirect effect. A
CI above zero indicates a significant positive indirect
effect. We computed bootstrapped 95% CI (2000
bootstrap samples). For the present analysis, we used a
nonparametric bootstrap procedure with the SAS (SAS
Institute, Cary, NC, USA) script provided by Preacher
and Hayes (2004).
In addition, we tested the indirect effect with
structural equation models (see details as follows).
Within the structural equation modeling framework,
the magnitude of an indirect effect was estimated
with an iterative algorithm (e.g., maximum likelihood), and the significance was tested by dividing
the magnitude of the indirect effect by its standard
error, producing a standard normally distributed variable Z (e.g., Iacobucci, Saldanha, & Deng, 2007;
Kline, 2011; MacKinnon, 2008). Testing indirect effects with structural equation models has been applied in different contexts (e.g., Ahearne, Mathieu,
& Rapp, 2005; Hilliard et al., 2013; King, King, Fairbank,
Keane, & Adams, 1998; Quilty, Godfrey, Kennedy, &
Bagby, 2010).
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
Testing the Model Fit with Structural Equation Models
Structural equation modeling can be used to test
whether a predefined structural model fits an observed
data set. In a first step, a covariance matrix of the observed data is estimated without any model assumptions. In a second step, a covariance matrix of the
observed data is estimated given the predefined model
structure (e.g., a complete mediation model X→Z→Y)
by searching for model parameters that minimize the
discrepancy between the model-free estimated covariance matrix and the model-dependent covariance matrix of the observed variables. In a third step, the
model fit is computed by comparing both covariance
matrices. A good model fit indicates that there is no substantial difference between the matrices.
We used two criteria to evaluate the fit of the models:
the root mean square error of approximation (RMSEA)
and the significance test of the estimated model parameters. The RMSEA is a parsimonious fit index that takes
the discrepancy between the observed and the modelimplied covariance matrices into account as well as the
model’s complexity. A smaller RMSEA indicates a better
model fit. Models with an RMSEA > 0.06 should be
rejected (Hu & Bentler, 1999). Models containing nonsignificant parameter estimates (including the indirect
effect from X to Y) were rejected because all models
are nested in either the partial mediation model or the
inverse mediation model. In other words, a model containing a zero parameter is equivalent to another more
parsimonious model. For example, a partial mediation
model with a zero path coefficient from the latent independent variable X to the latent dependent variable Y is
equivalent to the complete mediation model.
In addition, we used the χ 2 value and the Bayesian information criterion (BIC) for comparing models with
each other. The χ 2 difference test can be used to test
whether two nested models2 differ significantly. A
smaller χ 2 value indicates a better fit. Models that differ
in one degree of freedom differ significantly if they differ
at least by Δχ 2 = 3.84, models that differ in two degrees
of freedom differ significantly if they differ at least by
Δχ 2 = 5.99, and models that differ in three degrees of
freedom differ significantly if they differ by Δχ 2 = 7.81.
If two models differ significantly, the better fitting model
is accepted. If two models do not differ significantly, the
more parsimonious model (with more degrees of freedom) is accepted. The BIC value is a descriptive fit
2
Two models are nested if one model can be formulated as a special case
of the other model. For example, the reflection model can be formulated as a special case of the inverse mediation model where the path
between the independent variable and the proposed mediator is set
to zero.
Fig. 2: Example of a complete mediation model with three indicators for the
proposed mediator Z and three indicators for the dependent variable Y
index, which takes the parsimoniousness of the models
into account and can also be used to compare nonnested models. A ΔBIC > 10 indicates a meaningful difference (Raftey, 1995).
We estimated the fit of all 12 causal models to all data
sets. The structural models were specified as shown in
Figure 1. We modeled each construct as a latent variable
with three indicators, thus yielding sufficient degrees of
freedom for the parameter estimation. The measurement
models for the latent dependent variable and the latent
mediator variable were specified as τ-congeneric models.3
A graphical illustration of the complete mediation model is
shown in Figure 2. The model parameters were estimated
using the maximum likelihood function implemented in
the CALIS procedure in SAS 9.3 (SAS Institute).
RESULTS AND DISCUSSION
Tests of Indirect Effects (Bootstrap, Sobel, and
Structural Equation Models)
We estimated bootstrapped CI of the indirect effect in all
data sets. There were 1000 data sets in each class and 200
observations in each data set. We used the independent
variable X, the mean score of the three indicators of the
dependent variable Y, and the mean score of the three indicators of the proposed mediator Z for the analyses. The
results are summarized in Table 2.
3
A τ-congeneric measurement model specifies all indicators Yi of a scale
as a weighted linear combination of a latent construct variable τ and a
residual variable εi, Yi = λi*τ + εi.
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
Independence model
Simple effect (X→Y)
Simple effect (X→Z)
Simple effect (Z→Y)
Simple effect (Y→Z)
Complete mediation (X→Z→Y)
Common cause (X→Z, X→Y)
Common effect on Y (X→Y, Z→Y)
Reflection model (X→Y→Z)
Common effect on Z (X→Z, Y→Z)
Partial mediation (X→Z, Z→Y, X→Y)
Inverse mediation (X→Z, X→Y, Y→Z)
1
2
3
4
5
6
7
8
9
10
11
12
1
0
3
2
2
100
3
3
99
99
100
100
Percentage of how often indirect
effect was significanta
0.00 (0.01)
0.00 (0.01)
0.01 (0.07)
0.00 (0.12)
0.00 (0.09)
0.75 (0.15)
0.00 (0.07)
0.00 (0.12)
0.43 (0.10)
0.43 (0.10)
0.75 (0.15)
0.86 (0.13)
Indirect effect a*b
Note: SEM, structural equation model.
|Z| > 1.96 indicates a significant effect.
a
Indirect effect based on bootstrapped confidence intervals.
b
No indirect effect estimated because neither the complete mediation model nor the partial mediation model fits the data.
c
Indirect effect estimated based on the complete mediation model.
d
Indirect effect estimated based on the partial mediation model.
Model used to generate data
Class
Upper estimate
0.04 (0.02)
0.04 (0.02)
0.14 (0.08)
0.24 (0.13)
0.18 (0.10)
1.05 (0.17)
0.15 (0.08)
0.24 (0.13)
0.64 (0.12)
0.64 (0.12)
1.05 (0.17)
1.12 (0.15)
Lower estimate
0.04 (0.02)
0.04 (0.02)
0.14 (0.08)
0.24 (0.13)
0.19 (0.10)
0.48 (0.13)
0.14 (0.08)
0.24 (0.13)
0.23 (0.09)
0.23 (0.09)
0.48 (0.13)
0.62 (0.11)
0.01 (0.42)
0.01 (0.44)
0.00 (0.95)
0.01 (0.97)
0.02 (0.98)
5.09 (0.65)
0.00 (0.95)
0.01 (0.97)
4.09 (0.75)
4.09 (0.75)
5.09 (0.65)
6.61 (0.54)
Z-value (Sobel)
.b
.b
.b
0.02 (0.99)c
0.02 (0.99)c
5.48 (0.76)c
0.00 (0.98)d
.b
3.98 (0.69)d
3.69 (0.70)d
4.94 (0.59)d
6.88 (0.62)c
Z-value (SEM)
Table 2. Percentage of how often the test of indirect effect was significant (%), indirect effect, 95% confidence intervals of bootstrapped indirect effects (2000 samples), and Sobel’s Z and significance test via
structural equation modeling (mean score of 1000 data sets in each class, standard deviation in brackets)
D. Danner et al.
Mediation analysis with structural equation models
D. Danner et al.
Mediation analysis with structural equation models
As can be seen, the bootstrapped CI correctly indicated a mediation effect (a CI above zero) in 100% of
the data sets that were actually generated according to
the complete mediation structure (CI = [0.48; 1.05])
and in 100% of the data of the partial mediation structure (CI = [0.48; 1.05]). However, bootstrapping also
supported a mediation effect in 99% of the data sets that
were generated according to the reflection structure
(CI = [0.23; 0.64]) and the common-effect-on-Z structure (CI = [0.23; 0.64]) and 100% of the data sets of
the inverse mediation structure (CI = [0.62; 1.12]). As
evident from Table 2, the Sobel test and significance
tests via structural equation models yield the same pattern of results (cf. Hayes, 2013). Across all models and
analyses, we would falsely accept a (partial) mediation
in 31% of all cases when relying on the bootstrap test
of the indirect effect alone. These findings demonstrate
that observing a significant indirect effect—no matter
whether it was tested with a bootstrap procedure, a
Sobel test, or a structural equation model—does not unequivocally prove that the apparent mediation effect
was actually used to generate the data. Even in an experimental setting, a significant indirect effect can be
caused by different structures.
Table 3. Percentage of how often a causal model (columns) was accepted in the different data sets (rows) based on root mean square error of approximation (RMSEA)
Percentage model was accepted
1
2
3
4
5
6
7
8
9
10
11
12
Model used to generate data
1
2
3
4
5
6
7
8
9
10
11
12
Independence model
Simple effect (X→Y)
Simple effect (X→Z)
Simple effect (Z→Y)
Simple effect (Y→Z)
Complete mediation (X→Z, Z→Y)
Common cause (X→Z, X→Y)
Common effect on Y (X→Y, Z→Y)
Reflection model (X→Y, Y→Z)
Common effect on Z (X→Z, Y→Z)
Partial mediation (X→Z, Z→Y, X→Y)
Inverse mediation (X→Z, X→Y, Y→Z)
96
2
1
0
0
0
0
0
0
0
0
0
5
95
0
0
0
0
2
0
0
0
0
0
5
0
96
0
0
0
1
0
0
0
0
0
5
0
0
95
94
0
0
3
1
3
0
0
5
0
0
95
94
0
0
3
1
3
0
0
0
0
2
2
2
94
5
1
43
43
15
94
0
4
5
0
0
0
95
0
0
0
0
0
0
7
0
4
5
0
0
94
14
0
1
0
0
6
0
5
5
42
4
43
94
1
94
16
0
0
5
5
5
13
0
0
1
95
0
1
0
0
0
0
0
4
2
2
88
0
93
5
0
0
0
0
0
91
5
0
5
5
5
93
Note: There were 1000 data sets per class and N = 200 observations per data set. Models were accepted if RMSEA ≤ 0.06 and model parameters were
significant.
Bold figures indicate how often the correct model fit the data.
Table 4. Percentage of how often a causal model (columns) was accepted in the different data sets (rows) based on Bayesian information criterion (BIC)
Percentage model was accepted
Model used to generate data
1
2
3
4
5
6
7
8
9
10
11
12
Independence model
Simple effect (X→Y)
Simple effect (X→Z)
Simple effect (Z→Y)
Simple effect (Y→Z)
Complete mediation (X→Z, Z→Y)
Common cause (X→Z, X→Y)
Common effect on Y (X→Y, Z→Y)
Reflection model (X→Y, Y→Z)
Common effect on Z (X→Z, Y→Z)
Partial mediation (X→Z, Z→Y, X→Y)
Inverse mediation (X→Z, X→Y, Y→Z)
1
100
5
5
5
5
0
0
1
1
0
0
0
2
3
4
5
6
7
8
9
10
11
12
2
100
0
0
0
0
5
7
6
0
0
0
1
0
100
0
0
5
5
0
0
5
0
0
0
0
0
99
99
5
0
4
5
6
0
0
0
0
0
99
99
5
0
5
5
5
0
0
0
0
0
1
1
98
0
1
33
14
5
95
0
1
2
0
0
3
100
0
3
0
5
5
0
0
0
2
2
1
0
99
33
0
4
0
0
0
0
1
1
34
0
14
98
1
93
6
0
0
0
2
2
34
0
0
1
100
0
5
0
0
0
0
0
11
0
1
99
0
98
5
0
0
0
0
0
98
0
0
13
2
5
98
Note: There were 1000 data sets per class and N = 200 observations per data set. Models were accepted if ΔBIC ≤ 10 and model parameters were
significant.
Bold figures indicate how often the correct model fit the data.
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
Model Fit and Model Comparison
Considering the fit between the observed data and the
various causal models allows us to discard distinct
models as incompatible with the given covariance structure. Table 3 shows how often models were accepted in
the different data sets based on their RMSEA. A model
was accepted when RMSEA ≤ 0.06 (Hu & Bentler,
1999). In addition, Table 4 shows how often models
were accepted based on the χ 2 value. A model was accepted if there was no nested model that fitted the data
significantly better (Bollen, 1989). Table 5 shows how
often models were accepted based on their BIC. A
model was accepted if no other model had a BIC that
was 10 points smaller (Raftey, 1995).
As can be seen, no matter which fit index is used, the
number of viable causal models can substantially be reduced. For example, the RMSEA suggests that in the
data sets that were generated according to the complete
mediation model, a complete mediation model (94%),
an inverse mediation model (91%), or a reflection
model (42%) fit the data best. Similarly, the χ 2 values
suggest that a complete mediation model (91%) or an
inverse mediation model (68%) fits the data. The BIC
suggests that a complete mediation model (98%), an inverse mediation model (98%), or a reflection or
common-effect-on-Z model (34%) fit best.
Likewise, the RMSEA suggests that in the data
sets that were generated according to the common
cause model, a common cause model (99%) fits
the data best. Likewise, the BIC suggests that the
common cause model explains these data sets best
(100%). The χ 2 difference test suggests that not
only the common cause model fits the data best
(94%) but also the complete mediation model
(91%) or the reflection model (91%) fits the data.
This is due to the fact that the χ 2 difference test
can only distinguish between nested models, and
the common cause model, complete mediation
model, and the reflection model are not nested in
each other.
As can be seen in Tables 3, 4, and 5, this pattern
of results was the same for all generated data sets.
For each class of data sets, a class of causal models
can be rejected, whereas another class of viable
causal models remains. Therefore, it is not possible
to use the structural equation modeling approach
to completely eliminate the ambiguity that arises
from testing the indirect effect only. However, these
findings also show that structural equation modeling can reduce the number of possible causal explanations to a considerable degree, typically from 12
possible explanatory models down to two or three
viable models that are consistent with the data
(Tables 3, 4, and 5). For example, combining a test
of the indirect effect with assessing the fit of the
underlying causal models (via the RMSEA)
decreases the false alarm rate (for a mediation)
from 31% to 19% across all data sets (cf. Tables 2
and 3). In addition, structural equation modeling
reveals which alternative causal models can or cannot explain the observed data. It should be noted
that well-fitting models always imply covariance
matrices that are very similar to the empirical covariance of the manifest variables. Thus, similarity of
the covariance matrices is, not surprisingly, the key
2
Table 5. Percentage of how often a causal model (columns) was accepted in the different data sets (rows) based on χ difference test
Percentage model was accepted
Model used to generate data
1
2
3
4
5
6
7
8
9
10
11
12
Independence model
Simple effect (X→Y)
Simple effect (X→Z)
Simple effect (Z→Y)
Simple effect (Y→Z)
Complete mediation (X→Z, Z→Y)
Common cause (X→Z, X→Y)
Common effect on Y (X→Y, Z→Y)
Reflection model (X→Y, Y→Z)
Common effect on Z (X→Z, Y→Z)
Partial mediation (X→Z, Z→Y, X→Y)
Inverse mediation (X→Z, X→Y, Y→Z)
1
2
3
4
5
6
7
8
9
10
11
12
100
0
0
0
0
0
0
0
0
0
0
0
0
100
7
5
5
0
4
6
6
0
0
0
0
5
95
0
0
4
5
0
0
4
0
0
0
0
0
95
95
5
0
4
5
6
0
0
0
0
0
95
95
5
0
5
4
5
0
0
0
0
0
0
0
91
91
34
11
89
5
96
0
0
0
0
0
1
94
3
0
0
5
5
0
0
0
0
0
2
3
90
12
0
5
0
0
0
0
0
0
12
91
87
89
34
94
6
0
0
0
0
0
11
3
0
2
90
0
5
0
0
0
0
0
0
0
0
67
0
86
0
0
0
0
0
0
68
0
0
1
0
0
85
2
Note: There were 1000 data sets per class and N = 200 observations per data set. If two nested models differed significantly in their χ values, the model with
2
the smaller χ value was accepted. If two nested models did not differ significantly, the more parsimonious model was accepted.
Bold figures indicate how often the correct model fit the data.
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
to understanding the clusters of non-discriminable
causal models.4
Appendix B additionally shows the average fit indices
for the different data sets. These numbers suggest that
the number of eligible causal models can be further reduced by comparing the non-rejected models with each
other. For example, for the data that were generated according to the reflection model, the RMSEA suggests that
a reflection model, a partial mediation model, a complete
mediation model, or an inverse mediation model fit with
the data. The χ 2 values and the BIC further suggest that
the complete mediation model fit the data substantially
worse than a reflection model, a partial mediation model,
or an inverse mediation model. Hence, by combining the
RMSEA with the χ 2 and the BIC, the number of eligible
models may further be reduced.
(ii)
(iii)
(iv)
GENERAL DISCUSSION
A significant indirect effect does not prove that the data
have been generated by a causal mediation mechanism.
Even in experimental settings, other causal structures
than a mediation structure can also give rise to a significant test result of the indirect effect. However, as the
present article demonstrates, combining a significance
test of the indirect effect with evaluating the fit of alternative causal model can reduce the uncertainty within a
given trivariate theory space. Our simulation results illustrate that structural equation modeling allows researchers to reject specific classes of alternative causal
models and to concentrate on a clearly reduced set of viable models. Therefore, combining theoretical considerations, a significance test of the indirect effect, and
structural equation modeling can be very useful to reach
a better understanding of the mechanisms that may explain a given array of empirical evidence. We recommend a six-step approach to investigating mediation:
(i) Before data acquisition, limit the number of possible causal models by design: An experimental design allows excluding all models in which the
(v)
4
For the present analysis, we used a sample size of N = 200 observation
per simulated data set because this is the lower bound recommendation
for structural equation models (e.g., Hoyle, 1995) and, especially in experimental designs, it may be expensive to assess larger samples. As in
some experimental settings, it may not be possible to assess 200 participants, while in other settings, it may be possible to assess even larger
samples; we additionally run the analysis with N = 100 observations
per data set and N = 500 observations per data set, which revealed similar results. Likewise, we changed the cutoff value to RMSEA ≤ 0.10 and
alternatively used the cutoff value comparative fit index > 0.95, which
also reveal similar results.
(vi)
independent variable is affected by the proposed
mediator or the dependent variable.
Identify the causal models for your experimental
design. In a standard experimental design with
three constructs, there are 12 different causal
models (Figure 1).
Specify all remaining causal models as structural
equation models: Specify the proposed mediator
and the dependent variable as latent variables. Specify the relation between the latent variables according to the remaining causal models that you have
identified (an example of a complete mediation
model is shown in Figure 2). Additional models
can be constructed by changing the path between
the latent construct variables. Structural equation
models also allow testing whether an indirect effect is significant.
Investigate the fit of each causal model and compare the fit between models. By this means, the
number of eligible causal models can substantially
be reduced. The RMSEA allows deciding whether
a model sufficiently fits observed data or whether
a model should be rejected. Models with an
RMSEA > 0.06 can be rejected because of insufficient fit (Hu & Bentler, 1999). Models containing
nonsignificant parameters can be rejected because
a model containing a zero parameter is equivalent
to another more parsimonious model. The χ 2 difference test further allows deciding whether two
nested models fit the data equally well or whether
one model fits the data significantly better. The BIC
allows deciding whether non-nested models fit the
data equally well or whether one model fits the
data substantially better, where a ΔBIC > 10 suggests a meaningful difference. Using the χ 2
difference test and the BIC allows affirming
that the models that were rejected based on
their RMSEA do fit worse than the non-rejected
models and also further reducing the number
of eligible models by showing that some of the
non-rejected model fits the data better than other
non-rejected models.
Identify the remaining causal models that cannot
be rejected: The present study suggests that for
each causal structure, there are typically two or
three models between which cannot be discriminated by structural equation modeling.
Engage in deliberate attempts to reduce the number of remaining models: This can be carried out
by either a follow-up experiment where the
proposed mediator variable is manipulated
(e.g., Shrout & Bolger, 2002; Spencer, Zanna, &
Fong, 2005; Stone-Romero & Rosopa, 2011), a
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
longitudinal study (e.g., Cole & Maxwell, 2003), or
a theory-driven discourse.
Limitations
The Approach is Designed for an Experimental Setting
The basis for the present analysis approach was rejecting
several causal models by design. The experimental
manipulation of the independent variable allowed us
to reject all causal models in which the dependent variable or the proposed mediator affected the independent
variable. In a non-experimental design, one would not
be able to reject these models on a priori grounds. The
set of possible causal models would then grow from 12
classes of models up to at least 27 different classes of
models if the assumption is given up that one independent variable cannot be affected by the other variables.
This would increase the number of models that fits with
an observed data set, leaving various causal explanations for observed data. Likewise, there would be
several models that show an identical fit with the observed data (James et al., 2006; MacCallum et al.,
1993; Stelzl, 1986).
To be sure, an experimental manipulation is not the
only reason for excluding specific models. Given such
variables as biological sex or age, it is possible to exclude
all models that assign them the role of a dependent variable or mediator (e.g., MacCallum et al., 1993). For example, if somebody wants to investigate whether
gender role self-concept mediates the relation between
biological sex and dominant behavior, all causal models
could be excluded in which sex is affected by behavior
or gender role self-concept. Longitudinal design may
also sometimes allow one to discard certain models on
a priori ground. This highlights the usefulness of combining theory, design, and statistical methods and not
blindly applying multivariate statistics.
The Measurement of the Proposed Mediator and Dependent Variable Must be Valid
The results of the present analysis are only valid if the
indicators of the dependent variable and the indicators
of the proposed mediator are valid. For one thing, the
results of structural equation modeling will be biased if
the discriminant validity of the indicators of the proposed mediator and the dependent variable cannot be
established. Let us illustrate this complication with
reference to the well-being example. Suppose that we
want to investigate whether the relation between social
support and well-being is mediated by attribution style.
One of the items that we use to measure well-being is
Mediation analysis with structural equation models
“I am a great person.” One person may feel well and
therefore agree to this item. However, another person
who does not feel very well may still be convinced to
be a great person because he or she often achieves success. Such an item would be a blend of well-being and
attribution style, containing both well-being variance
and attribution style variance. Hence, this item will covary with the attribution style items and with the other
well-being items regardless of whether the attribution
style and well-being are related. This must bias the
structural equation modeling results because it artificially increases the association between the latent
well-being variable (the dependent variable) and the latent attribution style variable (the proposed mediator).
Therefore, it is necessary to ensure discriminant validity
of the manifest indicators.
Another problem arises if all indicators of the proposed mediator are correlated with another nonobserved construct. This problem was also discussed as
the spurious mediation problem (e.g., Fiedler et al.,
2011; MacKinnon et al., 2000). In particular, the validity of statistical testing of indirect effects has been criticized because the measurement of the proposed
mediator Z could alternatively be interpreted as a correlate of another potential mediator Z′. For one more illustration of this fundamental problem, suppose that
the persons’ attribution style is measured with items like
“Success depends on good relations,” “You have to be
lucky to be successful,” and “I cannot influence much
of what is happening to me.” These items may indeed
measure attribution style, but they may as well measure
other latent variables, such as optimism. Whenever
item overlap reduces the discriminant validity of latent
constructs, no statistical test can decide whether the effect of social support is mediated by one or the other
construct. This argument applies to regression analysis
with manifest variables as well as to structural equation
modeling.
Still, regarding the validity of single indicators, the
structural equation modeling approach is more robust
than the traditional regression approach.5 This is because the latent mediator variable is specified as the
common variance of all indicator variables. A serious
validity problem only arises if all or most indicators are
correlated with the same non-observed construct. No
validity problem exists if only one indicator is correlated
with another construct or if each indicator overlaps with
a different non-observed construct, because the
5
However, Ledgerwood and Shrout (2011) note that the standard errors of indirect effects based on latent variables can be greater than
the standard errors based on manifest variables, especially with heterogeneous items.
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
“unwanted” variance portions of these indicators would
be treated as specific variance components in the structural equation model and thus as part of the residual error variance. For example, assuming that we measured
attribution style with three items, one being Sartre’s
quote “I am the architect of my own self, my own character and destiny.” One could criticize that this particular item does not only measure attribution style but
also measure the person’s liking of Jean-Paul Sartre.
Using a traditional regression approach, the Sartre item
may affect the mean score of the indicators and hence,
bias the results of the mediation analysis. However,
using the structural equation modeling approach, the
Sartre-specific variance proportion would be treated as
measurement error in the model and hence, not bias
the results of the mediation analysis (see also StoneRomero & Rosopa, 2010; Shadish et al., 2002).
Structural Equation Modeling Cannot Exclude all
Alternative Causal Structures
The present results corroborate the contention that in
most cases, more than one causal model fits a data set.
For example, the common-effect-on-Z data set could
be fitted not only by the common-effect-on-Z model
but also by the inverse mediation model. Likewise,
there are models that are statistically equivalent. For example, model 4 (Z→Y) implies the same covariance
structure than model 5 (Y→Z) and thus cannot be distinguished by statistical procedures. Structural equation
modeling only allows reducing the number of possible
explanations for the indirect effect within the trivariate
framework of given variables. However, as a matter of
fact, researchers have to admit that no statistical method
can rule out all alternative models involving other variables not included in the trivariate framework. Therefore, as a matter of principle, researchers must always
go beyond statistics to complete the picture.
The Present Simulations Illustrate a Method But Do Not
Demonstrate General Rules
When we use our method, there remains a class of
models that cannot be further distinguished and that
all may explain the data equally well. Because the results of a simulation depend on the choice of the parameters, it is entirely possible that the class of remaining
models might have different members if we had chosen
different parameters. However, we are confident that
the main findings of our simulation may replicate for
other parameters, that is, the structural equation modeling approach helps to eliminate some although not all of
the alternative models. We therefore consider our
simulation study not as a demonstration of general rules
but as an illustration that our approach helps to decide
between concurrent models for which we could not decide if we rely only on the Sobel test or on the bootstrap
evaluation of the alleged mediation effect.
Beyond Statistical Testing
The approach we have described so far is a purely statistical one. Of course, there are ancillary and in several
settings preferable approaches for an analysis of mediation. For example, Spencer, Zanna, and Fong (2005)
suggested investigating mediation by manipulating the
mediator variable experimentally. This approach has
the great advantage that the direction of the relation between the proposed mediator and the dependent variable can be controlled. For example, if we could
experimentally control how persons attribute their successes and failures, we could exclude all causal models
that state that well-being affects attribution style. However, it may not always be possible to control constructs
such as attribution style (especially over a longer period
of time). In addition, it is not certain that the effect of the
experimental manipulation of the proposed mediator is
the same as the indirect effect of the mediator triggered
by the independent variable (Kenny, 2008). StoneRomero and Rosopa (2010) and Mathieu and Taylor
(2006) discuss further approaches to investigating mediation analysis with different designs, as the authors conclude the following: “ideally, the results of studies using
all such alternatives should converge (p. 700).”
It has also been suggested to measure the mediator
and the dependent variable at several measurement occasions or in a prescribed timely order (e.g., MacCallum
et al., 1993; MacKinnon, 2008; Cole & Maxwell, 2003).
The rationale behind this approach is that the variable
that is measured second cannot affect the variable that
was measured first. However, this may not be true in
all settings because the variables may have affected each
other before the first measurement took place (see also
MacKinnon, 2008). For example, social support may affect a person’s well-being after a first treatment. From
that moment on, persons may start to attribute their
successes to their capability. At the beginning of the second treatment session, the researcher assesses the persons’ attribution style and registers that the style has
changed. At the beginning of the third session, the researcher assesses the person’s well-being and registers
that their well-being has improved. Therefore, the
selected time interval is critical, and even if the variables
were measured in a prescribed timely order, the direction of the relation may not be clear (e.g., Cole &
Maxwell, 2003; MacKinnon, 2008).
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
CONCLUSION
Combining a significance test of the indirect effect
with evaluating the fit of alternative causal models
with structural equation models can improve mediation analyses. However, structural equation modeling
is not a silver bullet and has its limitations and weaknesses as does every method. The strength of the
present approach is that it can be combined with complementary approaches. For example, bootstrapping
or a Sobel test may indicate that an indirect effect is
significant, granted the mediation occurred. Structural
equation modeling can demonstrate that several alternative causal explanations for the significant indirect
effect can be rejected. Experimentally, manipulating
the mediator variable or measuring the proposed mediator before measuring the dependent variable may
further support this hypothesis. However, the most
important methodological tool for a scientific explanation consists of cleverly designed follow-up experiments informed by the insights gained from
structural equation modeling. We hope that such a
combined approach can improve the use of mediation
analysis in future research in social psychology and
thereby help scientists to gain a deeper understanding
of empirical reality.
ACKNOWLEDGEMENTS
This research was supported by grants awarded by the
Deutsche Forschungsgemeinschaft to the second
(HA3044/7-1) and third authors (Fi 294/23-1). We
gratefully thank Matthias Blümke, Jemaine Clement,
Sebastian Nagengast, Oliver Schilling, and three anonymous reviewers for helpful comments on an earlier
draft of this paper.
REFERENCES
Ahearne, M., Mathieu, J., & Rapp, A. (2005). To empower
or not to empower your sales force? An empirical examination of the influence of leadership empowerment
behavior on customer satisfaction and performance.
Journal of Applied Psychology, 90(5), 945–955. http://dx.
doi.org/10.1037/0021-9010.90.5.945
Arbuckle, J. L. (2013). Amos (version 22.0) [computer program]. Chicago: SPSS.
Baron, R. M., & Kenny, D. A. (1986). The moderator–mediator
variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182. http://dx.
doi.org/10.1037/0022-3514.51.6.1173
Bollen, K. A. (1989). Structural equations with latent variables.
Oxford England: John Wiley & Sons.
Mediation analysis with structural equation models
Cohen, J. (1990). Things I have learned (so far). American
Psychologist, 45, 1304–1312. http://dx.doi.org/10.1037/
0003-066X.45.12.1304
Cole, D. A., & Maxwell, S. E. (2003). Testing mediational
models with longitudinal data: Questions and tips in the
use of structural equation modeling. Journal of Abnormal
Psychology, 112(4), 558–577. http://dx.doi.org/10.1037/
0021-843X.112.4.558
Fiedler, K., Schott, M., & Meiser, T. (2011). What mediation
analysis can (not) do. Journal of Experimental Social
Psychology, 47(6), 1231–1236. http://dx.doi.org/10.1016/j.
jesp.2011.05.007
Frazier, P. A., Tix, A. P., & Barron, K. E. (2004). Testing moderator and mediator effects in counseling psychology
research. Journal of Counseling Psychology, 51(1), 115–134.
http://dx.doi.org/10.1037/0022-0167.51.1.115
Goodman, L. A. (1960). On the exact variance of products.
Journal of the American Statistical Association, 55,708–713.
http://dx.doi.org/10.1080/01621459.1960.10483369
Hayes, A. F. (2013). Introduction to mediation, moderation, and
conditional process analysis: A regression-based approach.
New York: Guilford Press.
Hayes, A. F., & Scharkow, M. (2013). The relative trustworthiness of inferential tests of the indirect effect in statistical
mediation analysis: Does method really matter? Psychological Science, 24(10), 1918–1927. http://dx.doi.org/10.1177/
0956797613480187
Hilliard, M. E., Holmes, C. S., Chen, R., Maher, K., Robinson,
E., & Streisand, R. (2013). Disentangling the roles of
parental monitoring and family conflict in adolescents’
management of type 1 diabetes. Health Psychology, 32(4),
388–396. http://dx.doi.org/10.1037/a0027811
Hoyle, R. H. (1995). Structural equation modeling: Concepts, issues, and applications. Thousand Oaks, CA US: Sage Publications, Inc.
Hoyle, R. H., & Smith, G. T. (1994). Formulating clinical research hypotheses as structural equation models: A conceptual overview. Journal of Consulting and Clinical Psychology,
62, 429–440.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in
covariance structure analysis: Conventional criteria versus
new alternatives. Structural Equation Modeling, 6, 1–55. doi:
10.1080/10705519909540118
Iacobucci, D., Saldanha, N., & Deng, X. (2007). A meditation
on mediation: Evidence that structural equations models
perform better than regressions. Journal of Consumer Psychology, 17(2), 139–153. http://dx.doi.org/10.1016/S10577408(07)70020-7
James, L. R., Mulaik, S. A., & Brett, J. M. (2006). A tale of two
methods. Organizational Research Methods, 9(2), 233–244.
http://dx.doi.org/10.1177/1094428105285144
Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating
mediation in treatment evaluations. Evaluation Review, 5(5),
602–619. http://dx.doi.org/10.1177/0193841X8100500502
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
Kenny, D. A. (2008). Reflections on mediation. Organizational
Research Methods, 11(2), 353–358. http://dx.doi.org/
10.1177/1094428107308978
King, L. A., King, D. W., Fairbank, J. A., Keane, T. M., &
Adams, G. A. (1998). Resilience–recovery factors in
post-traumatic stress disorder among female and male
Vietnam veterans: Hardiness, postwar social support,
and additional stressful life events. Journal of Personality
and Social Psychology, 74(2), 420–434. http://dx.doi.org/
10.1037/0022-3514.74.2.420
Kline, R. B. (2011). Principles and practice of structural equation
modeling (3rd ed.). New York, NY US: Guilford Press.
Ledgerwood, A., & Shrout, P. E. (2011). The trade-off
between accuracy and precision in latent variable models
of mediation processes. Journal of Personality and Social
Psychology, 101(6), 1174–1188. http://dx.doi.org/10.1037/
a0024776.
Lee, S., & Hershberger, S. (1990). A simple rule for generating
equivalent models in covariance structure modeling. Multivariate Behavioral Research, 25(3), 313–334. http://dx.doi.
org/10.1207/s15327906mbr2503_4
MacCallum, R. C., Wegener, D. T., Uchino, B. N., &
Fabrigar, L. R. (1993). The problem of equivalent models
in applications of covariance structure analysis. Psychological Bulletin, 114(1), 185–199. http://dx.doi.org/10.1037/
0033-2909.114.1.185
MacKinnon, D. P. (2008). Introduction to statistical mediation
analysis. New York, NY: Taylor & Francis Group/
Lawrence Erlbaum Associates.
MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000).
Equivalence of the mediation, confounding and suppression effect. Prevention Science, 1(4), 173–181. http://dx.doi.
org/10.1023/a:1026595011371
Mathieu, J. E. & Taylor, S. R. (2006). Clarifying conditions and decision points for mediational type
inferences in organizational behavior. Journal of Organizational Behavior, 27, 1031–1056. http://dx.doi.org/
10.1002/job.406
Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS
procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, &
Computers, 36(4), 717–731. http://dx.doi.org/10.3758/
bf03206553
Quilty, L. C., Godfrey, K. M., Kennedy, S. H., & Bagby,
R. M. (2010). Harm avoidance as a mediator of treatment response to antidepressant treatment of patients
with major depression. Psychotherapy and Psychosomatics, 79(2), 116–122. http://dx.doi.org/10.1159/
000276372
Raftey, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized
causal inference. Boston, MA US: Houghton, Mifflin and
Company.
Shrout, P. E., & Bolger, N. (2002). Mediation in experimental
and nonexperimental studies: New procedures and recommendations. Psychological Methods, 7(4), 422–445. http://
dx.doi.org/10.1037/1082-989X.7.4.422
Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equations models. In S. Leinhart
(Ed.), Sociological methodology (pp. 290–312). San Francisco:
Jossey-Bass.
Spencer, S. J., Zanna, M. P., & Fong, G. T. (2005). Establishing a causal chain: Why experiments are often more
effective than mediational analyses in examining psychological processes. Journal of Personality and Social
Psychology, 89(6), 845–851. http://dx.doi.org/10.1037/00223514.89.6.845
Stelzl, I. (1986). Changing a causal hypothesis without
changing the fit: Some rules for generating equivalent path
models. Multivariate Behavioral Research, 21(3), 309–331.
http://dx.doi.org/10.1207/s15327906mbr2103_3
Stone-Romero, E. F., & Rosopa, P. J. (2010). Research
design options for testing mediation models and their
implications for facets of validity. Journal of Managerial
Psychology, 25(7), 697–712. http://dx.doi.org/10.1108/
02683941011075256
Stone-Romero, E. F., & Rosopa, P. J. (2011). Experimental
tests of mediation models: Prospects, problems, and some
solutions. Organizational Research Methods, 14(4), 631–646.
http://dx.doi.org/10.1177/1094428110372673
APPENDIX A: TABLE A1
Equations used for generating the variables in the simulated data sets
Model class
Independence model
Simple effect (X→Y)
Simple effect (X→Z)
Simple effect (Z→Y)
Variable
Xij
Y1ij
Y2ij
Y3ij
Z1ij
Z2ij
Z3ij
Xij
Y1ij
Y2ij
Y3ij
Z1ij
Z2ij
Z3ij
Xij
Y1ij
Y2ij
Y3ij
Z1ij
Z2ij
Z3ij
Xij
Y1ij
Equation
[0;1]
1 * υij + 1 * ε1ij
1 * υij + 1 * ε2ij
1 * υij + 1 * ε3ij
1 * ωij + 1 * ε4ij
1 * ωij + 1 * ε5ij
1 * ωij + 1 * ε6ij
[0;1]
1 * Xij + 1 * υij + 1 * ε1ij
1 * Xij + 1 * υij + 1 * ε2ij
1 * Xij + 1 * υij + 1 * ε3ij
1 * ωij + 1 * ε4ij
1 * ωij + 1 * ε5ij
1 * ωij + 1 * ε6ij
[0;1]
1 * υij + 1 * ε1ij
1 * υij + 1 * ε2ij
1 * υij + 1 * ε3ij
1 * Xij + 1 * ωij + 1 * ε4ij
1 * Xij + 1 * ωij + 1 * ε5ij
1 * Xij + 1 * ωij + 1 * ε6ij
[0;1]
1 * ωij + 1 * υij + 1 * ε1ij
(Continues)
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
TABLE
A1 (Continued)
Table 1 (Continued)
Model class
Simple effect (Y→Z)
Complete mediation
TABLE
A1 (Continued)
Table 1 (Continued)
Variable
Z1ij
Z2ij
Z3ij
Xij
1 * ωij + 1 * υij + 1 * ε2ij
1 * ωij + 1 * υij + 1 * ε3ij
1 * ωij + 1 * ε4ij
1 * ωij + 1 * ε5ij
1 * ωij + 1 * ε6ij
[0;1]
1 * υij + 1 * ε1ij
1 * υij + 1 * ε2ij
1 * υij + 1 * ε3ij
1 * υij + 1 * ωij + 1 * ε4ij
1 * υij + 1 * ωij + 1 * ε5ij
1 * υij + 1 * ωij + 1 * ε6ij
[0;1]
1 * Xij + 1 * ωij + 1 *
υij + 1 * ε1ij
1 * Xij + 1 * ωij + 1 *
υij + 1 * ε2ij
1 * Xij + 1 * ωij + 1 *
υij + 1 * ε3ij
1 * Xij + 1 * ωij + 1 * ε4ij
1 * Xij + 1 * ωij + 1 * ε5ij
1 * Xij + 1 * ωij + 1 * ε6ij
[0;1]
Y1ij
Y2ij
Y3ij
Z1ij
Z2ij
Z3ij
Xij
1 * Xij + 1 * υij + 1 * ε1ij
1 * Xij + 1 * υij + 1 * ε2ij
1 * Xij + 1 * υij + 1 * ε3ij
1 * Xij + 1 * ωij + 1 * ε4ij
1 * Xij + 1 * ωij + 1 * ε5ij
1 * Xij + 1 * ωij + 1 * ε6ij
[0;1]
Y1ij
1 * Xij + 1 * ωij + 1 * υij + 1
* ε1ij
1 * Xij + 1 * ωij + 1 * υij + 1
* ε2ij
1 * Xij + 1 * ωij + 1 * υij
+ 1 * ε3ij
1 * ωij + 1 * ε4ij
1 * ωij + 1 * ε5ij
1 * ωij + 1 * ε6ij
Y2ij
Y3ij
Z1ij
Z2ij
Z3ij
Xij
Y1ij
Y2ij
Y3ij
Z1ij
Z2ij
Z3ij
Xij
Y1ij
Y2ij
Y3ij
Common cause
(X→Z, X→Y)
Common effect on Y
(X→Y, Z→Y)
Equation
Y2ij
Y3ij
Z1ij
Z2ij
Z3ij
(Continues)
Model class
Reflection model
(X→Y→Z)
Common effect on Z
(X→Z, Y →Z)
Partial mediation
(X→Z→Y, X→Y)
Variable
Xij
[0;1]
Y1ij
Y2ij
Y3ij
Z1ij
Z2ij
Z3ij
Xij
1 * Xij + 1 * υij + 1 * ε1ij
1 * Xij + 1 * υij + 1 * ε2ij
1 * Xij + 1 * υij + 1 * ε3ij
1 * Xij + 1 * υij + 1 * ωij + 1 * ε4ij
1 * Xij + 1 * υij + 1 * ωij + 1 * ε5ij
1 * Xij + 1 * υij + 1 * ωij + 1 * ε6ij
[0;1]
Y1ij
Y2ij
Y3ij
Z1ij
Z2ij
Z3ij
Xij
1 * υij + 1 * ε1ij
1 * υij + 1 * ε2ij
1 * υij + 1 * ε3ij
1 * Xij + 1 * υij + 1 * ωij + 1 * ε4ij
1 * Xij + 1 * υij + 1 * ωij + 1 * ε5ij
1 * Xij + 1 * υij + 1 * ωij + 1 * ε6ij
[0;1]
Y1ij
1 * Xij + 1 * Xij + 1 * ωij + 1 *
υij + 1 * ε1ij
1 * Xij + 1 * Xij + 1 * ωij + 1 *
υij + 1 * ε2ij
1 * Xij + 1 * Xij + 1 * ωij + 1 *
υij + 1 * ε3ij
1 * Xij + 1 * ωij + 1 * ε4ij
1 * Xij + 1 * ωij + 1 * ε5ij
1 * Xij + 1 * ωij + 1 * ε6ij
[0;1]
Y2ij
Y3ij
Inverse mediation
(X→Y→Z, X→Z)
Equation
Z1ij
Z2ij
Z3ij
Xij
Y1ij
Y2ij
Y3ij
Z1ij
Z2ij
Z3ij
1 * Xij + 1 * υij + 1 * ε1ij
1 * Xij + 1 * υij + 1 * ε2ij
1 * Xij + 1 * υij + 1 * ε3ij
1 * Xij + 1 * υij + 1 * Xij + 1 * ωij + 1
* ε4ij
1 * Xij + 1 * υij + 1 * Xij + 1 *
ωij + 1 * ε5ij
1 * Xij + 1 * υij + 1 * Xij + 1 *
ωij + 1 * ε6ij
Note: There were 1000 data sets in each class i and 200 observation in
each data set j. The terms υ, ω, and ε16 were normally distributed random
variables with M = 0 and standard deviation = 1.
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
APPENDIX B: TABLE B1
Average root mean square error of approximation (RMSEA) for the different causal models (columns) in the different data sets (rows) and standard deviation
in brackets
RMSEA for model
Model used to
generate data
1
1
Independence model
2
Simple effect (X→Y)
3
Simple effect (X→Z)
4
Simple effect (Z→Y)
5
Simple effect (Y→Z)
6
Complete mediation
(X→Z, Z→Y)
Common cause
(X→Z, X→Y)
Common effect on Y
(X→Y, Z→Y)
Reflection model
(X→Y, Y→Z)
Common effect on Z
(X→Z, Y→Z)
Partial mediation
(X→Z, Z→Y, X→Y)
Inverse mediation
(X→Z, X→Y, Y→Z)
7
8
9
10
11
12
0.02
(0.02)
0.11
(0.02)
0.11
(0.02)
0.16
(0.02)
0.16
(0.02)
0.21
(0.02)
0.15
(0.02)
0.18
(0.02)
0.21
(0.02)
0.18
(0.02)
0.25
(0.02)
0.25
(0.02)
2
3
4
5
6
7
8
9
10
11
12
0.02
(0.02)
0.02
(0.02)
.
0.01
(0.02)
0.11
(0.02)
0.02
(0.02)
0.17
(0.02)
0.17
(0.02)
0.19
(0.02)
0.11
(0.02)
.
0.02
(0.02)
0.11
(0.01)
.
0.02
(0.02)
0.11
(0.02)
.
.
.
.
.
.
.
.
.
0.02
(0.02)
.
0.01
(0.02)
.
.
.
.
.
0.02
(0.02)
0.02
(0.02)
0.11
(0.02)
0.15
(0.02)
0.10
(0.02)
0.11
(0.02)
0.10
(0.02)
0.16
(0.02)
0.16
(0.02)
.
.
.
0.18
(0.02)
0.17
(0.02)
0.02
(0.02)
.
0.02
(0.02)
0.11
(0.02)
0.12
(0.02)
0.02
(0.02)
0.09
(0.02)
.
0.02
(0.02)
0.02
(0.02)
0.06
(0.03)
0.10
(0.02)
0.06
(0.03)
0.02
(0.02)
0.10
(0.02)
0.02
(0.02)
0.08
(0.03)
0.02
(0.02)
0.02
(0.02)
0.02
(0.02)
0.09
(0.02)
0.11
(0.02)
.
.
0.02
(0.02)
0.02
(0.02)
0.11
(0.02)
0.15
(0.02)
0.10
(0.02)
0.11
(0.02)
0.10
(0.02)
0.16
(0.02)
0.16
(0.02)
0.02
(0.02)
0.02
(0.02)
.
.
.
.
0.02
(0.02)
0.02
(0.02)
.
0.17
(0.02)
0.17
(0.02)
0.20
(0.02)
0.11
(0.02)
0.17
(0.02)
0.19
(0.02)
0.20
(0.02)
0.20
(0.02)
0.23
(0.02)
0.20
(0.02)
0.17
(0.02)
0.23
(0.02)
0.20
(0.02)
0.02
(0.02)
0.02
(0.02)
0.02
(0.02)
0.02
(0.02)
0.10
(0.02)
0.17
(0.02)
0.06
(0.03)
0.06
(0.03)
0.08
(0.02)
0.02
(0.02)
0.17
(0.02)
0.18
(0.02)
0.17
(0.02)
0.17
(0.02)
0.11
(0.02)
0.16
(0.02)
0.11
(0.02)
0.02
(0.02)
0.16
(0.02)
0.11
(0.02)
.
.
0.02
(0.02)
.
0.02
(0.02)
0.02
(0.02)
0.02
(0.03)
0.02
(0.02)
0.01
(0.02)
0.02
(0.02)
Note: There were 1000 data sets per class and N = 200 observations per data set. The average RMSEA is reported if the model parameters are significant in
at least 5% of the simulated data sets.
TABLE B2
2
Average χ value for the different causal models (columns) in the different data sets (rows) and standard deviation in brackets
2
Model used to
generate data
1 Independence
model
2 Simple effect
(X→Y)
3 Simple effect
(X→Z)
4 Simple effect
(Z→Y)
5 Simple effect
(Y→Z)
6 Complete
mediation
(X→Z, Z→Y)
7 Common cause
(X→Z, X→Y)
8 Common effect
on Y (X→Y, Z→Y)
χ value for model
1
2
3
4
5
6
7
8
9
14.00
(4.73)
14.20
(5.56)
.
13.59
(4.48)
48.35
(11.23)
14.05
(5.27)
94.65
(15.80)
93.95
(16.03)
117.27
(18.08)
14.17
(4.71)
46.81
(11.49)
.
14.17
(4.71)
46.81
(11.49)
.
.
.
.
.
12.38
(5.03)
.
14.29
(5.38)
14.13
(5.45)
50.12
(12.20)
13.93
(5.77)
12.92
(4.69)
.
12.66
(5.23)
.
14.29
(5.38)
14.13
(5.45)
50.12
(12.20)
47.47
(9.93)
13.30
(5.72)
12.40
(3.73)
14.38
(5.33)
13.26
(5.27)
.
93.22
(17.17)
90.18
(16.50)
14.72
(4.99)
47.51
(11.76)
13.16
(6.49)
13.72
(5.19)
24.87
(8.45)
78.51
(14.42)
43.17
(11.39)
38.54
(14.42)
43.17
(11.39)
38.54
(11.26)
40.42
(11.38)
13.07
(5.15)
.
50.43
(13.36)
13.31
(5.26)
38.95
(10.92)
24.67
(8.10)
15.09
(5.43)
49.45
(12.50)
49.32
(12.02)
92.25
(16.74)
93.10
(17.02)
146.54
(22.86)
90.99
(14.76)
96.37
(16.36)
125.52
(19.11)
83.08
(16.17)
112.31
(18.25)
48.35
(12.00)
91.22
(16.63)
47.80
(11.95)
.
10
.
13.06
(4.82)
13.41
(4.62)
13.74
(5.97)
33.32
(10.15)
47.85
(10.58)
.
11
12
.
.
14.64
(13.39)
.
.
.
.
.
.
.
.
12.25
(5.09)
.
12.08
(4.70)
.
.
(Continues)
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Table
Mediation analysis with structural equation models
1
(Continued)
TABLE
B2 (Continued)
2
χ value for model
Model used to
generate data
9 Reflection model
(X→Y, Y→Z)
10 Common effect
on Z (X→Z, Y→Z)
11 Partial mediation
(X→Z, Z→Y,
X→Y)
12 Inverse mediation
(X→Z, X→Y,
Y→Z)
Model df
2
3
4
5
6
7
8
9
10
11
12
147.47
(22.87)
113.19
(18.22)
197.25
(25.82)
1
112.22
(18.13)
125.29
(17.88)
125.55
(19.09)
126.38
(19.45)
92.02
(16.97)
161.98
(20.18)
49.97
(12.48)
43.27
(11.30)
85.79
(15.41)
49.97
(12.48)
43.27
(11.30)
85.78
(15.41)
24.61
(8.76)
24.70
(8.43)
32.58
(10.34)
91.06
(16.84)
94.22
(16.21)
90.28
(16.47)
33.24
(10.15)
.
46.59
(11.97)
13.06
(5.22)
84.22
(15.02)
12.04
(5.02)
.
47.59
(11.92)
13.17
(5.37)
39.63
(10.68)
13.24
(5.25)
12.32
(5.11)
13.30
(6.35)
12.21
(4.24)
11.34
(4.30)
198.26
(25.58)
163.26
(20.03)
126.31
(19.44)
85.80
(15.29)
85.82
(15.29)
13.08
(5.30)
91.06
(16.85)
84.01
(14.73)
32.45
(10.67)
47.30
(12.28)
12.33
(4.16)
12.03
(5.08)
15
14
14
14
14
13
13
13
13
13
12
12
2
Note: There were 1000 data sets per class and N = 200 observations per data set. The average χ value is reported if the model parameters are significant in
at least 5% of the simulated data sets.
TABLE B3
Average Bayesian information criterion (BIC) value for the different causal models (columns) in the different data sets (rows) and standard deviation in
brackets
Model used to
generate data
1
2
3
4
5
6
7
8
9
10
11
Independence
model
Simple effect
(X→Y)
Simple effect
(X→Z)
Simple effect
(Z→Y)
Simple effect
(Y→Z)
Complete
mediation
(X→Z, Z→Y)
Common
cause (X→Z,
X→Y)
Common
effect on Y
(X→Y, Z→Y)
Reflection
model (X→Y,
Y→Z)
Common
effect on Z
(X→Z, Y→Z)
Partial
mediation
(X→Z, Z→Y,
X→Y)
BIC value for model
1
2
3
4
5
6
7
8
9
10
11
12
88.17
(4.73)
88.37
(5.56)
.
87.76
(4.48)
122.53
(11.23)
88.22
(5.27)
168.83
(15.80)
168.13
(16.03)
185.44
(18.08)
88.34
(4.71)
120.99
(11.49)
.
88.34
(4.71)
120.99
(11.49)
.
.
.
.
.
.
.
.
.
.
.
.
.
88.46
(5.38)
88.31
(5.45)
124.29
(12.20)
92.53
(4.82)
92.89
(4.62)
93.22
(5.97)
112.79
(10.15)
.
.
.
.
.
97.02
(5.09)
151.96
(16.17)
83.97
(5.43)
118.32
(12.50)
118.20
(12.02)
161.13
(16.74)
161.98
(17.02)
215.42
(22.86)
165.17
(14.76)
170.55
(16.36)
199.69
(19.11)
92.14
(5.23)
.
88.46
(5.38)
88.31
(5.45)
124.29
(12.20)
92.77
(5.27)
91.87
(3.73)
93.85
(5.33)
92.74
(5.27)
93.40
(5.77)
92.39
(4.69)
.
172.70
(17.17)
169.65
(16.50)
94.19
(4.99)
126.99
(11.76)
92.63
(6.49)
93.19
(5.19)
104.35
(8.45)
122.53 121.98 152.69
(12.00) (11.95) (14.42)
152.69
(14.42)
118.02
(11.26)
92.55
(5.15)
129.90
(13.36)
118.43
(10.92)
127.32
(10.58)
.
96.85
(4.70)
181.19
(18.25)
165.39
(16.63)
117.34
(11.39)
117.34
(11.39)
119.89
(11.38)
.
92.79
(5.26)
104.15
(8.10)
.
.
.
216.35
(22.87)
186.40 200.55 124.14
(18.13) (19.45) (12.48)
124.14
(12.48)
104.09
(8.76)
170.53
(16.84)
112.72
(10.03)
92.64
(5.37)
126.06
(11.97)
96.81
(5.02)
98.07
(6.35)
182.07
(18.22)
199.46 166.20 117.45
(17.88) (16.97) (11.30)
117.45
(11.30)
104.17
(8.43)
173.70
(16.21)
.
119.11
(10.68)
92.54
(5.22)
.
96.98
(4.24)
266.13
(25.82)
199.73 236.15 159.97
(19.09) (20.18) (15.41)
159.97
(15.41)
112.06
(10.43)
169.75
(16.47)
127.06
(11.92)
92.72
(5.25)
163.69
(15.02)
97.10
(5.11)
96.12
(4.30)
.
.
.
91.85
(5.03)
.
(Continues)
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
Table
1
(Continued)
TABLE
B2 (Continued)
BIC value for model
Model used to
generate data
12
Inverse
mediation
(X→Z, X→Y,
Y→Z)
1
267.13
(25.58)
2
3
4
237.18 200.49 160.00
(20.03) (19.44) (15.29)
5
6
7
8
9
10
11
12
160.00
(15.29)
92.56
(5.30)
170.54
(16.85)
163.48
(14.73)
111.93
(10.67)
126.77
(12.28)
97.11
(4.16)
96.81
(5.08)
Note: There were 1000 data sets per class and N = 200 observations per data set. The average BIC is reported if the model parameters are significant in at
least 5% of the simulated data sets.
APPENDIX C: GUIDELINE FOR APPLYING THE PRESENT APPROACH WITH AMOS
AMOS (Arbuckle, 2013) offers a graphical user interface where the user can specify a model by drawing a path diagram as shown in Figure 2:
(i) Specify a manifest variable for the independent variable X.
(ii) Specify a latent variable for the proposed mediator Z with three manifest variables.
(iii) Specify a latent variable for the dependent variable Y with three manifest variables.
(iv) Specify the relation between the latent variables according to the causal model shown in Figure 1. There will be one structural equation model
for each causal model. These models differ in the number and directing of the path between the latent variables. Add a residual variable for each
endogenous variable in the model. Endogenous variables are variables that are explained by other variables in the model (e.g., the dependent
variable in the simple effect on Y model or the proposed mediator Z and the dependent variable Y in the complete mediation model)
(v) Estimate the model parameters and the model fit for each causal model.
(vi) Investigate the fit of each causal model and compare the fit between models. Models with an RMSEA > 0.06 can be rejected because of insufficient fit (Hu & Bentler, 1999). Models containing nonsignificant parameters can be rejected because a model containing a zero parameter is
2
equivalent to another more parsimonious model. Nested models can be compared with the χ difference test, where a significant difference
suggests rejecting the worse fitting model. Non-nested models can be compared based on their BIC value where a ΔBIC > 10 suggests a meaningful difference.
(vii) Engage in further attempts to reject the remaining models by theoretical considerations or a follow-up study.
APPENDIX D: GUIDELINE FOR APPLYING THE PRESENT APPROACH WITH SAS
The syntax requires SAS 9.3 and a SAS data file containing an independent variable, three indicators for the dependent variable, and three indicators for the
dependent variable.
First, the location and the name of the SAS data file and the name of the variables must be specified. This can easily be carried out by modifying the last line
of the syntax. In the present example, “C:\” is the location of the SAS file, “data” is the name of the SAS file, “v1” is the name of the independent variable,
“v2”–“v4” are the names of the indicators for the dependent variables, and “v5”–“v7” are the names of the indicators of the proposed mediator variable:
%semmedðlocation ¼ C : ∖; file ¼ data; x ¼ v1; y1 ¼ v2; y2 ¼ v3; y3 ¼ v4; z1 ¼ v5; z2 ¼ v6; z3 ¼ v7Þ
2
The syntax produces a table containing the RMSEA, χ value, the df, and the BIC for each model. In addition, the script provides estimates of the indirect
effect based on the complete mediation model and the partial mediation model. After evaluating the results, engage in further attempts to reject the remaining models by theoretical considerations, specific model comparisons, or a follow-up study. The SAS syntax and an exemplary output are shown as
follows. The syntax can also be downloaded at http://www.gesis.org/fileadmin/upload/dienstleistung/methoden/spezielle_dienste/zis_ehes/semmed.sas
%macro semmed(location,file,x,y1,y2,y3,z1,z2,z3);
/*Creating the data file*/
libname library "&location."
data data;
set library.
x = &x.;
y1 = &y1.;
y2 = &y2.;
y3 = &y3.;
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
z1 = &z1.;
z2 = &z2.;
z3 = &z3.;
run;
%macro sem(model, y, z);
/*Specifying the structural equation model*/
ods exclude all;
proc calis data=data covariance alpharms=.05
outfit=fit outram=parameter;
var x y1-y3 z1-z3;
lineqs
y1 = f_y + e1, y2 = l2 f_y + e2, y3 = l3 f_y + e3,
z1 = f_z + e4, z2 = l5 f_z + e5, z3 = l6 f_z + e6,M
f_x = x + e0,
f_z = &y.,
f_y = &z.;
std
e0 e1 e2 e3 e4 e5 e6 e7 e8 = 0 e1_var e2_var e3_var e4_var e5_var e6_var e7_var e8_var;
effpart f_x -> f_y;
ods output EffectsOf=indirect_&model.;
run;
/*Significance of the structural paths*/
data parameter;
set parameter;
if _name_ = "a" or _name_ = "b" or _name_ = "c";
keep _name_ p;
p = 1-probnorm(_estim_/_stderr_);
run;
proc transpose data=parameter out=parameter; run;
/*RMSEA*/
data RMSEA;
set fit;
if fitindex = "RMSEA Estimate";
keep fitvalue;
run;
data RMSEA;
format model $7.;
merge RMSEA parameter indirect_&model.;
if a<.05 and b<.05 and c<.05 and pindirect<.05 then RMSEA=fitvalue;
model = "&model.";
keep model RMSEA;
run;
/*Chi2*/
data Chi2;
set fit;
if fitindex = "Chi-Square";
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
keep fitvalue;
run;
data Chi2;
merge Chi2 parameter indirect_&model.;
if a<.05 and b<.05 and c<.05 and pindirect<.05 then Chi2=fitvalue;
model = "&model.";
keep model Chi2;
run;
/*degrees of fredom*/
data df;
set fit;
if fitindex = "Chi-Square DF";
df = fitvalue;
model = "&model.";
keep model df;
run;
/*BIC*/
data BIC;
set fit;
if fitindex = "Schwarz Bayesian Criterion";
keep fitvalue;
run;
data BIC;
merge BIC parameter indirect_&model.;
if a<.05 and b<.05 and c<.05 and pindirect<.05 then BIC=fitvalue;
model = "&model.";
keep model BIC;
run;
data fit_&model.;
format model $37. RMSEA 3.2 Chi2 6.2 df 3.0 BIC 6.2;
merge RMSEA Chi2 df BIC;
by model;
run;
%mend;
/*Specifying the different causal models*/
%sem(model1, e7, e8);
%sem(model2, e7, c f_x + e8);
%sem(model3, a f_x + e7, e8);
%sem(model4, e7, b f_z + e8);
%sem(model5, b f_y + e7, e8);
%sem(model6, a f_x + e7, b f_z + e8);
%sem(model7, a f_x + e7, c f_x + e8);
%sem(model8, e7, c f_x + b f_z + e8);
%sem(model9, b f_y + e7, c f_x + e8);
%sem(model10, a f_x + b f_y + e7, e8);
%sem(model11, a f_x + e7, b f_z + c f_x + e8);
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
%sem(model12, a f_x + b f_y + e7, c f_x + e8);
data fit;
set fit_model1-fit_model12;
label model = "Model";
if model="model1" then model="Independence model";
if model="model2" then model="Simple effect (X->Y)";
if model="model3" then model="Simple effect (X->Z)";
if model="model4" then model="Simple effect (Z->Y)";
if model="model5" then model="Simple effect (Y->Z)";
if model="model6" then model="Complete mediation (X->Z, Z->Y)";
if model="model7" then model="Common cause (X->Z, X->Y)";
if model="model8" then model="Common effect on Y (X->Y, Z->Y)";
if model="model9" then model="Reflection model (X->Y, Y->Z)";
if model="model10" then model="Common effect on Z (X->Z, Y->Z)";
if model="model11" then model="Partial mediation (X->Z, Z->Y, X->Y)";
if model="model12" then model="Inverse mediation (X->Z, X->Y, Y->Z)";
run;
/*Indirect effect based on complete mediation model*/
data indirect_complete;
format sindirect 3.2 tindirect 3.2 pindirect 4.3;
label sindirect = "Std. indirect effect";
label tindirect = "t value";
label pindirect = "p value";
set indirect_model6;
keep sindirect tindirect pindirect;
run;
/*Indirect effect based on partial mediation model*/
data indirect_partial;
format sindirect 3.2 tindirect 3.2 pindirect 4.3;
label sindirect = "Std. indirect effect";
label tindirect = "t value";
label pindirect = "p value";
set indirect_model11;
keep sindirect tindirect pindirect;
run;
/*Creating output*/
ods exclude none;
ods options formdlim='-' nodate;
proc print data=fit noobs label;
title 'Model fit for causal models';
run;
proc print data=indirect_complete noobs label;
title 'Indirect effect based on complete mediation model’;
run;
proc print data=indirect_partial noobs label;
title 'Indirect effect based on partial mediation model’;
run;
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.
D. Danner et al.
Mediation analysis with structural equation models
%mend;
/*Last line of code where location, name of file, and name of variables can be specified*/
%semmed(location=C:\, file=data, x=v1, y1=v2, y2=v3, y3=v4, z1=v5, z2=v6, z3=v7);
Model fit for causal models
Model
RMSEA
Chi2
df
BIC
Independence model
Simple effect (X->Y)
Simple effect (X->Z)
Simple effect (Z->Y)
Simple effect (Y->Z)
Complete mediation (X->Z, Z->Y)
Common cause (X->Z, X->Y)
Common effect on Y (X->Y, Z->Y)
Reflection model (X->Y, Y->Z)
Common effect on Z (X->Z, Y->Z)
Partial mediation (X->Z, Z->Y, X->Y)
Inverse mediation (X->Z, X->Y, Y->Z)
.18
.17
16
.10
.10
04
.15
.
.07
.08
.
.05
106.46
93.22
84.42
40.84
40.84
17.04
71.17
.
25.72
29.97
.
16.90
15
14
14
14
14
13
13
13
13
13
12
12
175.34
167.39
158.59
115.02
115.02
96.51
150.65
.
105.20
109.44
.
101.67
Indirect effect based on complete mediation model
Std.
indirect
effect
.18
t
value
4.4
p
value
.000
Indirect effect based on partial mediation model
Std.
indirect
effec
.19
t
value
4.0
p
value
.000
European Journal of Social Psychology 00 (2015) 00-00 Copyright © 2015 John Wiley & Sons, Ltd.