Download Uncertainty analysis in carbon cycle models of forest ecosystems

e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
available at www.sciencedirect.com
journal homepage: www.elsevier.com/locate/ecolmodel
Uncertainty analysis in carbon cycle models of forest
ecosystems: Research needs and development of a
theoretical framework to estimate error propagation
Guy R. Larocque a,∗ , Jagtar S. Bhatti b , Robert Boutin a , Oleg Chertov c
a
Natural Resources Canada, Canadian Forest Service, Laurentian Forestry Centre, 1055 du P.E.P.S., P.O. Box 10380,
Stn. Ste-Foy, Quebec, Quebec G1V 4C7, Canada
b Natural Resources Canada, Canadian Forest Service, Northern Forestry Centre, 5320-122 Street, Edmonton, Alberta T6H 3S5, Canada
c Biological Research Institute, St. Petersburg State University, Peterhoff, 198904 St. Petersburg, Russia
a r t i c l e
i n f o
a b s t r a c t
Article history:
Few process-based models of the carbon (C) cycle of forest ecosystems integrate uncertainty
Published on line 4 September 2008
analysis into their predictions. There are two explanations as to why uncertainty estimates
in the predictions of these models have seldom been provided. First, as the development
Keywords:
of forest ecosystem process-based models has begun only recently, research efforts have
Uncertainty analysis
focused on theoretical development to improve realism rather than reducing the ampli-
Carbon cycle
tude of variation of the predictions. Second, there is still little information on uncertainty
Monte Carlo analysis
estimates in parameters and key variables for forest ecosystem models. As process-based
Error propagation
models usually contain several complex nonlinear relationships, the Monte Carlo method is
Process-based models
most commonly used to facilitate uncertainty analysis. However, its full potential for error
propagation analysis in process-based models of the C cycle of forest ecosystems remains to
be developed. In this paper, commonly used methods to address uncertainty in C cycle forest
ecosystem models are discussed and directions for further research are presented. Realizing
the full potential of uncertainty analysis for these model types will require obtaining better
estimates of the errors and distributions of key parameters for complex relationships in
ecophysiological processes by increasing sampling intensity and testing different sampling
designs. As the level of complexity of the type of relationships used in forest ecosystem
models varies substantially, the application of uncertainty analysis methods can be further facilitated by developing a model-driven decision support system based on different
analytical applications to derive optimum and efficient uncertainty analysis pathways.
Crown Copyright © 2008 Published by Elsevier B.V. All rights reserved.
1.
Introduction
Forest ecosystem process-based models are increasingly used
to better understand how the processes which govern the
dynamics of forest ecosystems interact. In particular, processbased models play a key role in predicting how the carbon (C)
∗
cycle of forest ecosystems will be affected by climate change
(Verbeeck et al., 2006). Despite the fact that many models
have been developed and calibrated, there is still much uncertainty associated with model predictions. In this regard, key
questions that are important for scientists and policy makers include changes in productivity, species migration, and
Corresponding author. Tel.: +1 418 648 5791; fax: +1 418 648 5849.
E-mail address: [email protected] (G.R. Larocque).
0304-3800/$ – see front matter. Crown Copyright © 2008 Published by Elsevier B.V. All rights reserved.
doi:10.1016/j.ecolmodel.2008.07.024
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
CO2 source-sink relationships in different forest ecosystems.
For the last question, it is important to determine if forests
will become sinks or sources of C under climate change. In
this regard, uncertainty analysis is important for the examination of the logical consistency of models (Smith and Heath,
2001), which influences the robustness of inferences that can
be drawn.
Even though the importance of uncertainty issues in
process-based models has been identified for some time, the
majority of C cycle models for forest ecosystems perform
deterministic simulations without uncertainty estimates of
the outputs (Smith and Heath, 2001; Verbeeck et al., 2006).
Models without sound estimates of uncertainty in the predictions are not considered as useful for decision making on
environmental issues as models that integrate uncertainty
analysis (Rowe, 1994; Morgan and Dowlatabadi, 1996; Cipra,
2000; Radtke et al., 2001; Allen et al., 2004). Also, the evaluation of uncertainty in model output responses is essential
for comparing the predictions of various models (Hollinger
and Richardson, 2005). There are several possible explanations
as to why very few C cycle forest ecosystem models provide
uncertainty estimates. One explanation may be that these
types of models are still in their infancy. Compared with other
research areas in the biological sciences, the development of
process-based models for forest ecosystems has begun only
recently. Relatively few mechanistic equations representing
the basic ecophysiological processes and the complexity of
the interactions involved in the dynamics of forest ecosystems have been developed and tested. Modelling efforts have
aimed at developing, testing, and refining the theoretical
basis of the mathematical relationships of the underlying
processes. Furthermore, emphasis was placed on the development of realistic models of ecosystem processes and less
importance was given to the amplitude of variation of the predictions. Another explanation is related to the availability of
experimental or historical data for the calibration and evaluation of forest ecosystem models. Uncertainty information
on key ecosystem variables or parameters is still greatly lacking or simply is not available (Smith and Heath, 2001; Garbey
et al., 2006). In addition, several datasets originating from
historical records or field experiments lack appropriate replication, which may underestimate uncertainty and limit sound
comparison of predicted and observed variations at different
stages of development to capture the longevity factor.
Uncertainty analysis evaluates the extent to which model
outputs are uncertain due to error propagation resulting from
variability in the input variables and parameters (Crosetto and
Tarantola, 2001). Parameter uncertainty stems mainly from
errors in the measurements used for parameterization, the
method used to scale point measurements to the scale the
model operates, or from parameter and input data estimation of semi-empirical process descriptions, e.g., not readily
measurable parameters or soil parameters derived from standard soil measurements (Zähle et al., 2005). Besides parameter
uncertainties, uncertainties related to input data (e.g., soil,
land use, climate input data, land management practices) and
scaling issues also have to be considered.
The variability in forest ecosystem models may originate from different sources (O’Neill and Rust, 1979; Kremer,
1983; Gardner et al., 1990; Parysow et al., 2000; Linkov and
401
Burmistrov, 2003; Garbey et al., 2006) and can be summarized
as:
1. Data uncertainty resulting from statistical errors associated with sampling methodology, field measurement
errors, instrument imprecision, or differences in spatial or
temporal scales;
2. Sensitivity to initial conditions;
3. Lack of understanding of the underlying processes,
resulting in the derivation of inaccurate or inadequate
mathematical representation in model structure;
4. Parameter estimates, which may be associated with the use
of parameter estimation methods or inaccurate assumptions about the parameter distribution;
5. Unknown or poorly constrained drivers; and
6. The amplitude of natural variation associated with the biological system under study.
The different sources of errors listed above and the increasing need to address uncertainty issues for C cycle forest
ecosystems models justify further examination of uncertainty
analysis methods in order to identify research priorities or
new directions. In this paper, we provide a synthesis of current
uncertainty analysis techniques for C cycle forest ecosystem
models, review case studies based on the use of the Monte
Carlo method, identify future research directions with respect
to data sampling and computing efficiency, and suggest a theoretical framework for further research on the development of
an integrated methodology to efficiently perform uncertainty
analysis for process-based models on the C cycle of forest
ecosystems.
2.
Current methods to address uncertainty
There is no single widely recognized classification of methods
for uncertainty analysis. In the literature, the most common
methods used to assess uncertainties in process-based models of forest ecosystems include sensitivity analysis, analytical
solution of differential equations, and Monte Carlo analysis
(Kremer, 1983; Annan, 2001; Xu et al., 2004). Non-probabilistic
methods like fuzzy sets, possibility theory, first-order analysis employing Taylor expansion, or screening designs are
applied in models that focus on the study or management
of natural resources, such as in hydrology. However, they are
not commonly applied in C cycle forest ecosystem models.
A distinction is often made between sensitivity analysis and
uncertainty analysis to highlight the fact that parameter sensitivity can be evaluated without providing error estimates.
In this regard, sensitivity analysis methods are classified into
local and global approaches (Saltelli et al., 1999). A common
local approach to sensitivity analysis consists in systematically varying model parameters one-by-one while keeping
the other parameters constant (Kremer, 1983; Elston, 1992),
thus allowing the modeller to identify parameters that have
the most influence on model output responses. Inferences
can be drawn about the effects of different combinations of
parameter values and the interpretation of the results (Elston,
1992). The percentage of variation in selected output variables
may be ranked. Usually, the confidence intervals or parameter
402
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
distributions are not considered (Kremer, 1983). If a model contains many parameters or interactive sub-models, sensitivity
analysis may become cumbersome in practice and require
excessive computer time, as the number of runs increases
substantially with the number of parameters and the range
of variation examined. However, several methods have been
developed to examine multiple parameter sensitivity, such as
the computation of Euclidean distance or the use of factorial
experiments (see Haefner, 1996). Common global sensitivity
analysis approaches are based on the use of error propagation methods, such as Monte Carlo analysis (Saltelli et al.,
1999).
Approaches based on the analysis of function derivatives
have also been applied. The analytical solution of differential equations consists in estimating uncertainty by analyzing
model output response from the partial derivatives of the
equation(s) with respect to the input parameters (Hammonds
et al., 1994). Error propagation equations are computed from
successively increasing the order of derivatives of the models evaluated (Haefner, 1996; Helton and Davis, 2003). The
Taylor series expansion may be used to evaluate error propagation. A major disadvantage of this class of approach, which
applies to linear models, resides in its applicability for complex models (Haefner, 1996; Parysow et al., 2000). The majority
of the relationships in C cycle forest ecosystem models consist of complex nonlinear relationships that can be solved only
through numerical solution. Consequently, using an analytical derivative-based approach is not appropriate for most C
cycle forest ecosystem models.
Monte Carlo analysis is a common method used to compute error propagation and evaluate the influential factors
that affect model uncertainty (Smith and Heath, 2001). For
instance, parameter values that may lead to unrealistic predictions can be identified (Haness et al., 1991). As Monte Carlo
analysis is based on relatively simple assumptions and does
not require the development of complex algorithms, it constitutes a major advantage when applied to complex models
(Haefner, 1996; Smith and Heath, 2001). Dufrêne et al. (2005)
concluded that the application of the Monte Carlo method
to the analysis of uncertainty of their model contributed to
“correct the biased results.” The basic approach consists in
running a model several times by randomly sampling the
parameters from a probability distribution function (Haefner,
1996; Annan, 1997, 2001; Xu et al., 2004). The random sampling
may be based on a uniform distribution, but a theoretical distribution is most often used, such as the normal distribution
or any distribution empirically estimated from experiments.
The outputs of the repeated simulations are then analyzed
by computing statistics, such as frequency distributions or
the means or standard deviations of the state variables of
interest. However, the Monte Carlo method requires that the
model be run many times, as the statistical validity of the
results increases with the sampling intensity (Haness et al.,
1991; Guan et al., 1997; Smith and Heath, 2001). For instance,
Dufrêne et al. (2005) executed their model 17,000 times in performing a Monte Carlo analysis (1000 simulations × 17 “key
parameters”). Thus, the execution time can be prohibitive for a
model that contains many parameters, even for powerful computers. Still with these limitations, the Monte Carlo method
is considered appropriate for providing sound estimates of
uncertainty, as long as the distribution of the parameters is
well identified (Kremer, 1983).
The two sampling methods commonly used for Monte
Carlo analysis are simple random sampling and Latin Hypercube Sampling (LHS) (see Hammonds et al., 1994; Helton and
Davis, 2003). In simple random sampling, the value of each
parameter is randomly selected from a probability distribution
function in successive runs of the model. The basic principle
of LHS is to partition the distribution area of each parameter into zones that have the same probability of occurrence. It
is a form of stratified random sampling without replacement
(Smith and Heath, 2001). The value for each parameter is randomly selected within each zone for every sampling iteration;
however, a specific zone is used only one time. Thus, the number of iterations performed is equal to the number of zones. An
important advantage of LHS is that it reduces the computation
time required (Smith and Heath, 2001).
3.
Monte Carlo applications for C cycle
forest ecosystem models
A literature search on C cycle models for forest ecosystems
that placed considerable emphasis on uncertainty analysis
resulted in relatively few papers. Among the studies that
included relatively detailed uncertainty analysis, the Monte
Carlo method was applied most often, sometimes in combination with sensitivity analysis. Case studies of the application
of the Monte Carlo method for forest ecosystem uncertainty
analysis are listed in Table 1. For most of these models, the
predictions of several state variables, such as net ecosystem
productivity or growth respiration, were involved in the uncertainty analysis. Three types of distributions were used in the
examples in Table 1: uniform, triangular and normal.
The simplest example in Table 1 of the application of the
Monte Carlo method may be found in the study by Medlyn
et al. (2005). Their objective was to evaluate the performance
of a model of net ecosystem exchange (NEE, the difference
between gross primary productivity and ecosystem respiration) using eddy covariance data. Both uniform and normal
distributions were used and results of the Monte Carlo simulations indicated wide confidence intervals (∼±100%). Despite
the fact that the observed mean NEE value was within the confidence interval of the predictions resulting from the Monte
Carlo simulations, both observed and predicted NEE values
differed by nearly a twofold factor. This difference was fairly
high given the fact that the errors in the parameters defined
for the Monte Carlo simulations were in general less than 5%.
Chertov et al. (2003), Komarov et al. (2003) and Shaw et
al. (2006) used the Monte Carlo method to quantify and analyze parameter uncertainty in the forest ecosystem model
EFIMOD. This model is a spatially explicit individual-tree
model that combines descriptions of ecophysiological, soil
processes, inter-tree, and tree–soil interactions. The model
was developed to describe biological turnover of C and N in the
“climate-forests-soil” system based on measured data from
permanent sample plots, standard forest and soil inventory
data, and measured climatic data. The outputs of the simulations are determined by initial conditions that are essentially
random due to the variability of some parameters. The param-
403
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
Table 1 – Examples of forest ecosystem carbon cycles modelling studies when the Monte Carlo method was used to
perform uncertainty analysis
Reference
Model name
Medlyn et al. (2005)
Main predicted variables
analyzed for uncertainty
analysis
Distribution type
Net ecosystem exchange
Normal, uniform
Comments
Latin Hypercube Sampling
Komarov et al. (2003)
EFIMOD2
Tree biomass
Soil organic matter
Forest floor mass
Nitrogen pool
Normal
Dufrêne et al. (2005)
CASTANEA
Net ecosystem exchange
Gross primary productivity
Autotrophic respiration
Heterotrophic respiration
Aerial wood production
Transpiration
Evapotranspiration
Normal
Joint sensitivity and
uncertainty analyses
Verbeeck et al. (2006)
FORUG
Net ecosystem exchange
Total ecosystem respiration
Gross primary productivity
Triangular, uniform
Multiple regression analysis; joint
sensitivity and uncertainty analyses
Smith and Heath
(2001)
FORCARB
Carbon content in trees,
understory/floor and soil
Normal
Latin Hypercube Sampling;
correlation analysis
Paul et al. (2003)
GRC3
Soil decomposition rate
Net primary productivity
Carbon allocation
Soil respiration
Litterfall rate
Litter humification rate
Litter decomposition rate
Triangular
Daily photosynthesis of unshaded
needles
Shading
Daily utilization of carbon pool
Growth respiration
Maintenance respiration
Needles
Stems
Coarse roots
Fine roots
Specific nitrogen uptake rate
Stem form coefficient
Wood density
Sapwood area/needle mass ratio
Transport root/needle mass ratio
Shoot growth per new foliage
Uniform
Koskela (2000)
Law et al. (2003, 2004)
Biome-BGC
Net ecosystem production
Uniform
Buckley and Roberts
(2005)
DESPOT
Tree height
Uniform
Joint sensitivity and
uncertainty analyses
Net primary productivity
Leaf area index
Gross primary productivity
eters that were accounted for in Monte Carlo simulations
using EFIMOD were: (i) the spatial distribution of trees, (ii) the
soil parameters of C and N pools, and (iii) climatic variables
such as photosynthetic active radiation or air temperature.
Several applications of EFIMOD for the simulation of various
scenarios under different natural conditions in Europe and
North America (see citations above) indicated that the largest
uncertainty was caused by soil parameters. These results can
be explained by the lack of soil inventory data in the majority
of countries where EFIMOD was used. The spatial distribution of trees also had a visible impact on the simulation
results, but significantly less than soil parameters. As the
availability of weather data was sufficient for the derivation
of reliable parameters in the climatic functions, the impact of
404
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
climate variation on the Monte Carlo simulation results was
minimal.
The model developed by Dufrêne et al. (2005), CASTANEA, is
characterized by a high degree of complexity. It belongs to the
class of models based on the representation of nearly all the
known fundamental processes in forest ecosystems, including
photosynthesis, growth and maintenance respiration, evapotranspiration, C allocation, etc. They conducted an integrated
sensitivity and uncertainty analysis by using the results of
sensitivity analysis, based on the computation of parameter specific variation percentages, to identify the model input
parameters that were likely to have the most effect on output uncertainty. The degree of uncertainty, expressed as the
coefficient of variation, differed among the main state variables: 29% for NEE, 10% for gross primary productivity and
autotrophic respiration, 2% for heterotrophic respiration, 14%
for stemwood production and 6% for evapotranspiration. Several reasons may explain the relatively large differences in
the coefficients of variation. For NEE, Dufrêne et al. (2005)
attributed the high percentage compared with the other fluxes
to the fact that this variable consisted of small values. It is also
possible that the differences in the amplitude of the errors for
the parameters associated with each output variable had an
effect on the corresponding uncertainty.
Verbeeck et al. (2006) also conducted an integrated sensitivity and uncertainty analysis on the C cycle forest ecosystem
model FORUG, which allowed them to select a subset of variables that were most likely to have an effect on uncertainty.
By doing so, they identified 10 parameters that were responsible for more than 90% of the uncertainty in predicted NEE.
However, they pointed out that the Monte Carlo method did
not provide an evaluation of the relative importance of each
parameter on the overall output uncertainty. Once the results
of the Monte Carlo simulations were obtained, they applied
a multiple linear regression analysis between the parameter
deviations (computed from the difference between randomly
selected and average parameters) and the variation in the
predicted variables. A similar ranking system based on correlation analysis was used by Smith and Heath (2001), Paul et al.
(2003) and Koskela (2000). Following Monte Carlo simulations
using LHS, Smith and Heath (2001) computed rank correlation coefficients between the distributions of predicted C pools
and randomly selected parameters to rank their relative influence on overall uncertainty on their model outputs. As they
further compared the LHS approach with sensitivity analysis, Smith and Heath (2001) concluded that the LHS allowed
them to draw better inferences on the interdependence among
parameters with regard to their respective uncertainties. For
their uncertainty analyses, Paul et al. (2003) and Koskela (2000)
used simple random sampling in their Monte Carlo simulations, but many more parameters were involved in these
uncertainty analyses compared with Smith and Heath (2001).
However, both Paul et al. (2003) and Koskela (2000) highlighted
the impact of parameter uncertainty on model outputs. The
application of the Monte Carlo method by Law et al. (2003,
2004) and Buckley and Roberts (2005) was less extensive or
complex than the other studies, as it was used merely to provide uncertainty estimates.
The case studies discussed above are good examples illustrating common use of the Monte Carlo method in obtaining
uncertainty estimates for model outputs. As such, the use
of Monte Carlo analysis in these case studies was consistent
with the usual expectations of uncertainty analysis. Uncertainty analysis may provide feedback that allows the modeller
to evaluate the relevancy of the underlying model structure representing the mechanisms involved and to identify
weaknesses in parameter estimates (Smith and Heath, 2001).
However, uncertainty analysis may also be used to draw additional conclusions from model output responses. For instance,
the use of uncertainty analysis in a model that simulates different scenarios of change in environmental conditions may
provide additional information on the amplitude of the differences in predicted changes.
The last point is illustrated by the results of uncertainty
analysis using a soil C model to simulate the effects of different scenarios of temperature increase in a mature balsam
fir (Abies balsamea (L.) Mill.) ecosystem located in the Canadian boreal forest in northern Quebec (Fig. 1). The Monte Carlo
method was applied for a mass-balance soil C model developed by Larocque et al. (2006). This model simulates the C
dynamics for the above- and below-ground litter pools and the
soil organic matter partitioned into active, slow and passive
pools in the organic and mineral layers. For each parameter,
the normal distribution was assumed and the random selection of each parameter during the 1000 simulation cycles was
constrained within the 95% confidence limits or within 10% of
variation. Results from both prescribed (without uncertainty
analysis) and Monte Carlo simulations are presented in Fig. 1.
In general, the predictions from the prescribed simulations
and the means of the Monte Carlo predictions were very close
for most of the simulation period, except for the active SOM
in the mineral layer (0–20 cm). The differences obtained in the
first few years for most of the pools and the respiration rate
were probably amplified by the random occurrences generated
by the application of the Monte Carlo method. For most of the
pools, the amplitude of the standard errors indicated relatively
large overlaps between the different scenarios of temperature
increase, particularly in the first 40 years of the simulations.
While differences among the scenarios were accentuated over
time for the litter, active and slow pools in the organic layer,
large overlaps between the standard errors for the active and
slow pools in the mineral layer remained important throughout the duration of the simulations. Except for the first 20
years, there was little overlap in soil respiration rate among
the scenarios of temperature increase. These results suggest
that significant differences among the scenarios of temperature increase could appear very slowly for some of the pools,
but changes in CO2 emissions through soil respiration could
occur more rapidly.
The EFIMOD model previously discussed was also used
to illustrate how uncertainty analysis may provide insightful information on the importance of the differences in the
predictions following the simulation of various scenarios of
changes in environmental conditions (Chertov et al., 2003;
Komarov et al., 2003). The model has a fixed procedure for
the Monte Carlo simulation, i.e., it is only possible to conduct
a random variation of soil parameters along with the spatial
distribution of trees. Monte Carlo simulations were performed
with 50 iterations for 100 years with an initial 50% random
variation in soil parameters and spatial distribution of trees.
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
405
Fig. 1 – Comparison of predicted carbon pools in a balsam fir (Abies balsamea (L.) Mill.) forest ecosystem located in the boreal
forest of northeastern Canada using a process-based model of the soil carbon cycle following prescribed and Monte Carlo
simulations. Error bars represent the standard deviations computed from the predictions using the Monte Carlo method.
The output data represented average values with associated
standard deviations. Canadian and Russian sites were chosen for the simulation study. The Canadian sites were from
the Candle Lake Boreal Forest Transect Case Study (BFTCS)
for black spruce and jack pine. This transect is located along
an ecoclimatic gradient of the boreal forest in Manitoba and
Saskatchewan. The Russian sites were Scots pine forests on
mesic sandy sites (typical for Central European Russia) with
a rather mild climate close to the border of a broad-leaved
forest zone. EFIMOD was used to simulate tree and stand
growth from young age to maturity for each of the Canadian
and Russian sites. Models used for the climate change scenarios were the Coupled Global Climate Model (CGCM) A2
in Canada (Price et al., 2004) and the Hadley Centre Coupled Model version 3 (HadCM3) A1Fi (Mitchell et al., 2004) in
Russia. Results comparing prescribed and Monte Carlo simulations in this example (Figs. 2–4) were basically the same as
in the previous case with balsam fir using the soil C model of
Larocque et al. (2006). For the important functional parameter – CO2 emissions from the soil – the predictions from the
406
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
Fig. 2 – Comparison of predicted carbon pools in a black spruce (Picea mariana (Mill.) BSP) forest ecosystem located in the
boreal forest of western Canada using a process-based model of the soil carbon cycle following prescribed and Monte Carlo
simulations. Error bars represent the standard deviations computed from the predictions using the Monte Carlo method.
prescribed simulations were fairly close to the means of the
Monte Carlo simulations. The fluctuations of the parameters
reflected variations in temperature and precipitation in the
climatic scenarios used. There was also a visible overlapping
of standard deviations along the time series with and without climate change for all C stocks and respiration, which
shows that tree productivity and changes in soil are strongly
linked to environmental changes. However, in the case of the
Russian site with Scots pine, changes in stand productivity
and soil parameters were more pronounced under climate
change scenarios due to ‘aridization’ in the southern part of
Russian European forests (Fig. 4). The growth of soil-C in the
Russian simulations was a result of rather high N input from
the atmosphere (10 kg m−2 year−1 ) and rather deep soil degradation during the half-millenium intensive exploitation due
to rotation of lands with extensive agriculture (especially in
the 20th century following three revolutions and two world
wars).
4.
Future directions
The studies reviewed and the case studies examined illustrate the potential of the Monte Carlo method to guide
modellers in identifying model weaknesses. Despite the fact
that the Monte Carlo method has been proven effective over
time, it has been seldom applied for C cycle forest ecosystem process-based models. As previously mentioned, the
fact that excessive computer time may be required to perform uncertainty analysis with the Monte Carlo method is
certainly an important reason for this. When models are
transferred to end-users, the simulations must be completed
efficiently within a short time period. Even though today’s
computers are very powerful, the use of uncertainty analysis methods for complex process-based models, such as the
Monte Carlo method, may require several hours, if not days,
to complete the necessary simulations. These reasons justify the identification of new research directions to facilitate
future use of uncertainty analysis in C cycle forest ecosystem
models.
4.1.
Data sampling
The issues concerning the lack of long-term historical data
(previously discussed in general terms to explain the lack
of uncertainty estimates in C cycle forest ecosystem models) most certainly remain relevant to explain the lack of
Monte Carlo analysis. As far as information on both ecophysiological processes and long-term changes in the C pools of
forest ecosystems are concerned, there are very few historical datasets of sufficiently long timescales that can be used
to derive appropriate uncertainty estimates. In addition, the
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
407
Fig. 3 – Comparison of predicted carbon pools in a jack pine (Pinus banksiana Lamb.) forest ecosystem located in the boreal
forest of western Canada using a process-based model of the soil carbon cycle following prescribed and Monte Carlo
simulations. Error bars represent the standard deviations computed from the predictions using the Monte Carlo method.
poor availability of portable instruments has been an important factor that limited the development of ecophysiological
datasets. With respect to soil C pool data, inadequate sampling intensity to account for the spatial variation across the
landscape and the vertical variation with horizon depth due
to microrelief, animal activity, windthrow, litter/coarse woody
debris input, human activity, effect of individual plants on
soil microclimate, and precipitation chemistry are important
sources of uncertainty. The technology for ecophysiological
measurements, such as photosynthesis, respiration or transpiration, has been developed only recently and the relatively
high cost has limited use. The availability issue is particularly important in the context of sound statistical sampling,
which is essential to ensuring that uncertainty estimates are
statistically valid. Regarding the longevity factor (the effect
of gradual change over time related to the long-lived nature
of forest ecosystems), the evaluation or uncertainty analysis
efforts that have been conducted using historical data have
been undertaken mostly with traditional inventory data, such
as annual volume increment (Medlyn et al., 2005). Thus, it
may be possible to capture the longevity factor using inventory data, but not to explain the processes governing tree and
stand growth. Radtke et al. (2001) point out that a major consequence of this situation is the lack of appropriate statistical
data on the distribution of many parameters (that compose
several mathematical relationships) describing basic ecophysiological processes. For their uncertainty analysis study using
the process-based model PnET-II (Aber et al., 1995), Radtke et
al. (2001) obtained information on probability distributions for
key parameters mostly from published studies. This approach
may lead to imprecise uncertainty estimates, as it is not
evident that the information obtained in this fashion adequately represents the population of interest (Radtke et al.,
2001). All these factors justify the continuation of research
efforts on uncertainty analysis for C cycle forest ecosystem
models in order to improve the methodologies of uncertainty
evaluation and considerably shorten model execution time.
Also, the probability distributions of key parameters must
be better described using appropriate statistical sampling
methods.
Major research efforts should aim at taking advantage of
recent technological advances in ecophysiological instrumentation to perform measurements on adequately replicated
experimental designs based on a chrono-sequence approach
to capture the longevity factor associated with age and succession. The chrono-sequence approach is based on the study
of the stand dynamics of the same forest type growing under
similar site conditions, but differing in age or successional status (Brubaker, 1980). Recent technological developments in gas
exchange instruments allow tree physiologists to take mea-
408
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
Fig. 4 – Comparison of predicted carbon pools in a Scots pine (Pinus sylvestris L.) forest ecosystem located in Russia using
EFIMOD, a forest ecosystem model, following prescribed and Monte Carlo simulations. Error bars represent the standard
deviations computed from the predictions using the Monte Carlo method.
surements for very short time periods, from hours to minutes
or seconds. For instance, Medlyn et al. (2005) performed model
evaluation and uncertainty analyses using eddy covariance
measurements. Despite the fact that their study was conducted in a young Sitka spruce (Picea sitchensis (Bong.) Carr.)
stand, they were able to evaluate the logical consistency and
uncertainty estimates of several components of a processbased model. Thus, the analytical procedures that they used
could be applied to forest ecosystems belonging to a chronosequence.
4.2.
Computing efficiency and statistical validity
As far as computing efficiency and statistical validity are concerned, there is a need to develop a historical database of
uncertainty analysis applications and test different sampling
designs. As previously discussed, there is a limited history
in applying the Monte Carlo method to model ecophysiological processes in forest ecosystems. This fact implies that
there is not much knowledge on the optimum number of execution cycles that must be completed to obtain statistically
valid results and even less information on the efficiency of
sampling designs. Thus, the forest ecosystem modellers who
used the Monte Carlo method have applied one of the basic
principles of statistics: the degree of confidence in the esti-
mates of population parameters, such as mean or standard
deviation, increases with sample size. As a consequence, the
majority of the studies that have applied the Monte Carlo
method to forest ecosystems models have performed simulations with many cycles. Using the most powerful PCs
currently available, it may require a few days to complete the
simulations.
Two solutions may be considered to reduce computing time. First, simulation experiments aimed at comparing
uncertainty estimates for different execution cycles should be
conducted in a systematic fashion to determine the minimum
number of execution cycles required to obtain statistically
valid results. This step could be conducted by applying
bootstrap methods to compare uncertainty estimates (see
Chernick, 1999). For statistical consistency, this task should
be undertaken using several process-based models commonly
used to simulate the C cycle of various types of forest ecosystems (e.g., Dufrêne et al., 2005; Komarov et al., 2003). These
ecosystems should differ in species composition and site characteristics, including soil and climatic conditions. Second,
the statistical designs based on stratified sampling methods, such as LHS, should be tested consistently. As stratified
sampling methods define specific boundaries of the random
selection of parameters during the Monte Carlo simulations,
they will reduce the number of simulation cycles needed to
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
obtain statistically valid results for C cycle forest ecosystem
models.
Computation efficiency can also be considered in terms
of model structure. For instance, Zhang et al. (2008)
compared several soil decomposition models differing substantially in terms of the number of pools and processes
addressed and the process formulations. Models with more
pools and processes (e.g., CENTURY, DOCMOD, SOMM) were
not necessarily better than models with fewer pools and
processes (e.g., FLDM, CANDY) in achieving good agreement between observed and predicted values. In general,
more complex models require more complicated model formulation, initialization, and parameter estimation efforts
which may lead to higher uncertainty and greater error
propagation.
A better knowledge of the effects of the number of
execution cycles and application of statistical designs on
uncertainty estimates can also contribute to improving model
comparison exercises. Smith et al. (1997) evaluated and compared the performance of nine different soil organic models
using 12 different datasets for grassland, arable cropland, and
woodland over a range of climatic conditions within the temperate region. The differences in the predictions were related
to model specific initial conditions as well as application to
ecosystems different from those for which the models were
developed. Smith et al. (1997) further observed that small
changes in initial condition input values could lead to very
different model output responses in all the cases. Some of
409
the conclusions with respect to the comparison of observed
and predicted values would have been different if an uncertainty analysis had been conducted, i.e., observed values
were provided with standard errors but there was no uncertainty estimate for the predictions. As a consequence, the
Smith et al. (1997) comparison of different soil organic models was not based on evaluating the degree to which there
was overlap between both observed and predicted errors. This
additional criterion strengthens the comparison standard. For
instance, the means of observed and predicted values may
substantially differ, but a large overlap between the respective errors may contribute to reducing the importance of the
differences with respect to the biological consistency of the
predictions.
4.3.
Model-driven decision support system framework
The application of uncertainty analysis in process-based
models of forest ecosystems can be improved by developing model-driven decision support systems (MDSS) based on
the use of different interactive analytical applications (e.g.,
Fig. 5). Examination of the literature shows that the application of uncertainty analysis in C cycle forest ecosystem
models has consisted for the most part in developing simple
subroutines or functions within the source code of models,
without integration of decision-support tools. For instance,
it appears that both sensitivity and uncertainty analyses
have been carried out “manually” and independently, which
Fig. 5 – Theoretical framework for the development of a model-driven decision support system to support the computation
of uncertainty analysis.
410
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
can be cumbersome for end-users, without the support of
software to automate the process. Software has been developed to facilitate sensitivity or uncertainty analyses, such
as SIMLAB (http://sensitivity-analysis.jrc.cec.eu.int) or GEMSA (http://ctcd.group.shef.ac.uk/gem.html). Both applications
contain rich features that can facilitate the undertaking of
sensitivity and uncertainty analyses. However, they still have
limited capabilities to process relatively complex information. There is a need to develop applications, such as expert
systems, that integrate complex sets of inference and mathematical rules to efficiently analyze quantitative information
from different sources.
The sensitivity analysis section of the suggested MDSS
framework design in Fig. 5 aims at selecting key parameters
for further uncertainty analysis. The process of sensitivity
analysis can be fully automated by developing the necessary
software components to evaluate the degree to which the outputs of a model vary with changes in parameters. Single or
multiple parameter variations could be allowed. Once this step
is completed, the outputs can be processed using different
analytical methods, such as the derivation of distance coefficients (e.g., Haefner, 1996), regression analysis (e.g., Helton and
Davis, 2003) or identifiability analysis (e.g., Brun et al., 2001,
2002). Regression analysis, which can be used by considering
sensitivity analysis as an experimental design, becomes practical to assess the relative contribution of several parameters
by computing, for instance, standardized regression coefficients. Several methods of identifiability analysis have been
developed which usually combine different analytical steps.
For instance, the method proposed by Brun et al. (2001, 2002)
consists of an iterative procedure that focuses on the ranking
of the parameters and the computation of multivariate coefficients, such as collinearity indices. The sensitivity indices
derived (Fig. 5) contribute to assisting the end user in selecting
the key parameters that should be submitted to uncertainty
analysis. Using scientific judgement, the user could: (1) let the
MDSS system decide (based on inference rules) which parameters should be included in the uncertainty analysis step, or
(2) make decisions by providing specific limits or conditions
on the basis of interpreting the sensitivity indices.
For the uncertainty analysis step, the suggested MDSS
framework also includes decision rules that determine the
most appropriate method for each model sub-component
(Fig. 5). The methods based on the analytical solution of differential equations (ASDE), such as the Taylor series expansion
or the first-order variance propagation analysis (see Kremer,
1983), can provide uncertainty estimates very efficiently compared with the Monte Carlo method, but only for linear
systems. When it is not possible to derive analytical solutions,
which is the case for many complex relationships based on the
representation of the dynamics of ecophysiological processes,
the Monte Carlo method is then applied. Thus, the proposed
MDSS framework could contain the necessary software to verify the model components for which ASDE can be applied. This
can be done using automatic differentiation software. Once
uncertainty outputs are estimated, further analysis, such as
error budget (see Gertner, 1990; Parysow et al., 2000) or neural network (see Guan et al., 1997), can be used. Both methods
estimate the absolute or relative contributions of the various
error sources.
5.
Conclusion
Major achievements have been accomplished in the development of process-based models for the simulation of the C
cycle in forest ecosystems. Like any other simulation model,
C cycle models consist of imperfect representations of reality.
The factors and the sources of error that influence parameter
uncertainty contribute to increasing errors in the predictions.
Therefore, it is important that scientists or policy makers who
rely on model predictions in their decision-making process be
able to evaluate the degree to which the results are uncertain.
However, the application of uncertainty analysis in C cycle
process-based models is not a common procedure because
of the lack of knowledge on the sources of uncertainty, the
complexity of the methodologies involved and the model execution time required. There are two kinds of research efforts
that need to be accomplished to further support uncertainty
analysis. First, it is important to increase scientific knowledge
on parameter variability by increasing the intensity of ecophysiological measurements in forest ecosystems that differ
in species composition and site conditions. Different sampling
designs must be tested and compared to improve statistical validity at given sites. Second, there is a need to think
differently in the development of new uncertainty analysis
software by combining different uncertainty analysis methods and automating further the analytical steps that are
involved.
Acknowledgements
Sincere thanks are extended to Dr. David Paré, with the Canadian Forest Service, Quebec, QC, for providing incubation data
used in some analyses and comments on an earlier version
of the manuscript. The comments of Thomas White, Canadian Forest Service, Victoria, B.C., were greatly appreciated.
We acknowledge the financial contribution of PERD, a research
program of the Government of Canada.
references
Aber, J.D., Ollinger, S.V., Federer, C.A., Reich, P.B., Goulden, M.L.,
Kicklighter, D.W., Melillo, J.M., Lathrop, R.G., 1995. Predicting
the effects of climate change on water yield and forest
production in the northeastern United States. Clim. Res. 5,
207–222.
Allen, M.R., Booth, B.B.B., Frame, D.J., Gregory, J.M., Kettleborough,
J.A., Smith, L.A., Satinforth, D.A., Stott, P.A., 2004.
Observational constraints on future climate: distinguishing
robust from model-dependent statements of uncertainty in
climate forecasting. In: Contribution to the IPCC Workshop on
Communicating Uncertainty and Risk.
Annan, J.D., 1997. On repeated parameter sampling in Monte
Carlo simulations. Ecol. Model. 97, 111–115.
Annan, J.D., 2001. Modelling under uncertainty: Monte Carlo
methods for temporally varying parameters. Ecol. Model. 136,
297–302.
Brubaker, L.B., 1980. Long-term forest dynamics. In: West, D.C.,
Shugart, H.H., Botkin, D.B. (Eds.), Forest Succession, Concepts
and Application. Springer-Verlag, New York, pp. 95–106.
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
Brun, R., Kühni, M., Siegrist, H., Gujer, W., Reichert, P., 2002.
Practical identifiability of ASM2d parameters—systematic
selection and tuning of parameter subsets. Water Res. 36,
4113–4127.
Brun, R., Reichert, P., Kunsch, H.R., 2001. Practical identifiability
analysis of large environmental simulation models. Water
Resour. Res. 37, 1015–1030.
Buckley, T.N., Roberts, D.W., 2005. DESPOT, a process-based tree
growth model that allocates carbon to maximize carbon gain.
Tree Physiol. 26, 129–144.
Chernick, M.R., 1999. Bootstrap Methods. A Practitioner’s Guide.
John Wiley & Sons, Inc., Toronto.
Chertov, O., Komarov, A., Kolström, M., Pitkänen, S., Strandman,
H., Zudin, S., Kellomäki, S., 2003. Modelling the long-term
dynamics of populations and communities of trees in boreal
forests based on competition for light and nitrogen. For. Ecol.
Manage. 176, 355–369.
Cipra, B., 2000. Revealing uncertainties in computer models.
Science 287, 960–961.
Crosetto, M., Tarantola, S., 2001. Uncertainty and sensitivity
analysis: tools for GIS-based model implementation. Int. J.
Geogr. Inf. Sci. 15, 415–437.
Dufrêne, E., Davi, H., François, C., le Maire, G., Le Dantec, V.,
Granier, A., 2005. Modelling carbon and water cycles in a
beech forest. Part I. Model description and uncertainty
analysis on modelled NEE. Ecol. Model. 185, 407–436.
Elston, D.A., 1992. Sensitivity analysis in the presence of
correlated parameter estimates. Ecol. Model. 64, 11–22.
Garbey, C., Garbey, M., Muller, S., 2006. Using modeling to
improve models. Ecol. Model. 197, 303–319.
Gardner, R.H., Dale, V.H., O’Neil, R.V., 1990. Error propagation and
uncertainty in process modeling. In: Dixon, R.K., Meldahl, R.S.,
Ruark, G.A., Warren, W.G. (Eds.), Process Modelling of Forest
Growth Responses to Environmental Stress. Timber Press,
Portland, OR, pp. 208–219.
Gertner, G., 1990. Error budgets: a means of assessing component
variability and identifying efficient ways to improve model
predictive ability. In: Dixon, R.K., Meldahl, R.S., Ruark, G.A.,
Warren, W.G. (Eds.), Process Modelling of Forest Growth
Responses to Environmental Stress. Timber Press, Portland,
OR, pp. 220–225.
Guan, B.T., Gertner, G.Z., Parysow, P., 1997. A framework for
uncertainty assessment of mechanistic forest growth models:
a neural network example. Ecol. Model. 98, 47–58.
Haefner, J.W., 1996. Modeling Biological Systems. Chapman &
Hall, New York.
Hammonds, J.S., Hoffman, F.O., Bartell, S.M., 1994. An
Introductory Guide to Uncertainty Analysis in Environmental
and Health Risk Assessment. ES/ER/TM-35/R1, Oak Ridge
National Laboratory, Oak Ridge, TN.
Haness, S.J., Roberts, L.A., Warwick, J.J., Cale, W.G., 1991. Testing
the utility of first order uncertainty analysis. Ecol. Model. 58,
1–23.
Helton, J.C., Davis, F.J., 2003. Latin hypercube sampling and the
propagation of uncertainty in analyses of complex systems.
Reliability Eng. Syst. Saf. 81, 23–69.
Hollinger, D.Y., Richardson, A.D., 2005. Uncertainty in eddy
covariance measurements and its application to physiological
models. Tree Physiol. 25, 873–885.
Komarov, A., Chertov, O., Zudin, S., Nadporozhskaya, M.,
Mikhailov, A., Bykhovets, S., Zudina, E., Zoubkova, E., 2003.
EFIMOD 2—a model of growth and cycling of elements in
boreal forest ecosystems. Ecol. Model. 170, 373–392.
Koskela, J., 2000. A process-based growth model for the grass
stage pine seedlings. Silva Fenn. 34, 3–20.
Kremer, J.N., 1983. Ecological implications of parameter
uncertainty in stochastic simulation. Ecol. Model. 18, 187–207.
Larocque, G.R., Boutin, R., Paré, D., Robitaille, G., Lacerte, V., 2006.
Assessing a new soil carbon model to simulate the effect of
411
temperature increase on the soil carbon cycle in three eastern
Canadian forest types characterized by different climatic
conditions. Can. J. Soil Sci. 86, 187–202.
Law, B.E., Sun, O.J., Campbell, J., Van Tuyl, S., Thornton, P.E., 2003.
Changes in carbon storage and fluxes in a chronosequence of
ponderosa pine. Global Change Biol. 9, 510–524.
Law, B.E., Turner, D., Campbell, J., Sun, O.J., Van Tuyl, S., Ritts,
W.D., Cohen, W.B., 2004. Disturbance and climate effects on
carbon stocks and fluxes across Western Oregon USA. Global
Change Biol. 10, 1429–1444.
Linkov, I., Burmistrov, D., 2003. Model uncertainty and choices
made by modelers: lessons learned from the International
Atomic Energy Agency model intercomparisons. Risk Anal. 23,
1297–1308.
Medlyn, B.E., Robinson, A.P., Clement, R., McMurtrie, R.E., 2005.
On the validation of models of forest CO2 exchange using
eddy covariance data: some perils and pitfalls. Tree Physiol.
25, 839–857.
Mitchell, T.D., Carter, T.R., Jones, P.D., Hulme, M., New, M., 2004. A
comprehensive set of high-resolution grids of monthly
climate for Europe and the globe: the observed record
(1901–2000) and 16 scenarios (2001–2100). Tyndall Centre for
Climate Change Research, Working Paper 55. 30 pp.
http://www.cru.uea.ac.uk/∼timm/grid/TYN SC 2 0.html.
Morgan, M.G., Dowlatabadi, H., 1996. Learning from
integrated assessment of climate change. Clim. Change 34,
337–368.
O’Neill, R.V., Rust, B., 1979. Aggregation error in ecological
models. Ecol. Model. 7, 91–105.
Parysow, P., Gertner, G., Westervelt, J., 2000. Efficient
approximation for building error budgets for process models.
Ecol. Model. 135, 111–125.
Paul, K.I., Polglase, P.J., Richards, G.P., 2003. Predicted change in
soil carbon following afforestation or reforestation, and
analysis of controlling factors by linking a C accounting model
(CAMFor) to models of forest growth (3PG), litter
decomposition (GENDEC) and soil C turnover (RothC). For.
Ecol. Manage. 177, 485–501.
Price, D.T., McKenney, D.W., Papadopol, P., Logan, T., Hutchinson,
M.F., 2004. High resolution future scenario climate data for
North America. In: Proc. Amer. Meteor. Soc. 26th Conference
on Agricultural and Forest Meteorology, Vancouver, B.C.,
August 23–26, 13 pp (CD-ROM).
Radtke, P.J., Burk, T.E., Bolstad, P.V., 2001. Estimates of the
distributions of forest ecosystem model inputs for
deciduous forests of eastern North America. Tree Physiol. 21,
505–512.
Rowe, W.D., 1994. Understanding uncertainty. Risk Anal. 14,
743–750.
Saltelli, A., Torantola, S., Chan, K.P.S., 1999. A quantitative
model-independent method for global sensitivity analysis of
model output. Technometrics 41, 39–56.
Shaw, C., Chertov, O., Komarov, A., Bhatti, J., Nadporozskaya, M.,
Apps, M., Bykhovets, S., Mikhailov, A., 2006. Application of the
forest ecosystem model EFIMOD 2 to jack pine along the
Boreal Forest Transect Case Study. Can. J. Soil Sci. 86,
171–185.
Smith, J.E., Heath, L.S., 2001. Identifying influences on model
uncertainty: an application using a forest carbon budget
model. Environ. Manage. 27, 253–267.
Smith, P., Smith, J.U., Powlson, D.S., McGill, W.B., Aha, J.R.M.,
Chertove, O.G., Coleman, K., Franko, U., Frolking, S., Jenkinson,
D.G., Jensen, L.S., Kelly, R.H., Klein-Gunnewiek, H., Somarov,
A., Li, C., Molina, J.A.E., Meuller, T., Parton, W.J., Thornley,
J.H.M., Whitmore, A.P., 1997. A comparison of the performance
of nine soil organic matter models using datasets from seven
long-term experiments. Geoderma 81, 153–225.
Verbeeck, H., Samson, R., Verdonck, F., Lemeur, R., 2006.
Parameter sensitivity and uncertainty of the forest carbon
412
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412
flux model FORUG: a Monte Carlo analysis. Tree Physiol. 26,
807–817.
Xu, C., He, H.S., Hu, Y., Chang, Y., Larsen, D.R., Li, X., Bu, R., 2004.
Assessing the effect of cell-level uncertainty on a forest
landscape model simulation in northeastern China. Ecol.
Model. 180, 57–72.
Zähle, S., Sitch, S., Smith, B., Hattermann, F., 2005. Effects of
parameter uncertainties on the modeling of terrestrial
biosphere dynamics. Global Biogeochem. Cycles 19 (GB3020),
1–16.
Zhang, C.F., Meng, F.-R., Bhatti, J.S., Trofymow, J.A., Arp, P.A., 2008.
Modeling forest leaf-litter decomposition and N
mineralization in litterbags, placed across Canada: a 5-model
comparison. Ecol. Model. 219, 342–360.