Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 available at www.sciencedirect.com journal homepage: www.elsevier.com/locate/ecolmodel Uncertainty analysis in carbon cycle models of forest ecosystems: Research needs and development of a theoretical framework to estimate error propagation Guy R. Larocque a,∗ , Jagtar S. Bhatti b , Robert Boutin a , Oleg Chertov c a Natural Resources Canada, Canadian Forest Service, Laurentian Forestry Centre, 1055 du P.E.P.S., P.O. Box 10380, Stn. Ste-Foy, Quebec, Quebec G1V 4C7, Canada b Natural Resources Canada, Canadian Forest Service, Northern Forestry Centre, 5320-122 Street, Edmonton, Alberta T6H 3S5, Canada c Biological Research Institute, St. Petersburg State University, Peterhoff, 198904 St. Petersburg, Russia a r t i c l e i n f o a b s t r a c t Article history: Few process-based models of the carbon (C) cycle of forest ecosystems integrate uncertainty Published on line 4 September 2008 analysis into their predictions. There are two explanations as to why uncertainty estimates in the predictions of these models have seldom been provided. First, as the development Keywords: of forest ecosystem process-based models has begun only recently, research efforts have Uncertainty analysis focused on theoretical development to improve realism rather than reducing the ampli- Carbon cycle tude of variation of the predictions. Second, there is still little information on uncertainty Monte Carlo analysis estimates in parameters and key variables for forest ecosystem models. As process-based Error propagation models usually contain several complex nonlinear relationships, the Monte Carlo method is Process-based models most commonly used to facilitate uncertainty analysis. However, its full potential for error propagation analysis in process-based models of the C cycle of forest ecosystems remains to be developed. In this paper, commonly used methods to address uncertainty in C cycle forest ecosystem models are discussed and directions for further research are presented. Realizing the full potential of uncertainty analysis for these model types will require obtaining better estimates of the errors and distributions of key parameters for complex relationships in ecophysiological processes by increasing sampling intensity and testing different sampling designs. As the level of complexity of the type of relationships used in forest ecosystem models varies substantially, the application of uncertainty analysis methods can be further facilitated by developing a model-driven decision support system based on different analytical applications to derive optimum and efficient uncertainty analysis pathways. Crown Copyright © 2008 Published by Elsevier B.V. All rights reserved. 1. Introduction Forest ecosystem process-based models are increasingly used to better understand how the processes which govern the dynamics of forest ecosystems interact. In particular, processbased models play a key role in predicting how the carbon (C) ∗ cycle of forest ecosystems will be affected by climate change (Verbeeck et al., 2006). Despite the fact that many models have been developed and calibrated, there is still much uncertainty associated with model predictions. In this regard, key questions that are important for scientists and policy makers include changes in productivity, species migration, and Corresponding author. Tel.: +1 418 648 5791; fax: +1 418 648 5849. E-mail address: [email protected] (G.R. Larocque). 0304-3800/$ – see front matter. Crown Copyright © 2008 Published by Elsevier B.V. All rights reserved. doi:10.1016/j.ecolmodel.2008.07.024 e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 CO2 source-sink relationships in different forest ecosystems. For the last question, it is important to determine if forests will become sinks or sources of C under climate change. In this regard, uncertainty analysis is important for the examination of the logical consistency of models (Smith and Heath, 2001), which influences the robustness of inferences that can be drawn. Even though the importance of uncertainty issues in process-based models has been identified for some time, the majority of C cycle models for forest ecosystems perform deterministic simulations without uncertainty estimates of the outputs (Smith and Heath, 2001; Verbeeck et al., 2006). Models without sound estimates of uncertainty in the predictions are not considered as useful for decision making on environmental issues as models that integrate uncertainty analysis (Rowe, 1994; Morgan and Dowlatabadi, 1996; Cipra, 2000; Radtke et al., 2001; Allen et al., 2004). Also, the evaluation of uncertainty in model output responses is essential for comparing the predictions of various models (Hollinger and Richardson, 2005). There are several possible explanations as to why very few C cycle forest ecosystem models provide uncertainty estimates. One explanation may be that these types of models are still in their infancy. Compared with other research areas in the biological sciences, the development of process-based models for forest ecosystems has begun only recently. Relatively few mechanistic equations representing the basic ecophysiological processes and the complexity of the interactions involved in the dynamics of forest ecosystems have been developed and tested. Modelling efforts have aimed at developing, testing, and refining the theoretical basis of the mathematical relationships of the underlying processes. Furthermore, emphasis was placed on the development of realistic models of ecosystem processes and less importance was given to the amplitude of variation of the predictions. Another explanation is related to the availability of experimental or historical data for the calibration and evaluation of forest ecosystem models. Uncertainty information on key ecosystem variables or parameters is still greatly lacking or simply is not available (Smith and Heath, 2001; Garbey et al., 2006). In addition, several datasets originating from historical records or field experiments lack appropriate replication, which may underestimate uncertainty and limit sound comparison of predicted and observed variations at different stages of development to capture the longevity factor. Uncertainty analysis evaluates the extent to which model outputs are uncertain due to error propagation resulting from variability in the input variables and parameters (Crosetto and Tarantola, 2001). Parameter uncertainty stems mainly from errors in the measurements used for parameterization, the method used to scale point measurements to the scale the model operates, or from parameter and input data estimation of semi-empirical process descriptions, e.g., not readily measurable parameters or soil parameters derived from standard soil measurements (Zähle et al., 2005). Besides parameter uncertainties, uncertainties related to input data (e.g., soil, land use, climate input data, land management practices) and scaling issues also have to be considered. The variability in forest ecosystem models may originate from different sources (O’Neill and Rust, 1979; Kremer, 1983; Gardner et al., 1990; Parysow et al., 2000; Linkov and 401 Burmistrov, 2003; Garbey et al., 2006) and can be summarized as: 1. Data uncertainty resulting from statistical errors associated with sampling methodology, field measurement errors, instrument imprecision, or differences in spatial or temporal scales; 2. Sensitivity to initial conditions; 3. Lack of understanding of the underlying processes, resulting in the derivation of inaccurate or inadequate mathematical representation in model structure; 4. Parameter estimates, which may be associated with the use of parameter estimation methods or inaccurate assumptions about the parameter distribution; 5. Unknown or poorly constrained drivers; and 6. The amplitude of natural variation associated with the biological system under study. The different sources of errors listed above and the increasing need to address uncertainty issues for C cycle forest ecosystems models justify further examination of uncertainty analysis methods in order to identify research priorities or new directions. In this paper, we provide a synthesis of current uncertainty analysis techniques for C cycle forest ecosystem models, review case studies based on the use of the Monte Carlo method, identify future research directions with respect to data sampling and computing efficiency, and suggest a theoretical framework for further research on the development of an integrated methodology to efficiently perform uncertainty analysis for process-based models on the C cycle of forest ecosystems. 2. Current methods to address uncertainty There is no single widely recognized classification of methods for uncertainty analysis. In the literature, the most common methods used to assess uncertainties in process-based models of forest ecosystems include sensitivity analysis, analytical solution of differential equations, and Monte Carlo analysis (Kremer, 1983; Annan, 2001; Xu et al., 2004). Non-probabilistic methods like fuzzy sets, possibility theory, first-order analysis employing Taylor expansion, or screening designs are applied in models that focus on the study or management of natural resources, such as in hydrology. However, they are not commonly applied in C cycle forest ecosystem models. A distinction is often made between sensitivity analysis and uncertainty analysis to highlight the fact that parameter sensitivity can be evaluated without providing error estimates. In this regard, sensitivity analysis methods are classified into local and global approaches (Saltelli et al., 1999). A common local approach to sensitivity analysis consists in systematically varying model parameters one-by-one while keeping the other parameters constant (Kremer, 1983; Elston, 1992), thus allowing the modeller to identify parameters that have the most influence on model output responses. Inferences can be drawn about the effects of different combinations of parameter values and the interpretation of the results (Elston, 1992). The percentage of variation in selected output variables may be ranked. Usually, the confidence intervals or parameter 402 e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 distributions are not considered (Kremer, 1983). If a model contains many parameters or interactive sub-models, sensitivity analysis may become cumbersome in practice and require excessive computer time, as the number of runs increases substantially with the number of parameters and the range of variation examined. However, several methods have been developed to examine multiple parameter sensitivity, such as the computation of Euclidean distance or the use of factorial experiments (see Haefner, 1996). Common global sensitivity analysis approaches are based on the use of error propagation methods, such as Monte Carlo analysis (Saltelli et al., 1999). Approaches based on the analysis of function derivatives have also been applied. The analytical solution of differential equations consists in estimating uncertainty by analyzing model output response from the partial derivatives of the equation(s) with respect to the input parameters (Hammonds et al., 1994). Error propagation equations are computed from successively increasing the order of derivatives of the models evaluated (Haefner, 1996; Helton and Davis, 2003). The Taylor series expansion may be used to evaluate error propagation. A major disadvantage of this class of approach, which applies to linear models, resides in its applicability for complex models (Haefner, 1996; Parysow et al., 2000). The majority of the relationships in C cycle forest ecosystem models consist of complex nonlinear relationships that can be solved only through numerical solution. Consequently, using an analytical derivative-based approach is not appropriate for most C cycle forest ecosystem models. Monte Carlo analysis is a common method used to compute error propagation and evaluate the influential factors that affect model uncertainty (Smith and Heath, 2001). For instance, parameter values that may lead to unrealistic predictions can be identified (Haness et al., 1991). As Monte Carlo analysis is based on relatively simple assumptions and does not require the development of complex algorithms, it constitutes a major advantage when applied to complex models (Haefner, 1996; Smith and Heath, 2001). Dufrêne et al. (2005) concluded that the application of the Monte Carlo method to the analysis of uncertainty of their model contributed to “correct the biased results.” The basic approach consists in running a model several times by randomly sampling the parameters from a probability distribution function (Haefner, 1996; Annan, 1997, 2001; Xu et al., 2004). The random sampling may be based on a uniform distribution, but a theoretical distribution is most often used, such as the normal distribution or any distribution empirically estimated from experiments. The outputs of the repeated simulations are then analyzed by computing statistics, such as frequency distributions or the means or standard deviations of the state variables of interest. However, the Monte Carlo method requires that the model be run many times, as the statistical validity of the results increases with the sampling intensity (Haness et al., 1991; Guan et al., 1997; Smith and Heath, 2001). For instance, Dufrêne et al. (2005) executed their model 17,000 times in performing a Monte Carlo analysis (1000 simulations × 17 “key parameters”). Thus, the execution time can be prohibitive for a model that contains many parameters, even for powerful computers. Still with these limitations, the Monte Carlo method is considered appropriate for providing sound estimates of uncertainty, as long as the distribution of the parameters is well identified (Kremer, 1983). The two sampling methods commonly used for Monte Carlo analysis are simple random sampling and Latin Hypercube Sampling (LHS) (see Hammonds et al., 1994; Helton and Davis, 2003). In simple random sampling, the value of each parameter is randomly selected from a probability distribution function in successive runs of the model. The basic principle of LHS is to partition the distribution area of each parameter into zones that have the same probability of occurrence. It is a form of stratified random sampling without replacement (Smith and Heath, 2001). The value for each parameter is randomly selected within each zone for every sampling iteration; however, a specific zone is used only one time. Thus, the number of iterations performed is equal to the number of zones. An important advantage of LHS is that it reduces the computation time required (Smith and Heath, 2001). 3. Monte Carlo applications for C cycle forest ecosystem models A literature search on C cycle models for forest ecosystems that placed considerable emphasis on uncertainty analysis resulted in relatively few papers. Among the studies that included relatively detailed uncertainty analysis, the Monte Carlo method was applied most often, sometimes in combination with sensitivity analysis. Case studies of the application of the Monte Carlo method for forest ecosystem uncertainty analysis are listed in Table 1. For most of these models, the predictions of several state variables, such as net ecosystem productivity or growth respiration, were involved in the uncertainty analysis. Three types of distributions were used in the examples in Table 1: uniform, triangular and normal. The simplest example in Table 1 of the application of the Monte Carlo method may be found in the study by Medlyn et al. (2005). Their objective was to evaluate the performance of a model of net ecosystem exchange (NEE, the difference between gross primary productivity and ecosystem respiration) using eddy covariance data. Both uniform and normal distributions were used and results of the Monte Carlo simulations indicated wide confidence intervals (∼±100%). Despite the fact that the observed mean NEE value was within the confidence interval of the predictions resulting from the Monte Carlo simulations, both observed and predicted NEE values differed by nearly a twofold factor. This difference was fairly high given the fact that the errors in the parameters defined for the Monte Carlo simulations were in general less than 5%. Chertov et al. (2003), Komarov et al. (2003) and Shaw et al. (2006) used the Monte Carlo method to quantify and analyze parameter uncertainty in the forest ecosystem model EFIMOD. This model is a spatially explicit individual-tree model that combines descriptions of ecophysiological, soil processes, inter-tree, and tree–soil interactions. The model was developed to describe biological turnover of C and N in the “climate-forests-soil” system based on measured data from permanent sample plots, standard forest and soil inventory data, and measured climatic data. The outputs of the simulations are determined by initial conditions that are essentially random due to the variability of some parameters. The param- 403 e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 Table 1 – Examples of forest ecosystem carbon cycles modelling studies when the Monte Carlo method was used to perform uncertainty analysis Reference Model name Medlyn et al. (2005) Main predicted variables analyzed for uncertainty analysis Distribution type Net ecosystem exchange Normal, uniform Comments Latin Hypercube Sampling Komarov et al. (2003) EFIMOD2 Tree biomass Soil organic matter Forest floor mass Nitrogen pool Normal Dufrêne et al. (2005) CASTANEA Net ecosystem exchange Gross primary productivity Autotrophic respiration Heterotrophic respiration Aerial wood production Transpiration Evapotranspiration Normal Joint sensitivity and uncertainty analyses Verbeeck et al. (2006) FORUG Net ecosystem exchange Total ecosystem respiration Gross primary productivity Triangular, uniform Multiple regression analysis; joint sensitivity and uncertainty analyses Smith and Heath (2001) FORCARB Carbon content in trees, understory/floor and soil Normal Latin Hypercube Sampling; correlation analysis Paul et al. (2003) GRC3 Soil decomposition rate Net primary productivity Carbon allocation Soil respiration Litterfall rate Litter humification rate Litter decomposition rate Triangular Daily photosynthesis of unshaded needles Shading Daily utilization of carbon pool Growth respiration Maintenance respiration Needles Stems Coarse roots Fine roots Specific nitrogen uptake rate Stem form coefficient Wood density Sapwood area/needle mass ratio Transport root/needle mass ratio Shoot growth per new foliage Uniform Koskela (2000) Law et al. (2003, 2004) Biome-BGC Net ecosystem production Uniform Buckley and Roberts (2005) DESPOT Tree height Uniform Joint sensitivity and uncertainty analyses Net primary productivity Leaf area index Gross primary productivity eters that were accounted for in Monte Carlo simulations using EFIMOD were: (i) the spatial distribution of trees, (ii) the soil parameters of C and N pools, and (iii) climatic variables such as photosynthetic active radiation or air temperature. Several applications of EFIMOD for the simulation of various scenarios under different natural conditions in Europe and North America (see citations above) indicated that the largest uncertainty was caused by soil parameters. These results can be explained by the lack of soil inventory data in the majority of countries where EFIMOD was used. The spatial distribution of trees also had a visible impact on the simulation results, but significantly less than soil parameters. As the availability of weather data was sufficient for the derivation of reliable parameters in the climatic functions, the impact of 404 e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 climate variation on the Monte Carlo simulation results was minimal. The model developed by Dufrêne et al. (2005), CASTANEA, is characterized by a high degree of complexity. It belongs to the class of models based on the representation of nearly all the known fundamental processes in forest ecosystems, including photosynthesis, growth and maintenance respiration, evapotranspiration, C allocation, etc. They conducted an integrated sensitivity and uncertainty analysis by using the results of sensitivity analysis, based on the computation of parameter specific variation percentages, to identify the model input parameters that were likely to have the most effect on output uncertainty. The degree of uncertainty, expressed as the coefficient of variation, differed among the main state variables: 29% for NEE, 10% for gross primary productivity and autotrophic respiration, 2% for heterotrophic respiration, 14% for stemwood production and 6% for evapotranspiration. Several reasons may explain the relatively large differences in the coefficients of variation. For NEE, Dufrêne et al. (2005) attributed the high percentage compared with the other fluxes to the fact that this variable consisted of small values. It is also possible that the differences in the amplitude of the errors for the parameters associated with each output variable had an effect on the corresponding uncertainty. Verbeeck et al. (2006) also conducted an integrated sensitivity and uncertainty analysis on the C cycle forest ecosystem model FORUG, which allowed them to select a subset of variables that were most likely to have an effect on uncertainty. By doing so, they identified 10 parameters that were responsible for more than 90% of the uncertainty in predicted NEE. However, they pointed out that the Monte Carlo method did not provide an evaluation of the relative importance of each parameter on the overall output uncertainty. Once the results of the Monte Carlo simulations were obtained, they applied a multiple linear regression analysis between the parameter deviations (computed from the difference between randomly selected and average parameters) and the variation in the predicted variables. A similar ranking system based on correlation analysis was used by Smith and Heath (2001), Paul et al. (2003) and Koskela (2000). Following Monte Carlo simulations using LHS, Smith and Heath (2001) computed rank correlation coefficients between the distributions of predicted C pools and randomly selected parameters to rank their relative influence on overall uncertainty on their model outputs. As they further compared the LHS approach with sensitivity analysis, Smith and Heath (2001) concluded that the LHS allowed them to draw better inferences on the interdependence among parameters with regard to their respective uncertainties. For their uncertainty analyses, Paul et al. (2003) and Koskela (2000) used simple random sampling in their Monte Carlo simulations, but many more parameters were involved in these uncertainty analyses compared with Smith and Heath (2001). However, both Paul et al. (2003) and Koskela (2000) highlighted the impact of parameter uncertainty on model outputs. The application of the Monte Carlo method by Law et al. (2003, 2004) and Buckley and Roberts (2005) was less extensive or complex than the other studies, as it was used merely to provide uncertainty estimates. The case studies discussed above are good examples illustrating common use of the Monte Carlo method in obtaining uncertainty estimates for model outputs. As such, the use of Monte Carlo analysis in these case studies was consistent with the usual expectations of uncertainty analysis. Uncertainty analysis may provide feedback that allows the modeller to evaluate the relevancy of the underlying model structure representing the mechanisms involved and to identify weaknesses in parameter estimates (Smith and Heath, 2001). However, uncertainty analysis may also be used to draw additional conclusions from model output responses. For instance, the use of uncertainty analysis in a model that simulates different scenarios of change in environmental conditions may provide additional information on the amplitude of the differences in predicted changes. The last point is illustrated by the results of uncertainty analysis using a soil C model to simulate the effects of different scenarios of temperature increase in a mature balsam fir (Abies balsamea (L.) Mill.) ecosystem located in the Canadian boreal forest in northern Quebec (Fig. 1). The Monte Carlo method was applied for a mass-balance soil C model developed by Larocque et al. (2006). This model simulates the C dynamics for the above- and below-ground litter pools and the soil organic matter partitioned into active, slow and passive pools in the organic and mineral layers. For each parameter, the normal distribution was assumed and the random selection of each parameter during the 1000 simulation cycles was constrained within the 95% confidence limits or within 10% of variation. Results from both prescribed (without uncertainty analysis) and Monte Carlo simulations are presented in Fig. 1. In general, the predictions from the prescribed simulations and the means of the Monte Carlo predictions were very close for most of the simulation period, except for the active SOM in the mineral layer (0–20 cm). The differences obtained in the first few years for most of the pools and the respiration rate were probably amplified by the random occurrences generated by the application of the Monte Carlo method. For most of the pools, the amplitude of the standard errors indicated relatively large overlaps between the different scenarios of temperature increase, particularly in the first 40 years of the simulations. While differences among the scenarios were accentuated over time for the litter, active and slow pools in the organic layer, large overlaps between the standard errors for the active and slow pools in the mineral layer remained important throughout the duration of the simulations. Except for the first 20 years, there was little overlap in soil respiration rate among the scenarios of temperature increase. These results suggest that significant differences among the scenarios of temperature increase could appear very slowly for some of the pools, but changes in CO2 emissions through soil respiration could occur more rapidly. The EFIMOD model previously discussed was also used to illustrate how uncertainty analysis may provide insightful information on the importance of the differences in the predictions following the simulation of various scenarios of changes in environmental conditions (Chertov et al., 2003; Komarov et al., 2003). The model has a fixed procedure for the Monte Carlo simulation, i.e., it is only possible to conduct a random variation of soil parameters along with the spatial distribution of trees. Monte Carlo simulations were performed with 50 iterations for 100 years with an initial 50% random variation in soil parameters and spatial distribution of trees. e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 405 Fig. 1 – Comparison of predicted carbon pools in a balsam fir (Abies balsamea (L.) Mill.) forest ecosystem located in the boreal forest of northeastern Canada using a process-based model of the soil carbon cycle following prescribed and Monte Carlo simulations. Error bars represent the standard deviations computed from the predictions using the Monte Carlo method. The output data represented average values with associated standard deviations. Canadian and Russian sites were chosen for the simulation study. The Canadian sites were from the Candle Lake Boreal Forest Transect Case Study (BFTCS) for black spruce and jack pine. This transect is located along an ecoclimatic gradient of the boreal forest in Manitoba and Saskatchewan. The Russian sites were Scots pine forests on mesic sandy sites (typical for Central European Russia) with a rather mild climate close to the border of a broad-leaved forest zone. EFIMOD was used to simulate tree and stand growth from young age to maturity for each of the Canadian and Russian sites. Models used for the climate change scenarios were the Coupled Global Climate Model (CGCM) A2 in Canada (Price et al., 2004) and the Hadley Centre Coupled Model version 3 (HadCM3) A1Fi (Mitchell et al., 2004) in Russia. Results comparing prescribed and Monte Carlo simulations in this example (Figs. 2–4) were basically the same as in the previous case with balsam fir using the soil C model of Larocque et al. (2006). For the important functional parameter – CO2 emissions from the soil – the predictions from the 406 e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 Fig. 2 – Comparison of predicted carbon pools in a black spruce (Picea mariana (Mill.) BSP) forest ecosystem located in the boreal forest of western Canada using a process-based model of the soil carbon cycle following prescribed and Monte Carlo simulations. Error bars represent the standard deviations computed from the predictions using the Monte Carlo method. prescribed simulations were fairly close to the means of the Monte Carlo simulations. The fluctuations of the parameters reflected variations in temperature and precipitation in the climatic scenarios used. There was also a visible overlapping of standard deviations along the time series with and without climate change for all C stocks and respiration, which shows that tree productivity and changes in soil are strongly linked to environmental changes. However, in the case of the Russian site with Scots pine, changes in stand productivity and soil parameters were more pronounced under climate change scenarios due to ‘aridization’ in the southern part of Russian European forests (Fig. 4). The growth of soil-C in the Russian simulations was a result of rather high N input from the atmosphere (10 kg m−2 year−1 ) and rather deep soil degradation during the half-millenium intensive exploitation due to rotation of lands with extensive agriculture (especially in the 20th century following three revolutions and two world wars). 4. Future directions The studies reviewed and the case studies examined illustrate the potential of the Monte Carlo method to guide modellers in identifying model weaknesses. Despite the fact that the Monte Carlo method has been proven effective over time, it has been seldom applied for C cycle forest ecosystem process-based models. As previously mentioned, the fact that excessive computer time may be required to perform uncertainty analysis with the Monte Carlo method is certainly an important reason for this. When models are transferred to end-users, the simulations must be completed efficiently within a short time period. Even though today’s computers are very powerful, the use of uncertainty analysis methods for complex process-based models, such as the Monte Carlo method, may require several hours, if not days, to complete the necessary simulations. These reasons justify the identification of new research directions to facilitate future use of uncertainty analysis in C cycle forest ecosystem models. 4.1. Data sampling The issues concerning the lack of long-term historical data (previously discussed in general terms to explain the lack of uncertainty estimates in C cycle forest ecosystem models) most certainly remain relevant to explain the lack of Monte Carlo analysis. As far as information on both ecophysiological processes and long-term changes in the C pools of forest ecosystems are concerned, there are very few historical datasets of sufficiently long timescales that can be used to derive appropriate uncertainty estimates. In addition, the e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 407 Fig. 3 – Comparison of predicted carbon pools in a jack pine (Pinus banksiana Lamb.) forest ecosystem located in the boreal forest of western Canada using a process-based model of the soil carbon cycle following prescribed and Monte Carlo simulations. Error bars represent the standard deviations computed from the predictions using the Monte Carlo method. poor availability of portable instruments has been an important factor that limited the development of ecophysiological datasets. With respect to soil C pool data, inadequate sampling intensity to account for the spatial variation across the landscape and the vertical variation with horizon depth due to microrelief, animal activity, windthrow, litter/coarse woody debris input, human activity, effect of individual plants on soil microclimate, and precipitation chemistry are important sources of uncertainty. The technology for ecophysiological measurements, such as photosynthesis, respiration or transpiration, has been developed only recently and the relatively high cost has limited use. The availability issue is particularly important in the context of sound statistical sampling, which is essential to ensuring that uncertainty estimates are statistically valid. Regarding the longevity factor (the effect of gradual change over time related to the long-lived nature of forest ecosystems), the evaluation or uncertainty analysis efforts that have been conducted using historical data have been undertaken mostly with traditional inventory data, such as annual volume increment (Medlyn et al., 2005). Thus, it may be possible to capture the longevity factor using inventory data, but not to explain the processes governing tree and stand growth. Radtke et al. (2001) point out that a major consequence of this situation is the lack of appropriate statistical data on the distribution of many parameters (that compose several mathematical relationships) describing basic ecophysiological processes. For their uncertainty analysis study using the process-based model PnET-II (Aber et al., 1995), Radtke et al. (2001) obtained information on probability distributions for key parameters mostly from published studies. This approach may lead to imprecise uncertainty estimates, as it is not evident that the information obtained in this fashion adequately represents the population of interest (Radtke et al., 2001). All these factors justify the continuation of research efforts on uncertainty analysis for C cycle forest ecosystem models in order to improve the methodologies of uncertainty evaluation and considerably shorten model execution time. Also, the probability distributions of key parameters must be better described using appropriate statistical sampling methods. Major research efforts should aim at taking advantage of recent technological advances in ecophysiological instrumentation to perform measurements on adequately replicated experimental designs based on a chrono-sequence approach to capture the longevity factor associated with age and succession. The chrono-sequence approach is based on the study of the stand dynamics of the same forest type growing under similar site conditions, but differing in age or successional status (Brubaker, 1980). Recent technological developments in gas exchange instruments allow tree physiologists to take mea- 408 e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 Fig. 4 – Comparison of predicted carbon pools in a Scots pine (Pinus sylvestris L.) forest ecosystem located in Russia using EFIMOD, a forest ecosystem model, following prescribed and Monte Carlo simulations. Error bars represent the standard deviations computed from the predictions using the Monte Carlo method. surements for very short time periods, from hours to minutes or seconds. For instance, Medlyn et al. (2005) performed model evaluation and uncertainty analyses using eddy covariance measurements. Despite the fact that their study was conducted in a young Sitka spruce (Picea sitchensis (Bong.) Carr.) stand, they were able to evaluate the logical consistency and uncertainty estimates of several components of a processbased model. Thus, the analytical procedures that they used could be applied to forest ecosystems belonging to a chronosequence. 4.2. Computing efficiency and statistical validity As far as computing efficiency and statistical validity are concerned, there is a need to develop a historical database of uncertainty analysis applications and test different sampling designs. As previously discussed, there is a limited history in applying the Monte Carlo method to model ecophysiological processes in forest ecosystems. This fact implies that there is not much knowledge on the optimum number of execution cycles that must be completed to obtain statistically valid results and even less information on the efficiency of sampling designs. Thus, the forest ecosystem modellers who used the Monte Carlo method have applied one of the basic principles of statistics: the degree of confidence in the esti- mates of population parameters, such as mean or standard deviation, increases with sample size. As a consequence, the majority of the studies that have applied the Monte Carlo method to forest ecosystems models have performed simulations with many cycles. Using the most powerful PCs currently available, it may require a few days to complete the simulations. Two solutions may be considered to reduce computing time. First, simulation experiments aimed at comparing uncertainty estimates for different execution cycles should be conducted in a systematic fashion to determine the minimum number of execution cycles required to obtain statistically valid results. This step could be conducted by applying bootstrap methods to compare uncertainty estimates (see Chernick, 1999). For statistical consistency, this task should be undertaken using several process-based models commonly used to simulate the C cycle of various types of forest ecosystems (e.g., Dufrêne et al., 2005; Komarov et al., 2003). These ecosystems should differ in species composition and site characteristics, including soil and climatic conditions. Second, the statistical designs based on stratified sampling methods, such as LHS, should be tested consistently. As stratified sampling methods define specific boundaries of the random selection of parameters during the Monte Carlo simulations, they will reduce the number of simulation cycles needed to e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 obtain statistically valid results for C cycle forest ecosystem models. Computation efficiency can also be considered in terms of model structure. For instance, Zhang et al. (2008) compared several soil decomposition models differing substantially in terms of the number of pools and processes addressed and the process formulations. Models with more pools and processes (e.g., CENTURY, DOCMOD, SOMM) were not necessarily better than models with fewer pools and processes (e.g., FLDM, CANDY) in achieving good agreement between observed and predicted values. In general, more complex models require more complicated model formulation, initialization, and parameter estimation efforts which may lead to higher uncertainty and greater error propagation. A better knowledge of the effects of the number of execution cycles and application of statistical designs on uncertainty estimates can also contribute to improving model comparison exercises. Smith et al. (1997) evaluated and compared the performance of nine different soil organic models using 12 different datasets for grassland, arable cropland, and woodland over a range of climatic conditions within the temperate region. The differences in the predictions were related to model specific initial conditions as well as application to ecosystems different from those for which the models were developed. Smith et al. (1997) further observed that small changes in initial condition input values could lead to very different model output responses in all the cases. Some of 409 the conclusions with respect to the comparison of observed and predicted values would have been different if an uncertainty analysis had been conducted, i.e., observed values were provided with standard errors but there was no uncertainty estimate for the predictions. As a consequence, the Smith et al. (1997) comparison of different soil organic models was not based on evaluating the degree to which there was overlap between both observed and predicted errors. This additional criterion strengthens the comparison standard. For instance, the means of observed and predicted values may substantially differ, but a large overlap between the respective errors may contribute to reducing the importance of the differences with respect to the biological consistency of the predictions. 4.3. Model-driven decision support system framework The application of uncertainty analysis in process-based models of forest ecosystems can be improved by developing model-driven decision support systems (MDSS) based on the use of different interactive analytical applications (e.g., Fig. 5). Examination of the literature shows that the application of uncertainty analysis in C cycle forest ecosystem models has consisted for the most part in developing simple subroutines or functions within the source code of models, without integration of decision-support tools. For instance, it appears that both sensitivity and uncertainty analyses have been carried out “manually” and independently, which Fig. 5 – Theoretical framework for the development of a model-driven decision support system to support the computation of uncertainty analysis. 410 e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 can be cumbersome for end-users, without the support of software to automate the process. Software has been developed to facilitate sensitivity or uncertainty analyses, such as SIMLAB (http://sensitivity-analysis.jrc.cec.eu.int) or GEMSA (http://ctcd.group.shef.ac.uk/gem.html). Both applications contain rich features that can facilitate the undertaking of sensitivity and uncertainty analyses. However, they still have limited capabilities to process relatively complex information. There is a need to develop applications, such as expert systems, that integrate complex sets of inference and mathematical rules to efficiently analyze quantitative information from different sources. The sensitivity analysis section of the suggested MDSS framework design in Fig. 5 aims at selecting key parameters for further uncertainty analysis. The process of sensitivity analysis can be fully automated by developing the necessary software components to evaluate the degree to which the outputs of a model vary with changes in parameters. Single or multiple parameter variations could be allowed. Once this step is completed, the outputs can be processed using different analytical methods, such as the derivation of distance coefficients (e.g., Haefner, 1996), regression analysis (e.g., Helton and Davis, 2003) or identifiability analysis (e.g., Brun et al., 2001, 2002). Regression analysis, which can be used by considering sensitivity analysis as an experimental design, becomes practical to assess the relative contribution of several parameters by computing, for instance, standardized regression coefficients. Several methods of identifiability analysis have been developed which usually combine different analytical steps. For instance, the method proposed by Brun et al. (2001, 2002) consists of an iterative procedure that focuses on the ranking of the parameters and the computation of multivariate coefficients, such as collinearity indices. The sensitivity indices derived (Fig. 5) contribute to assisting the end user in selecting the key parameters that should be submitted to uncertainty analysis. Using scientific judgement, the user could: (1) let the MDSS system decide (based on inference rules) which parameters should be included in the uncertainty analysis step, or (2) make decisions by providing specific limits or conditions on the basis of interpreting the sensitivity indices. For the uncertainty analysis step, the suggested MDSS framework also includes decision rules that determine the most appropriate method for each model sub-component (Fig. 5). The methods based on the analytical solution of differential equations (ASDE), such as the Taylor series expansion or the first-order variance propagation analysis (see Kremer, 1983), can provide uncertainty estimates very efficiently compared with the Monte Carlo method, but only for linear systems. When it is not possible to derive analytical solutions, which is the case for many complex relationships based on the representation of the dynamics of ecophysiological processes, the Monte Carlo method is then applied. Thus, the proposed MDSS framework could contain the necessary software to verify the model components for which ASDE can be applied. This can be done using automatic differentiation software. Once uncertainty outputs are estimated, further analysis, such as error budget (see Gertner, 1990; Parysow et al., 2000) or neural network (see Guan et al., 1997), can be used. Both methods estimate the absolute or relative contributions of the various error sources. 5. Conclusion Major achievements have been accomplished in the development of process-based models for the simulation of the C cycle in forest ecosystems. Like any other simulation model, C cycle models consist of imperfect representations of reality. The factors and the sources of error that influence parameter uncertainty contribute to increasing errors in the predictions. Therefore, it is important that scientists or policy makers who rely on model predictions in their decision-making process be able to evaluate the degree to which the results are uncertain. However, the application of uncertainty analysis in C cycle process-based models is not a common procedure because of the lack of knowledge on the sources of uncertainty, the complexity of the methodologies involved and the model execution time required. There are two kinds of research efforts that need to be accomplished to further support uncertainty analysis. First, it is important to increase scientific knowledge on parameter variability by increasing the intensity of ecophysiological measurements in forest ecosystems that differ in species composition and site conditions. Different sampling designs must be tested and compared to improve statistical validity at given sites. Second, there is a need to think differently in the development of new uncertainty analysis software by combining different uncertainty analysis methods and automating further the analytical steps that are involved. Acknowledgements Sincere thanks are extended to Dr. David Paré, with the Canadian Forest Service, Quebec, QC, for providing incubation data used in some analyses and comments on an earlier version of the manuscript. The comments of Thomas White, Canadian Forest Service, Victoria, B.C., were greatly appreciated. We acknowledge the financial contribution of PERD, a research program of the Government of Canada. references Aber, J.D., Ollinger, S.V., Federer, C.A., Reich, P.B., Goulden, M.L., Kicklighter, D.W., Melillo, J.M., Lathrop, R.G., 1995. Predicting the effects of climate change on water yield and forest production in the northeastern United States. Clim. Res. 5, 207–222. Allen, M.R., Booth, B.B.B., Frame, D.J., Gregory, J.M., Kettleborough, J.A., Smith, L.A., Satinforth, D.A., Stott, P.A., 2004. Observational constraints on future climate: distinguishing robust from model-dependent statements of uncertainty in climate forecasting. In: Contribution to the IPCC Workshop on Communicating Uncertainty and Risk. Annan, J.D., 1997. On repeated parameter sampling in Monte Carlo simulations. Ecol. Model. 97, 111–115. Annan, J.D., 2001. Modelling under uncertainty: Monte Carlo methods for temporally varying parameters. Ecol. Model. 136, 297–302. Brubaker, L.B., 1980. Long-term forest dynamics. In: West, D.C., Shugart, H.H., Botkin, D.B. (Eds.), Forest Succession, Concepts and Application. Springer-Verlag, New York, pp. 95–106. e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 Brun, R., Kühni, M., Siegrist, H., Gujer, W., Reichert, P., 2002. Practical identifiability of ASM2d parameters—systematic selection and tuning of parameter subsets. Water Res. 36, 4113–4127. Brun, R., Reichert, P., Kunsch, H.R., 2001. Practical identifiability analysis of large environmental simulation models. Water Resour. Res. 37, 1015–1030. Buckley, T.N., Roberts, D.W., 2005. DESPOT, a process-based tree growth model that allocates carbon to maximize carbon gain. Tree Physiol. 26, 129–144. Chernick, M.R., 1999. Bootstrap Methods. A Practitioner’s Guide. John Wiley & Sons, Inc., Toronto. Chertov, O., Komarov, A., Kolström, M., Pitkänen, S., Strandman, H., Zudin, S., Kellomäki, S., 2003. Modelling the long-term dynamics of populations and communities of trees in boreal forests based on competition for light and nitrogen. For. Ecol. Manage. 176, 355–369. Cipra, B., 2000. Revealing uncertainties in computer models. Science 287, 960–961. Crosetto, M., Tarantola, S., 2001. Uncertainty and sensitivity analysis: tools for GIS-based model implementation. Int. J. Geogr. Inf. Sci. 15, 415–437. Dufrêne, E., Davi, H., François, C., le Maire, G., Le Dantec, V., Granier, A., 2005. Modelling carbon and water cycles in a beech forest. Part I. Model description and uncertainty analysis on modelled NEE. Ecol. Model. 185, 407–436. Elston, D.A., 1992. Sensitivity analysis in the presence of correlated parameter estimates. Ecol. Model. 64, 11–22. Garbey, C., Garbey, M., Muller, S., 2006. Using modeling to improve models. Ecol. Model. 197, 303–319. Gardner, R.H., Dale, V.H., O’Neil, R.V., 1990. Error propagation and uncertainty in process modeling. In: Dixon, R.K., Meldahl, R.S., Ruark, G.A., Warren, W.G. (Eds.), Process Modelling of Forest Growth Responses to Environmental Stress. Timber Press, Portland, OR, pp. 208–219. Gertner, G., 1990. Error budgets: a means of assessing component variability and identifying efficient ways to improve model predictive ability. In: Dixon, R.K., Meldahl, R.S., Ruark, G.A., Warren, W.G. (Eds.), Process Modelling of Forest Growth Responses to Environmental Stress. Timber Press, Portland, OR, pp. 220–225. Guan, B.T., Gertner, G.Z., Parysow, P., 1997. A framework for uncertainty assessment of mechanistic forest growth models: a neural network example. Ecol. Model. 98, 47–58. Haefner, J.W., 1996. Modeling Biological Systems. Chapman & Hall, New York. Hammonds, J.S., Hoffman, F.O., Bartell, S.M., 1994. An Introductory Guide to Uncertainty Analysis in Environmental and Health Risk Assessment. ES/ER/TM-35/R1, Oak Ridge National Laboratory, Oak Ridge, TN. Haness, S.J., Roberts, L.A., Warwick, J.J., Cale, W.G., 1991. Testing the utility of first order uncertainty analysis. Ecol. Model. 58, 1–23. Helton, J.C., Davis, F.J., 2003. Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems. Reliability Eng. Syst. Saf. 81, 23–69. Hollinger, D.Y., Richardson, A.D., 2005. Uncertainty in eddy covariance measurements and its application to physiological models. Tree Physiol. 25, 873–885. Komarov, A., Chertov, O., Zudin, S., Nadporozhskaya, M., Mikhailov, A., Bykhovets, S., Zudina, E., Zoubkova, E., 2003. EFIMOD 2—a model of growth and cycling of elements in boreal forest ecosystems. Ecol. Model. 170, 373–392. Koskela, J., 2000. A process-based growth model for the grass stage pine seedlings. Silva Fenn. 34, 3–20. Kremer, J.N., 1983. Ecological implications of parameter uncertainty in stochastic simulation. Ecol. Model. 18, 187–207. Larocque, G.R., Boutin, R., Paré, D., Robitaille, G., Lacerte, V., 2006. Assessing a new soil carbon model to simulate the effect of 411 temperature increase on the soil carbon cycle in three eastern Canadian forest types characterized by different climatic conditions. Can. J. Soil Sci. 86, 187–202. Law, B.E., Sun, O.J., Campbell, J., Van Tuyl, S., Thornton, P.E., 2003. Changes in carbon storage and fluxes in a chronosequence of ponderosa pine. Global Change Biol. 9, 510–524. Law, B.E., Turner, D., Campbell, J., Sun, O.J., Van Tuyl, S., Ritts, W.D., Cohen, W.B., 2004. Disturbance and climate effects on carbon stocks and fluxes across Western Oregon USA. Global Change Biol. 10, 1429–1444. Linkov, I., Burmistrov, D., 2003. Model uncertainty and choices made by modelers: lessons learned from the International Atomic Energy Agency model intercomparisons. Risk Anal. 23, 1297–1308. Medlyn, B.E., Robinson, A.P., Clement, R., McMurtrie, R.E., 2005. On the validation of models of forest CO2 exchange using eddy covariance data: some perils and pitfalls. Tree Physiol. 25, 839–857. Mitchell, T.D., Carter, T.R., Jones, P.D., Hulme, M., New, M., 2004. A comprehensive set of high-resolution grids of monthly climate for Europe and the globe: the observed record (1901–2000) and 16 scenarios (2001–2100). Tyndall Centre for Climate Change Research, Working Paper 55. 30 pp. http://www.cru.uea.ac.uk/∼timm/grid/TYN SC 2 0.html. Morgan, M.G., Dowlatabadi, H., 1996. Learning from integrated assessment of climate change. Clim. Change 34, 337–368. O’Neill, R.V., Rust, B., 1979. Aggregation error in ecological models. Ecol. Model. 7, 91–105. Parysow, P., Gertner, G., Westervelt, J., 2000. Efficient approximation for building error budgets for process models. Ecol. Model. 135, 111–125. Paul, K.I., Polglase, P.J., Richards, G.P., 2003. Predicted change in soil carbon following afforestation or reforestation, and analysis of controlling factors by linking a C accounting model (CAMFor) to models of forest growth (3PG), litter decomposition (GENDEC) and soil C turnover (RothC). For. Ecol. Manage. 177, 485–501. Price, D.T., McKenney, D.W., Papadopol, P., Logan, T., Hutchinson, M.F., 2004. High resolution future scenario climate data for North America. In: Proc. Amer. Meteor. Soc. 26th Conference on Agricultural and Forest Meteorology, Vancouver, B.C., August 23–26, 13 pp (CD-ROM). Radtke, P.J., Burk, T.E., Bolstad, P.V., 2001. Estimates of the distributions of forest ecosystem model inputs for deciduous forests of eastern North America. Tree Physiol. 21, 505–512. Rowe, W.D., 1994. Understanding uncertainty. Risk Anal. 14, 743–750. Saltelli, A., Torantola, S., Chan, K.P.S., 1999. A quantitative model-independent method for global sensitivity analysis of model output. Technometrics 41, 39–56. Shaw, C., Chertov, O., Komarov, A., Bhatti, J., Nadporozskaya, M., Apps, M., Bykhovets, S., Mikhailov, A., 2006. Application of the forest ecosystem model EFIMOD 2 to jack pine along the Boreal Forest Transect Case Study. Can. J. Soil Sci. 86, 171–185. Smith, J.E., Heath, L.S., 2001. Identifying influences on model uncertainty: an application using a forest carbon budget model. Environ. Manage. 27, 253–267. Smith, P., Smith, J.U., Powlson, D.S., McGill, W.B., Aha, J.R.M., Chertove, O.G., Coleman, K., Franko, U., Frolking, S., Jenkinson, D.G., Jensen, L.S., Kelly, R.H., Klein-Gunnewiek, H., Somarov, A., Li, C., Molina, J.A.E., Meuller, T., Parton, W.J., Thornley, J.H.M., Whitmore, A.P., 1997. A comparison of the performance of nine soil organic matter models using datasets from seven long-term experiments. Geoderma 81, 153–225. Verbeeck, H., Samson, R., Verdonck, F., Lemeur, R., 2006. Parameter sensitivity and uncertainty of the forest carbon 412 e c o l o g i c a l m o d e l l i n g 2 1 9 ( 2 0 0 8 ) 400–412 flux model FORUG: a Monte Carlo analysis. Tree Physiol. 26, 807–817. Xu, C., He, H.S., Hu, Y., Chang, Y., Larsen, D.R., Li, X., Bu, R., 2004. Assessing the effect of cell-level uncertainty on a forest landscape model simulation in northeastern China. Ecol. Model. 180, 57–72. Zähle, S., Sitch, S., Smith, B., Hattermann, F., 2005. Effects of parameter uncertainties on the modeling of terrestrial biosphere dynamics. Global Biogeochem. Cycles 19 (GB3020), 1–16. Zhang, C.F., Meng, F.-R., Bhatti, J.S., Trofymow, J.A., Arp, P.A., 2008. Modeling forest leaf-litter decomposition and N mineralization in litterbags, placed across Canada: a 5-model comparison. Ecol. Model. 219, 342–360.