Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Review Stochastic formulation of ecological models and their applications Andrew J. Black1 and Alan J. McKane2 1 2 School of Mathematical Sciences, The University of Adelaide, Adelaide, SA 5005, Australia Theoretical Physics Division, School of Physics and Astronomy, University of Manchester, Manchester, M13 9PL, UK The increasing use of computer simulation by theoretical ecologists started a move away from models formulated at the population level towards individual-based models. However, many of the models studied at the individual level are not analysed mathematically and remain defined in terms of a computer algorithm. This is not surprising, given that they are intrinsically stochastic and require tools and techniques for their study that may be unfamiliar to ecologists. Here, we argue that the construction of ecological models at the individual level and their subsequent analysis is, in many cases, straightforward and leads to important insights. We discuss recent work that highlights the importance of stochastic effects for parameter ranges and systems where it was previously thought that such effects would be negligible. Why should ecological models be individual based and stochastic? Ecology was the first area of biology where quantitative models were constructed, and it is still one of the most mathematically well developed of the biological sciences. Early models were formulated at the population level, frequently as differential equations, because these were the only tools available at the time (Box 1). The advent of electronic computers led to a huge increase in the variety and complexity of models that can be studied, most naturally formulated at the individual rather than the population level. This is clearly a better starting point for building ecological models as it explicitly recognises the discreteness of the population and the stochastic nature of the dynamics. As such, these types of model can capture a larger range of phenomena than can basic populationlevel models (PLMs; see Glossary). The main drawback of such an approach is that the results are frequently numerical in nature, making any theoretical understanding difficult. In this review, we highlight the recent trend whereby ecological models are formulated as stochastic Markov processes so they can be studied analytically as well as numerically [1–5]. We refer to these as individual-based models (IBMs). This is to be contrasted with what we term ‘agent-based models’, which although based on individuals, are often defined in terms of algorithms, thus precluding any analysis. These two approaches, along with PLMs, are summarised in Box 1. Glossary Discretise: the procedure whereby one replaces a continuous variable, such as the fraction of the population that consists of a particular type in a PLM, by a discrete variable, such as the number of individuals of that type. Individual-based model (IBM) or individual-level model (ILM): a model in which the basic entities are individuals. This description in terms of discrete variables is in contrast to PLMs, where the description is in terms of continuous population densities. The former are sometimes referred to as a microscopic description and the latter as a macroscopic description. Limit cycle: a periodic cycling in the macroscopic dynamics. Limit cycles are deterministic in that they remain in phase, as contrasted with quasi-cycles, which are phase forgetting. Macroscopic: description of a system on a large scale, so that the detailed (microscopic) structure is not observable. In ecology, this will typically be a description at the population level involving population densities, rather than individuals. Markov process: a stochastic process where the probability of making a transition to a new state only depends on the current state of the system, and not on its past history. Master equation: the equation for a continuous-time Markov process that specifies how the probability of the system being in a given state changes with time. More specifically, it gives an expression for dPðn;tÞ in terms of transitions dt into the state n minus transitions out of this state. Microscopic: description of a system at a small scale in terms of the entities making up the system. In ecology, this will typically be a description at the individual level. The processes via which the population of individuals evolves will then be probabilistic. Phenomenological: refers to the process of constructing a model. Such a model does not have a microscopic basis, but is simply postulated in a consistent manner using observed phenomena and general principles. Power spectrum (or power spectral density): a real positive function of the frequencies that appear in a time series. If all frequencies are equally represented (white noise) then it is a constant (i.e. flat), whereas if the signal is sinusoidal (i.e. only one frequency, v0, is present) then it is a spike at v0. For a stochastic or noisy sinusoidal signal, where the typical frequency is v0, but where there is a range of frequencies within a band v0 Dv, the power spectrum will have a peak at v0 and a width of the order of Dv. Quasi-cycles: a stochastic phenomenon where demographic noise resulting from the individual nature of the population excites the system, giving rise to noisy cycling in the dynamics. In general, the power spectrum of these will be peaked with a finite width, indicating a range of excited frequencies. Resonance: the tendency of a system to oscillate at greater amplitude at some frequencies rather than at others. If the system is subject to external forces at these frequencies, large amplitude oscillations will be produced. Stability matrix: a matrix of partial derivatives (the Jacobian) evaluated at a fixed point of a set of differential equations. If all its eigenvalues have a negative real part, then the fixed point is stable. State variables: for an IBM, these are the number of different types of individual at a given time: ni individuals of type i, where i = 1,2,. . .. This is frequently written as a vector n ¼ ðn1 ; n2 ; . . .Þ. The i could label species, locations in space, age, or any other trait or attribute. For a PLM, the state variables are continuous and denoted as xi, with the entire state written as x ¼ ðx 1 ; x 2 ; . . .Þ. Stochastic process: a process where the rule for making a transition to a new state of the system at time t+1 (or t+dt, if time is continuous) from the current state at time t is a stochastic (i.e. random) variable. So, unlike a deterministic process, only the probability of being in a given state n at time t can be specified. This is denoted by P(n,t). Stochastic simulation algorithm (SSA) or the Gillespie algorithm: a Monte Carlo scheme for the simulation of a stochastic Markov chain. This scheme is exact; the algorithm is derived from the same assumptions as the master equation. Corresponding author: McKane, A.J. ([email protected]). 0169-5347/$ – see front matter ß 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.tree.2012.01.014 Trends in Ecology and Evolution, June 2012, Vol. 27, No. 6 337 Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6 White noise: the term ‘noise’ refers to a stochastic (random) process and ‘white’ to one where all frequencies are equally represented. It draws its name from the idea that colours of different frequencies are equally represented in white light. We first review what constitutes an IBM; we look at the construction of IBMs and discuss setting up the (stochastic) dynamics of the system. This is illustrated using several different types of ecological model. The fundamental equation governing the dynamics of the stochastic process is the so-called ‘master equation’, which is simply a continuous time version of a Markov chain. However, when the IBM is viewed in this way, powerful techniques can be used to analyse the master equation, leading to quantitative predictions for several interesting ecological problems. We emphasise the importance of deriving PLMs from IBMs, and discuss a variety of examples drawn from the literature. We show how the concrete predictions that Box 1. Three approaches to ecological modelling Population-level models PLMs are typically formulated as ordinary differential equations for the fraction of the population that is of a certain kind. For example, Levins’ original metapopulation model is written as Equation I [78]: dQ ¼ cQð1 QÞ eQ; dt [I] where Q is the fraction of patches that are occupied and c and e are effective parameters. This is a phenomenological equation; that is, written down consistently, but not derived from a more basic model. The variables, such as Q, are continuous, implicitly assuming an infinite population size, and thus the results are deterministic (nonrandom). The advantage of this approach is that the well-developed mathematics of dynamical systems is applicable. This is still considered the dominant methodology, with many textbooks available [40,47]. If one starts with this approach, then the question of how to include stochastic effects arises. Some authors discretise the equations whereas others add stochastic (noise) terms on to the equations. Both of these approaches are ill defined. Agent-based models Agent-based models have a finite number of individuals in the system, each typically having many attributes or degrees of freedom [79,80]. These are frequently defined in terms of computer algorithms ready for simulation. The advantages of models of this type are the potentially unlimited amounts of detail that can be included and the ease of programming. The main disadvantage is that the algorithmic approach can only produce numerical results; in general, they are too complex for mathematical analysis. can be obtained from IBMs using these tools can elucidate well-known, but poorly characterised, phenomena. There are many empirical studies that have attributed their findings to stochastic effects [6–11], making it an important long-term goal to develop a sound theoretical understanding of such effects. One of the major points to emerge from this line of research is the importance of stochastic effects in large populations, not just small ones, as has been traditionally assumed in the study of extinction effects. Creating stochastic models Beginning with IBMs, and deducing PLMs from them, is the approach taken in many of the physical sciences. In chemistry, for example, reaction kinetics at the molecular level yield rate equations at the macroscale and, in physics, statistical mechanical models at the microscale lead to a thermodynamical description at the macroscale. In ecological IBMs, random events (e.g. birth, death and predation) at the level of individuals (the microscale) give rise to macroscopic dynamics for large populations of individuals. In the limit of an infinite population, these dynamics become deterministic and can be described by a set of ordinary differential equations: a PLM. As we discuss later, when the population is large but still finite, these dynamics are often still strongly stochastic. Before we do this, we first discuss the construction of IBMs and some methods of analysis. Suppose that one wishes to describe an ecological system that contains m different species, and the only way that individuals will be labelled is as belonging to a given species. If the population size is assumed to be finite, equal to N, then the state of the system is given by a vector of integers n ¼ ðn1 ; . . . ; nm Þ, where ni is the number of individuals of species i at that time. This could also be the number of individuals of species i per unit area, or some other measure. Given that the basic idea of model specification can be illustrated on a system with just one species, we focus on a system with n identical individuals of type A. Suppose also that the only processes are death and (asexual) birth. We represent these as Equations 1 and 2, respectively: d A!E; b A þ E!A þ A Individual-based models IBMs are also composed of a finite number of individuals, but typically with a smaller number of attributes. The rules for the time evolution of the individuals within the system are formulated probabilistically, usually as a Markov process. Therefore, state variables are discrete instead of continuous and, by definition, the model is stochastic. In an IBM version of the metapopulation model (Equation I), n would be the number of patches and the dynamics of colonisation and extinction of patches are random events that depend on n (explicit details of this model are given in Box 2). The population is usually divided up into a small number of classes, reducing the degrees of freedom; thus, all members of the same class are indistinguishable. The probabilistic formulation of IBMs means they can be simulated exactly with Monte Carlo techniques [12], but most importantly they can be analysed analytically via the master equation. The corresponding PLM to a given IBM can also be analytically derived. 338 [1] [2] Equation 1 indicates that an individual of type A dies at a rate d to give a vacancy (E for ‘empty’); Equation 2 shows that if there is a vacancy, another individual of the same type is born at a rate b. The rate at which the number of individuals reduces from n to (n–1) is then T(n 1|n) = dn and the rate at which it increases from n to (n+1) is shown in Equation 3: n [3] Tðn þ 1jnÞ ¼ bn 1 N This last result assumes a well-mixed population and is proportional to the probability of an individual and a vacancy existing in the system. These are the fundamental ingredients that underpin the dynamics. We should Review emphasise that these rates depend on N and thus so do the dynamics of the model. This is not true of a PLM. Several examples of IBMs of increasing complexity are given in Box 2. Analysis of stochastic models Once a model has been specified, the complete stochastic dynamics are encoded analytically by the master equation, or individual realisations can be obtained by Monte Carlo simulation, which outputs a stochastic time series. These are the two main ways of investigating and solving these models (i.e. obtaining a complete description of the dynamics). The stochastic simulation algorithm (SSA) developed by Gillespie provides a method for the exact simulation of a master equation [12]. This type of simulation is the usual way of investigating these models, mainly because of its simplicity to program, but an important point is that the SSA and master equation are both derived from the same underlying Markovian assumptions, so that there is an exact correspondence between the two. Thus, any model that can be simulated in this way can be written as a master equation and vice versa. Of course, there are the usual drawbacks to just using simulations to investigate a model, such as the costs in terms of computing power and time, the numerical nature of the results and the difficulty in investigating rare events [13,14]. In general, simulations provide individual realisations of a stochastic process. The master equation (also known as Kolmogorov’s forward equation) describes analytically the temporal evolution of the probability density of being in a particular state n, thus encoding the full stochastic dynamics [12,15]. If it could be solved (for the probability density), then one would have a complete description of the properties and the dynamics of the stochastic system [4,16]. One calculation that is straightforward and can always be carried out is to determine the macroscopic behaviour from the master equation, which is the N ! 1 limit, as discussed in Box 3, and which yields the PLM corresponding to the original IBM. The master equation cannot be solved analytically for most cases of interest, and so several approximation methods have been developed [17]. These can be extended if the system involves a network [18–21]. If the population and the number of degrees of freedom are small, then it can be solved numerically [22]. Epidemic models with household structure are another interesting example where a master equation is used to describe the spread of infection within a household, but situated within an infinite population of such households, which allows the derivation of several population-level results [23,24]. Other ecological systems have this kind of hierarchical structure (e.g. involving collections of local populations) for which similar techniques might be applicable. We cannot discuss all these methods in this review, so we concentrate on techniques that set up an expansion of the , where N is, for master equation in increasing powers of p1ffiffiffi N example, the total population. If N is large, which is typical in ecological systems, then solutions of the master equation can be approximated by just two terms, which correspond to a deterministic part plus a stochastic correction that Trends in Ecology and Evolution June 2012, Vol. 27, No. 6 describes the effects of demographic stochasticity. This approximation is well known in the literature, and goes under several names: the van Kampen system size expansion [15], the Gaussian or diffusion approximation, or Kurtz’s theorem [25,26]. The method is discussed in more detail in Box 3, but essentially consists of approximating the probability density in the master equation with a Gaussian distribution. The power of this technique is that, starting from the master equation, it provides a very general method of deriving equations describing the macroscopic stochastic dynamics. Population level descriptions follow from individual level descriptions In an individual-based formulation, it is always the case that a finite population of N individuals is dealt with, which evolves via several random processes (e.g. birth, death and predation). For small N, the dynamics will be random. When the population becomes large, the dynamics will start to become deterministic (non-random). One way of investigating this behaviour is just to simulate large systems and average the results. Far more insight can be found by analytically deriving the corresponding PLM (a set of differential equations) from the IBM in the limit of infinite N (Boxes 1 and 3). This methodology clearly has several advantages. First, the predictions of the IBM and PLM can be compared exactly; thus, it is possible to see under what conditions a PLM will be a valid description of a given system. It is also helpful in writing down a PLM when the ecology of the system is complicated [27]. In general, it is easier to consider the events that define a system, write down an IBM and then derive the corresponding PLM. Often these derived equations may differ from the naively postulated equations in a simple population-level approach [28–31]. This also has the advantage that the explicit microscopic basis for the PLM is known and so more than just a phenomenological analysis is carried out. Unfortunately, this procedure is often reversed: a wellstudied PLM is discretised, interpreted as an IBM and then simulated to investigate various stochastic effects. This is exactly the wrong way round: in general, there will be many IBMs (sometimes infinitely many) giving a particular PLM, and so the PLM should always be derived from the IBM [31,32]. The reason why many IBMs give the same PLM is that IBMs define a stochastic process and PLMs are equations for the averages of this process; however, many stochastic processes have the same averages. Applications of IBMs described by master equations The modelling approach we have described has been applied to several problems over the past few years, and there is a clear trend towards the increasing use of this methodology. For example, in the case of the neutral model discussed in Box 2, the quantity that has been studied most intensively is the species abundance distribution. This is a plot of the number of species with 1,2,. . .,m individuals in a given local community. It can be calculated exactly in the classic model starting from the master equation [33], and fits well with empirical data [34–38]. Rather than discuss more specific examples, we review three broad aspects. 339 Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6 Box 2. Examples of ecological IBMs An advantage of IBMs is that specifying them is simple and they are straightforward to motivate biologically, with additional features being easily incorporated. Here, we give several examples, three of which are illustrated in Figure I. Table I. Neutral model. Event Birth–death Transition 1m A þ B! A þ A, 1m A þ B! B þ B; Neutral models The ‘classic’ neutral theory, introduced by Hubbell [81], consists of a local community with birth–death processes of the Moran type [82] and immigration from a metacommunity, which acts as a well-mixed source pool of potential immigrants [83,84]. Given that all individuals are assumed to have the same birth and death rates, we may fix our attention on one species, A, and denote all other species as B. Then the transitions are as given in Table I. Here, the rates do not include the combinatoric factor that gives the probability of choosing individuals of the desired species to interact. The quantity m is the probability that the replacement of an individual is the result of an immigration event, rather than by a birth–death event. The probability of a particular immigrant being chosen is given by the relative abundance of that particular species in the metacommunity, which can also be described as an IBM with birth–death and speciation [83]. This model is only implicitly spatial, but it can be extended in many ways; for example, by making it explicitly spatial [84], incorporating a chain of islands [85] or a network structure [86]. Metapopulation models Metapopulations are sets of fragmented local populations connected by migration [87]. At the simplest level of description, the patches can be thought of as being either occupied or unoccupied [2,16]. If A labels the occupied patches and E the unoccupied ones, then the possible events are as detailed in Table II. This leads to a simple one-dimensional master equation, where the state variable is the number of occupied patches. The same transition scheme applies to many models, such as logistic growth and the SIS model [17]. Again, this simple model has been elaborated on; for instance, by making the number of colonisable patches into a dynamic variable [2,88]. Spatial predator–prey model An example of a reaction–diffusion equation showing Turing patterns can be found in the description of plankton–herbivore dynamics given (a) m Immigration Table II. Metapopulation model. Event Colonisation Transition Extinction A!E, Migration m c A þ E!A þ A; e E!A: Table III. Predator-prey model. Event Linear growth Extinction Transition b P!P þ P; e Allee effect P þ P!P þ P þ P; p1 Predation 1 P þ H!H; Predation 2 P þ H!H þ H; Competition H þ H!H: p2 d by Levin and Segal [89]. An IBM that gives this model in the deterministic limit is given in [5]; within a patch, there are five processes between the plankton, P, and the herbivores, H (Table III). The model can simply be taken to describe the dynamics on a large well-mixed patch, but it can be made spatial by allowing both the P and H to jump between patches on a regular lattice. The introduction of spatial variation introduces no new points of principle in the modelling procedure, although the analysis typically becomes more complex [34,90]. (c) Colonisation m A!B, B!A: p Competition Predation redation 1 Allee effect Predation 2 Growth of P Immigration N patches Reproduction (b) Homogeneous patch Immigration Migration Migration Lattice of patches TRENDS in Ecology & Evolution Figure I. Three examples of ecological individual-based models (IBMs). (a) A simple metapopulation model. (b) A neutral island chain [85]. (c) The Levin–Segal model [5,89], which describes the explicitly spatial plankton–herbivore dynamics. 340 Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6 Box 3. The master equation and the emergence of macroscopic dynamics The transition rates T(n + 1jn) and T(n - 1jn) define the model. From these, the probability of finding n individuals in the system at time t, P(n,t) can be calculated from Equation I: replace the spike with a Gaussian. The system would now be stochastic, being described by a probability distribution function, although of the simplest kind. The width of the Gaussian would be expected to scale as p1ffiffiffi which means effectively writing Nn ¼ x þ pjffiffiffi , N N where x ¼ hNni and j is a stochastic variable. This is the content of the van Kampen system-size expansion. It turns out to be an excellent approximation (far better than might naively be expected) away from boundaries. Furthermore, given that Gaussian probability distributions are derived from linear stochastic differential equations, the whole theory is linear and so can be treated exactly, even though the underlying stochastic processes may be highly nonlinear. For most interesting applications, more than one degree of freedom is needed, but the whole theory generalises in this case to a set of stochastic differential equations for ji : (Equation II) dPðn; tÞ ¼ T ðnjn þ 1ÞPðn þ 1; tÞ þ T ðnjn 1ÞPðn 1; tÞ dt ½T ðn 1jnÞ þ T ðn þ 1jnÞPðn; tÞ: [I] This is the equation that governs the dynamics of the system. It is the equation for a Markov chain in continuous time, and is called the master equation by physical scientists. Although we have only discussed the case of a single species, all the formalism naturally generalises. We can replace n by n everywhere in the master equation where n is a vector with components that can represent any set of species in a large collection of spatial patches and having any number of other attributes. The master equation contains far more information than the usual differential equations written down phenomenologically in terms of the population density. The latter can be found by simply taking the average of Nn ; all the detail of the stochastic fluctuations is lost. Doing this for the metapopulation model described in Box 2 gives the wellknown logistic equation [40]. In the deterministic description, the probability distribution, P(n,t), is a spike at n ¼ hni. The simplest approximation beyond this is to dji X ¼ ai j j j ðtÞ þ hi ðtÞ; dt j [II] where hi(t) is a Gaussian white noise with zero mean, which is the only remnant of the demographic stochasticity of the system. It has a correlation function given by a matrix bij. So the whole theory is given by two matrices aij and bij, which can be systematically calculated from the transition rates that define the model. Figure I illustrates this theory using the metapopulation model described in Box 2. (a) 100 (b) 0.20 Patches 80 P 60 0.15 0.10 40 0.05 20 0 0.00 0 5 10 15 100 5 20 Time Time 50 Patches 10 Patches (c) 0 (d) 0.06 1400 1200 1000 800 600 400 200 0 0 0.04 P 0.02 0.00 5 10 15 Time 20 1500 5 1000 Time 10 0 500 Patches TRENDS in Ecology & Evolution Figure I. Emergence of macroscopic behaviour. The number of occupied patches for the metapopulation model for (a) N = 100, and (c) N = 1500 carrying capacity. The broken red lines show the deterministic result and the blue lines are from stochastic simulations. (b) and (d) show the evolution of the probability density for N = 100 and N = 1500, respectively. The peak of the distributions move with the deterministic result and the width is given by the solution of the stochastic differential equation 1 (Equation II). As the system gets larger, the amplitude of the fluctuations decreases as N 2 . In the limit N!1, the probability density becomes a spike. Quasi-cycles A common feature of some IBMs (with two or more degrees of freedom) is the ability of the demographic stochasticity to excite macroscopic-scale coherent oscillations, known as quasi-cycles [1,39]. An example of these is shown in Figure 1. A prerequisite for quasi-cycles to occur in a given IBM is that there is a stable fixed point in the corresponding PLM that is approached in an oscillatory manner. Expressed mathematically, this means that the stability matrix at this fixed point must have at least one pair of complex eigenvalues. However, although the oscillations in the deterministic system are damped and so die away, in the full (stochastic) system the demographic stochasticity acts as a forcing term and sustains the oscillations. From a mathematical point of view, this effect is described by Equation II of Box 3. A physical analogy is a pendulum that is lightly damped, so eventually any oscillations will die away. However, if it is randomly bombarded 341 Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6 (b) 120 100 1500 80 Power Infectives (a) 2000 1000 60 40 500 20 0 0 5 10 15 0 0.0 20 Time (years) 0.2 0.4 0.6 0.8 1.0 Frequency (1/years) TRENDS in Ecology & Evolution Figure 1. An example of quasi-cycles in a stochastic model. The blue and purple lines in (a) show two realisations of a stochastic individual-based model (IBM) [the susceptible–infectious–recovered (SIR) model [3]] that are started from the same initial condition. The broken red line shows the result using the corresponding populationlevel model (PLM). Whereas the oscillations are damped, tending to a stable fixed point in the population-level version, they are maintained in the stochastic IBM. These stochastic oscillations are different from limit cycles in that they are phase forgetting. This can be seen in the two realisations that, although started from the same initial condition, soon go out of phase. This is reflected in the power spectrum of the time series, shown in (b), which has a finite width. All frequencies shown in the spectrum are excited in the time-series, but the position of the peak indicates the dominant frequencies that are amplified the most. by weak white noise, these oscillations will be sustained. In addition, the amplitude of these oscillations will be much bigger than might naively be expected, because in among all the frequencies contained in the white noise will be one that corresponds to the natural frequency of the pendulum. This resonance effect has been called ‘stochastic amplification’. Quasi-cycles differ from limit cycles in that they are a stochastic phenomenon and provide a simple robust mechanism that generates cyclic behaviour without the need for additional complexities, as are often assumed, to generate cyclic behaviour in deterministic PLMs [40]. Figure 1 shows two realisations of a stochastic model and it is clearly seen that, after some time, these go out of phase, unlike true limit cycles. One of the most powerful tools for analysing such oscillations is the power spectrum [10,41]. This shows how the different frequencies that make up the time-series are distributed. Quasi-cycles are characterised by a strongly peaked power spectrum, which has a finite width. The width is important as it shows how coherent the cycles are: the smaller the width, the longer cycles remain in phase. The definition and meaning of the power spectrum are discussed in Box 4. An analytic approach using the master equation is particularly useful for the characterisation of these cycles. From the stochastic differential equations derived using the system-size expansion, one can easily derive an analytic expression for the power spectrum, again described in more mathematical detail in Box 4. This allows the easy investigation of this phenomenon throughout a wide parameter range as well as a vital check against the results of simulations. Other related questions that have been addressed are: what is the connection of quasi-cycles with limit cycles, which are the conventional way of describing oscillations in nonlinear dynamical systems [3,42]?; and are there characteristic signatures of the two types of oscillation that can be used to tell them apart in ecological time series [43]? Spatial patterns In 1952, Turing predicted that systems that consisted of agents of at least two different species that reacted together and spatially diffused could, under certain circumstances, give rise to patterns, now known as Turing patterns [44]. Field observations are now providing evidence that Turing 342 patterns are found in ecosystems [45,46]. This would be expected on purely theoretical grounds, given that spatial ecosystem dynamics is frequently described by reaction– diffusion equations [47]. However, just as demographic stochasticity leads to stochastic oscillations where a deterministic analysis finds only static behaviour, it also leads to Box 4. The power spectrum The stochastic cycles found in the IBMs discussed in this review do not have a single period, but a distribution of periods centred about an average value. This, together with the fluctuations in the amplitude of the cycles, means that when the time series from a large number of realisations, ji(t), are averaged over, they average out to zero. Yet, a single realisation corresponds to what will be seen in an experiment, and so some averaging mechanism is needed that will not wipe out the cycles. The simplest way to achieve this is to take the Fourier transform of the time series [41] (Equation I): j̃i ðvÞ ¼ Z þ1 ji ðtÞe ivt dt [I] 1 The frequencies centred around that corresponding to the average period will have the largest magnitude, and so taking theDmodulus 2 E squared of the Fourier transform and then averaging: j̃i ðvÞ ; should give a smooth function spread about the characteristic frequency of the system. This is the power spectrum of the fluctuations. The spread of the power spectrum will show how coherent the fluctuations are. If they were deterministic and sinusoidal, the power spectrum would be a spike. Although some ecologists were well aware of the existence of sustained oscillations, the effect has not been studied within a welldefined mathematical scheme; neither were the power spectra analysed carefully to learn more about the nature of the fluctuations. A range of phenomena have been revealed through more detailed recent studies. First, the height of the peaks of the power spectra (reflecting the amplitude of the oscillations) are much larger than might be expected owing to a resonance effect: all frequencies are present in the white noise h(t) and this resonates with the characteristic frequency of the system. Second, this characteristic frequency is not the same as the frequency of the decay of perturbations to the deterministic system [1]. The latter is given by jImlj; where l is a complex eigenvalue of the Jacobian of fluctuations about the stationary state. Instead, the peak of the qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðIm lÞ2 ðRe lÞ2 , and the power spectrum is approximately at precise value also depends on the bij. Third, it was not clear how the stochastic cycles related to conventional (deterministic) limit cycles, but this has now been understood [42]. Finally, the mathematical systematisation has led to the effect being found in other areas; for example, stochastic Turing patterns [5,51]. Review Trends in Ecology and Evolution June 2012, Vol. 27, No. 6 (a) (b) 700 700 600 600 500 500 400 400 300 300 88 80 72 56 48 Population 64 40 200 200 32 100 100 24 0 0 100 200 300 400 500 600 700 0 16 0 100 200 300 400 500 600 700 TRENDS in Ecology & Evolution Figure 2. Quasi-pattern formation in the stochastic Levin–Segal model as described in Box 2. (a) shows a population map of the model in the quasi-pattern phase. (b) shows the same data, but randomised. If the patchiness were a statistical artefact, then it would be preserved in the randomised version. Reproduced, with permission, from [5]. stochastic Turing patterns in spatial systems, where reaction–diffusion equations would indicate that none should exist. This has been shown in a variety of systems [5,48–50]; moreover, the range of parameters for which stochastic Turing patterns exist is usually much larger than for conventional Turing patterns, which have restrictions such as the diffusion constants of the two species being at least an order of magnitude different to each other. An example of this phenomenon is shown in Figure 2. The mechanism that gives rise to stochastic Turing patterns is essentially the same as that which gives rise to the stochastic oscillations discussed earlier. Now the model is spatial, and so the noise is a function of time and space, or of frequency, v, and wavenumber k, in the Fourier representation. The noise is still white, so all frequencies and wavenumbers are excited. We can now repeat the argument given above for the origin of stochastic oscillations, replacing v with k. If there is a peak in the power spectrum at a non-zero k, say kmax, then there is a natural wavenumber of the system that has been excited by the noise and is now sustained rather than decaying. In the conventional Turing description, this decaying mode would signify that the homogeneous state was stable to perturbations at k = kmax. The noise produces sustained oscillations (in space), which gives rise to spatial patterns with a characteristic wavenumber kmax. It is probable that at least some of the Turing patterns observed in nature are of this type [5,34,51]. Recurrent epidemics IBMs have a long history in epidemiological modelling, beginning with Bartlett’s pioneering work [52]. Many epidemic models are most intuitively formulated on the individual level and master equation methods have been a popular tool to investigate static properties [53,54]. The dynamics of childhood diseases, such as measles, whooping cough and rubella [55–57], provide particularly interesting case studies because they are subject to both external forcing [58], owing to the aggregation of children in schools, and demographic stochasticity. The resulting range of dynamics observed in both time-series case report data and simple models is rich and understanding the mechanistic basis for it is still an active research topic [59]. There are several potential mechanisms that could give rise to the observed dynamics. One of the most fascinating questions is whether these mechanisms are deterministic in origin, for example where noise causes the system to switch between deterministic states, which could be understood using just a deterministic model [60]. This is in contrast to the situation where noise plays an active role that cannot be explained with a deterministic model [61– 63]. As typical of the whole field, older studies have tended to use an ad hoc collection of methods to try and ‘disentangle’ stochastic and deterministic elements [9], with a reliance on simulations, leading to much debate and confusion over the supposed role of stochasticity [62]. Newer studies using an IBM approach have started to provide a more systematic account of these models and their dynamics, where the macroscopic dynamics, including deterministic elements, is instead viewed as emergent from the microscopic dynamics. In a previous subsection, we discussed the phenomenon of quasi-cycles about a deterministic fixed point. In models of recurrent epidemics, there is a strong forcing on the system, which in turn leads to a limit cycle in the deterministic dynamics, possibly with multiple stable attractors [60,63]. The previous analysis can be carried over, allowing one to derive the power spectrum of the forced model [3,42], which again affords a view of the stochastic dynamics ideally suited for understanding the underlying processes. In these models, the noise excites the transient dynamics about the limit cycle, so the resulting macroscopic time series are a superposition of these and the deterministic limit cycle. The relative proportions of the two periodicities reflected in the power spectrum then depend on the stability of the limit cycle and the size of the system. Such an analysis can encompass and explain the differing periodicities of whooping cough and measles before and after mass vaccination [3,60,64,65]. 343 Review Concluding remarks Our aim in this article was to highlight the trend towards models in ecology being individual based. There are several advantages in using stochastic models of this kind: they are easier to construct than are PLMs; they are more intuitive; and they predict phenomena that deterministic models miss. The importance of stochastic effects has long been known, going back to the earliest Monte Carlo simulations of epidemic spreading and fade-out [52], but computational investigations have outpaced the theoretical understanding of such models. Approaches based on the master equation are starting to remedy this and we have given here an overview of the philosophy and methodology of these. We expect that future research will continue to develop techniques based on this starting point. Several other areas are already using these techniques successfully; for instance, coevolutionary dynamics [66–68], cellular biology and genetics [69–71]. The construction of biological models is now driving the use of new techniques, such as incorporating non-Markov processes into the formalism [64,72] and parameter estimation, which follows elegantly on from the master equation and Gaussian approximation [73]. We expect that the future trend will be towards whole classes of model that interpolate between these different traditions of model building. Thus, agent-based models would be simplified by construction of a chain of related models that would terminate with a relatively simple IBM, which in turn could be studied analytically and its deterministic limit compared with the differential equations that are generally postulated phenomenologically. The range of models continues to grow and diversify. Some are now becoming significantly more complicated and are beginning to bridge the gap with agent-based models [74– 77]. This would pave the way to a more unified and coherent approach to model building in ecology, leading to greater predictive power. Acknowledgements A.J.B. acknowledges support from the EPSRC and the Australian Research Council’s Discovery Projects funding scheme (project number DP110102893). References 1 McKane, A.J. and Newman, T.J. (2005) Predator–prey cycles from resonant amplification of demographic stochasticity. Phys. Rev. Lett. 94, 218102 2 Ross, J.V. (2006) Stochastic models for mainland–island metapopulations in static and dynamic landscapes. Bull. Math. Biol. 68, 417–449 3 Black, A.J. and McKane, A.J. (2010) Stochastic amplification in an epidemic model with seasonal forcing. J. Theor. Biol. 267, 85–94 4 Shahrezaei, V. and Swain, P.S. (2008) Analytical distributions for stochastic gene expression. Proc. Natl. Acad. Sci. U.S.A. 105, 17256–17261 5 Butler, T. and Goldenfeld, N. (2011) Fluctuation-driven Turing patterns. Phys. Rev. E 84, 011112 6 Higgins, K. et al. (1997) Stochastic dynamics and deterministic skeletons: population behavior of Dungeness crab. Science 276, 1431–1435 7 Stenseth, N.C. et al. (1998) Phase- and density-dependent population dynamics in Norwegian lemmings: interaction between deterministic and stochastic processes. Proc. R. Soc. B 265, 1957–1968 8 Bjornstad, O.N. and Grenfell, B.T. (2001) Noisy clockwork: time series analysis of population fluctuations in animals. Science 293, 638–643 344 Trends in Ecology and Evolution June 2012, Vol. 27, No. 6 9 Rohani, P. et al. (2002) The interplay between determinism and stochasticity in childhood diseases. Am. Nat. 159, 469–481 10 Reuman, D.C. et al. (2006) Power spectra reveal the influence of stochasticity on nonlinear population dynamics. Proc. Nat. Acad. Sci. U.S.A. 103, 18860–18865 11 Krkosek, M. et al. (2011) Cycles, stochasticity and density dependence in pink salmon population dynamics. Proc. R. Soc. B 278, 2060–2068 12 Gillespie, D.T. (1992) Markov Processes: An Introduction for Physical Scientists, Academic Press 13 Assaf, M. and Meerson, B. (2010) Extinction of metastable stochastic populations. Phys. Rev. E 81, 021116 14 Ovaskainen, O. and Meerson, B. (2010) Stochastic models of population extinction. Trends Ecol. Evol. 25, 643–652 15 van Kampen, N.G. (1992) Stochastic Processes in Physics and Chemistry, Elsevier 16 Alonso, D. and McKane, A.J. (2002) Extinction dynamics in mainlandisland metapopulations: an N-patch stochastic model. Bull. Math. Biol. 64, 913–958 17 Newman, T.J. et al. (2004) Extinction times and moment closure in the stochastic logistic process. Theor. Popul. Biol. 65, 115–126 18 Simoes, M. et al. (2008) Stochastic fluctuations in epidemics on networks. J. R. Soc. Interface 5, 555–566 19 Rozhnova, G. and Nunes, A. (2009) Cluster approximations for infection dynamics on random networks. Phys. Rev. E 80, 051915 20 Sharkey, K.J. (2008) Deterministic epidemiological models at the individual level. J. Math. Biol. 57, 311–331 21 Dangerfield, C.E. et al. (2009) Integrating stochasticity and network structure into an epidemic model. J. R. Soc. Interface 6, 761–774 22 Keeling, M.J. and Ross, J.V. (2008) On methods for studying stochastic disease dynamics. J. R. Soc. Interface 5, 171–181 23 Goshal, G. et al. (2004) SIS epidemics with household structure: the self-consistent field method. Math. Biosci. 190, 71–85 24 Ross, J.V. et al. (2010) Calculation of disease dynamics in a population of households. PLoS ONE 5, e9666 25 Kurtz, T. (1978) Strong approximation theorems for density dependent Markov chains. Stoch. Process Appl. 6, 223–240 26 Baxendale, P.H. and Greenwood, P.E. (2010) Sustained oscillations for density dependent Markov processes. J. Math. Biol. 63, 433–457 27 Wang, Z. and Goldenfeld, N. (2010) Fixed points and limit cycles in the population dynamics of lysogenic viruses and their hosts. Phys. Rev. E 82, 011918 28 Goutsias, J. (2007) Classical versus stochastic kinetics modeling of biochemical reaction systems. Biophys. J. 92, 2350–2365 29 Grima, R. (2009) Noise-induced breakdown of the Michaelis-Menten equation in steady-state conditions. Phys. Rev. Lett. 102, 218103 30 Campillo, F. et al. (2011) Stochastic modeling of the chemostat. Ecol. Model. 222, 2676–2689 31 McKane, A.J. and Newman, T.J. (2004) Stochastic models in population biology and their deterministic analogs. Phys. Rev. E 70, 041902 32 Durrett, R. and Levin, S. (1994) The importance of being discrete (and spatial). Theor. Popul. Biol. 46, 363–394 33 McKane, A.J. et al. (2004) Analytic solution of Hubbell’s model of local community dynamics. Theor. Popul. Biol. 65, 67–73 34 Antonovics, J. and Edwards, M. (2011) Spatio-temporal dynamics of bumblebee nest parasites (Bombus subgenus Psythirus ssp.) and their hosts (Bombus spp.). J. Anim. Ecol. 80, 999–1011 35 Volkov, I. et al. (2005) Density dependence explains tree species abundance and diversity in tropical forests. Nature 438, 658–661 36 Muneepeerakul, R. et al. (2008) Neutral metacommunity models predict fish diversity patterns in Mississippi–Missouri basin. Nature 453, 220–222 37 Rosindell, J. et al. (2010) Protracted speciation revitalizes the neutral theory of biodiversity. Ecol. Lett. 13, 716–727 38 Vanpeteghem, D. and Haegeman, B. (2010) An analytic approach to spatio-temporal dynamics of neutral community models. J. Math. Biol. 61, 323–357 39 Kuske, R. et al. (2007) Sustained oscillations via coherence resonance in SIR. J. Theor. Biol. 245, 459–469 40 Brauer, F. and Castillo-Chavez, C. (2001) Mathematical Models in Population Biology and Epidemiology, Springer 41 Priestley, M.B. (1982) Spectral Analysis and Time Series, Academic Press Review 42 Boland, R.P. et al. (2009) Limit cycles, complex Floquet multipliers and intrinsic noise. Phys. Rev. E 79, 051131 43 Pineda-Krch, M. et al. (2007) A tale of two cycles – distinguishing quasicycles and limit cycles in finite predator–prey populations. Oikos 116, 53–64 44 Turing, A.M. (1952) The chemical basis of morphogenesis. Philos. Trans. R. Soc. B 237, 37–72 45 Maron, J.L. and Harrison, S. (1997) Spatial pattern formation in an insect host-parasitoid system. Science 278, 1619–1621 46 Reitkerk, M. and van de Koppel, J. (2008) Regular pattern formation in real ecosystems. Trends Ecol. Evol. 23, 169–175 47 Murray, J.D. (2002) Mathematical Biology, II: Spatial Models and Biomedical Applications, Springer-Verlag 48 Butler, T. and Goldenfeld, N. (2009) Robust ecological pattern formation induced by demographic noise. Phys. Rev. E 80, 030902 49 Biancalani, T. et al. (2010) Stochastic Turing patterns in the Brusselator model. Phys. Rev. E 81, 046215 50 Scott, M. et al. (2010) Approximating intrinsic noise in continuous multispecies models. Proc. R. Soc. A 467, 718–737 51 Woolley, T.E. et al. (2011) Power spectra methods for a stochastic description of diffusion on deterministically growing domains. Phys. Rev. E 84, 021915 52 Bartlett, M.S. (1960) Stochastic Population Models in Ecology and Epidemiology, Methuen 53 Nasell, I. (2002) Stochastic models of some endemic infections. Math. Biosci. 179, 1–19 54 Stollenwerk, N. and Jansen, V.A.A. (2003) Meningitis, pathogenicity near criticality: the epidemiology of meningococcal disease as a model for accidental pathogens. J. Theor. Biol. 222, 347–359 55 Bauch, C.T. and Earn, D.J.D. (2003) Transients and attractors in epidemics. Proc. R. Soc. Lond. B 270, 1573–1578 56 Keeling, M.J. and Rohani, P. (2007) Modelling Infectious Diseases in Humans and Animals, Princeton University Press 57 Metcalf, C.J.E. et al. (2011) The epidemiology of rubella in Mexico: seasonality, stochasticity and regional variation. Epidemiol. Infect. 139, 1029–1038 58 Altizer, S. et al. (2006) Seasonality and the dynamics of infectious diseases. Ecol. Lett. 9, 467–484 59 Mantilla-Beniers, N.B. et al. (2010) Decreasing stochasticity through enhanced seasonality in measles epidemics. J. R. Soc. Interface 7, 727–739 60 Earn, D. et al. (2000) A simple model for complex dynamical transitions in epidemics. Science 287, 667–670 61 Bauch, C.T. and Earn, D.J.D. (2003) Interepidemic intervals in forced and unforced SEIR models. Fields Inst. Comm. 36, 33–44 62 Coulson, T. et al. (2004) Skeletons, noise and population growth: the end of an old debate? Trends Ecol. Evol. 19, 359–364 63 Nguyen, H.T.H. and Rohani, P. (2008) Noise, nonlinearity and seasonality: the epidemics of whooping cough revisited. J. R. Soc. Interface 5, 403–413 64 Black, A.J. and McKane, A.J. (2010) Stochasticity in staged models of epidemics: quantifying the dynamics of whooping cough. J. R. Soc. Interface 7, 1219–1227 65 Rohani, P. et al. (1999) Opposite patterns of synchrony in sympatric disease metapopulations. Science 286, 968–971 66 Traulsen, A. et al. (2005) Coevolutionary dynamics: from finite to infinite populations. Phys. Rev. Lett. 95, 238701 Trends in Ecology and Evolution June 2012, Vol. 27, No. 6 67 Reichenbach, T. et al. (2007) Mobility promotes and jeopardizes biodiversity in rock–paper–scissors games. Nature 448, 1046–1049 68 Altrock, P.M. et al. (2010) Stochastic slowdown in evolutionary processes. Phys. Rev. E 82, 011925 69 Thomas, P. et al. (2010) Stochastic theory of large-scale enzymereaction networks: finite copy number corrections to rate equation models. J. Chem. Phys. 133, 195101 70 Assaf, M. et al. (2011) Determining the stability of genetic switches: explicitly accounting for mRNA noise. Phys. Rev. Lett. 106, 248102 71 Waclaw, B. et al. (2011) Dynamical phase transition in a model for evolution with migration. Phys. Rev. Lett. 105, 268101 72 Lloyd, A.L. (2001) Destabilization of epidemic models with the inclusion of realistic distributions of infectious periods. Proc. R. Soc. Lond. B 268, 985–993 73 Ross, J.V. et al. (2009) On parameter estimation in population models II: multi-dimensional processes and transient dynamics. Theor. Popul. Biol. 75, 123–132 74 Powell, C.R. and Boland, R.P. (2009) The effects of stochastic population dynamics on food web structure. J. Theor. Biol. 257, 170–180 75 Murase, Y. et al. (2010) Effects of demographic stochasticity on biological community assembly on evolutionary time scales. Phys. Rev. E 81, 041908 76 Galla, T. (2010) Independence and interdependence in the nest-site choice by honeybee swarms: agent-based models, analytical approaches and pattern formation. J. Theor. Biol. 262, 186–196 77 Spence, L.A. et al. (2011) Disturbance affects short-term facilitation, but not long-term saturation, of exotic plant invasion in New Zealand forest. Proc. R. Soc. B 278, 1457–1466 78 Levins, R. (1969) Some demographic and genetic consequences of environmental heterogeneity for biological control. Bull. Entomol. Soc. Am. 15, 227–240 79 Grimm, V. et al. (2005) Pattern-oriented modeling of agent-based complex systems: lessons from ecology. Science 310, 987–991 80 Ferguson, N.M. et al. (2005) Strategies for containing an emerging influenza pandemic in Southeast Asia. Nature 437, 209–214 81 Hubbell, S.P. (2001) The Unified Neutral Theory of Biodiversity and Biogeography, Princeton University Press 82 Moran, P.A.P. (1962) The Statistical Processes of Evolutionary Theory, Oxford University Press 83 Alonso, D. et al. (2006) The merits of neutral theory. Trends Ecol. Evol. 21, 451–457 84 Rosindell, J. et al. (2011) The unified neutral theory of biodiversity and biogeography at age ten. Trends Ecol. Evol. 26, 340–348 85 Warren, P.B. (2010) Biodiversity on island chains: neutral model simulations. Phys. Rev. E 82, 051922 86 Economo, E.P. and Keitt, T.H. (2008) Species diversity in neutral metacommunities: a network approach. Ecol. Lett. 11, 52–62 87 Hanski, I. (1999) Metapopulation Ecology, Oxford University Press 88 Ross, J.V. et al. (2008) Metapopulation persistence in a dynamic landscape: more habitat or better stewardship? Ecol. Appl. 18, 590–598 89 Levin, S.A. and Segel, L.A. (1976) Hypothesis for origin of planktonic patchiness. Nature 259, 659 90 Waddell, J.N. et al. (2010) Demographic stochasticity versus spatial variation in the competition between fast and slow dispersers. Theor. Popul. Biol. 77, 279–286 345