Download Stochastic formulation of ecological models and their applications

Review
Stochastic formulation of ecological
models and their applications
Andrew J. Black1 and Alan J. McKane2
1
2
School of Mathematical Sciences, The University of Adelaide, Adelaide, SA 5005, Australia
Theoretical Physics Division, School of Physics and Astronomy, University of Manchester, Manchester, M13 9PL, UK
The increasing use of computer simulation by theoretical ecologists started a move away from models formulated at the population level towards individual-based
models. However, many of the models studied at the
individual level are not analysed mathematically and
remain defined in terms of a computer algorithm. This is
not surprising, given that they are intrinsically stochastic and require tools and techniques for their study that
may be unfamiliar to ecologists. Here, we argue that
the construction of ecological models at the individual
level and their subsequent analysis is, in many cases,
straightforward and leads to important insights. We
discuss recent work that highlights the importance of
stochastic effects for parameter ranges and systems
where it was previously thought that such effects would
be negligible.
Why should ecological models be individual based and
stochastic?
Ecology was the first area of biology where quantitative
models were constructed, and it is still one of the most
mathematically well developed of the biological sciences.
Early models were formulated at the population level,
frequently as differential equations, because these were
the only tools available at the time (Box 1). The advent of
electronic computers led to a huge increase in the variety
and complexity of models that can be studied, most naturally formulated at the individual rather than the population level. This is clearly a better starting point for
building ecological models as it explicitly recognises the
discreteness of the population and the stochastic nature of
the dynamics. As such, these types of model can capture a
larger range of phenomena than can basic populationlevel models (PLMs; see Glossary). The main drawback
of such an approach is that the results are frequently
numerical in nature, making any theoretical understanding difficult.
In this review, we highlight the recent trend whereby
ecological models are formulated as stochastic Markov
processes so they can be studied analytically as well as
numerically [1–5]. We refer to these as individual-based
models (IBMs). This is to be contrasted with what we term
‘agent-based models’, which although based on individuals,
are often defined in terms of algorithms, thus precluding
any analysis. These two approaches, along with PLMs, are
summarised in Box 1.
Glossary
Discretise: the procedure whereby one replaces a continuous variable, such as
the fraction of the population that consists of a particular type in a PLM, by a
discrete variable, such as the number of individuals of that type.
Individual-based model (IBM) or individual-level model (ILM): a model in which
the basic entities are individuals. This description in terms of discrete variables
is in contrast to PLMs, where the description is in terms of continuous
population densities. The former are sometimes referred to as a microscopic
description and the latter as a macroscopic description.
Limit cycle: a periodic cycling in the macroscopic dynamics. Limit cycles are
deterministic in that they remain in phase, as contrasted with quasi-cycles,
which are phase forgetting.
Macroscopic: description of a system on a large scale, so that the detailed
(microscopic) structure is not observable. In ecology, this will typically be a
description at the population level involving population densities, rather than
individuals.
Markov process: a stochastic process where the probability of making a
transition to a new state only depends on the current state of the system, and
not on its past history.
Master equation: the equation for a continuous-time Markov process that
specifies how the probability of the system being in a given state changes with
time. More specifically, it gives an expression for dPðn;tÞ
in terms of transitions
dt
into the state n minus transitions out of this state.
Microscopic: description of a system at a small scale in terms of the entities
making up the system. In ecology, this will typically be a description at the
individual level. The processes via which the population of individuals evolves
will then be probabilistic.
Phenomenological: refers to the process of constructing a model. Such a
model does not have a microscopic basis, but is simply postulated in a
consistent manner using observed phenomena and general principles.
Power spectrum (or power spectral density): a real positive function of the
frequencies that appear in a time series. If all frequencies are equally
represented (white noise) then it is a constant (i.e. flat), whereas if the signal
is sinusoidal (i.e. only one frequency, v0, is present) then it is a spike at v0. For a
stochastic or noisy sinusoidal signal, where the typical frequency is v0, but
where there is a range of frequencies within a band v0 Dv, the power
spectrum will have a peak at v0 and a width of the order of Dv.
Quasi-cycles: a stochastic phenomenon where demographic noise resulting
from the individual nature of the population excites the system, giving rise to
noisy cycling in the dynamics. In general, the power spectrum of these will be
peaked with a finite width, indicating a range of excited frequencies.
Resonance: the tendency of a system to oscillate at greater amplitude at some
frequencies rather than at others. If the system is subject to external forces at
these frequencies, large amplitude oscillations will be produced.
Stability matrix: a matrix of partial derivatives (the Jacobian) evaluated at a
fixed point of a set of differential equations. If all its eigenvalues have a
negative real part, then the fixed point is stable.
State variables: for an IBM, these are the number of different types of
individual at a given time: ni individuals of type i, where i = 1,2,. . .. This is
frequently written as a vector n ¼ ðn1 ; n2 ; . . .Þ. The i could label species,
locations in space, age, or any other trait or attribute. For a PLM, the state
variables are continuous and denoted as xi, with the entire state written as
x ¼ ðx 1 ; x 2 ; . . .Þ.
Stochastic process: a process where the rule for making a transition to a new
state of the system at time t+1 (or t+dt, if time is continuous) from the current
state at time t is a stochastic (i.e. random) variable. So, unlike a deterministic
process, only the probability of being in a given state n at time t can be
specified. This is denoted by P(n,t).
Stochastic simulation algorithm (SSA) or the Gillespie algorithm: a Monte
Carlo scheme for the simulation of a stochastic Markov chain. This scheme is
exact; the algorithm is derived from the same assumptions as the master
equation.
Corresponding author: McKane, A.J. ([email protected]).
0169-5347/$ – see front matter ß 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.tree.2012.01.014 Trends in Ecology and Evolution, June 2012, Vol. 27, No. 6
337
Review
Trends in Ecology and Evolution June 2012, Vol. 27, No. 6
White noise: the term ‘noise’ refers to a stochastic (random) process and
‘white’ to one where all frequencies are equally represented. It draws its name
from the idea that colours of different frequencies are equally represented in
white light.
We first review what constitutes an IBM; we look at the
construction of IBMs and discuss setting up the (stochastic)
dynamics of the system. This is illustrated using several
different types of ecological model. The fundamental equation governing the dynamics of the stochastic process is the
so-called ‘master equation’, which is simply a continuous
time version of a Markov chain. However, when the IBM is
viewed in this way, powerful techniques can be used to
analyse the master equation, leading to quantitative predictions for several interesting ecological problems.
We emphasise the importance of deriving PLMs from
IBMs, and discuss a variety of examples drawn from the
literature. We show how the concrete predictions that
Box 1. Three approaches to ecological modelling
Population-level models
PLMs are typically formulated as ordinary differential equations for
the fraction of the population that is of a certain kind. For example,
Levins’ original metapopulation model is written as Equation I [78]:
dQ
¼ cQð1 QÞ eQ;
dt
[I]
where Q is the fraction of patches that are occupied and c and e are
effective parameters. This is a phenomenological equation; that is,
written down consistently, but not derived from a more basic model.
The variables, such as Q, are continuous, implicitly assuming an
infinite population size, and thus the results are deterministic (nonrandom). The advantage of this approach is that the well-developed
mathematics of dynamical systems is applicable. This is still considered the dominant methodology, with many textbooks available
[40,47]. If one starts with this approach, then the question of how
to include stochastic effects arises. Some authors discretise the
equations whereas others add stochastic (noise) terms on to the
equations. Both of these approaches are ill defined.
Agent-based models
Agent-based models have a finite number of individuals in the
system, each typically having many attributes or degrees of freedom
[79,80]. These are frequently defined in terms of computer
algorithms ready for simulation. The advantages of models of this
type are the potentially unlimited amounts of detail that can be
included and the ease of programming. The main disadvantage is
that the algorithmic approach can only produce numerical results; in
general, they are too complex for mathematical analysis.
can be obtained from IBMs using these tools can elucidate well-known, but poorly characterised, phenomena.
There are many empirical studies that have attributed
their findings to stochastic effects [6–11], making it an
important long-term goal to develop a sound theoretical
understanding of such effects. One of the major points to
emerge from this line of research is the importance of
stochastic effects in large populations, not just small
ones, as has been traditionally assumed in the study of
extinction effects.
Creating stochastic models
Beginning with IBMs, and deducing PLMs from them, is
the approach taken in many of the physical sciences. In
chemistry, for example, reaction kinetics at the molecular
level yield rate equations at the macroscale and, in physics,
statistical mechanical models at the microscale lead to a
thermodynamical description at the macroscale. In ecological IBMs, random events (e.g. birth, death and predation)
at the level of individuals (the microscale) give rise to
macroscopic dynamics for large populations of individuals.
In the limit of an infinite population, these dynamics
become deterministic and can be described by a set of
ordinary differential equations: a PLM. As we discuss
later, when the population is large but still finite, these
dynamics are often still strongly stochastic. Before we do
this, we first discuss the construction of IBMs and some
methods of analysis.
Suppose that one wishes to describe an ecological system that contains m different species, and the only way
that individuals will be labelled is as belonging to a given
species. If the population size is assumed to be finite, equal
to N, then the state of the system is given by a vector of
integers n ¼ ðn1 ; . . . ; nm Þ, where ni is the number of individuals of species i at that time. This could also be the
number of individuals of species i per unit area, or some
other measure.
Given that the basic idea of model specification can be
illustrated on a system with just one species, we focus on a
system with n identical individuals of type A. Suppose also
that the only processes are death and (asexual) birth. We
represent these as Equations 1 and 2, respectively:
d
A!E;
b
A þ E!A þ A
Individual-based models
IBMs are also composed of a finite number of individuals, but
typically with a smaller number of attributes. The rules for the time
evolution of the individuals within the system are formulated
probabilistically, usually as a Markov process. Therefore, state
variables are discrete instead of continuous and, by definition, the
model is stochastic. In an IBM version of the metapopulation model
(Equation I), n would be the number of patches and the dynamics of
colonisation and extinction of patches are random events that
depend on n (explicit details of this model are given in Box 2).
The population is usually divided up into a small number of
classes, reducing the degrees of freedom; thus, all members of the
same class are indistinguishable. The probabilistic formulation of
IBMs means they can be simulated exactly with Monte Carlo
techniques [12], but most importantly they can be analysed
analytically via the master equation. The corresponding PLM to a
given IBM can also be analytically derived.
338
[1]
[2]
Equation 1 indicates that an individual of type A dies at
a rate d to give a vacancy (E for ‘empty’); Equation 2 shows
that if there is a vacancy, another individual of the same
type is born at a rate b. The rate at which the number of
individuals reduces from n to (n–1) is then T(n 1|n) = dn
and the rate at which it increases from n to (n+1) is shown
in Equation 3:
n
[3]
Tðn þ 1jnÞ ¼ bn 1 N
This last result assumes a well-mixed population and is
proportional to the probability of an individual and a
vacancy existing in the system. These are the fundamental
ingredients that underpin the dynamics. We should
Review
emphasise that these rates depend on N and thus so do the
dynamics of the model. This is not true of a PLM. Several
examples of IBMs of increasing complexity are given in
Box 2.
Analysis of stochastic models
Once a model has been specified, the complete stochastic
dynamics are encoded analytically by the master equation, or individual realisations can be obtained by Monte
Carlo simulation, which outputs a stochastic time series.
These are the two main ways of investigating and solving
these models (i.e. obtaining a complete description of the
dynamics).
The stochastic simulation algorithm (SSA) developed by
Gillespie provides a method for the exact simulation of a
master equation [12]. This type of simulation is the usual
way of investigating these models, mainly because of its
simplicity to program, but an important point is that the
SSA and master equation are both derived from the same
underlying Markovian assumptions, so that there is an
exact correspondence between the two. Thus, any model
that can be simulated in this way can be written as a
master equation and vice versa. Of course, there are the
usual drawbacks to just using simulations to investigate a
model, such as the costs in terms of computing power and
time, the numerical nature of the results and the difficulty
in investigating rare events [13,14].
In general, simulations provide individual realisations
of a stochastic process. The master equation (also known as
Kolmogorov’s forward equation) describes analytically the
temporal evolution of the probability density of being in a
particular state n, thus encoding the full stochastic dynamics [12,15]. If it could be solved (for the probability
density), then one would have a complete description of the
properties and the dynamics of the stochastic system
[4,16]. One calculation that is straightforward and can
always be carried out is to determine the macroscopic
behaviour from the master equation, which is the
N ! 1 limit, as discussed in Box 3, and which yields
the PLM corresponding to the original IBM.
The master equation cannot be solved analytically for
most cases of interest, and so several approximation methods have been developed [17]. These can be extended if the
system involves a network [18–21]. If the population and
the number of degrees of freedom are small, then it can be
solved numerically [22]. Epidemic models with household
structure are another interesting example where a master
equation is used to describe the spread of infection within a
household, but situated within an infinite population of
such households, which allows the derivation of several
population-level results [23,24]. Other ecological systems
have this kind of hierarchical structure (e.g. involving
collections of local populations) for which similar techniques might be applicable.
We cannot discuss all these methods in this review, so we
concentrate on techniques that set up an expansion of the
, where N is, for
master equation in increasing powers of p1ffiffiffi
N
example, the total population. If N is large, which is typical
in ecological systems, then solutions of the master equation
can be approximated by just two terms, which correspond
to a deterministic part plus a stochastic correction that
Trends in Ecology and Evolution June 2012, Vol. 27, No. 6
describes the effects of demographic stochasticity. This
approximation is well known in the literature, and goes
under several names: the van Kampen system size expansion [15], the Gaussian or diffusion approximation, or
Kurtz’s theorem [25,26]. The method is discussed in more
detail in Box 3, but essentially consists of approximating
the probability density in the master equation with a
Gaussian distribution. The power of this technique is that,
starting from the master equation, it provides a very
general method of deriving equations describing the macroscopic stochastic dynamics.
Population level descriptions follow from individual
level descriptions
In an individual-based formulation, it is always the case
that a finite population of N individuals is dealt with,
which evolves via several random processes (e.g. birth,
death and predation). For small N, the dynamics will be
random. When the population becomes large, the dynamics
will start to become deterministic (non-random). One way
of investigating this behaviour is just to simulate large
systems and average the results. Far more insight can be
found by analytically deriving the corresponding PLM (a
set of differential equations) from the IBM in the limit of
infinite N (Boxes 1 and 3).
This methodology clearly has several advantages. First,
the predictions of the IBM and PLM can be compared
exactly; thus, it is possible to see under what conditions
a PLM will be a valid description of a given system. It is
also helpful in writing down a PLM when the ecology of the
system is complicated [27]. In general, it is easier to
consider the events that define a system, write down an
IBM and then derive the corresponding PLM. Often these
derived equations may differ from the naively postulated
equations in a simple population-level approach [28–31].
This also has the advantage that the explicit microscopic
basis for the PLM is known and so more than just a
phenomenological analysis is carried out.
Unfortunately, this procedure is often reversed: a wellstudied PLM is discretised, interpreted as an IBM and
then simulated to investigate various stochastic effects.
This is exactly the wrong way round: in general, there will
be many IBMs (sometimes infinitely many) giving a particular PLM, and so the PLM should always be derived
from the IBM [31,32]. The reason why many IBMs give the
same PLM is that IBMs define a stochastic process and
PLMs are equations for the averages of this process; however, many stochastic processes have the same averages.
Applications of IBMs described by master equations
The modelling approach we have described has been applied to several problems over the past few years, and there
is a clear trend towards the increasing use of this methodology. For example, in the case of the neutral model discussed in Box 2, the quantity that has been studied most
intensively is the species abundance distribution. This is a
plot of the number of species with 1,2,. . .,m individuals in a
given local community. It can be calculated exactly in the
classic model starting from the master equation [33], and
fits well with empirical data [34–38]. Rather than discuss
more specific examples, we review three broad aspects.
339
Review
Trends in Ecology and Evolution June 2012, Vol. 27, No. 6
Box 2. Examples of ecological IBMs
An advantage of IBMs is that specifying them is simple and they are
straightforward to motivate biologically, with additional features
being easily incorporated. Here, we give several examples, three of
which are illustrated in Figure I.
Table I. Neutral model.
Event
Birth–death
Transition
1m
A þ B! A þ A,
1m
A þ B! B þ B;
Neutral models
The ‘classic’ neutral theory, introduced by Hubbell [81], consists of a
local community with birth–death processes of the Moran type [82]
and immigration from a metacommunity, which acts as a well-mixed
source pool of potential immigrants [83,84]. Given that all individuals
are assumed to have the same birth and death rates, we may fix our
attention on one species, A, and denote all other species as B. Then
the transitions are as given in Table I.
Here, the rates do not include the combinatoric factor that gives the
probability of choosing individuals of the desired species to interact.
The quantity m is the probability that the replacement of an individual
is the result of an immigration event, rather than by a birth–death
event. The probability of a particular immigrant being chosen is given
by the relative abundance of that particular species in the metacommunity, which can also be described as an IBM with birth–death and
speciation [83]. This model is only implicitly spatial, but it can be
extended in many ways; for example, by making it explicitly spatial
[84], incorporating a chain of islands [85] or a network structure [86].
Metapopulation models
Metapopulations are sets of fragmented local populations connected
by migration [87]. At the simplest level of description, the patches can
be thought of as being either occupied or unoccupied [2,16]. If A
labels the occupied patches and E the unoccupied ones, then the
possible events are as detailed in Table II.
This leads to a simple one-dimensional master equation, where the
state variable is the number of occupied patches. The same transition
scheme applies to many models, such as logistic growth and the SIS
model [17]. Again, this simple model has been elaborated on; for
instance, by making the number of colonisable patches into a
dynamic variable [2,88].
Spatial predator–prey model
An example of a reaction–diffusion equation showing Turing patterns
can be found in the description of plankton–herbivore dynamics given
(a)
m
Immigration
Table II. Metapopulation model.
Event
Colonisation
Transition
Extinction
A!E,
Migration
m
c
A þ E!A þ A;
e
E!A:
Table III. Predator-prey model.
Event
Linear growth
Extinction
Transition
b
P!P þ P;
e
Allee effect
P þ P!P þ P þ P;
p1
Predation 1
P þ H!H;
Predation 2
P þ H!H þ H;
Competition
H þ H!H:
p2
d
by Levin and Segal [89]. An IBM that gives this model in the
deterministic limit is given in [5]; within a patch, there are five
processes between the plankton, P, and the herbivores, H (Table III).
The model can simply be taken to describe the dynamics on a large
well-mixed patch, but it can be made spatial by allowing both the P
and H to jump between patches on a regular lattice. The introduction
of spatial variation introduces no new points of principle in the
modelling procedure, although the analysis typically becomes more
complex [34,90].
(c)
Colonisation
m
A!B, B!A:
p
Competition
Predation
redation 1
Allee effect
Predation 2
Growth of P
Immigration
N patches
Reproduction
(b)
Homogeneous patch
Immigration
Migration
Migration
Lattice of patches
TRENDS in Ecology & Evolution
Figure I. Three examples of ecological individual-based models (IBMs). (a) A simple metapopulation model. (b) A neutral island chain [85]. (c) The Levin–Segal model
[5,89], which describes the explicitly spatial plankton–herbivore dynamics.
340
Review
Trends in Ecology and Evolution June 2012, Vol. 27, No. 6
Box 3. The master equation and the emergence of macroscopic dynamics
The transition rates T(n + 1jn) and T(n - 1jn) define the model. From
these, the probability of finding n individuals in the system at time t,
P(n,t) can be calculated from Equation I:
replace the spike with a Gaussian. The system would now be
stochastic, being described by a probability distribution function,
although of the simplest kind. The width of the Gaussian would be
expected to scale as p1ffiffiffi
which means effectively writing Nn ¼ x þ pjffiffiffi
,
N
N
where x ¼ hNni and j is a stochastic variable. This is the content of the
van Kampen system-size expansion. It turns out to be an excellent
approximation (far better than might naively be expected) away from
boundaries. Furthermore, given that Gaussian probability distributions are derived from linear stochastic differential equations, the
whole theory is linear and so can be treated exactly, even though the
underlying stochastic processes may be highly nonlinear.
For most interesting applications, more than one degree of freedom
is needed, but the whole theory generalises in this case to a set of
stochastic differential equations for ji : (Equation II)
dPðn; tÞ
¼ T ðnjn þ 1ÞPðn þ 1; tÞ þ T ðnjn 1ÞPðn 1; tÞ
dt
½T ðn 1jnÞ þ T ðn þ 1jnÞPðn; tÞ:
[I]
This is the equation that governs the dynamics of the system. It is the
equation for a Markov chain in continuous time, and is called the
master equation by physical scientists. Although we have only
discussed the case of a single species, all the formalism naturally
generalises. We can replace n by n everywhere in the master equation
where n is a vector with components that can represent any set of
species in a large collection of spatial patches and having any number
of other attributes.
The master equation contains far more information than the usual
differential equations written down phenomenologically in terms of
the population density. The latter can be found by simply taking the
average of Nn ; all the detail of the stochastic fluctuations is lost. Doing
this for the metapopulation model described in Box 2 gives the wellknown logistic equation [40].
In the deterministic description, the probability distribution, P(n,t),
is a spike at n ¼ hni. The simplest approximation beyond this is to
dji X
¼
ai j j j ðtÞ þ hi ðtÞ;
dt
j
[II]
where hi(t) is a Gaussian white noise with zero mean, which is the only
remnant of the demographic stochasticity of the system. It has a
correlation function given by a matrix bij. So the whole theory is given
by two matrices aij and bij, which can be systematically calculated from
the transition rates that define the model. Figure I illustrates this theory
using the metapopulation model described in Box 2.
(a) 100
(b)
0.20
Patches
80
P
60
0.15
0.10
40
0.05
20
0
0.00
0
5
10
15
100
5
20
Time
Time
50 Patches
10
Patches
(c)
0
(d) 0.06
1400
1200
1000
800
600
400
200
0
0
0.04
P
0.02
0.00
5
10
15
Time
20
1500
5
1000
Time
10 0
500
Patches
TRENDS in Ecology & Evolution
Figure I. Emergence of macroscopic behaviour. The number of occupied patches for the metapopulation model for (a) N = 100, and (c) N = 1500 carrying capacity. The
broken red lines show the deterministic result and the blue lines are from stochastic simulations. (b) and (d) show the evolution of the probability density for N = 100 and
N = 1500, respectively. The peak of the distributions move with the deterministic result and the width is given by the solution of the stochastic differential equation
1
(Equation II). As the system gets larger, the amplitude of the fluctuations decreases as N 2 . In the limit N!1, the probability density becomes a spike.
Quasi-cycles
A common feature of some IBMs (with two or more degrees
of freedom) is the ability of the demographic stochasticity
to excite macroscopic-scale coherent oscillations, known as
quasi-cycles [1,39]. An example of these is shown in
Figure 1. A prerequisite for quasi-cycles to occur in a given
IBM is that there is a stable fixed point in the corresponding PLM that is approached in an oscillatory manner.
Expressed mathematically, this means that the stability
matrix at this fixed point must have at least one pair of
complex eigenvalues. However, although the oscillations in
the deterministic system are damped and so die away, in
the full (stochastic) system the demographic stochasticity
acts as a forcing term and sustains the oscillations.
From a mathematical point of view, this effect is described by Equation II of Box 3. A physical analogy is a
pendulum that is lightly damped, so eventually any oscillations will die away. However, if it is randomly bombarded
341
Review
Trends in Ecology and Evolution June 2012, Vol. 27, No. 6
(b) 120
100
1500
80
Power
Infectives
(a) 2000
1000
60
40
500
20
0
0
5
10
15
0
0.0
20
Time (years)
0.2
0.4
0.6
0.8
1.0
Frequency (1/years)
TRENDS in Ecology & Evolution
Figure 1. An example of quasi-cycles in a stochastic model. The blue and purple lines in (a) show two realisations of a stochastic individual-based model (IBM) [the
susceptible–infectious–recovered (SIR) model [3]] that are started from the same initial condition. The broken red line shows the result using the corresponding populationlevel model (PLM). Whereas the oscillations are damped, tending to a stable fixed point in the population-level version, they are maintained in the stochastic IBM. These
stochastic oscillations are different from limit cycles in that they are phase forgetting. This can be seen in the two realisations that, although started from the same initial
condition, soon go out of phase. This is reflected in the power spectrum of the time series, shown in (b), which has a finite width. All frequencies shown in the spectrum are
excited in the time-series, but the position of the peak indicates the dominant frequencies that are amplified the most.
by weak white noise, these oscillations will be sustained. In
addition, the amplitude of these oscillations will be much
bigger than might naively be expected, because in among
all the frequencies contained in the white noise will be one
that corresponds to the natural frequency of the pendulum.
This resonance effect has been called ‘stochastic amplification’. Quasi-cycles differ from limit cycles in that they are a
stochastic phenomenon and provide a simple robust mechanism that generates cyclic behaviour without the need for
additional complexities, as are often assumed, to generate
cyclic behaviour in deterministic PLMs [40].
Figure 1 shows two realisations of a stochastic model
and it is clearly seen that, after some time, these go out of
phase, unlike true limit cycles. One of the most powerful
tools for analysing such oscillations is the power spectrum
[10,41]. This shows how the different frequencies that
make up the time-series are distributed. Quasi-cycles
are characterised by a strongly peaked power spectrum,
which has a finite width. The width is important as it
shows how coherent the cycles are: the smaller the width,
the longer cycles remain in phase. The definition and
meaning of the power spectrum are discussed in Box 4.
An analytic approach using the master equation is
particularly useful for the characterisation of these cycles.
From the stochastic differential equations derived using
the system-size expansion, one can easily derive an analytic expression for the power spectrum, again described in
more mathematical detail in Box 4. This allows the easy
investigation of this phenomenon throughout a wide parameter range as well as a vital check against the results of
simulations. Other related questions that have been
addressed are: what is the connection of quasi-cycles with
limit cycles, which are the conventional way of describing
oscillations in nonlinear dynamical systems [3,42]?; and
are there characteristic signatures of the two types of
oscillation that can be used to tell them apart in ecological
time series [43]?
Spatial patterns
In 1952, Turing predicted that systems that consisted of
agents of at least two different species that reacted together
and spatially diffused could, under certain circumstances,
give rise to patterns, now known as Turing patterns [44].
Field observations are now providing evidence that Turing
342
patterns are found in ecosystems [45,46]. This would be
expected on purely theoretical grounds, given that spatial
ecosystem dynamics is frequently described by reaction–
diffusion equations [47]. However, just as demographic
stochasticity leads to stochastic oscillations where a deterministic analysis finds only static behaviour, it also leads to
Box 4. The power spectrum
The stochastic cycles found in the IBMs discussed in this review do
not have a single period, but a distribution of periods centred about
an average value. This, together with the fluctuations in the
amplitude of the cycles, means that when the time series from a
large number of realisations, ji(t), are averaged over, they average
out to zero. Yet, a single realisation corresponds to what will be seen
in an experiment, and so some averaging mechanism is needed that
will not wipe out the cycles. The simplest way to achieve this is to
take the Fourier transform of the time series [41] (Equation I):
j̃i ðvÞ ¼
Z
þ1
ji ðtÞe ivt dt
[I]
1
The frequencies centred around that corresponding to the average
period will have the largest magnitude, and so taking theDmodulus
2 E
squared of the Fourier transform and then averaging: j̃i ðvÞ ;
should give a smooth function spread about the characteristic
frequency of the system. This is the power spectrum of the
fluctuations. The spread of the power spectrum will show how
coherent the fluctuations are. If they were deterministic and
sinusoidal, the power spectrum would be a spike.
Although some ecologists were well aware of the existence of
sustained oscillations, the effect has not been studied within a welldefined mathematical scheme; neither were the power spectra
analysed carefully to learn more about the nature of the fluctuations.
A range of phenomena have been revealed through more detailed
recent studies. First, the height of the peaks of the power spectra
(reflecting the amplitude of the oscillations) are much larger than
might be expected owing to a resonance effect: all frequencies are
present in the white noise h(t) and this resonates with the
characteristic frequency of the system. Second, this characteristic
frequency is not the same as the frequency of the decay of
perturbations to the deterministic system [1]. The latter is given by
jImlj; where l is a complex eigenvalue of the Jacobian of
fluctuations about the stationary state. Instead, the peak of the
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
ðIm lÞ2 ðRe lÞ2 , and the
power spectrum is approximately at
precise value also depends on the bij. Third, it was not clear how the
stochastic cycles related to conventional (deterministic) limit cycles,
but this has now been understood [42]. Finally, the mathematical
systematisation has led to the effect being found in other areas; for
example, stochastic Turing patterns [5,51].
Review
Trends in Ecology and Evolution June 2012, Vol. 27, No. 6
(a)
(b)
700
700
600
600
500
500
400
400
300
300
88
80
72
56
48
Population
64
40
200
200
32
100
100
24
0
0
100
200
300
400
500
600
700
0
16
0
100
200
300
400
500
600
700
TRENDS in Ecology & Evolution
Figure 2. Quasi-pattern formation in the stochastic Levin–Segal model as described in Box 2. (a) shows a population map of the model in the quasi-pattern phase. (b) shows
the same data, but randomised. If the patchiness were a statistical artefact, then it would be preserved in the randomised version. Reproduced, with permission, from [5].
stochastic Turing patterns in spatial systems, where reaction–diffusion equations would indicate that none should
exist. This has been shown in a variety of systems [5,48–50];
moreover, the range of parameters for which stochastic
Turing patterns exist is usually much larger than for conventional Turing patterns, which have restrictions such as
the diffusion constants of the two species being at least an
order of magnitude different to each other. An example of
this phenomenon is shown in Figure 2.
The mechanism that gives rise to stochastic Turing
patterns is essentially the same as that which gives rise
to the stochastic oscillations discussed earlier. Now the
model is spatial, and so the noise is a function of time and
space, or of frequency, v, and wavenumber k, in the Fourier
representation. The noise is still white, so all frequencies
and wavenumbers are excited. We can now repeat the
argument given above for the origin of stochastic oscillations, replacing v with k. If there is a peak in the power
spectrum at a non-zero k, say kmax, then there is a natural
wavenumber of the system that has been excited by the
noise and is now sustained rather than decaying. In the
conventional Turing description, this decaying mode would
signify that the homogeneous state was stable to perturbations at k = kmax. The noise produces sustained oscillations
(in space), which gives rise to spatial patterns with a
characteristic wavenumber kmax. It is probable that at
least some of the Turing patterns observed in nature are
of this type [5,34,51].
Recurrent epidemics
IBMs have a long history in epidemiological modelling,
beginning with Bartlett’s pioneering work [52]. Many
epidemic models are most intuitively formulated on the
individual level and master equation methods have been
a popular tool to investigate static properties [53,54]. The
dynamics of childhood diseases, such as measles, whooping cough and rubella [55–57], provide particularly interesting case studies because they are subject to both
external forcing [58], owing to the aggregation of children
in schools, and demographic stochasticity. The resulting
range of dynamics observed in both time-series case
report data and simple models is rich and understanding
the mechanistic basis for it is still an active research topic
[59].
There are several potential mechanisms that could give
rise to the observed dynamics. One of the most fascinating
questions is whether these mechanisms are deterministic
in origin, for example where noise causes the system to
switch between deterministic states, which could be understood using just a deterministic model [60]. This is in
contrast to the situation where noise plays an active role
that cannot be explained with a deterministic model [61–
63]. As typical of the whole field, older studies have tended
to use an ad hoc collection of methods to try and ‘disentangle’ stochastic and deterministic elements [9], with a reliance on simulations, leading to much debate and confusion
over the supposed role of stochasticity [62]. Newer studies
using an IBM approach have started to provide a more
systematic account of these models and their dynamics,
where the macroscopic dynamics, including deterministic
elements, is instead viewed as emergent from the microscopic dynamics.
In a previous subsection, we discussed the phenomenon
of quasi-cycles about a deterministic fixed point. In models
of recurrent epidemics, there is a strong forcing on the
system, which in turn leads to a limit cycle in the deterministic dynamics, possibly with multiple stable attractors
[60,63]. The previous analysis can be carried over, allowing
one to derive the power spectrum of the forced model [3,42],
which again affords a view of the stochastic dynamics
ideally suited for understanding the underlying processes.
In these models, the noise excites the transient dynamics
about the limit cycle, so the resulting macroscopic time
series are a superposition of these and the deterministic
limit cycle. The relative proportions of the two periodicities
reflected in the power spectrum then depend on the stability of the limit cycle and the size of the system. Such an
analysis can encompass and explain the differing periodicities of whooping cough and measles before and after mass
vaccination [3,60,64,65].
343
Review
Concluding remarks
Our aim in this article was to highlight the trend towards
models in ecology being individual based. There are several
advantages in using stochastic models of this kind: they are
easier to construct than are PLMs; they are more intuitive;
and they predict phenomena that deterministic models
miss. The importance of stochastic effects has long been
known, going back to the earliest Monte Carlo simulations
of epidemic spreading and fade-out [52], but computational
investigations have outpaced the theoretical understanding of such models. Approaches based on the master equation are starting to remedy this and we have given here an
overview of the philosophy and methodology of these. We
expect that future research will continue to develop techniques based on this starting point. Several other areas are
already using these techniques successfully; for instance,
coevolutionary dynamics [66–68], cellular biology and genetics [69–71]. The construction of biological models is now
driving the use of new techniques, such as incorporating
non-Markov processes into the formalism [64,72] and parameter estimation, which follows elegantly on from the
master equation and Gaussian approximation [73].
We expect that the future trend will be towards whole
classes of model that interpolate between these different
traditions of model building. Thus, agent-based models
would be simplified by construction of a chain of related
models that would terminate with a relatively simple IBM,
which in turn could be studied analytically and its deterministic limit compared with the differential equations
that are generally postulated phenomenologically. The
range of models continues to grow and diversify. Some
are now becoming significantly more complicated and are
beginning to bridge the gap with agent-based models [74–
77]. This would pave the way to a more unified and coherent approach to model building in ecology, leading to
greater predictive power.
Acknowledgements
A.J.B. acknowledges support from the EPSRC and the Australian
Research Council’s Discovery Projects funding scheme (project number
DP110102893).
References
1 McKane, A.J. and Newman, T.J. (2005) Predator–prey cycles from
resonant amplification of demographic stochasticity. Phys. Rev. Lett.
94, 218102
2 Ross, J.V. (2006) Stochastic models for mainland–island
metapopulations in static and dynamic landscapes. Bull. Math. Biol.
68, 417–449
3 Black, A.J. and McKane, A.J. (2010) Stochastic amplification in an
epidemic model with seasonal forcing. J. Theor. Biol. 267, 85–94
4 Shahrezaei, V. and Swain, P.S. (2008) Analytical distributions
for stochastic gene expression. Proc. Natl. Acad. Sci. U.S.A. 105,
17256–17261
5 Butler, T. and Goldenfeld, N. (2011) Fluctuation-driven Turing
patterns. Phys. Rev. E 84, 011112
6 Higgins, K. et al. (1997) Stochastic dynamics and deterministic
skeletons: population behavior of Dungeness crab. Science 276,
1431–1435
7 Stenseth, N.C. et al. (1998) Phase- and density-dependent population
dynamics in Norwegian lemmings: interaction between deterministic
and stochastic processes. Proc. R. Soc. B 265, 1957–1968
8 Bjornstad, O.N. and Grenfell, B.T. (2001) Noisy clockwork: time
series analysis of population fluctuations in animals. Science 293,
638–643
344
Trends in Ecology and Evolution June 2012, Vol. 27, No. 6
9 Rohani, P. et al. (2002) The interplay between determinism and
stochasticity in childhood diseases. Am. Nat. 159, 469–481
10 Reuman, D.C. et al. (2006) Power spectra reveal the influence of
stochasticity on nonlinear population dynamics. Proc. Nat. Acad.
Sci. U.S.A. 103, 18860–18865
11 Krkosek, M. et al. (2011) Cycles, stochasticity and density dependence
in pink salmon population dynamics. Proc. R. Soc. B 278, 2060–2068
12 Gillespie, D.T. (1992) Markov Processes: An Introduction for Physical
Scientists, Academic Press
13 Assaf, M. and Meerson, B. (2010) Extinction of metastable stochastic
populations. Phys. Rev. E 81, 021116
14 Ovaskainen, O. and Meerson, B. (2010) Stochastic models of population
extinction. Trends Ecol. Evol. 25, 643–652
15 van Kampen, N.G. (1992) Stochastic Processes in Physics and
Chemistry, Elsevier
16 Alonso, D. and McKane, A.J. (2002) Extinction dynamics in mainlandisland metapopulations: an N-patch stochastic model. Bull. Math. Biol.
64, 913–958
17 Newman, T.J. et al. (2004) Extinction times and moment closure in the
stochastic logistic process. Theor. Popul. Biol. 65, 115–126
18 Simoes, M. et al. (2008) Stochastic fluctuations in epidemics on
networks. J. R. Soc. Interface 5, 555–566
19 Rozhnova, G. and Nunes, A. (2009) Cluster approximations for
infection dynamics on random networks. Phys. Rev. E 80, 051915
20 Sharkey, K.J. (2008) Deterministic epidemiological models at the
individual level. J. Math. Biol. 57, 311–331
21 Dangerfield, C.E. et al. (2009) Integrating stochasticity and network
structure into an epidemic model. J. R. Soc. Interface 6, 761–774
22 Keeling, M.J. and Ross, J.V. (2008) On methods for studying stochastic
disease dynamics. J. R. Soc. Interface 5, 171–181
23 Goshal, G. et al. (2004) SIS epidemics with household structure: the
self-consistent field method. Math. Biosci. 190, 71–85
24 Ross, J.V. et al. (2010) Calculation of disease dynamics in a population
of households. PLoS ONE 5, e9666
25 Kurtz, T. (1978) Strong approximation theorems for density dependent
Markov chains. Stoch. Process Appl. 6, 223–240
26 Baxendale, P.H. and Greenwood, P.E. (2010) Sustained oscillations for
density dependent Markov processes. J. Math. Biol. 63, 433–457
27 Wang, Z. and Goldenfeld, N. (2010) Fixed points and limit cycles in the
population dynamics of lysogenic viruses and their hosts. Phys. Rev. E
82, 011918
28 Goutsias, J. (2007) Classical versus stochastic kinetics modeling of
biochemical reaction systems. Biophys. J. 92, 2350–2365
29 Grima, R. (2009) Noise-induced breakdown of the Michaelis-Menten
equation in steady-state conditions. Phys. Rev. Lett. 102, 218103
30 Campillo, F. et al. (2011) Stochastic modeling of the chemostat. Ecol.
Model. 222, 2676–2689
31 McKane, A.J. and Newman, T.J. (2004) Stochastic models in
population biology and their deterministic analogs. Phys. Rev. E 70,
041902
32 Durrett, R. and Levin, S. (1994) The importance of being discrete (and
spatial). Theor. Popul. Biol. 46, 363–394
33 McKane, A.J. et al. (2004) Analytic solution of Hubbell’s model of local
community dynamics. Theor. Popul. Biol. 65, 67–73
34 Antonovics, J. and Edwards, M. (2011) Spatio-temporal dynamics of
bumblebee nest parasites (Bombus subgenus Psythirus ssp.) and their
hosts (Bombus spp.). J. Anim. Ecol. 80, 999–1011
35 Volkov, I. et al. (2005) Density dependence explains tree species
abundance and diversity in tropical forests. Nature 438, 658–661
36 Muneepeerakul, R. et al. (2008) Neutral metacommunity models
predict fish diversity patterns in Mississippi–Missouri basin. Nature
453, 220–222
37 Rosindell, J. et al. (2010) Protracted speciation revitalizes the neutral
theory of biodiversity. Ecol. Lett. 13, 716–727
38 Vanpeteghem, D. and Haegeman, B. (2010) An analytic approach to
spatio-temporal dynamics of neutral community models. J. Math. Biol.
61, 323–357
39 Kuske, R. et al. (2007) Sustained oscillations via coherence resonance
in SIR. J. Theor. Biol. 245, 459–469
40 Brauer, F. and Castillo-Chavez, C. (2001) Mathematical Models in
Population Biology and Epidemiology, Springer
41 Priestley, M.B. (1982) Spectral Analysis and Time Series, Academic
Press
Review
42 Boland, R.P. et al. (2009) Limit cycles, complex Floquet multipliers and
intrinsic noise. Phys. Rev. E 79, 051131
43 Pineda-Krch, M. et al. (2007) A tale of two cycles – distinguishing quasicycles and limit cycles in finite predator–prey populations. Oikos 116,
53–64
44 Turing, A.M. (1952) The chemical basis of morphogenesis. Philos.
Trans. R. Soc. B 237, 37–72
45 Maron, J.L. and Harrison, S. (1997) Spatial pattern formation in an
insect host-parasitoid system. Science 278, 1619–1621
46 Reitkerk, M. and van de Koppel, J. (2008) Regular pattern formation in
real ecosystems. Trends Ecol. Evol. 23, 169–175
47 Murray, J.D. (2002) Mathematical Biology, II: Spatial Models and
Biomedical Applications, Springer-Verlag
48 Butler, T. and Goldenfeld, N. (2009) Robust ecological pattern
formation induced by demographic noise. Phys. Rev. E 80, 030902
49 Biancalani, T. et al. (2010) Stochastic Turing patterns in the
Brusselator model. Phys. Rev. E 81, 046215
50 Scott, M. et al. (2010) Approximating intrinsic noise in continuous
multispecies models. Proc. R. Soc. A 467, 718–737
51 Woolley, T.E. et al. (2011) Power spectra methods for a stochastic
description of diffusion on deterministically growing domains. Phys.
Rev. E 84, 021915
52 Bartlett, M.S. (1960) Stochastic Population Models in Ecology and
Epidemiology, Methuen
53 Nasell, I. (2002) Stochastic models of some endemic infections. Math.
Biosci. 179, 1–19
54 Stollenwerk, N. and Jansen, V.A.A. (2003) Meningitis, pathogenicity
near criticality: the epidemiology of meningococcal disease as a model
for accidental pathogens. J. Theor. Biol. 222, 347–359
55 Bauch, C.T. and Earn, D.J.D. (2003) Transients and attractors in
epidemics. Proc. R. Soc. Lond. B 270, 1573–1578
56 Keeling, M.J. and Rohani, P. (2007) Modelling Infectious Diseases in
Humans and Animals, Princeton University Press
57 Metcalf, C.J.E. et al. (2011) The epidemiology of rubella in Mexico:
seasonality, stochasticity and regional variation. Epidemiol. Infect.
139, 1029–1038
58 Altizer, S. et al. (2006) Seasonality and the dynamics of infectious
diseases. Ecol. Lett. 9, 467–484
59 Mantilla-Beniers, N.B. et al. (2010) Decreasing stochasticity through
enhanced seasonality in measles epidemics. J. R. Soc. Interface 7,
727–739
60 Earn, D. et al. (2000) A simple model for complex dynamical transitions
in epidemics. Science 287, 667–670
61 Bauch, C.T. and Earn, D.J.D. (2003) Interepidemic intervals in forced
and unforced SEIR models. Fields Inst. Comm. 36, 33–44
62 Coulson, T. et al. (2004) Skeletons, noise and population growth: the
end of an old debate? Trends Ecol. Evol. 19, 359–364
63 Nguyen, H.T.H. and Rohani, P. (2008) Noise, nonlinearity and
seasonality: the epidemics of whooping cough revisited. J. R. Soc.
Interface 5, 403–413
64 Black, A.J. and McKane, A.J. (2010) Stochasticity in staged models of
epidemics: quantifying the dynamics of whooping cough. J. R. Soc.
Interface 7, 1219–1227
65 Rohani, P. et al. (1999) Opposite patterns of synchrony in sympatric
disease metapopulations. Science 286, 968–971
66 Traulsen, A. et al. (2005) Coevolutionary dynamics: from finite to
infinite populations. Phys. Rev. Lett. 95, 238701
Trends in Ecology and Evolution June 2012, Vol. 27, No. 6
67 Reichenbach, T. et al. (2007) Mobility promotes and jeopardizes
biodiversity in rock–paper–scissors games. Nature 448, 1046–1049
68 Altrock, P.M. et al. (2010) Stochastic slowdown in evolutionary
processes. Phys. Rev. E 82, 011925
69 Thomas, P. et al. (2010) Stochastic theory of large-scale enzymereaction networks: finite copy number corrections to rate equation
models. J. Chem. Phys. 133, 195101
70 Assaf, M. et al. (2011) Determining the stability of genetic switches:
explicitly accounting for mRNA noise. Phys. Rev. Lett. 106, 248102
71 Waclaw, B. et al. (2011) Dynamical phase transition in a model for
evolution with migration. Phys. Rev. Lett. 105, 268101
72 Lloyd, A.L. (2001) Destabilization of epidemic models with the
inclusion of realistic distributions of infectious periods. Proc. R. Soc.
Lond. B 268, 985–993
73 Ross, J.V. et al. (2009) On parameter estimation in population models
II: multi-dimensional processes and transient dynamics. Theor. Popul.
Biol. 75, 123–132
74 Powell, C.R. and Boland, R.P. (2009) The effects of stochastic
population dynamics on food web structure. J. Theor. Biol. 257,
170–180
75 Murase, Y. et al. (2010) Effects of demographic stochasticity on
biological community assembly on evolutionary time scales. Phys.
Rev. E 81, 041908
76 Galla, T. (2010) Independence and interdependence in the nest-site
choice by honeybee swarms: agent-based models, analytical
approaches and pattern formation. J. Theor. Biol. 262, 186–196
77 Spence, L.A. et al. (2011) Disturbance affects short-term facilitation,
but not long-term saturation, of exotic plant invasion in New Zealand
forest. Proc. R. Soc. B 278, 1457–1466
78 Levins, R. (1969) Some demographic and genetic consequences of
environmental heterogeneity for biological control. Bull. Entomol.
Soc. Am. 15, 227–240
79 Grimm, V. et al. (2005) Pattern-oriented modeling of agent-based
complex systems: lessons from ecology. Science 310, 987–991
80 Ferguson, N.M. et al. (2005) Strategies for containing an emerging
influenza pandemic in Southeast Asia. Nature 437, 209–214
81 Hubbell, S.P. (2001) The Unified Neutral Theory of Biodiversity and
Biogeography, Princeton University Press
82 Moran, P.A.P. (1962) The Statistical Processes of Evolutionary Theory,
Oxford University Press
83 Alonso, D. et al. (2006) The merits of neutral theory. Trends Ecol. Evol.
21, 451–457
84 Rosindell, J. et al. (2011) The unified neutral theory of biodiversity and
biogeography at age ten. Trends Ecol. Evol. 26, 340–348
85 Warren, P.B. (2010) Biodiversity on island chains: neutral model
simulations. Phys. Rev. E 82, 051922
86 Economo, E.P. and Keitt, T.H. (2008) Species diversity in neutral
metacommunities: a network approach. Ecol. Lett. 11, 52–62
87 Hanski, I. (1999) Metapopulation Ecology, Oxford University Press
88 Ross, J.V. et al. (2008) Metapopulation persistence in a dynamic
landscape: more habitat or better stewardship? Ecol. Appl. 18,
590–598
89 Levin, S.A. and Segel, L.A. (1976) Hypothesis for origin of planktonic
patchiness. Nature 259, 659
90 Waddell, J.N. et al. (2010) Demographic stochasticity versus spatial
variation in the competition between fast and slow dispersers. Theor.
Popul. Biol. 77, 279–286
345