Download Doubly stochastic processes: an approach for understanding central

PROCEEDINGS OF THE 3RD INTERNATIONAL CONFERENCE ON APPLIED MATHEMATICS, SIMULATION, MODELLING (ASM'09)
PROCEEDINGS OF THE 3RD INTERNATIONAL CONFERENCE ON CIRCUITS, SYSTEMS AND SIGNALS (CSS'09)
Doubly stochastic processes: an approach for
understanding central nervous system activity
Janet A. Best
constitute part of the neural signal [8]. When the neuronal
activity is stochastic, the transitions themselves may also be
stochastic, for example as fluctuations in spike threshold.
Doubly stochastic processes are a natural tool for
understanding many types of neuronal processing in which (i)
neurons fire irregularly, (ii) the firing rate changes in response
to inputs, and (iii) there are underlying state changes (likely
changing the stochastic firing rate of inputs) that can be
modeled stochastically. Likely applications include firing of
globus pallidum neurons [9] and modifying synapses during
learning [10].
Doubly stochastic processes themselves are not new; they
have been investigated in apurely mathematical context [11][13]. While they have been applied in computational
neuroscience [14], [15] such use has been relatively
infrequent.
Even if one agrees that doubly stochastic processes are a
natural tool in these research areas, two natural questions arise.
First, what kind of evidence in data would be necessary to
prove that these processes are doubly stochastic? Secondly, if
they are doubly stochastic, how does that help one understand
neural processing and biological function? Below we describe
new work on sleep-wake transitions where this research
program has been carried out and where both questions have
been answered.
Abstract— In this paper we argue that doubly stochastic
processes are a natural tool for understanding certain types of
information processing in the central nervous system. Doubly
stochastic processes themselves are not new and have been
investigated in a mathematical context; however, they have not been
widely applied in neuroscience. We begin by pointing out that the
brain is very stochastic: both individual neurons and physiological
neural networks exhibit a variety of stochastic firing patterns. We
then consider transitions from one firing pattern to another in
individual cells and argue that the transitions themselves are likely to
be stochastic. Even if one agrees that doubly stochastic processes are
a natural tool in these research areas, two natural questions arise.
First, what kind of evidence in data would be necessary to prove that
these processes are doubly stochastic? Secondly, if they are doubly
stochastic, how does that help one understand neural processing and
biological function? We describe new work on sleep-wake transitions
where this research program has been carried out and where both
questions have been answered.
Keywords— Doubly stochastic, Poisson process, power law,
sleep-wake.
I. INTRODUCTION
T
HE brain is a very stochastic place: both individual neurons
and physiological neural networks exhibit a variety of
stochastic firing patterns [1] and stochastic processes have
been widely applied to model phenomena in neurobiology.
Of course, the effects of stochasticity can also vary widely
and may depend upon details of the system, resulting for
instance in stochastic resonance [2], increased reliability [3] or
diminished stability [4]. Though we are still learning to
interpret neural codes and still debating the roles of
irregularity, what is clear is that the healthy brain has ways to
regulate variability in neuronal activity: the spikes of an
individual neuron or of a population of neurons can be very
regular or very irregular [5]-[7]. In this short paper, we focus
on the irregular case: instances in which the activity of a cell or
of a population of neurons is well-modeled by a stochastic
process. We interpret the fact that brain has the ability to
avoid such irregularity as an indication that the apparent
stochasticity may have some role and thus merits attention.
In many cases, transitions between distinct activity patterns
II. A DOUBLY STOCHASTIC MODEL OF SLEEP-WAKE
REGULATION
A. Developmental Changes in Sleep-Wake Regulation
The physiological complexity of sleep invites mathematical
approaches and contributions: mathematical models of the
neural interactions may help untangle causality or at least
plausibility. A number of mathematical models have been
developed to address aspects of sleep regulation.
The basic neural circuitry underlying sleep-wake transitions
is complex and remains to be fully identified. A reduced
circuitry regulates sleep in infant mammals compared to
adults: infant sleep-wake cycles arise from a brain stem
network; during maturation, a gradual strengthening of
modulatory inputs from more rostral areas parallels a
consolidation of sleep and wake states into longer episodes
[16],[17].
Blumberg and collaborators have studied these
developmental changes in several model species, particularly
in young rats. They observe that sleep and wake episodes are
very brief for newborn animals, with exponentially-distributed
Manuscript received December 15, 2009. The research was supported by
the Air Force Office of Scientific Research grant FA9550-06-1-0033 and by
the National Science Foundation under agreement 0112050.
J. A. Best is with the Mathematics Department, Ohio State University,
Columbus, OH 43210 USA (e-mail: [email protected]).
ISBN: 978-960-474-147-2
155
ISSN: 1790-5117
PROCEEDINGS OF THE 3RD INTERNATIONAL CONFERENCE ON APPLIED MATHEMATICS, SIMULATION, MODELLING (ASM'09)
PROCEEDINGS OF THE 3RD INTERNATIONAL CONFERENCE ON CIRCUITS, SYSTEMS AND SIGNALS (CSS'09)
data as described above.
Based on inferences from lesion and other experiments (see
for example [16]), the model has mutually-inhibitory
interactions between the wake-active dorsolateral pontine
tegmentum (DLPT) and the sleep-active nucleus pontis oralis
(NPO), and the firing of neurons in each nucleus are modeled
as inhomogeneous Poisson processes. The power law arises
due to a mechanism involving strengthening of an excitatory
connection to DLPT from the locus coeruleus (LC) [22]. Fig. 1
depicts this network.
duration t. In other words, letting p(r) denote the probability
density function of durations between r and r+dr, so
that P(t) ≡
∫
∞
t
p(r)dr is the cumulative distribution of
episode durations, they observe that P(t)~exp(-t/τ). Here, τ is
the expected value of the distribution and the characteristic
time scale of the behavioral state; typically τsleep ≠ τwake. The
exponential shape of these two distributions persists during the
first two postnatal weeks, with increasing time constants τ
reflecting consolidation into longer episodes. Around the
beginning of the third postnatal week, an additional process
C. Doubly Stochastic Point Process Models and
Bifurcation Theory
We model the neuronal network governing sleep-wake as a set
of three interacting cell populations. The firing times of the ith population are given by a Poisson process with intensity λi(t)
which satisfies the stochastic differential equation
n
λi′(t) = f i ( λi ) + ∑ gij ( λi )∑δ (T jk − t)
j
j=1
for 1 ≤ i ≤ 3. Since λi(t) is itself stochastic, the overall process
is a doubly stochastic process. Here f gives the firing rate in
the absence of inputs, and gij describes the excitatory or
inhibitory impact of population j on population i. The model
describes the time evolution of the intensity process λi for each
population. The intensity is dependent on the random variables
Tjkj which are the times of firing of the j-th population.
Because the full stochastic system is difficult to analyze, we
have derived a system of ordinary differential equations that
approximate (in a manner that can be stated precisely) the
stochastic system for sufficiently short periods of time. Briefly,
Fig. 1. The brain stem network. Arrows represent excitatory
connections, circles denote inhibitory connections. Dashed arrows
represent inputs from unspecified rostral nuclei, responsible for bout
consolidation.
E[ λ ′ | Λ t ] = f (λ (t)) +
∑ g (λ(t))λ (t)
j
j
(2)
j ∈input
becomes evident, transforming the distribution of wake
episodes such that the probability of ending a wake episode
decreases across the duration of the episode. By postnatal day
21, the length of wake episodes are well-approximated by a
power law distribution, P(t) ~ t-α, as has been observed in adult
mammals of several species [18]. The exponential distribution
for sleep bouts appears to remain throughout the life of the
animal.
where the expectation is conditioned on Λt, the history of
the process up to time t. Tuckwell and others have studies
similar questions in the case of stochastic diffusion; in their
case, conditioning on the past was unnecessary [23].
If we make the assumption that λ'(t) = E[λ' | Λt], we obtain a
system of three deterministic equations that has a bifurcation
diagram as shown in Fig. 2. We now address the significance
of the different parts of the diagram. The vertical axis of the
diagram corresponds to the firing rate of a wake-active cell
population during wakefulness. The bifurcation parameter, on
the horizontal axis, α represents the maximal extent of positive
feedback during an individual wake bout; α increases with age.
For animals less than 14 days old, α<αSN, so that there is only
one (stable) equilibrium for the firing rate, corresponding to a
constant probability of terminating the wake bout and therefore
an exponential distribution of wake bout durations. As α
increases beyond αSN, the system begins to have two stable
equilibria for firing rate, separated by an unstable fixed point.
As a wake bout begins, the firing rate is near the lower stable
fixed point; however, as wakefulness continues, positive
feedback increases, resulting in a biased random walk towards
the upper stable fixed point. Note that, if the wake bout
continues long enough, the firing rate can remain near the
B. Noisy Data; Mathematical Model
All biological data contain variation from many sources,
resulting in a noisy appearance. Yet the data on sleep and
wake bout lengths are not merely noisy. Data showing, for
example, that the distribution of wake bouts in adult rats is
well fit by a power law (over three orders of magnitude)
indicate that there are underlying stochastic processes at work.
Therefore we have been investigating the nature of stochastic
mechanisms that may be giving rise to the observed data. The
underlying scientific rationale is that the mechanisms causing
the bout distribution are exactly the mechanisms of transition
between sleep and wakefulness that we seek to understand.
We have constructed mathematical models for the
developing infant neural circuitry [19]-[21]. These models,
summarized below, capture well the features of experimental
ISBN: 978-960-474-147-2
(1)
kj
156
ISSN: 1790-5117
PROCEEDINGS OF THE 3RD INTERNATIONAL CONFERENCE ON APPLIED MATHEMATICS, SIMULATION, MODELLING (ASM'09)
PROCEEDINGS OF THE 3RD INTERNATIONAL CONFERENCE ON CIRCUITS, SYSTEMS AND SIGNALS (CSS'09)
analyze mathematically, we have illustrated here that pairing
with deterministic systems can aide in analysis, and we
continue to work on the mathematical theory to support and
extend this approach. Depending on one’s purpose, it may be
adequate to approximate a doubly stochastic process with a
simpler inhomogenous stochastic process or a mixture of
simpler stochastic processes though such an approximation
may forfeit the opportunity of a mechanistic connection for the
model. However, for the model described above we have been
able to develop a mixed Poisson process model that yields
complementary biological insights; see [21] for details.
A natural question concerns what sort of evidence would
support the occurrence of doubly stochastic processes in the
central nervous system. We note that other authors [14] have
also found, in a comparison of stochastic models, that doubly
stochastic models fit the irregular activity of some neurons
better than several alternative stochastic models.
In this paper we have focused only upon doubly stochastic
Poisson processes. However, one must expect that many other
types of doubly stochastic processes occur in the central
nervous system. Another natural class of processes that may
be described as doubly stochastic is that of stochastic
processes on random graphs; such processes may also have
significant applications in neuroscience. In the event that a
neuronal network changes stochastically, for example synaptic
plasticity in learning, the activity on the network may be a
doubly stochastic process. In related work, we are studying
the probabilistic spread of neuronal activity through a random
Fig. 2. Bifurcation diagram for the deterministic system. Here λ
network.
represents the firing frequency of DLPT during wakefulness; α
represents the strength of mutual excitation between DLPT and LC and
For sleep-wake cycling described above, the doubly
increases with development. A saddle-node bifurcation occurs at αSN.
stochastic model provides a framework for understanding how
The arrow indicates the increase in intensity λ during a wake bout, due the observed data can arise from the interactions of groups of
to mutual excitation.
neurons. In particular, the model suggests how the postnatal
distribution will depend upon details of the stochastic process
development of the neuronal circuitry may drive the changes in
governing the biased random walk. In ongoing work, we are
sleep-wake dynamics observed through the shifting shape of
studying the conditions under which this walk gives rise to
bout distributions .
power laws or other prescribed distributions for bout length.
This work has significance in a number of areas unrelated to
sleep-wake cycling.
Intervals of behavior and their
distribution have been long studied by ethologists [25]; typical
REFERENCES
approaches have involved fitting the distribution with a [1] H. C. Tuckwell, Stochastic Processes in the Neurosciences.
Philadelphia, PA: Society for Industrial and Applied Mathematics,
mixture of exponential processes. While a mixture of
1989.
exponentials is well-known to be able to approximate other
[2] A. Longtin, “Stochastic resonance in neuron models,” Journal of
distributions including power law, such an approach is
Statistical Physics, vol. 70, pp. 309-327, 1993.
statistical only and does not shed light on the mechanisms [3] S. Tanabe, “Noise-enhanced neuronal reliability,” Phys Review E, vol.
64, 041904, 2001.
generating the observed behavior. Our goal is to develop
[4] E. T. Rolls, M. Loh, G. Deco, G. Winterer, “Computational models of
methods for investigating the biological mechanisms.
schizophrenia and dopamine modulation in the prefrontal cortex,”
upper fixed point, resulting again in an additional set of
exponentially-distributed wake bouts, but these with larger
mean than those corresponding to the lower fixed point. In
fact, these two ranges of exponentially-distributed wake bouts
(separated by a range with power-law distribution) have
recently been reported experimentally in adult animals [24].
The distribution of wake bout durations corresponding to
lengths between these two exponential ranges will clearly have
a heavy tail compared to the shorter-duration exponential
bouts. The precise shape and width of this part of the
[5]
III. CONCLUSION
In this paper we have argued that activity in the brain
frequently appears stochastic and that in fact doubly stochastic
mathematical models are often a natural tool that can provide
real insights into biological mechanisms. We should note in
this context that deterministic systems can also give rise to
irregular activity [9].
Though doubly stochastic processes can be difficult to
ISBN: 978-960-474-147-2
[6]
[7]
157
Nature Reviews Neurosci, vol. 9, pp. 696-709, 2008.
Y. Prut, S. I. Perlmutter, “Firing properties of spinal interneurons during
voluntary movement. 1. state-dependent regularity of firing,” The
Journal of Neurosci, vol. 23, pp. 9600-9610, 2003.
S.-L. Shin, S. Rotter, A. Aertsen, E. De Schutter, “Stochastic description
of complex and simple spike firing in cerebellar Purkinje cells,”
European J of Neurosci, vol. 25, pp. 785-794, 2007.
M. Rudolph, A. Destexhe, “The discharge variability of neocortical
neurons during high-conductance states,” Neuroscience, v. 119, pp.
855-873, 2003.
ISSN: 1790-5117
PROCEEDINGS OF THE 3RD INTERNATIONAL CONFERENCE ON APPLIED MATHEMATICS, SIMULATION, MODELLING (ASM'09)
PROCEEDINGS OF THE 3RD INTERNATIONAL CONFERENCE ON CIRCUITS, SYSTEMS AND SIGNALS (CSS'09)
[8]
J. Deppisch, H.-U. Bauer, T. Schillen, P. Konig, K. Pawelzik, T. Geisel,
“Alternating oscillatory and stochastic states in a network of spiking
neurons,” Network: Computation in Neural Systems, vol. 4, pp. 243257, 1993.
[9] J. A. Best, C. Park, D. Terman, C Wilson, “Transitions between
irregular and rhythmic firing patterns in excitatory-inhibitory neuronal
networks,” J Computational Neurosci, vol. 23, pp. 217-235, 2007.
[10] H. S. Seung, “Learning in spiking neural networks by reinforcement of
stochastic synaptic transmission,” Neuron, vol. 40, pp. 1063-1073,
2003.
[11] D. R. Cox, “Some statistical methods related with series of events,” J
Royal Statistical Soc. B, v. 17, pp. 129-164, 1955.
[12] J. Mecke, “Eine characteristische Eigenschaft der doppelt
stochastischen Poissonschen Prozesse,” Z. Wahrscheinlichkeitstheorie
verw. Geb., vol. 11, pp. 74-81, 1968.
[13] J. Grandell, Doubly Stochastic Poisson Processes. Berlin: SpringerVerlag, 1976.
[14] S. Shinomoto, T. Yasuhiro, “Modeling spiking behavior of neurons with
time-dependent Poisson processes,” Physical Review E, vol. 64,
041910, 2001.
[15] B. E. A. Saleh, M. C. Teich, “Multiplication and Refractoriness in the
Cat's Retinal-Ganglion-Cell Discharge at Low Light Levels,”
Cybernetics, vol. 52, pp.101-107, 1985.
[16] K. Karlsson, A. Gall, E. Mohns, A. Seelke, M. Blumberg, “The neural
substrates of infant sleep in rats,” PLOS Biology, vol. 3, pp. 0891-0901,
2005.
[17] M. Blumberg, A. Seelke, S. Lowen, K. Karlsson, “Dynamics of sleepwake cyclicity in developing rats,” PNAS, vol. 102, pp. 14860-14864,
2005.
[18] C. C. Lo, T. Chou, T. Penzel, T. Scammell, R. Strecker, H. Stanley, P.
Ivanov, “Common scale-invariant patterns of sleep-wake transitions
across mammalian species,” PNAS, vol. 101, pp. 17545-17548, 2004.
[19] J. Best, C. Diniz Behn, G. Poe, V. Booth, “Neuronal models for sleepwake regulation and synaptic reorganization in the sleeping
hippocampus,” Journal of Biological Rhythms, vol. 22, pp. 220-232,
2007.
[20] B. Joshi, “A doubly stochastic Poisson process model for wake-sleep
cycling,” Ph.D. dissertation, Dept. Math., Ohio State University,
Columbus, Ohio, USA, 2009.
[21] B. Joshi, J. Best, “A doubly stochastic model for the development of
sleep-wake regulation,” unpublished.
[22] A. Gall, B. Joshi, J. Best, M. Blumberg, “Developmental emergence of
power-law wake behavior depends upon the functional integrity of the
locus coeruleus,” Sleep, vol. 32, pp. 920-926, 2009.
[23] R. Rodriguez, H. Tuckwell, “Statistical properties of stochastic
nonlinear dynamical models of single spiking neurons and neural
networks,” Physical Review E, vol. 54, pp. 5585-5590, 1996.
[24] S. Simasko, S. Mukherjee, “Novel analysis of sleep patterns in rats
separates periods of vigilance cycling from long-duration wake events,”
Behavioral Brain Research, vol. 196, pp. 228-236, 2009.
[25] P. W. Colgan, Quantitative Ethology, New York: John Wiley & Sons,
1978.
ISBN: 978-960-474-147-2
158
ISSN: 1790-5117