Download Does the Conventional Leaky Integrate-and

Does the Conventional Leaky Integrate-and-Fire Neuron
Synchronize Spikes in Multiple Firing Mode?
(Synchronization/ Integrate and Fire Neuron/ Spike time Distribution/ Refractory Period/ Neural Network)
Baktash Babadi, Ehsan Arabzadeh, Arash Yazdanbakhsh, Shahin Rouhani
Abstract
The importance of synchronization of firing in a neuron pool is widely emphasized in many
studies. In most of the computational studies concerned with the synchronization mechanism in
feed-forward neural pools, only a single spike (first spike) of each neuron is analyzed. In this
paper we argue that a more realistic setting is when each neuron fires multiply. It is shown that
unlike the simple case of single firings for each neuron, in the more realistic condition, the
synchronization of firings in successive pools of leaky integrate and fire neurons is impossible.
We have confirmed this through the results of our simulation as well as analytic results. Finally we
present two possibilities in order to explain the physiological experiments where presence of
synchronization is evident.
1. Introduction
Considerable evidence indicates that neurons in different cortical areas are capable of producing
synchronous action potentials on the time scale of millisecond (Bair 1994, Bair 1996, Marsalek
1997) and the synchronization may play functional roles in neural processing (Abeles 1993, Prut
1998, Riehle 1997, Singer 1993).
Apart from the physiological evidence, synchronous firing in a group of neurons is of great
importance from computational viewpoint. A neuron that receives many simultaneous inputs, is
more likely to generate an action potential than one which receives the same inputs distributed
over a wider time range. Thus, it is believed that synchronous activity provides an efficient mean
to increase the reliability of responses and to eliminate noise in neural assemblies (Diesmann et
al 1999). Besides, synchronization can be assumed as a mechanism by which spatially separated
neurons, responding to the same stimulus, bind together to make up a functional group (Usher
1993, Engel 1991).
1
A number of studies have addressed the mechanism of synchronization in a feed-forward neural
network (Hermann et al 1995, Diesmann 1999, Marsalek 1997, Feng 1997, Burkitt 1999). In
some of these studies, the problem is considered in the neuron pool level (Hermann 1995,
Diesmann 1999), while the others approach the problem in a single neuron level (Marsalek 1997,
Feng 1997, Burkitt 1999), which is then generalized to the pool level (Diesmann 1999).
The most commonly used pulse generating neuron in the studies is the Integrate-and-Fire (I&F)
neuron model. Although the (I&F) model is one of the oldest proposed neuron models (Lapicqe
1907), it mimics the most important properties of the real neuron, such as temporal summation of
inputs and firing due to reaching a threshold (Koch 1999), and has been widely used in modeling
the neural structures (Tuckwell 1988). Despite the simplicity of the I&F model in comparison with
the detailed descriptive Hodgkin-Huxley neuron model (Hodgkin, Huxley 1952), it has been
shown that many biophysically detailed and biologically plausible Hodgkin-Huxley-type neural
networks can be transformed into I&F form by a piece-wise continuous change of variables
(Hoppensteadt , Izhikevich 1997).
The method common in the cited studies (Hermann et al 1995, Marsalek 1997, Feng 1997,
Burkitt 1999, Diesmann 1999), is to present a number of spikes with a known temporal
distribution (a pulse packet) as an input to a pulse generating neuron (or neuron pool) and
investigating the spike response of the neuron (or neuron pool). It showed that the temporal
variance of the spike response is less than the temporal variance of the input pulse packet in
time, i.e. the output pulse packet is more synchronized than the input pulse packet.
Some of these studies (Marsalek 1997, Feng 1997, Diesmann 1999) have assumed that a
neuron generates a single spike, and analyzed the variance of this solitary spike in the recipient
neuron. The other studies, on the other hand have inferred the distribution of spikes through
evaluating the variance of the first spike times, and assumed that this variance is a good
representative of the general distribution of spikes (Burkitt 1999). Thus the common trend in the
cited studies is single spike analysis. But in the more realistic case where the neuron generates
multiple spikes in a short time interval, what will be the out put spike time distribution as a whole?
This article addresses this question. In the next section, the time distribution of output spikes in a
single neuron in response to an incoming pulse packet is evaluated analytically with minimal
approximations. In the third section, the analytical results are confirmed through a computer
simulation. Finally, the efficiency of the conventional I&F feed forward network, in issuing the
synchronization phenomena is discussed.
2
2. Analysis of Output Spikes Distribution
Here, we consider a leaky I&F neuron with a large number of input connections (Fig. 1).
Fig.1, A pool of N Integrate and fire Neurons, feeds its
output to a neuron. The received input pulse packet has
a normal distribution in time. In response, the recipient
neuron generates an output pulse packet.
The membrane potential of the leaky integrate-and-fire neuron in its sub-threshold regime is
governed by:
dV (t )  V (t )  input (t )

,
dt

(1)
where input (t ) is the sum of excitatory and inhibitory spikes arriving to the neuron at time t and
 is the membrane time constant. Whenever the changing membrane potential reaches a
constant value (threshold), a spike is generated as:
Output (t )   (t  t s ) ,
where,  (t ) is the impulse function and
(2)
t s is the time in which the neuron has reached the
threshold. Immediately after reaching the threshold, the membrane potential is rendered to its
resting potential, which for the sake of simplicity is set to zero. The resting potential is also
maintained for a time window named its refractory period (r).
The recipient neuron receives its input from a pool of firing neurons (Fig. 1), so its input can be
represented by:
3
N
input (t )   w j  (t  t sj ) ,
(3)
j 1 t sj
j
where t s (s=1,2,.. n j ) are the spike times of the feeding neuron j and N is the total number of
neurons in the feeding pool.
For a moment, suppose that the neuron is not allowed to fire (Fig.2). Substituting equation (3) in
(1) and solving the resultant differential equation in terms of t yields:
V (t ) 
1

N

j 1 t sj
w j (t  t )e
j
s
( t sj t )

.
(4)
Suppose that the neurons in the feeding pool have produced an overall number of
nin spikes with
a normal time distribution of:

1
 in (t ) 
2  in
e
t2
2 in 2
,
(5)
where,  in is the standard deviation of the spike times in the input layer (Fig.1).
Due to a large enough
nin , one can change the above discrete summation (equation 4) into
integration:
V (t ) 
nin w


t

 in ( x)e
( x t )

dx .
(6)
As we have a narrow distribution for connecting weights, we could assume w j independent of x,
w (mean weight) is put out of integral as the representative of w j s. Fig. 2 plots the
membrane potential V (t ) versus time.
hence
4
Fig. 2, The membrane voltage of an Integrate-and-Fire neuron in response to its feeding inputs, which are
normally distributed in time. This neuron is not allowed to fire, hence, the voltage dynamic is the result of
incoming input and membrane leakage without resetting potential to zero after firing. This can be considered
as the graphical presentation of equation (4).
Now we consider the case where the threshold is present and the neuron fires consecutively in
times t1 , t 2 ,..., t no u t , in response to the incoming pulse packet (Fig.3). n out is the total number of
generated spikes. Note that here, the neuron is allowed to generate multiple spikes in response
to its incoming pulse packet. Hermann 1995, Feng 1997, Marsalek 1997, Burkitt 1999 and
Diesmann 1999 have analyzed the problem in case the neuron generates only a single spike.
Fig. 3, The neuron fires whenever reaches the threshold, then the membrane potential resets to zero. Thereafter, the
neuron gains the remaining pulse packet to produce the next spike. This procedure continues until the potential cannot
reach the threshold; and decay is the case after weak peak value.
5
If each neuron generates multiple spikes in response to its input pulse packet, let us consider the
time distribution of spikes. After each spike is generated, the membrane potential is set to zero
and remains there for its refractory period, r. So in each inter-spike interval t k  t  t k 1 , the
membrane potential is zero for
interval
t k  t  (t k  r ) and the integration takes place over the time
t k  r  t  t k 1 , so in order to show the membrane potential in the latter interval we
rewrite equation (6) as:
U (t k , t ) 
where,
nw


t
t k 1  r
 in (t )e
( t  t )

dt  ,
(7)
U (t k , t ) represents the membrane potential in t k  r  t  t k 1 . Considering equation (6)
we can rewrite equation (7) as (see Appendix A):
U (t k , t )  V (t )  e
where, A 
rn in w

( t k t )

V (t k )  A   in (t k )  e
( t k t )

,
(8)
.
Note again that V (t ) represents the membrane potential if firing is not allowed, while
U (t k , t ) is
the potential in the actual case where the neuron could have generated spikes.
Obviously, the membrane potential reaches the threshold th, in the firing times t1 , t 2 ,..., t no ut .
So, for each t k  {t1 , t 2 ,..., t nou t } ,
V (t k )  th  e
( t k 1 t k )

U (t k 1 , t k ) =th. Using equation (8), we have:
V (t k 1 )  A   in (t k 1 )  e
( t k 1 t k )

.
By iterating equation (9) on itself and taking into account that
k
V (t k )   e
( ti t k )
i 1

( A in (t i )  th) .
(9)
V (t1 )  th , it can be written as:
(10)
Assuming that the number of generated spikes ( n out ) is considerably large and the spikes are
close together, we rewrite the above summation in terms of integration:
t
V (t )  nout  pout (t )( Ain (t )  th)e
t  t


dt  ,
(11)
pout (t ) is the time density distribution of the generated spikes of the neuron.
pout (t )dt  represents the ratio of generated spikes in dt  . So, we have to multiply it by nout to
reach the number of spikes in each integration interval ( dt  ).
where
Comparing Equation (11) and (6) yields:
6
A in (t )  r nout pout (t )( A in (t )  th) .
(12)
Now, it is possible to obtain the time distribution of the generated spikes in terms of the input
spikes parameters:
1
pout (t ) 
 in (t )
(13)
thA rnout  rnout  in (t )
1
The variance of the output spikes is (see Appendix B):

2
 out
  t 2 p out (t )dt   in2 (1 

rnin w
K) ,
th
(14)
where K is positive.
Equation (14) shows that
2
 out
  in2 .
The order of the refractory period is small ( ~ 10
Patton 1989), so, at best (when refractory period is negligible) 
2
out

2
in .
3
sec,
This result is
interesting, because, in any case there remains no route for such a model to compress its output
spike packet compared to its input pulse packet.
3. Simulation
We assess the above discussion through a computer simulation on a PC. In the simulation we
studied the time dependent behavior of a single leaky integrate and fire neuron. The model
neuron receives a packet of 1000 spikes as input, which are normally distributed in time around
t=0 with a standard deviation of 10 msec. To be a close approximation to the biological reality, the
membrane time constant is set to 20 msec, the threshold is set to 20 mV above the resting
potential (as the resting potential is assumed zero for simplicity, the threshold is equal to 20 mV
here) and the refractory period is assumed equal to 1.75 msec (McCormik et al 1985). Regarding
equation (6), each spike raises the EPSP by the value of
w

. So, the mean input connection
weight is set to 20 mV.msec to mach the intracellular recordings which revealed a nearly 1 mV of
EPSP rise per a single input spike (Mason et al 1991). The membrane potential change of the
model neuron was approximated by piece wise linear solution of the differential equation (1).
The simulation results can be seen in Fig. 4. Each input is presented by a vertical bar, and
obviously, the inter spike intervals (ISI) are the distance between the bars. Fig. 4a shows the
input spikes to a neuron. Regarding the normal distribution of input spikes in time, the vertical
bars are denser in the center.
Fig. 4b,c shows the output spikes generated by the neuron in response to the input pulse (spike)
packet shown in Fig. 4a. As mentioned by Mason et al 1991, approximately 20 spikes are needed
to trigger the neuron’s action potential. So, it was somehow predictable that the output firing
pattern should be sparser than its corresponding input.
Fig. 4b illustrates the output spiking pattern in the case where no refractory period is
implemented. The time window of firings is trimmed from both sides, and the generated spikes
are sparser in time.
7
Now consider the case in Fig. 4c, where the refractory period is added and compare it with Fig.
4b. In the presence of refractory period the firing pattern is sparser because in the refractory
period, a number of incoming spikes are neglected, so the ‘effective’ input to the neuron will be
smaller, resulting in a sparser output spike pattern.
Note that the narrowing of the output time band does not necessarily imply a decrease in the
standard deviation of spikes in time, which is usually considered the criteria of synchronization in
the literature. There is a tradeoff between the time window narrowing and the sparseness of the
generated spikes, in contributing to synchronization, i.e. standard deviation in time. In other
words, in a constant time band when spikes become sparser, the inter-spike intervals become
greater, so the standard deviation increases, which is more striking in the presence of refractory
period, as is shown analytically by equation (14).
a
b
c
Fig. 4, a) The input that a single model neuron receives from its connections. A vertical bar presents each input spike
which are normally distributed in time. Input bars are denser in the center and sparser in the periphery. The neuron sums
them up temporarily according to its dynamic. b) Generated spikes by the recipient neuron with no refractory period.
Because ~20 input spikes are needed to trigger an out put spike the trimming of the time window from both side occurs.
For the same reason compared with 4a the spikes are sparser and the standard deviation is larger. c) Output pulse
packet when refractory period is present. It is sparser and its standard deviation in time is even greater than the case in
4b (see equation 14).
4. Discussion
This study was aimed at evaluating the capability of the leaky integrate-and-fire neuron to
synchronize its output spikes in comparison with its input spikes, when it generates multiple
spikes. Our results show that in such a neuron, the output spikes variance is equal or greater
than the input spikes variance, which means that it fails to synchronize its output spikes. This
result can be generalized to the recipient neuron pool level, consisting of identical independent
neurons i.e. the time density distribution of the total generated spikes in the pool is equal to that
of single neurons (See Appendix C).
But as mentioned before, synchronization is of great importance in the neural assemblies. Given
that the cortical neurons operate in a noisy environment, in the absence of a synchronizing
mechanism, there will be a permanent tendency for desynchronizing the spikes in cortical neural
8
assemblies. To put it another way, even in case of  out   in , one will face a progressive
2
2
asynchrony, because of an inevitable noise. Some of the sources of this noise, which tend to
desynchronize the generated spikes of a neuron pool, could be listed as:
1- The differences between the axonal and dendritic lengths and diameters in different
neurons of a pool (Manor et al, 1991). (Geometrical noise)
2- Variation of the delay between pre-synaptic spike arrival and post-synaptic channel
opening, in different synapses. (Synaptic noise)
3- The noise due to spontaneous firings of the neurons, which is often treated as a Poisson
process. (Spontaneous noise)
Thus, if the cortical neural groups are assumed to be arranged in a feed-forward manner (Abeles
1991), in the absence of a synchronizing mechanism in the single neuron level, the activity
(spiking) pattern of successive pools desynchronizes or rounds off through the hierarchy.
Yet, this is not the case in the cortex; it has been shown that in visual cortical hierarchy, the
frequency rise time of the neurons in successive neuron pools remains relatively constant and
accurate in time (Marsalek 1997).
Taking into account the above biological fact, a minimal degree of synchronization is needed at
least to oppose the noise disturbance to prevent the rounding off of the packet of the spikes.
As a conclusion, the above mismatch between the modeling results and the experimental data
can be solved considering either or both of the following assumptions:
a) The leaky integrate-and-fire neuron is an oversimplified model to present the
synchronization phenomena in the single neuron level.
b) The interconnections between the neurons in a pool (intra-pool connections) are
responsible for synchronizing the firing activity of the neurons in the pool, so the feedforward structure appears to be inappropriate as a model for cortical neural arrangement.
Appendix A
Equation (7) can be written as:
U (t k , t ) 
nw


t

 in (t )e
( t  t )

dt  
nw

e
t


t k 1  r

t
 in (t )e  dt  ,
(15)
Assuming that the refractory period r is considerably short, equation (15) can be approximated by
Taylor expansion as:
U (t k , t ) 
nw


t

 in (t )e
( t  t )

dt  
nw

e
t


t k 1

t
 in (t )e  dt  
rn w

e
t k 1 t

 in (t k 1 ) .
(16)
Considering equation (6) yields:
9
U (t k , t )  V (t )  e
( t k t )

V (t k )  A   in (t k )  e
( t k t )

,
which is exactly equation (8).
Appendix B
1
By replacing helping constants m  thA rn out and
l  rnout  in (t ) in equation (13) one can
obtain:
pout (t ) 
 in (t )
.
m  l in (t )
(17)
By differentiation in terms of t, one reaches to:
 (t ) 
pout
m in (t )
 m t  in (t )
 m t pout (t )
,


2
2
2
[m  l in (t )]
 in [m  l in (t )]
 in 2 [m  l in (t )]
(18)
which yields:
 (t ) 
t pout (t )   in2 pout
where
l in2
F (t ) ,
m
(19)
 (t ) .
F (t )   in (t ) pout
By definition of the variance:


2
out

t

2

p out (t )dt   t [t p out (t )]dt ,
(20)

and substituting equation (19) in (20):
2
 out
 t ( in2 pout (t ) 

l in2
l 2
F (t ))]     in2 pout (t ) dt  in
m
m


 F (t )dt .
(21)

The left phrase of the right side is zero, so:

2
out


2
in


l in2
pout (t ) dt 
m

 F (t )dt .
(22)


Given that
pout (t ) is a density distribution function,

p out (t ) dt  1 , so:

2
 out
  in2 [1 

l
F (t )dt ]
m 
(23)
10
 (t ) , is zero at t=0,
F (t ) is a positive bell shaped function, because its derivative,  in (t ) pout

positive for t < 0, negative for t > 0, and near zero at   and
  . So
 F (t )dt is positive too,

which we will show it by K. Substituting the values of l and m in the above equation (23), Results
in:
2
 out
  in2 (1 
rnin w
K) ,
th
which is exactly the equation (14).
Appendix C
Let us suppose that the recipient pool consists of N neurons. As the neurons are identical and the
distribution of the connecting weights is narrow, the output spikes time distribution for all the
neurons in this pool are identical and equal to p out (t ) . So, in a small time interval dt, the total
N
number of generated spikes by the pool is (
n
i 1
i
out
) p out (t ) dt . On the other hand, if we consider
 out (t ) , in the small time interval
pool
pool
nout
 out (t ) dt , where nout
is the
the time density distribution of the pool spikes -as a whole- as
dt, the total number of generated spikes by the pool will be
total generated spikes by the pool. Given that nout 
pool
N
n
i 1
i
out
, yields:
 out (t )  Pout (t ) .
References
Abeles M., Bergman H., Margalit E., Vaadia E., Spatiotemporal Firing Patterns in the Frontal
Cortex of Behaving Monkeys, J. Neurophysiol., 70, 1629-1638, 1993.
Bair W, Koch C, Newsome WT, Britten KH, Reliable temporal modulation in cortical spike trains in
awake monkey. Society for Neuroscience, Abstracts 20(2):1279 (#525.7), 1994.
Bair W, Koch C., Temporal precision of spike trains in extrastriate cortex, Neural Computation,
8:1185-1202, 1996.
Burkitt, A. N., and Clark, G. M., Analysis of integrate-and-fire neurons: synchronization of synaptic
input and spike output, Neural Computation 11(4), 871-901, 1999.
Diesmann M., Gewaltig M., Aertsen A., Stable Propagation of Synchronous Spiking in Cortical
Neural Networks, Nature, Vol. 402, 2 Dec. 1999.
Engel, A. K., Konig, P, Kreiter, A. K., & Singer, W., Interhemispheric synchronization of oscillatory
neuronal responses in cat visual cortex, Science, 252: 1177-79, 1991
Feng J., Behaviors of Spike Output Jitter in the Integrate-and-Fire Model, Physical Review
Letters, Vol. 79, Number 22, Dec. 1997.
11
Herrmann M., Hertz J. A., Prugel-Bennett A., Analysis of Synfire Chain ,Network, 6:403--414,
1995.
Izhikevich E.M., Weakly Pulse-Coupled Oscillators, FM Interactions, Synchronization, and
Oscillatory Associative Memory, IEEE Transactions On Neural Networks, 10:508-526, 1999
Koch C., Simplified Models of Individual Neurons, in Biophysics of Computation: Information
Processing in Single Neurons, Oxford University Press, 1999
Lapique, L. Recherches quantitatives sur l’excitation electrique des nerfs traitee comme une
polarization. J. Physiol. Paris 9:620-635, 1907.
Manor Y., Koch C., Segev I., Effect of geometrical irregularities on propagation delay in axonal
trees, Biophys. J., 60: 1424-1437, 1991.
Marsalek P., Koch C., Maunsell J., On the relationship between synaptic input and spike output
jitter in individual neurons, Proc. Natl. Acad. Sci., Vol 94, pp 735-740, Jan. 1997
Mason A., Nicoll A. and Stratford K. Synaptic Transmission between Individual Pyramidal
Neurons of the Rat Visual Cortex in vitro. The Journal of Neuroscience, 11:72-84., 1991.
McCormick D. A., Connors B. W., Lighthall J. W., and Prince D. A., Comparative
electrophysiology of pyramidal and sparsely spiny stellate neurons of the neocortex. J.
Neurophysiol., 54:782-805, 1985.
Patton, H. D. Fuchs, A. F., Textbook of Physiology: Vol. 1 Nervous System, 1989.
Prut Y. et al Spatiotemporal structure of cortical activity: Properties and behavioral relevance. J.
Neurophysiol., 79, 2857-2874, 1998.
Riehle A, Grun S, Diesmann M and Aertsen A (1997): Spike synchronization and rate modulation
differentially involved in motor cortical function. Science 278, 1950-1953.
Singer W., Synchronization of cortical activity and its putative role in information processing and
learning, Annu. Rev. Physiol. 55: 349-374, 1993.
Usher M., Schuster H. G., Neibur E., Dynamics of populations of integrate-and-fire neurons,
partial synchronization and memory, Neural Comput. 5: 570-586, 1993.
12