Download Modeling Short-Term Synaptic Depression in Silicon

LETTER
Communicated by Kwabena Boahen
Modeling Short-Term Synaptic Depression in Silicon
Malte Boegerhausen
[email protected]
Pascal Suter
[email protected]
Shih-Chii Liu
[email protected]
Institute of Neuroinformatics, University of Zurich and ETH Zurich,
Winterthurerstrasse 190, CH-8057 Zurich, Switzerland
We describe a model of short-term synaptic depression that is derived
from a circuit implementation. The dynamics of this circuit model is similar to the dynamics of some theoretical models of short-term depression except that the recovery dynamics of the variable describing the depression is nonlinear and it also depends on the presynaptic frequency.
The equations describing the steady-state and transient responses of this
synaptic model are compared to the experimental results obtained from
a fabricated silicon network consisting of leaky integrate-and-Žre neurons and different types of short-term dynamic synapses. We also show
experimental data demonstrating the possible computational roles of depression. One possible role of a depressing synapse is that the input can
quickly bring the neuron up to threshold when the membrane potential
is close to the resting potential.
1 Introduction
Short-term synaptic dynamics has been observed in different parts of the
cortical system (Stratford, Tarczy-Hornoch, Martin, Bannister, & Jack, 1998;
Varela et al., 1997; Markram, Wang, & Tsodyks, 1998; Reyes et al., 1998).
The functionality of the short-term dynamics has been implicated in various cortical models (Senn, Segev, & Tsodyks, 1998; Chance, Nelson, & Abbott, 1998; Matveev & Wang, 2000). The properties of a network with dynamic synapses have been investigated by Tsodyks, Pawelzik, and Markram
(1998), and Maass and Zador (1999) and the use of such a network in time
series processing by Maass and Zador (1999) and Natschlager, Maass, and
Zador (2001). The introduction of these dynamic synapses into hardware
implementations of recurrent neuronal networks allows a wide range of
operating regimes, especially in the case of time-varying inputs.
In this work, we describe a model that was derived from a circuit implementation of short-term depression and facilitation. The dynamics of this
Neural Computation 15, 331–348 (2003)
c 2002 Massachusetts Institute of Technology
°
332
M. Boegerhausen, P. Suter, and S. Liu
circuit, initially proposed by Rasche and Hahnloser (2001), was not analyzed in their work. Our work is presented in two parts. We Žrst describe
the model of this synaptic circuit and compare the differential equations
describing the depression with the equations of one of the theoretical models frequently used in network simulations (Abbott, Sen, Varela, & Nelson, 1997; Varela et al., 1997). We analyze the transient and steady-state
responses of this synaptic model in response to inputs of different statistical
distributions.
Next, we compare the theoretical results with experimental responses
obtained from a silicon network of leaky integrate-and-Žre neurons together
with different short-term dynamic synapses. We also show experimental
data from the chip that demonstrate the possible computational roles of
depression. From these experiments, we postulate that one possible role of
depression is to bring the neuron’s response up to threshold quickly if the
membrane potential of the neuron was close to the resting potential. We
also mapped a proposed cortical model of direction selectivity that uses
depressing synapses onto this chip. The results are qualitatively similar to
the results obtained in the original work (Chance et al., 1998).
The similarity of the circuit responses to the responses from Abbott and
colleagues’ model means that we can use these VLSI networks of integrateand-Žre (IF) neurons as an alternative to computer simulations of dynamical
networks composed of large numbers of integrate-and-Žre (IF) neurons using synapses with different time constants. The outputs of such networks
can also be used as an interface with wetware. An infrastructure for a reprogrammable, reconŽgurable, multichip neuronal system is being developed
along with a user-deŽned interface so that the system is easily accessible to
a naive user.
2 Comparisons Between Models of Depression
Two theoretical models describing synaptic depression and facilitation used
frequently in network simulations are the phenomenological model of Tsodyks and Markram (1997) and the model from Abbott et al. (1997). Because
these models are equivalent if the inactivation synaptic time constant in
Tsodyks and Markram’s model is small when compared with the recovery
time constant of the depression, we compare the circuit model only with the
model from Abbott and colleagues. Here, we describe only the circuit model
for synaptic depression. The equivalent model for facilitation is described
elsewhere (Liu, in press).
2.1 Theoretical Model for Depression Model. In the model from Abbott et al. (1997), the synaptic strength is described by gD (t), where D is a
variable between 0 and 1 that describes the amount of depression (D D 1
means no depression) and g is the maximum synaptic strength. The recovery
Modeling Short-Term Synaptic Depression in Silicon
333
dynamics of D is
td
dD
D 1 ¡ D,
dt
(2.1)
where td is the recovery time of the depression. The update equation for D
right after a spike at time t D tsp is
C
¡
) D dD (tsp
),
D(tsp
(2.2)
where d (d < 1) is the amount by which D is decreased right after the spike
and tsp is the time of the spike. The average steady-state value of depression
for a regular spike train with a rate r is
Dss D
1 ¡ e¡1/ (rtd )
.
1 ¡ de ¡1 / (rtd )
(2.3)
2.2 Circuit Model of Depressing Synapse. In this model of synaptic
depression derived from the circuit implementation, the equation that describes the recovery dynamics of the depressing variable, D, is nonlinear.
This nonlinearity comes about because there is no compact circuitry to implement a linear resistor whose value can be adjusted easily after fabrication.
A transistor can act as a linear resistor as long as the terminal voltages satisfy
certain criteria. Additional circuitry would be needed to satisfy these criteria for a wide range of voltages, and it would increase the size of the circuit.
One alternative is to replace the exponential dynamics in equation 2.1 with
diode dynamics, which is easily obtained with a single diode-connected
transistor. The current through the diode has a nonlinear decay instead of
an exponential decay in the case of a linear resistor.
Because of the different dynamics, the equation describing the recovery
of D (derived from the circuit in the region where a transistor operates in
the subthreshold region or the current is exponential in the gate voltage of
the transistor) can be formulated as
dD
1k
D M (1 ¡ D / ),
dt
(2.4)
where 1 / M is the equivalent of td in equation 2.1 and k (a transistor parameter) is less than 1. The maximum value of D is 1. The update equation
remains as before:
C
¡
) D dD (tsp
).
D(tsp
(2.5)
2.2.1 Circuit. Equations 2.4 and 2.5 are derived from the circuit in Figure 1. The operation of this circuit is described in the caption, and the detailed analysis leading to the differential equations for D is described in the
334
M. Boegerhausen, P. Suter, and S. Liu
Figure 1: Schematic for a depressing synapse circuit and responses to a regular
input spike train. (a) Depressing synapse circuit. The voltage Va determines the
synaptic conductance g, while the synaptic term gD or Isyn is exponential in the
voltage, Vx . The subcircuit consisting of transistors, M1 , M2 , and M3 controls
the dynamics of Isyn. The presynaptic input goes to the gate terminal of M3 ,
which acts like a switch. When there is a presynaptic spike, a quantity of charge
(determined by Vd ) is removed from the node Vx . In between spikes, Vx recovers
to the voltage, Va , through the diode-connected transistor, M1 . When there is no
spike, Vx is around Va . When the presynaptic input comes from a regular spike
train, Vx decreases with each spike and recovers between spikes. It reaches a
steady-state value, as shown in b. During the spike, transistor M4 turns on, and
the synaptic weight current Isyn charges up the membrane potential of the neuron
through the current-mirror circuit consisting of M6 , M7 , and the capacitor C2 .
We can convert the Isyn current source into a synaptic current Id with some gain
and a “time constant” by adjusting the voltage Vgain . The decay dynamics of Id
is given by Id (t) D
Isyn (t D tsp )
1C
AIsyn (tD tsp )
QT
k (Vdd ¡Vgain ) / UT
where QT D C2 UT and A D e
. In a
normal synapse circuit (i.e., without short-term dynamics), Vx is controlled by
an external bias voltage. (b) Input spike train at a frequency of 20 Hz (bottom
curve) and corresponding response Vx (top curve) of the circuit for Vd D 0.3 V.
The diode-connected transistor M1 has nonlinear dynamics. This dynamics is
discussed in the text. The recovery time of the variable D varies depending on
the distance of the present value of Vx from Va . The recovery rate of D increases
for a larger difference between Vx and Va .
appendix. The voltage Vx in Figure 1 codes for gD. The conductance g is
set by Va , while the dynamics of D is set by both Vd and Va . The time taken
for the present value of Vx to return to the value of Va is determined by the
current dynamics of the diode-connected transistor M1 and Va . As shown
in the appendix, the recovery time constant (1 / M) of D is set by Va .
Modeling Short-Term Synaptic Depression in Silicon
335
The synaptic weight is described by the current, Isyn in Figure 1a,
Isyn (t) D Ion ek Vx / UT D gD (t),
where g D Ion ek Va / UT is the synaptic strength, D(t) is
k (Vdd ¡Vx ) / UT
Irf D Iop e
(2.6)
k (V
e
dd ¡Va ) / UT Iop
,
Irf (t)
and
. The recovery time constant (1 / M) of D is set by Va
Iopk ¡(1¡k ) (V ¡Va ) / UT
dd
).
QT e
(M D
The synaptic current, Id to the neuron, is then
a current source Isyn , which lasts for the duration of the pulse width of
the presynaptic spike. However, we can set a longer time constant for the
synaptic current through Vgain . The equation describing this dependence
(that is, the equation of the current in a current-mirror circuit) is given in
the caption of Figure 1.
It is difŽcult to compute a closed-form solution for equation 2.4 for any
value of k (a transistor parameter which is less than 1). This value also
changes under different operating conditions and between transistors fabricated in different processes. Hence, we solve for D(t) in the case of k D 0.5
given that the last spike occurred at t D t0 1 :
dD / dt D M (1 ¡ D2 )
) D(t) D
D
D(t 0 ) C D(t0 )e2Mt ¡ 1 C e2Mt
¡D(t0 ) C D(t0 )e2Mt C 1 C e2Mt
D(t0 ) cosh (Mt) C sinh (Mt)
.
cosh (Mt) C D(t0 ) sinh (Mt)
(2.7)
When D is far from its recovered value of 1, we can approximate its recovery
dynamics by dD / dt D M (irrespective of k ), and solving for D(t), we get
D(t) D Mt C D(t 0 ).
In this regime, D(t) follows a linear trajectory. Note that the same is true of
equation 2.1 when t ¿ td .
3 Neuron Circuit
The dynamics of the neuron circuit of Figure 2 is similar to that of a leaky IF
neuron with a constant leak. The circuit was previously described in Indiveri
(2000) and Liu et al. (2001). It is a modiŽed version of previous designs
(Mead, 1989; Van Schaik, 2001) and also includes the circuitry that models
Žring-rate adaptation (Boahen, 1997a, 1997b) frequently seen in pyramidal
1
Note that if k =1, the equation reduces to equation 2.1.
336
M. Boegerhausen, P. Suter, and S. Liu
C1
Vb
V thresh
Id
V leak
Ileak
Cm
Vo
Vm
M5
V pw
M6
Vo
Vo
Vm
M1
Vo
M2
Vca
M4
M3
V refr
Ipw
Ca
Vt
Spike adaptation
circuitry
Figure 2: Schematic of the leaky integrate-and-Žre neuron circuit. In this Žgure,
Vrefr, sets the refractory period, Vthresh sets the threshold voltage, Vpw sets the
pulse width of the spike, Vleak sets the leak current Ileak, and VCa and Vt set the
output spike adaptation dynamics.
cells in the cortex. The equation describing the depolarization of the soma
potential Vm is
Cm
dV m (t)
D i(t) ¡ Ileak ¡ Iahp ,
dt
Vm (t) < Vthresh ,
(3.1)
where i(t) is the synaptic current to the soma, Ileak is the leakage current, and
Iahp is the after-hyperpolarization potassium (K) current, which causes the
adaptation in the Žring rate of the cells. The synaptic current can be set to a
point current source, that is, i(t) D Idd (t ¡tjk ), or it can have a “time constant”
and gain that is determined by the current-mirror circuit consisting of M6 ,
M7 , and C2 in Figure 1. The time constant and gain are controlled by the
Isyn (tD tsp )
voltage Vgain . The decay dynamics of Id is of the form Id (t) D
AIsyn (tD tsp ) ,
kV
dd ¡Vgain
1C
QT
UT
where QT D C2 UT and A D e
.
The operation of the circuit is as follows. When Vm (t) increases above
Vthresh at ts D t (ts is the time of the output spike), it increases by a step
increment determined by the amplitude of the spike and the capacitive
divider set by C1 and Cm . The output Vo becomes active at this time and
turns on the discharging current path through transistors M5 and M6 . The
time during which Vo remains high, TP , depends on the time taken for Vm to
discharge below Vthresh . In this design, the pulse width TP is determined by
the rate at which Vm is discharged, which in turn depends on the difference
between the input current Id , the leak current Ileak, and the current Ipw . In
other designs, Vm is reset immediately below Vthresh when Vo becomes active
because either the input current is blocked from charging the membrane or
Modeling Short-Term Synaptic Depression in Silicon
337
the current Ipw is much larger than the input current. The refractory period,
TR , is determined by Vrefr , which keeps Vo high so that Id cannot charge up
the membrane. The spike output is taken from the node VNo .
We ignore the Iahp current in determining the Žring rate dependence on
the input current. The time needed for the neuron to charge up to threshold
Vthresh
with the input current is TI D Cm i(t)¡I
. The spiking rate is given by
rD
leak
1
TI C TP C TR .
4 Comparison Between Models
We compare the two models by looking at how D changes in response to
a Poisson-distributed train whose frequency varied from 40 Hz to 1 Hz, as
shown in Figure 3. We used a simple linear differential equation to describe
the dynamics of the membrane potential Vm ,
tm
dV m (t)
D Ri(t) ¡ Vm (t),
dt
where tm is the membrane time constant and i(t) is the synaptic current.
We ran an optimization algorithm on the parameters in the two models
so that the least-square error between the excitatory postsynaptic potential
(EPSP) outputs of both models was at a minimum. In this case, the EPSP
responses were identical (see Figure 3b) and the corresponding D values (see
Figure 3c) were almost identical except in the region where D was close to
the maximum value. We performed the same comparison with Tsodyks and
Markram’s model, and the results were similar. Hence, the circuit model can
be used to describe short-term synaptic depression in a network simulation.
However, the nonlinear recovery dynamics of the circuit model leads a
different functional dependence of the average steady-state EPSP on the
frequency of a regular input spike train.
5 Transient Response
The data in the Žgures in the remainder of this article are obtained from
a fabricated silicon network of aVLSI IF neurons of the type described in
section 3 together with different types of synapses.
We Žrst measured the transient response of the neuron when stimulated
by a 10 Hz spike train through the depressing synapse. We tuned the parameters of the synapse and the leak current so that the membrane potential did
not build up to threshold. These data are shown in Figure 4a for a regular
train and in Figure 4b for a Poisson-distributed train.
The measured EPSP amplitude, DVm can be Žtted with equation 2.6,
DVm (t)
D
dt
dt
(Isyn (t) ¡ Ileak ) D
(gD (t) ¡ Ileak ),
Cm
Cm
(5.1)
338
M. Boegerhausen, P. Suter, and S. Liu
Figure 3: Comparison between the outputs of the two models of depression. An
optimization algorithm was used to determine the parameters of the models
so that the least-square error between the EPSPs from the two models was
at a minimum. The corresponding D distribution is shown in c. (a) Poissondistributed input with an initial frequency of 40 Hz and an end frequency of
1 Hz. (b) The EPSP responses of both models were identical. (c) The D values
were almost identical except in the region when D is close to 1. Parameters used
in the simulations: tm D 20 ms, d D 0.6, k D 0.7, M D 2.20 s ¡1 .
where Cm is the membrane capacitance, Ileak is the leak current, dt is the pulse
width of the presynaptic spike, and D(t) is computed using equation 2.7.
Equation 5.1 Žts the measured data well (see Figure 4).
6 Steady-State Response
The equation describing the dependence of the steady-state values of D on
the presynaptic frequency can easily be determined in the case of a regular
spiking input of rate r by using equations 2.5 and 2.7. From these equations,
we get
p
(¡1 C d) (1 C e2M/ r ) C 4d(¡1 C e2M/ r ) 2 C (1¡d) 2 (1 C e2M/ r )2
Dss D
. (6.1)
2d(¡1 C e2M/ r )
Modeling Short-Term Synaptic Depression in Silicon
339
Figure 4: Transient responses to a (a) 10 Hz regular spike train and a (b) 10 Hz
Poisson-distributed train. The input is the bottom curve of each plot. In a, the
amplitude of the EPSP decreases with each incoming input spike, clearly showing the effect of synaptic depression. In b, the EPSP amplitude depends on the
occurrence of the previous spike. The asterisks are the Žts of the circuit model
to the peak value of each EPSP. The Žts give a d value of 0.79.
340
M. Boegerhausen, P. Suter, and S. Liu
If we use the reduced dynamics expression dD / dt D M, we replace equation 2.7 with equation 3.1 and obtain a simpler expression for Dss :
Dss D
M
.
(1 ¡ d)r
(6.2)
This equation shows that the steady-state D, and, hence, the steady-state
EPSP amplitude, is inversely dependent on the presynaptic rate r. The form
of the curve is similar to the results obtained by Abbott et al. (1997), where
the data can be Žtted with equation 2.3.
From the chip, we measured the steady-state EPSP amplitudes using
the two-input spike distributions changing over a frequency range of 3
Hz to 50 Hz in steps of 1 Hz. Each frequency interval lasted 15 s, and
the EPSP amplitude was averaged in the last 5 s to obtain the steady-state
value. Four separate trials were performed; the resulting mean and the
variance of the measurements are shown in Figure 5 for both the regular train
and the Poisson-distributed train. We Žtted the experimental data using
equations 5.1 and 6.1. The parameters from the Žts to the data in Figure 5a
were then used to generate the Žtted curve to the data from the Poissondistributed train. The values from the Žts give recovery time constants from
1 to 3 s and Dss values varying between 0.02 and 0.04.
7 Role of Synaptic Depression
Different computational roles have been proposed for networks that incorporate synaptic depression. In this section, we describe some measurements
that illustrate the postulated roles of depression.
7.1 Gain Control. Depressing synapses have been implicated in cortical
gain control (Abbott et al., 1997). A depressing synapse acts like a transient
detector to changes in frequency (or a Žrst derivative Žlter) similar to the
way in which the photoreceptor responds with a higher gain to contrast and
is less responsive to background illumination. A synapse with short-term
depression responds equally to equal percentage rate changes in its input
on different Žring rates. We demonstrate the gain control mechanism of
short-term depression by measuring the neuron’s response to step changes
in input frequency from 10 Hz to 20 Hz to 40 Hz. Each step change is the
same rate change in input frequency. These results are shown in Figure 7.1a
for a regular train and in Figure 6b for a Poisson-distributed train. Each
frequency epoch lasted 3 s, so the synaptic strength should have reached
steady state before the next step increase in input frequency. For both Žgures
in Figure 7.1, the top curve shows the response of the neuron when stimulated by the input (bottom curve) through a depressing synapse (top curve)
and a nondepressing synapse (middle curve). We tuned the parameters of
Modeling Short-Term Synaptic Depression in Silicon
341
Figure 5: Dependence of the steady-state EPSP amplitude on the input frequency for different values of depression. The data were measured from the
fabricated circuit. The solid lines are Žts using equations 5.1 and 6.1. The parameters for the Žts are determined using the data in a. (a) Steady-state EPSP
amplitude versus frequency for a regular train. From the retrieved values of d
(between 0.2 and 0.6), we Žnd that Dss varies between 0.02 to 0.04. (b) Steadystate EPSP amplitude versus frequency for a Poisson-distributed train.
342
M. Boegerhausen, P. Suter, and S. Liu
both types of synapses in Figure 6b so that the steady-state Žring rate of the
neuron was similar during the 20 Hz input period.
Figure 6a shows clearly the transient increase in the Žring rate of a neuron when stimulated through a depressing synapse right after each step
increase in input frequency and the subsequent adaptation of its Žring rate
to a steady-state value. The steady-state Žring rate of the neuron with a depressing synapse is less dependent on the absolute input frequency when
compared to the Žring rate of the neuron when stimulated through the
nondepressing synapse. In the latter case, the Žring rate of the neuron is
approximately linear in the input rate.
The data shown in Figure 6a were obtained using a Poisson-distributed
input. One obvious difference in the responses between the depressing and
nondepressing synapse is that the neuron quickly reached threshold for a
10 Hz input for the depressing-synapse case, while the neuron remained
subthreshold in the nondepressing synapse case until the input increased
to 20 Hz. This suggests that a depressing synapse can be used to drive a
neuron quickly to threshold when the membrane potential of the neuron is
far from the threshold.
7.2 Phase Advance. Another feature of the depressing synapse is the
phase advance of the membrane response of the neuron. This advance increases when the depression increases (see Figure 7). This feature was exploited in a model that described the direction-selective responses of visual
cortical neurons (Chance et al., 1998). In this model, the neuron was driven
by the output of a set of lateral geniculate nucleus (LGN) cells through
depressing synapses and through an adjacent spatially shifted set of LGN
cells through nondepressing synapses. We have attempted the same experiment using a neuron on our chip by using spikes recorded from an LGN
Figure 6: Facing page. Response of neuron to a step change in input frequency
(bottom curve) when stimulated through a depressing synapse (top curve) and
a nondepressing synapse (middle curve). The neuron was stimulated for three
frequency intervals (10 Hz to 20 Hz to 40 Hz) lasting 3 s each. (a) Response of
neuron using a regular spiking input. The steady-state Žring rate of the neuron
increased almost linearly with the input frequency when stimulated through the
nondepressing synapse. In the depressing synapse curve, there is a transient increase in the neuron’s Žring rate before the rate adapted to steady state. The
steady-state output is less dependent on the absolute input frequency. (b) Response of neuron using a Poisson-distributed input. The parameters for both
types of synapses were tuned so that the steady-state Žring rates were about the
same at the end of each frequency interval for both synapses. Notice that during
the 10 Hz interval, the neuron quickly built up to threshold if it was stimulated
through the depressing synapse, but it stayed subthreshold when it was driven
through to the nondepressing synapse until the input frequency reached 20 Hz.
Modeling Short-Term Synaptic Depression in Silicon
343
344
M. Boegerhausen, P. Suter, and S. Liu
Figure 7: Phase advance in the membrane response for different levels of depression in the synapse. The neuron was driven through the synapse by a
Poisson-distributed input depicted at the bottom of the Žgure. The neuron shows
the largest phase advance when the synapse has the largest depression value
(Vd D 0.3V has the largest d value in the plot).
cell in the cat visual cortex during stimulation with a drifting sinusoidal
grating (courtesy of K. Martin). An example of the direction-selective response is shown in Figure 8 (Liu, 2001). The direction-selective results were
qualitatively similar to the data in Chance et al. (1998).
8 Conclusion
We described a model of synaptic depression derived from a circuit implementation. This circuit model has nonlinear recovery dynamics for the
depression variable in contrast to current theoretical models of dynamic
synapses. However, it gives qualitatively similar results when compared
to the model of Abbott and colleagues (1997). Measured data from a chip
with aVLSI IF neurons and dynamic synapses show that this network can
be used to simulate the responses of dynamic networks with short-term dynamical synapses. Experimental results suggest that depressing synapses
can be used to drive a neuron quickly up to threshold if its membrane potential is at the resting potential. The silicon networks provide an alternative to
computer simulation of spike-based processing models with different time
constant synapses because they run in real time and the computational time
does not scale with the size of the neuronal network.
Modeling Short-Term Synaptic Depression in Silicon
345
Figure 8: Response to a 1 Hz drifting sinusoidal grating. The input spikes from
the LGN cell are depicted at the bottom of the curve. The top curve shows the
response of the neuron when the grating drifted in the preferred direction. The
sharp excursions at the top of the potential are the output spikes of the neuron.
The middle curve shows the response to the stimulus in the null direction. The
membrane potential did not build up to threshold.
Appendix
The current equation for a p-type Želd-effect transistor (pFET) in subthreshold where the bulk of the transistor is at the voltage Vdd is
Iop ek (Vdd ¡Vg ) / UT (e ¡(Vdd ¡Vs ) / UT ¡ e (Vdd ¡Vd ) / UT ),
where Vg is the voltage at the gate, and Vd and Vs are the voltages at the
drain and source terminals of a transistor; k describes the efŽciency of the
gate in controlling the current in the channel and is less than 1 for a transistor
operating in subthreshold. The voltages in the exponent are normalized in
terms of the thermal voltage, UT . The differential equation for the current in
a diode-capacitor circuit (e.g., M1 and C in Figure 1) when there is no input
current is given by
Iop ek (Vdd ¡Vx ) / UT (e¡(Vdd ¡Va ) / UT ¡ e ¡(Vdd ¡Vx ) / UT ) D C
dV x
.
dt
(A.1)
346
M. Boegerhausen, P. Suter, and S. Liu
By introducing the current variable, Irf D Iop ek (Vdd ¡Vx ) / UT , we solve for the
differential, dI rf :
dI rf D ¡
k
UT
Irf dV x ) dV x D ¡
UT dI rf
.
k Irf
By changing variables from Vx to Irf in equation A.1, we get,
QT
Á
1
d
k
Irf
0
!
"
)
D dt @e¡(Vdd ¡Va / UT
By substituting D D
k (V
e
dd ¡Va ) / UT Iop
Irf
Iop
¡
Irf
#1/k 1
A.
(A.2)
into equation A.2, we get
Iopk ¡(1¡k ) (V ¡Va ) / UT
dD
dd
(1 ¡ D1/k ) D M(1 ¡ D1 /k ),
e
D
dt
QT
(A.3)
Iopk
where M D QT e¡(1¡k ) (Vdd ¡Va ) / UT . In steady state, D D 1.
The update dynamics for D can be inferred from the dynamics of Ir ,
which are described by
Ir (tnC ) D (1 C a)Ir (tn¡ ),
(A.4)
where tnC is the time right after the nth spike and tn¡ is the time right before
the nth spike. The a factor is determined by a voltage parameter and the
pulse width of the presynaptic spike. The corresponding equation for D is
D(tnC ) D dD(tn¡ ),
where d is
1
1C a
(A.5)
in equation 2.5. 2
Acknowledgments
This work was supported in part by the Swiss National Foundation Research
SPP grant. We also acknowledge Kevan Martin, Pamela Baker, and Ora
Ohana for many discussions on dynamic synapses.
2
In equation A.5, for d D 1, a has to be 0. But a is never zero due to leakage currents.
To compute d, we Žrst solve for 1 C a D e (k Idt ) / QT , where I is set by the voltage parameter
Vd , dt is the pulse width of the input spike, and k is a transistor parameter. The smallest I
is the leakage current, so for this value, we approximate a D (k Ir0dt ) / QT .
Modeling Short-Term Synaptic Depression in Silicon
347
References
Abbott, L., Sen, K., Varela, J., & Nelson, S. (1997).Synaptic depression and cortical
gain control. Science, 275, 220–223.
Boahen, K. A. (1997a). The retinomorphic approach: Pixel-parallel adaptive ampliŽcation, Žltering, and quantization. Analog Integrated Circuits and Signal
Processing, 13, 53–68.
Boahen, K. A. (1997b). Retinomorphic vision systems: Reverse engineering the vertebrate retina. Unpublished doctoral dissertation, California Institute of Technology, Pasadena, CA.
Chance, F., Nelson, S., & Abbott, L. (1998). Synaptic depression and the temporal
response characteristics of V1 cells. Journal of Neuroscience, 18, 4785–4799.
Indiveri, G. (2000). Modeling selective attention using a neuromorphic aVLSI
device. Neural Computation, 12, 2857–2880.
Liu, S.-C. (2001, Oct. 19). Simple cortical modeling with aVLSI spiking neurons and
dynamic synapses. Paper presented at the ZNZ Symposium, University of
Zurich.
Liu, S.-C. (in press). Analog VLSI circuits for short-term dynamic synapses.
EURASIP Journal on Applied Signal Processing: Special Issue.
Liu, S.-C., Kramer, J., Indiveri, G., Delbrück, T., Burg, T., & Douglas, R. (2001).
Orientation-selective aVLSI spiking neurons. Neural Networks: Special Issue,
14(6/7), 629–643.
Maass, W., & Zador, A. (1999). Computing and learning with dynamic synapses.
In W. Maass & C. M. Bishop (Eds.), Pulsed neural networks (pp. 157–178).
Cambridge, MA: MIT Press.
Markram, H., Wang, Y., & Tsodyks, M. (1998). Differential signaling via the
same axon of neocortical pyramidal neurons. Proc. Natl. Acad. Sci. USA, 95,
5323–5328.
Matveev, V., & Wang, X. (2000). Differential short-term synaptic plasticity and
transmission of complex spike trains: To depress or to facilitate? Cerebral
Cortex, 10, 1143–1153.
Mead, C. (1989). Analog VLSI and neural systems. Reading, MA: Addison-Wesley.
Natschlager, T., Maass, W., & Zador, A. (2001). EfŽcient temporal processing
with biologically realistic dynamic synapses. Network, 12, 75–87.
Rasche, C., & Hahnloser, R. (2001). Silicon synaptic depression. Biological Cybernetics, 84, 57–62.
Reyes, A., Lujan, R., Rozov, A., Burnashev, N., Somogyi, P., & Sakmann, B. (1998).
Target-cell-speciŽc facilitation and depression in neocortical circuits. Nature
Neuroscience, 1, 279–285.
Senn, W., Segev, I., & Tsodyks, M. (1998). Reading neuronal synchrony with
depressing synapses. Neural Computation, 10, 815–819.
Stratford, K., Tarczy-Hornoch, K., Martin, K., Bannister, N., & Jack, J. (1998).
Excitatory synaptic inputs to spiny stellate cells in cat visual cortex. Nature,
382, 258–261.
Tsodyks, M., & Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proc. Natl.
Acad. Sci. USA, 94(2), 719–723.
348
M. Boegerhausen, P. Suter, and S. Liu
Tsodyks, M., Pawelzik, K., & Markram, H. (1998). Neural networks with dynamic synapses. Neural Computation, 10, 821–835.
Van Schaik, A. (2001). Building blocks for electronic spiking neural networks.
Neural Networks, 14(6/7), 617–628.
Varela, J., Sen, K., Gibson, J., Fost, J., Abbott, L., and Nelson, S. (1997). A quantitative description of short-term plasticity at excitatory synapses in layer 2/3
of rat primary visual cortex. Journal of Neuroscience, 17, 7926–7940.
Received June 14, 2002; accepted July 24, 2002.