Download A current-mode conductance-based silicon neuron for Address

A current-mode conductance-based silicon neuron
for Address-Event neuromorphic systems
Paolo Livi
Giacomo Indiveri
ETH Zurich, Bio Engineering Laboratory
Institute of Neuroinformatics
Department Biosystems, Science and Engineering (BSSE)
University of Zurich, ETH Zurich
Mattenstr. 26, CH-4058 Basel, Switzerland
Winterthurerstr. 190, CH-8057 Zurich, Switzerland
E-mail: [email protected]
E-mail: [email protected]
bstract— Silicon neuron circuits emulate the electrophysiological behavior of real neurons. Many circuits can be
integrated on a single Very Large Scale Integration VLSI) device,
and form large networks of spiking neurons. Connectivity among
neurons can be achieved by using time multiplexing and fast
asynchronous digital circuits. As the basic characteristics of the
silicon neurons are determined at design time, and cannot be
changed after the chip is fabricated, it is crucial to implement a
circuit which represents an accurate model of real neurons, but at
the same time is compact, low-power and compatible with asynchronous logic. Here we present a current-mode conductancebased neuron circuit, with spike-frequency adaptation, refractory
period, and bio-physically realistic dynamics which is compact,
low-power and compatible with fast asynchronous digital circuits.
I. I NTRODUCTION
Many VLSI models of spiking neurons have been developed in the past [1]–[9], and many are still being actively
investigated [10]–[13]. A common goal is to integrate large
numbers of these circuits on single chips, or even full wafers,
and create large networks of neurons, densely interconnected.
In these systems, the strategy used to connect multiple neurons
with each other is to use asynchronous digital circuits that map
and route the spikes as they are generated to other neurons on
different chips or other areas of the same chip/wafer [14],
[15]. It is therefore crucial to develop compact low-power
circuits, that implement faithful models of real neurons, but
that can also produce extremely fast digital pulses required
by the asynchronous circuits that manage the communication
infrastructure.
A family of spiking neuron models that allows us to
implement large, massively parallel networks of neurons is
the Integrate-and-Fire (IDF) model. IDF neurons integrate presynaptic input currents and generate a voltage pulse analogous
to an action potential when the integrated voltage reaches a
spiking threshold. Networks of IDF neurons have been shown
to exhibit a wide range of useful computational properties,
including feature binding, segmentation, pattern recognition,
onset detection, input prediction, etc..
It has been recently argued however that simple IDF
models do not produce a rich enough range of behaviors
useful for investigating the computational properties of large
neural networks, thus compromising the usefulness of ded-
978-1-4244-3828-0/09/$25.00 ©2009 IEEE
icated hardware implementations [13], [16], [17]. Wijekoon
and Dudek recently proposed an alternative phenomenological
VLSI model, loosely based on the Izhikevich model [16],
which can produce a wide range of spiking patterns, using a
very small number of transistors [13]. This circuit is also low
power, but it operates in “scaled–time” (i.e. on microsecond
scale, rather than millisecond scale). Therefore it cannot be
used to implement artificial or hybrid systems that interact with
the world in real–time. In addition both this phenomenological
model, and the vast majority of IDF neuron circuits previously
proposed do not exhibit conductance-based behavior, which
is crucial for implementing bio-physically realistic models of
neurons.
There is a class of VLSI conductance-based silicon neurons
that has been proposed in the past [2], [6], [7]. These indeed
implement faithful models of real neurons, but they use a
considerable amount of silicon real-estate (i.e. many transistors
and large capacitors). A new generation of bio-physically
realistic circuits has been recently proposed [10], [11], which
emulates the dynamics of neuronal proteic channels, reproduces faithful action potential traces, and use a considerable
lower number of transistors per neuron model. But these
circuits do not produce fast digital pulses, and cannot be easily
interfaced to asynchronous digital circuits.
We propose a phenomenological silicon neuron circuit,
based on a variant of a low-power IDF model [18], which
is conductance-based, compact, real–time (with bio-physically
realistic temporal dynamics), and compatible with the asynchronous Address-Event Representation (AER) [19]. The circuit uses a current-mode approach, similar to that proposed
in [9], [12], but rather than using conventional log-domain
filters [20], it uses the diff-pair integrator (DPI) [21] for implementing tunable dynamic conductances. The subthreshold
log-domain circuits used in [9], [12] require p-FETs with
isolated wells, while the DPI filters can be implemented using
both n-type and p-type variants of the circuit, and can be
designed with more compact layouts. An additional advantage
of the DPI circuit over the standard current-mode log-domain
circuit [20], is given by the possibility to control the circuit’s
gain with an additional independent bias voltage. A detailed
analysis of both circuits and advantages of one over the other
is presented in [22].
2898
A. Circuit operation
Fig. 1.
circuit.
Schematic diagram of the Differential-Pair Integrator (DPI) neuron
In addition to the conductance-based behavior, the circuit implements a series of functionalities which reproduce
many important features observed in real neurons: a positivefeedback mechanism, which reproduces the effect of Sodium
activation and inactivation channels in real neurons; a refractory period mechanism for limiting the maximum possible firing rate of the neuron; and a spike-frequency adaptation mechanism, which effectively introduces a second slow variable in
the model, potentially allowing for subthreshold resonances
and oscillatory behaviors [16].
The circuit’s positive feedback mechanism drastically reduces the switching time of the neuron’s MOSFETs, and
makes the model extremely low-power. In the next Section we
describe the circuit and explain its operational principles; in
Section III we present experimental results; and in Section IV
we draw the conclusions and describe future work on it.
II. T HE SILICON NEURON CIRCUIT
The silicon neuron circuit schematic is shown in Fig. 1. The
circuit comprises:
• An input DPI circuit [21] (M1-M4,M22), which models
the neuron’s leak conductance, and provides exponential
subthreshold dynamics in response to constant input
currents.
• An integrating capacitor Cmem , which represents the neuron’s membrane capacitance.
• A second instance of a DPI (M5-M10), which models the
neuron’s Calcium conductance, and implements the spike
frequency adaptation mechanism.
• An inverting amplifier (M15-M17) with positive feedback
(M11-M13).
• A starved inverter with controllable slew rate (M18-M21),
which can be used to set arbitrary refractory periods.
• The neuron’s reset transistor (M14).
The read-out transistor M22 is used in simulations and for
the analytical derivations, but was actually not included in the
final layout of the circuit.
The input current Iin j is summed to a constant background
current (set by Vrest ) which can be used to model spontaneous
activity. Input currents are integrated by the DPI, increasing
the membrane voltage Vmem . As Vmem approaches the switching voltage of the inverting amplifier, the feedback current
I f b starts to flow through M11-M13, increasing Vmem more
sharply. This positive feedback has the effect of making the
amplifier M15-M16 switch very rapidly, reducing dramatically
its power dissipation. When Vmem increases enough to make
the first inverter switch the voltage VO1 is brought to ground
and Vspk is driven to Vdd . Then the membrane capacitor Cmem
is discharged back to ground through the reset transistor M14,
VO1 rises back sharply to Vdd , and the voltage Vspk is slowly
reset to zero, at a rate controlled by the bias voltage Vr f and
the size of Cr f . The neuron’s refractory period lasts as long as
Vspk is sufficiently high to keep the reset transistor on. During
the spike emission period (while VO1 is low), a current with
amplitude set by Vadap is sourced into the adaptation DPI, with
a gain set by the gate bias voltage Vthra , and a time constant
set by Valk . The adaptation current Iadap increases with every
spike, following the same first-order dynamics of Imem . As a
consequence, given the negative-feedback property of Iadap ,
the neuron’s response to a step input current is characterized
by an initial output firing rate proportional to the input current,
which gradually decreases until an equilibrium is reached, thus
reproducing the spike-frequency adaptation behavior observed
in real neurons.
The subthreshold behavior of the neuron can be derived
analytically, by assuming that the relevant transistors operate
in the weak-inversion (or subthreshold) regime [23]. For
this analysis we neglect the adaptation current Iadap , as this
becomes non-negligible only after the first spike. In weakinversion, the drain current of a saturated n-FET changes
exponentially with its gate voltage [23]. In particular:
Imem = I0 e
κVmem
UT
(1)
where I0 is the n-FET’s leakage current, κ is the subthreshold slope factor and UT is the thermal voltage [23]. The
subthreshold branch current IM3 of the differential pair is given
by:
IM3 = Iin
e
κVdd −Vmem )
UT
(2)
κVdd −Vmem )
κVdd −Vthr )
UT
e
+ e UT
where Iin = Irest +Iin j . If we assume that the subthreshold slope
coefficients κ of n and p-FETs are equal, we arbitrarily define
κVthr
Ig = I0 e UT , and take into account eq. (1), we can rewrite IM3
as:
1
IM3 = Iin
(3)
1 + Imem
Ig
Kirchhoff’s current law on the Vmem node yields:
Cmem
2899
d
Vmem = IM3 − Iτ + I f b
dt
(4)
0.35
0.3
0.25
VCa (V)
0.4
0.2
0.15
0.1
0.05
0
0
20
40
60
Time (ms)
80
100
80
100
(a)
Fig. 2. Silicon neuron membrane current Imem in response to a constant step
input current, for different values of the Vthr bias setting.
14
12
κ 2 Vmem
κ+1 UT
10
κVτ
Imem (μA)
where I f b ≈ IM16 = I0 e
and Iτ = I0 e UT .
If we differentiate eq. (1) with respect to Vmem and combine
it with the eq. (4) we obtain:
IM3 I f b
d
(5)
τ Imem = −Imem 1 −
−
dt
Iτ
Iτ
Cmem
where τ = UTκI
. Replacing IM3 from eq. (3) into eq. (5)
τ
yields:
4
0
−2
0
20
40
60
Time (ms)
(b)
(6)
g
This is a first-order non-linear differential equation. However, for small values of Vmem (e.g. at the beginning of an
action potential) the effect of the DPI dominates on the positive
feedback, and we can neglect the current I f b . Moreover, the
I
right-hand term of eq. (6) can be reduced to Iin Iτg , if Ig Imem .
In these conditions eq. (6) simplifies to:
Ig
d
Imem + Imem = Iin
dt
Iτ
6
2
I
mem
Ifb
d
I
τ Imem + Imem 1 −
) = Iin tauImem
dt
Iτ
1+ I
τ
8
(7)
Thus, for small values of Vmem , and sufficiently large values
of Imem , such that Imem Ig the silicon neuron exhibits a
classical RC-filter type behavior.
III. E XPERIMENTAL R ESULTS
The results described here were obtained using SPICE
simulations. However a small 445 194mm2 prototype chip,
containing 32 of these neurons, has already been designed
and fabricated using a 035μm AMS CMOS technology.
The SPICE simulations were carried out using 035μm AMS
process parameters and 33V power supply setting. In Fig. 2
we plot the transient simulations results, for a step input
current and different settings of the Vthr bias parameter. There
are two important aspects to note in this data. The first one
is the RC profile of the membrane current Imem , when the
current is low; this effect is due to the charge phase of the
DPI circuit and its response characteristic is clearly visible,
until the effect of the positive feedback starts to dominate and
Fig. 3. Spike-frequency adaptation: (a) adaptation internal signal related to
the Calcium concentration present in real neurons; (b) membrane “potential”
signal traces in response to a step input current
finally generates a spike. The second aspect we would like to
point out is the effect of the bias voltage Vthr : higher values of
this bias increase the value of the DPI’s steady state output,
as predicted by eq. (7). The more this bias is increased, the
more current is injected into the integrating capacitor Cmem ,
resulting in a sharper growing of the membrane current. A
sharper increase of this current means that the Vmem voltage
reaches the inverting amplifier’s threshold voltage in a shorter
time, resulting in an earlier spike generation. This can clearly
be seen in Fig. 2, where the neuron membrane current Imem is
plotted for different values of the bias Vthr , in response to the
same step input current.
As mentioned in Section I, an important feature of this
circuit is the presence of a second, slower variable (e.g.
VCa ), which allows the circuit to implement a spike frequency
adaptation mechanism. We activated the adaptation DPI (M5M10) by decreasing the value of Vadap below Vdd and carried
out transient simulations, observing the behavior of the neuron
for multiple output spikes.
In Fig. 3(a) we show the Vca voltage measured in these
simulations: the voltage steadily increases with every spike,
until it reaches an equilibrium, at which the neuron’s output
spike frequency is maximally reduced. In Fig. 3(b) we plot the
membrane current Imem as a function of time, measured in the
same simulation run. As shown, the neuron’s spike frequency
2900
TABLE I
S ILICON NEURON CIRCUIT SPECIFICATIONS .
Cmem area
Cmem capacitance (MOSCAP)
Silicon neuron layout area
Supply voltage
Power consumption/spike (300ns pulse)
Power consumption/spike (100ms, including integration phase)
100μm2
05pF
913μm2
33V
7pJ
267pJ
decreases as VCa increases, until the steady state is reached.
SPICE simulations were carried out also to evaluate the
circuit’s power dissipation characteristics. The digital spikes
produced by the neuron (e.g. derived from the VO1 node) are
extremely narrow, lasting about 300ns. The circuit’s simulated
average power consumption measured during this period is
approximately 7pJ/spike. This measure is commonly used to
describe circuit performance in the literature (e.g. Wijekoon
and Dudek report 85pJ/spike [13]), and to our knowledge,
this is the best figure ever reported. However, a more realistic
measure is the average power dissipation measured during
the whole current integration and action-potential generation
phase. For example, given an average firing rate of 10Hz, the
power dissipation should be measured over 100ms (e.g. from
the end of one spike, to the end of the subsequent spike). In
this case, the (simulated) average power consumption of our
neuron circuit is approximately 267pJ.
The power consumption specifications and other characteristics of the circuit, for the specific implementation made using
a standard 4-metal, double-poly 035μm CMOS process, are
summarize in Table I.
IV. C ONCLUSIONS AND OUTLOOK
We designed and implemented a low-power current-mode
conductance-based neuron circuit, with refractory period and
spike frequency adaptation mechanisms. We described its
properties by means of formal analysis and SPICE simulations.
The results shown in each case are consistent with each other
and demonstrate the expected RC-type response to a constant
current injection, in the membrane current temporal profile.
Although we have not yet demonstrated that the proposed
circuits can produce oscillatory behaviors, such as bursting
activation patterns, the positive feedback and the spike frequency adaptation mechanisms implemented make this VLSI
model equivalent to the “adaptive exponential integrate-andfire” (aEIF) model recently proposed in [17]. In [17] the
authors demonstrate that aEIF models can accurately predict
the spike trains of detailed Hodgkin-Huxley type models,
driven by realistic conductance-based synaptic inputs. Therefore we argue that the circuit described in this work can
implement faithful models of real neurons, while at the same
time satisfying the compactness, low-power, and compatibility
with asynchronous logic constraints.
ACKNOWLEDGMENT
We would like to acknowledge Elisabetta Chicca for guidance and support during the design and layout phase of the
chip produced, and Andre van Schaik for fruitful discussions
on integrate-and-fire neurons. This work was supported by the
DAISY (FP6-2005-015803) EU grant.
R EFERENCES
[1] C. Mead, Analog VLSI and Neural Systems. Reading, MA: AddisonWesley, 1989.
[2] M. Mahowald and R. Douglas, “A silicon neuron,” Nature, vol. 354, pp.
515–518, 1991.
[3] A. van Schaik, “Building blocks for electronic spiking neural networks,”
Neural Networks, vol. 14, no. 6–7, pp. 617–628, Jul–Sep 2001.
[4] K. Hynna and K. Boahen, “Space–rate coding in an adaptive silicon
neuron,” Neural Networks, vol. 14, pp. 645–656, 2001.
[5] G. Indiveri, “A low-power adaptive integrate-and-fire neuron circuit,” in
Proc. IEEE International Symposium on Circuits and Systems. IEEE,
May 2003, pp. IV–820–IV–823.
[6] L. Alvado, J. Tomas, S. Saighi, S. Renaud-Le Masson, T. Bal, A. Destexhe, and G. Le Masson, “Hardware computation of conductance-based
neuron models,” Neurocomputing, vol. 58–60, pp. 109–115, 2004.
[7] M. Simoni, G. Cymbalyuk, M. Sorensen, and S. Calabrese, R.L. DeWeerth, “A multiconductance silicon neuron with biologically matched
dynamics,” IEEE Transactions on Biomedical Engineering, vol. 51,
no. 2, pp. 342–354, February 2004.
[8] J. Schemmel, K. Meier, and E. Mueller, “A new VLSI model of neural
microcircuits including spike time dependent plasticity,” in Proceedings
of the IEEE International Joint Conference on Neural Networks, vol. 3.
IEEE, July 2004, pp. 1711–1716.
[9] J. Arthur and K. Boahen, “Recurrently connected silicon neurons with
active dendrites for one-shot learning,” in IEEE International Joint
Conference on Neural Networks, vol. 3, July 2004, pp. 1699–1704.
[10] E. Farquhar and P. Hasler, “A bio-physically inspired silicon neuron,”
IEEE Transactions on Circuits and Systems???I, vol. 52, no. 3, pp. 477–
488, March 2005.
[11] K. M. Hynna and K. Boahen, “Neuronal ion-channel dynamics in silicon,” in 2006 IEEE International Symposium on Circuits and Systems,
May 2006, pp. 3614–3617.
[12] J. Arthur and K. Boahen, “Synchrony in silicon: The gamma rhythm,”
IEEE Transactions on Neural Networks, vol. 18, pp. 1815–1825, 2007.
[13] J. Wijekoon and P. Dudek, “Compact silicon neuron circuit with spiking
and bursting behaviour,” Neural Networks, vol. 21, no. 2–3, pp. 524–534,
March–April 2008.
[14] E. Chicca, A. M. Whatley, V. Dante, P. Lichtsteiner, T. Delbrück,
P. Del Giudice, R. J. Douglas, and G. Indiveri, “A multi-chip pulsebased neuromorphic infrastructure and its application to a model of
orientation selectivity,” IEEE Transactions on Circuits and Systems I,
Regular Papers, vol. 5, no. 54, pp. 981–993, 2007.
[15] J. Schemmel, J. Fieres, and K. Meier, “Wafer-scale integration of
analog neural networks,” in Proceedings of the IEEE International Joint
Conference on Neural Networks, 2008, (In Press).
[16] E. Izhikevich, “Simple model of spiking neurons,” IEEE Transactions
on Neural Networks, vol. 14, no. 6, pp. 1569–1572, 2003.
[17] R. Brette and W. Gerstner, “Adaptive exponential integrate-and-fire
model as an effective description of neuronal activity,” Journal of
Neurophysiology, vol. 94, pp. 3637–3642, 2005.
[18] G. Indiveri, E. Chicca, and R. Douglas, “A VLSI array of low-power
spiking neurons and bistable synapses with spike–timing dependent
plasticity,” IEEE Transactions on Neural Networks, vol. 17, no. 1, pp.
211–221, Jan 2006.
[19] K. A. Boahen, “Point-to-point connectivity between neuromorphic chips
using address-events,” IEEE Transactions on Circuits and Systems II,
vol. 47, no. 5, pp. 416–34, 2000.
[20] D. R. Frey, “Log-domain filtering: An approach to current-mode filtering,” IEE Proceedings G: Circuits, Devices and Systems, vol. 140, no. 6,
pp. 406–416, December 1993.
[21] C. Bartolozzi, S. Mitra, and G. Indiveri, “An ultra low power current–
mode filter for neuromorphic systems and biomedical signal processing,”
in IEEE Proceedings on Biomedical Circuits and Systems 4BioCAS06),
2006, pp. 130–133.
[22] C. Bartolozzi and G. Indiveri, “Synaptic dynamics in analog VLSI,”
Neural Computation, vol. 19, no. 10, pp. 2581–2603, Oct 2007.
[23] S.-C. Liu, J. Kramer, G. Indiveri, T. Delbrück, and R. Douglas, Analog
VLSI:Circuits and Principles. MIT Press, 2002.
2901