Download High-Speed and High-Precision Current Winner-Take

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—II: EXPRESS BRIEFS, VOL. 52, NO. 3, MARCH 2005
131
High-Speed and High-Precision Current
Winner-Take-All Circuit
Alexander Fish, Student Member, IEEE, Vadim Milrud, and Orly Yadid-Pecht, Senior Member, IEEE
Abstract—A CMOS high-performance current-mode winnertake-all circuit is presented. The circuit employs a novel technique
for inhibitory and excitatory feedbacks based on input currents
average computation, achieving both high speed and high precision. The circuit is designed for operation with a wide range
of input current values, allowing its integration with circuits
operating both in subthreshold and in strong inversion regions.
Two circuits, each for a different range of input currents, have
been implemented in a standard 0.35- m CMOS process available
through MOSIS and are operated via a 3.3-V supply. Their operation is discussed, simulation results are reported and preliminary
measurements from a test chip are presented.
Index Terms—Analog circuits, analog integrated circuits,
CMOS analog integrated circuits, neural networks, winner take
all (WTA).
I. INTRODUCTION
A
winner-take-all (WTA) circuit, which identifies the
highest signal intensity among multiple inputs, is one of
the most important building blocks in neural networks, fuzzy
systems and nonlinear filters. Many WTA circuit implementations have been proposed in the literature [1]–[19]. The MOS
implementation of the WTA function was first introduced by
Lazzaro et al. [1]. This very compact circuit optimizes power
consumption and silicon area usage. It is asynchronous, processes all input currents in parallel and provides output voltages
in real time. The first true current-mode (CM) WTA circuit,
producing an output current that is proportional to the value
of the winning current, was introduced by Andreou et al. [2]
and Boahen et al. [3]. In 1993, the use of positive feedback to
improve the performance of a CM WTA system was reported
by Pouliquen et al. [4]. Several modifications to Lazzaro’s
design have been suggested in the past [5]–[7]. The circuit has
been modified by Starzyk and Fang [5] by improving precision
and speed performance. In 1995, DeWeerth and Morris [6]
have added distributed hysteresis using a resistive network.
Distributed hysteresis allows the winning input to shift between
adjacent locations maintaining its winning status, without
having to reset the network. Additional modifications that
endow the Lazzaro’s WTA with hysteretic and lateral inhibition
and excitation properties have been proposed by Indiveri [7].
Voltage-mode WTA works based on Lazzaro’s circuit were
Manuscript received March 9, 2004; revised July 13, 2004. This paper was
recommended by Associate Editor A. G. Andreou.
A. Fish and V. Milrud are with the VLSI Systems Center, Beer-Sheva 84105,
Israel (e-mail: [email protected]; [email protected]).
O. Yadid-Pecht is with VLSI Systems Center, Beer-Sheva 84105, Israel, and
also with the Department of Electrical and Computer Engineering, University
of Calgary, AB T2N1N4, Canada (e-mail: [email protected]).
Digital Object Identifier 10.1109/TCSII.2004.842062
presented [8], [9]. In 1995, this circuit was modified with
feedback and inhibition by Wilson and DeWeerth [8] and later,
in 1999, by Kalim and Wilson [9]. The performances of all
WTA circuits can be measured in terms of speed, accuracy
and power consumption. For example, some of them were designed to achieve high speed and high accuracy in high-speed,
high-precision applications [10], [11], while [12], [13] were
designed for pulse-coded neural networks.
The fact that the WTA operation is fully parallel and can be
distributed makes these circuits very useful in visual attention
and tracking systems, where the localization of the most salient
regions in an image and selection of salient targets in the field
of view (FOV) are very important tasks. In these systems WTA
networks are able to select and lock onto the input with the
strongest amplitude, and to track it as it shifts from one pixel
to its neighbor [20]–[29].
Although WTA circuits are very useful in many applications
and are very popular, they suffer from matching problems, especially in systems with a large number of inputs. A complete analysis of device mismatch limitations in large WTA circuits was
first presented by Kumar et al. in 1993 [30], [31] and was a subject of further research in the later works [10], [32]. A floatinggate calibration technique to improve the performance/matching
of a WTA circuit was presented by Miwa et al. [33] in 1994.
This paper presents a novel high-speed and high-precision
CM WTA circuit. The circuit employs inhibitory and local excitatory feedbacks based on input currents average computation,
enhancing precision and speed performance of the circuit. Local
excitatory feedback provides a hysteretic mechanism that prevents the selection of other potential winners unless they are
stronger than the selected one by a set hysteretic current. The
proposed circuit can be useful for integration with circuits operating in the strong inversion region and supplying input currents
of 3 –50 A, as well as for subthreshold applications with inputs of 0–50 nA. Not as the previously presented WTA circuits,
the proposed circuit achieves a very high speed [32 ns for high
currents of 3–50 A—(measured) and 34 ns for subthreshold
currents—(simulated)] in case when a very small difference between two input currents is applied (30 nA for high currents
and 1.8 nA for subthreshold applications). These circuit performances are the direct result from very strong feedbacks applied
in the circuit, causing however, higher power dissipation (87.5
W for high-input currents and 22.5 W for subthreshold currents), compared to existing low-performance WTA circuits, operating in subthreshold and optimizing power consumption [1],
[5], [7]. The power consumption of the proposed circuit can be
significantly reduced by decreasing the feedbacks values, while
degrading the circuit performance.
1057-7130/$20.00 © 2005 IEEE
132
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—II: EXPRESS BRIEFS, VOL. 52, NO. 3, MARCH 2005
Fig. 1. Cells 1 and k (out of n) of the proposed WTA circuit.
Section II presents the proposed circuit architecture and describes its operation. The performance of the implemented circuits including simulation results, preliminary measurements
from a test chip, as well a comparison to the existing WTA circuits are shown in Section III. Section IV concludes the paper.
II. CIRCUIT DESCRIPTION
Fig. 1 shows cells and (out of the interacting cells) of the
,
WTA circuit. Cell receives a unidirectional input current
. This output has a high
and produces an output voltage
is identified as winner and
digital value if the input current
low, otherwise. The WTA circuit operates as follows: the drains
transistors of all cells of the array are connected to the
of
transistors by a single common wire with voltage
drains of
. The circuit starts competition by applying
“1” for
a short period of time. This way the excitatory feedback
and inhibitory feedback
are cancelled. Assuming that
“1” is applied, the
all cells in the array are identical and
, through
, is equal to the average of all input
current
currents of the array, neglecting small deviations in the referis copied to
by the pMOS curenced input currents.
and
) and is compared with the input current mirror (
copied by the nMOS current mirror (
and
).
rent
If
then
, assuming the same drivof
and
transistors. An increase in
ability factor
relative to
causes a decrease in
input current
due to the Early effect. This way, during the reset phase, input
currents of all cells are compared to the average of all input
for every
currents of the array, producing a unique output
cell. The cell having the highest input current value produces
voltage.
the smallest
“0”, the exWith the completion of the reset phase, i.e.
and the inhibitory feedback
citatory feedback
are produced. The
node inputs to the gate of
pMOS
(highest input current)
transistor; thus, the cell with smaller
produces higher current
through
and
. This curand
), crerent is copied by the nMOS current mirror (
. On the other hand,
is
ating the excitatory feedback
and
), resulting
copied by the nMOS current mirror (
in inhibitory feedback
. The
is added to the
flowing through
and
is added to the average of
transistor to COM node,
all input current by connection
value. This way, every cell produces a new
increasing the
voltage value, according to the comparison between curand a new value of
current, that is now
rent
given by
(1)
where
is the average of all input currents of the array and
N is the number of array cells. For the cell, having the highest
and
input current, the difference between
growths, decreasing
value.
and
tranFor example, for the same dimensions of
is equal to
. The difference
sistors, the value of
between the input current
of the cell and
without
feedback (during reset phase) can be calculated as
(2)
FISH et al.: HIGH-SPEED AND HIGH-PRECISION CURRENT WTA CIRCUIT
After feedback is applied, the difference
133
is given by
(3)
because
, (3) can be rewritten as
(4)
Thus, the difference between
and
Fig. 2.
is given by
Transient response of WTA circuit.
TABLE I
SIMULATION RESULTS OF FABRICATED CIRCUITS
(5)
is the average of excitatory feedback currents
where
of all cells.
It can be easily seen, that if cell has the highest input and
thus the maximum
, then
, causing
to decrease. A decrease in
causes
to increase and
so on. At the same time, cells, having small inputs have
, causing their
to be increased. This computation
phase is finished when only one cell is identified as a winner,
“1” at the inverter output. All other cells
producing
“0”.
are identified as losers with
As was mentioned above, excitatory feedback provides a hysteretic mechanism that prevents the selection of other potential
winners unless they are stronger than the selected one by a set
hysteretic current. The value of this hysteretic current can be
calculated in the following way: the excitatory and inhibitory
feedbacks of the all looser cells are approximately equal to zero.
Thus, assuming cell is a winner, the hysteretic current value
can be expressed as
(6)
Note, that in some applications (operating in strong inversion)
this hysteresis can be useful, allowing circuit operation without
need to reset, while in others, where small currents are input to
the circuit, a reset of the circuit is required each time before the
competition start.
III. PERFORMANCE, SIMULATION AND PRELIMINARY
MEASUREMENTS FROM TEST CHIP
Two circuits, each designed for a different range of input currents, having eight cells were designed and fabricated in the
0.35- m, n-well, 4-metal, CMOS technology process supported
by MOSIS. To verify the above analysis, the performance of
both WTA circuits has been simulated with the SPECTRES Cadence simulator using TSMC 0.35- m CMOS process parameters. The preliminary measurements from a test chip were performed for the high-input currents. The supply voltage is 3.3 V.
A. Simulation Results
Fig. 2 shows an example of the transient responses of
voltages for two cells. For this simulation
A and
A, while
have different values, smaller
. Because
is greater than
, cell is supthan
posed to be the winner and cell 2 the loser. Two phases can be
clearly identified in Fig. 2: the reset phase, where excitatory and
inhibitory feedbacks are cancelled, and a computation phase,
and
are
where the winner is found. It can be seen that
constant during the reset phase. After the feedbacks are applied,
rapidly falls, causing cell to be a winner. Note, that the
presented voltages are the cell outputs before the inverter, thus
the circuit real response time is faster than indicated in Fig. 2.
Table I summarizes the main characteristics of the circuit, as
were carried out by simulations. Note that mismatch problems
between different devices are not taken in account in these simulation results, but can be clearly identified by measurements of
a test chip, as shown in the next subsection. Monte Carlo simulations and analysis of the proposed circuit operation with a
large number of inputs are subjects for further research.
B. Preliminary Measurements From a Test Chip
To determine the properties of the fabricated WTA circuit, a
specific test board has been designed. This board is designed to
provide all control signals for the circuit as well high-precision
currents in very wide range. At this stage the board set-up is not
finished yet, therefore only preliminary measurements for high
input currents results are provided. Generally, there is a good
134
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—II: EXPRESS BRIEFS, VOL. 52, NO. 3, MARCH 2005
TABLE II
MAIN CHARACTERISTICS OF FABRICATED CIRCUITS WITH COMPARISON TO EXISTING WTA IMPLEMENTATIONS
agreement between the measurement process and the simulation
process. Two main sets of measurements where applied on the
test chip to determine its properties: 1) “typical” delay measurements—during these measurements the difference between the
two highest closest input currents was about 10% (the same conditions as presented in simulation, shown in Fig. 2), and 2) precision measurements and delay measurements in case when the
difference between two highest closest input currents is equal
to the value of measured precision. Note, that this is the worst
case for delay measurements. In order to check the influence of
the device mismatch on the circuit performance, the two most
and
), while
had
far inputs were tested (
different values, smaller than
and
. Following the test
chip measurements, the measured mismatch is equivalent to the
difference of 10 nA in input currents. Table II summarizes the
main characteristics of the fabricated circuit, comparing the proposed circuit with existing WTA circuits [5], [18], [32].
IV. CONCLUSION
We have presented a high-speed and high-precision CM
WTA circuit, described its operation and compared it to the
WTA circuits previously published in the literature. A prototype
chip with two WTA circuits, each for a different range of input
currents, have been implemented in a standard 0.35- m CMOS
process available through MOSIS. The circuit employs inhibitory and local excitatory feedbacks based on input currents
average computation. The simulation results and preliminary
measurements from a test chip show that the proposed circuit
achieves very high speed and high precision and is suitable for
a wide range of input currents.
Subthreshold matching issues (including Monte Carlo simulations), circuit operation with a large number of inputs are subjects for further research.
ACKNOWLEDGMENT
The authors wish to thank A. Slepoy for his help with the
simulations and layout for this work.
REFERENCES
[1] J. Lazzaro, S. Ryckebusch, M. A. Mahowald, and C. A. Mead,
Winner-Take-All Networks of O(n) Complexity, D. S. Touretzky,
Ed. San Mateo, CA: Morgan Kaufmann, 1989, vol. 1, pp. 703–711.
[2] A. G. Andreou, K. A. Boahen, A. Pavasovic, P. O. Pouliquen, R. E.
Jenkins, and K. Strohbehn, “Current-mode subthreshold MOS circuits
for analog VLSI neural systems,” IEEE Trans. Neural Netw., vol. 2, no.
2, pp. 205–213, Mar. 1991.
[3] K. A. Boahen, A. G. Andreou, P. O. Pouliquen, and R. E. Jenkins,
“Current-mode based analog circuits for synthetic neural systems,” U.S.
Patent 5 206 541, Apr. 27, 1993.
[4] P. O. Pouliquen, A. G. Andreou, K. Strohbehn, and R. E. Jenkins, “An
associative memory integrated system for character recognition,” in
Proc. 36th Midwest Symp. Circuits Systems, Detroit, MI, Aug. 1993,
pp. 762–765.
[5] J. A. Startzyk and X. Fang, “CMOS current-mode winner-take-all circuit
with both excitatory and inhibitory feedback,” Electron. Lett., vol. 29,
no. 10, pp. 908–910, 1993.
[6] S. P. DeWeerth and T. G. Morris, “CMOS current-mode winner-take-all
circuit with distributed hysteresis,” Electron. Lett., vol. 31, no. 13, pp.
1051–1053, 1995.
[7] G. Indiveri, “A current-mode hysteretic winner-take-all network, with
excitatory and inhibitory coupling,” Analog Integr. Circuits Signal
Process., vol. 28, pp. 279–291, 2001.
FISH et al.: HIGH-SPEED AND HIGH-PRECISION CURRENT WTA CIRCUIT
[8] D. M. Wilson and S. P. DeWeerth, “Winning isn’t everything,” in Proc.
IEEE ISCAS’95, 1995, pp. 105–108.
[9] R. Kalim and D. M. Wilson, “Semi-parallel rank-order filtering in analog
VLSI,” in Proc. IEEE ISCAS’99, vol. 2, 1999, pp. 232–235.
[10] T. Serrano and B. Linares-Barranco, “A modular current-mode highprecision winner-take-all circuit,” IEEE Trans. Circuits Syst. II, vol. 42,
no. 2, pp. 132–134, Feb. 1995.
[11] A. Nakada, M. Konda, T. Morimoto, T. Yonezawa, T. Shibata, and
T. Ohmi, “Fully-parallel VLSI implementation of vector quantization
processor using neuron-MOS technology,” IEICE Trans. Electron., vol.
E82C, no. 9, pp. 1730–1738, Sep. 1999.
[12] J. L. Meador and P. D. Hylander, “Pulse coded winner-take-all networks,” in Silicon Implementation of Pulse Coded Neural Networks, M.
E. Zaghloul, J. L. Meador, and R. W. Newcomb, Eds. Norwell, MA:
Kluwer, 1994, ch. 5, pp. 79–99.
[13] E. I. ElMasry, H. K. Yang, and M. A. Yakout, “Implementations of artificial neural networks using current-mode pulse width modulation technique,” IEEE Trans. Neural Netw., vol. 8, no. 3, pp. 532–548, May 1997.
[14] J. Choi and B. J. Sheu, “A high-precision VLSI winner-take-all circuit
for self-organizing neural networks,” IEEE J. Solid-State Circuits, vol.
28, no. 5, pp. 576–584, May 1993.
[15] H. M. Yu, R. S. Miyaoka, and T. K. Lewellen, “A high-speed and
high-precision winner-select-output (WSO) ASIC,” IEEE Trans. Nuclear Sci., vol. 45, no. 3, pp. 772–776, Jun. 1998.
[16] K. T. Lau and S. T. Lee, “A CMOS winner-takes-all circuit for self-organizing neural networks,” Int. J. Electron., vol. 84, no. 2, pp. 131–136,
1998.
[17] Y. He and E. Sanchez-Sinencio, “Min-net winner-take-all CMOS implementation,” Electron. Lett., vol. 29, no. 14, p. 3, 1993.
[18] A. Demosthenous, S. Smedley, and J. Taylor, “A CMOS analog winnertake-all network for large-scale applications,” IEEE Trans. Circuits Syst.
I, vol. 45, no. 3, pp. 300–304, Mar. 1998.
[19] P. O. Pouliquen, A. G. Andreou, and K. Strohbehn, “Winner-takes-all
associative memory: a hamming distance vector quantizer,” in Analog
Integr. Circuits Signal Process.. Norwell, MA: Kluwer, 1997, vol. 13,
pp. 211–222.
[20] Y. Fang, M. A. Cohen, and T. G. Kincaid, “Dynamics of a winnertake-all neural network,” Neural Netw., vol. 9, no. 7, pp. 1141–1154,
1996.
[21] G. Indiveri, “A neuromorphic VLSI device for implementing 2D selective attention systems,” IEEE Trans. Neural Netw., vol. 12, no. 6, pp.
1455–1463, Nov. 2001.
135
[22] T. G. Morris, T. K. Horiuchi, and P. DeWeerth, “Object-based selection
within an analog VLSI visual attention system,” IEEE Trans. Circuits
Syst. II, Analog Digit. Signal Process., vol. 45, no. 12, pp. 1564–1572,
Dec. 1998.
[23] T. G. Moris and S. P. DeWeerth, “A smart-scanning analog VLSI visualattention system,” Analog Integr. Circuits Signal Process., vol. 21, pp.
67–78, 1999.
[24]
, “Analog VLSI excitatory feedback circuits for attentional shifts
and tracking,” Analog Integr. Circuits Signal Process., vol. 13, no. 1/2,
pp. 79–92, 1997.
[25] T. Horiuchi and E. Niebur, “Conjunction search using a 1-d analog VLSI
based attentional search/tracking chip,” in Proc. 1999 Conf. Advanced
Research VLSI, 1999, pp. 276–290.
[26] T. G. Moris, S. P. DeWeerth, T. K. Horiuchi, and C. Koch, “Analog VLSI
circuits for attention-based, visual tracking,” in Proc. Neural Information Processing System Conf., Denver, CO, Dec. 1996.
[27] G. Indiveri, “Neuromorphic analog VLSI sensor for visual tracking: circuits and application examples,” IEEE Trans. Circuits Syst. II, Analog
Digit. Signal Process., vol. 46, no. 11, pp. 1337–1347, Nov. 1999.
[28] C. S. Wilson, T. G. Morris, and P. DeWeerth, “A two-dimensional, object-based analog VLSI visual attention system,” in Proc. 20th Anniversary Conference on Advanced Research in VLSI, vol. 20, Mar. 1999, pp.
291–308.
[29] V. Brajovic and T. Kanade, “Computational sensor for visual tracking
with attention,” IEEE J. Solid-State Circuits, vol. 33, no. 8, Aug. 1998.
[30] N. Kumar, P. O. Pouliquen, and A. G. Andreou, “Device mismatch limitations on the performance of a Hamming distance classifier,” in Proc.
1993 IEEE Int. Workshop on Defect and Fault Tolerance in VLSI Systems, R. Stefanelli, Ed., 1993, pp. 327–334.
, “Device mismatch limitations on the performance of an associa[31]
tive memory system,” in Proc. 36th Midwest Symp. Circuits Systems,
Detroit, MI, Aug. 1993, pp. 570–573.
[32] T. Serrano-Gotarredona and B. Linares-Barranco, “A high-precision
current-mode WTA-MAX circuit with multichip capability,” IEEE J.
Solid-State Circuits, vol. 33, pp. 280–286, Feb. 1998.
[33] H. Miwa, K. Yang, P. O. Pouliquen, N. Kumar, and A. G. Andreou,
“Storage enhancement techniques for digital memory based, analog
computational engines,” in Proc. 1994 Int. Symp. Circuits Syst., vol. 5,
London, U.K., Jun. 1994, pp. 45–48.