Awinner-take-all (WTA) circuit, which identifies the

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II: EXPRESS BRIEFS, VOL. 52, NO. 3, MARCH 2005 131 High-Speed and High-Precision Current Winner-Take-All Circuit Alexander Fish, Student Member, IEEE, Vadim Milrud, and Orly Yadid-Pecht, Senior Member, IEEE Abstract A CMOS high-performance current-mode winnertake-all circuit is presented. The circuit employs a novel technique for inhibitory and excitatory feedbacks based on input currents average computation, achieving both high speed and high precision. The circuit is designed for operation with a wide range of input current values, allowing its integration with circuits operating both in subthreshold and in strong inversion regions. Two circuits, each for a different range of input currents, have been implemented in a standard 0.35- m CMOS process available through MOSIS and are operated via a 3.3-V supply. Their operation is discussed, simulation results are reported and preliminary measurements from a test chip are presented. Index Terms Analog circuits, analog integrated circuits, CMOS analog integrated circuits, neural networks, winner take all (WTA). I. INTRODUCTION Awinner-take-all (WTA) circuit, which identifies the highest signal intensity among multiple inputs, is one of the most important building blocks in neural networks, fuzzy systems and nonlinear filters. Many WTA circuit implementations have been proposed in the literature [1] [19]. The MOS implementation of the WTA function was first introduced by Lazzaro et al. [1]. This very compact circuit optimizes power consumption and silicon area usage. It is asynchronous, processes all input currents in parallel and provides output voltages in real time. The first true current-mode (CM) WTA circuit, producing an output current that is proportional to the value of the winning current, was introduced by Andreou et al. [2] and Boahen et al. [3]. In 1993, the use of positive feedback to improve the performance of a CM WTA system was reported by Pouliquen et al. [4]. Several modifications to Lazzaro s design have been suggested in the past [5] [7]. The circuit has been modified by Starzyk and Fang [5] by improving precision and speed performance. In 1995, DeWeerth and Morris [6] have added distributed hysteresis using a resistive network. Distributed hysteresis allows the winning input to shift between adjacent locations maintaining its winning status, without having to reset the network. Additional modifications that endow the Lazzaro s WTA with hysteretic and lateral inhibition and excitation properties have been proposed by Indiveri [7]. Voltage-mode WTA works based on Lazzaro s circuit were Manuscript received March 9, 2004; revised July 13, 2004. This paper was recommended by Associate Editor A. G. Andreou. A. Fish and V. Milrud are with the VLSI Systems Center, Beer-Sheva 84105, Israel (e-mail: afish@ee.bgu.ac.il; milrud@ee.bgu.ac.il). O. Yadid-Pecht is with VLSI Systems Center, Beer-Sheva 84105, Israel, and also with the Department of Electrical and Computer Engineering, University of Calgary, AB T2N1N4, Canada (e-mail: oyp@ee.bgu.ac.il). Digital Object Identifier 10.1109/TCSII.2004.842062 presented [8], [9]. In 1995, this circuit was modified with feedback and inhibition by Wilson and DeWeerth [8] and later, in 1999, by Kalim and Wilson [9]. The performances of all WTA circuits can be measured in terms of speed, accuracy and power consumption. For example, some of them were designed to achieve high speed and high accuracy in high-speed, high-precision applications [10], [11], while [12], [13] were designed for pulse-coded neural networks. The fact that the WTA operation is fully parallel and can be distributed makes these circuits very useful in visual attention and tracking systems, where the localization of the most salient regions in an image and selection of salient targets in the field of view (FOV) are very important tasks. In these systems WTA networks are able to select and lock onto the input with the strongest amplitude, and to track it as it shifts from one pixel to its neighbor [20] [29]. Although WTA circuits are very useful in many applications and are very popular, they suffer from matching problems, especially in systems with a large number of inputs. A complete analysis of device mismatch limitations in large WTA circuits was first presented by Kumar et al. in 1993 [30], [31] and was a subject of further research in the later works [10], [32]. A floatinggate calibration technique to improve the performance/matching of a WTA circuit was presented by Miwa et al. [33] in 1994. This paper presents a novel high-speed and high-precision CM WTA circuit. The circuit employs inhibitory and local excitatory feedbacks based on input currents average computation, enhancing precision and speed performance of the circuit. Local excitatory feedback provides a hysteretic mechanism that prevents the selection of other potential winners unless they are stronger than the selected one by a set hysteretic current. The proposed circuit can be useful for integration with circuits operating in the strong inversion region and supplying input currents of 3 50 A, as well as for subthreshold applications with inputs of 0 50 na. Not as the previously presented WTA circuits, the proposed circuit achieves a very high speed [32 ns for high currents of 3 50 A (measured) and 34 ns for subthreshold currents (simulated)] in case when a very small difference between two input currents is applied (30 na for high currents and 1.8 na for subthreshold applications). These circuit performances are the direct result from very strong feedbacks applied in the circuit, causing however, higher power dissipation (87.5 W for high-input currents and 22.5 W for subthreshold currents), compared to existing low-performance WTA circuits, operating in subthreshold and optimizing power consumption [1], [5], [7]. The power consumption of the proposed circuit can be significantly reduced by decreasing the feedbacks values, while degrading the circuit performance. 1057-7130/$20.00 2005 IEEE

132 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II: EXPRESS BRIEFS, VOL. 52, NO. 3, MARCH 2005 Fig. 1. Cells 1 and k (out of n) of the proposed WTA circuit. Section II presents the proposed circuit architecture and describes its operation. The performance of the implemented circuits including simulation results, preliminary measurements from a test chip, as well a comparison to the existing WTA circuits are shown in Section III. Section IV concludes the paper. II. CIRCUIT DESCRIPTION Fig. 1 shows cells and (out of the interacting cells) of the WTA circuit. Cell receives a unidirectional input current, and produces an output voltage. This output has a high digital value if the input current is identified as winner and low, otherwise. The WTA circuit operates as follows: the drains of transistors of all cells of the array are connected to the drains of transistors by a single common wire with voltage. The circuit starts competition by applying 1 for a short period of time. This way the excitatory feedback and inhibitory feedback are cancelled. Assuming that all cells in the array are identical and 1 is applied, the current, through, is equal to the average of all input currents of the array, neglecting small deviations in the referenced input currents. is copied to by the pmos current mirror ( and ) and is compared with the input current copied by the nmos current mirror ( and ). If then, assuming the same drivability factor of and transistors. An increase in input current relative to causes a decrease in due to the Early effect. This way, during the reset phase, input currents of all cells are compared to the average of all input currents of the array, producing a unique output for every cell. The cell having the highest input current value produces the smallest voltage. With the completion of the reset phase, i.e. 0, the excitatory feedback and the inhibitory feedback are produced. The node inputs to the gate of pmos transistor; thus, the cell with smaller (highest input current) produces higher current through and. This current is copied by the nmos current mirror ( and ), creating the excitatory feedback. On the other hand, is copied by the nmos current mirror ( and ), resulting in inhibitory feedback. The is added to the flowing through and is added to the average of all input current by connection transistor to COM node, increasing the value. This way, every cell produces a new voltage value, according to the comparison between current and a new value of current, that is now given by where is the average of all input currents of the array and N is the number of array cells. For the cell, having the highest input current, the difference between and growths, decreasing value. For example, for the same dimensions of and transistors, the value of is equal to. The difference between the input current of the cell and without feedback (during reset phase) can be calculated as (1) (2)

FISH et al.: HIGH-SPEED AND HIGH-PRECISION CURRENT WTA CIRCUIT 133 After feedback is applied, the difference is given by (3) because, (3) can be rewritten as (4) Thus, the difference between and is given by Fig. 2. Transient response of WTA circuit. where is the average of excitatory feedback currents of all cells. It can be easily seen, that if cell has the highest input and thus the maximum, then, causing to decrease. A decrease in causes to increase and so on. At the same time, cells, having small inputs have, causing their to be increased. This computation phase is finished when only one cell is identified as a winner, producing 1 at the inverter output. All other cells are identified as losers with 0. As was mentioned above, excitatory feedback provides a hysteretic mechanism that prevents the selection of other potential winners unless they are stronger than the selected one by a set hysteretic current. The value of this hysteretic current can be calculated in the following way: the excitatory and inhibitory feedbacks of the all looser cells are approximately equal to zero. Thus, assuming cell is a winner, the hysteretic current value can be expressed as Note, that in some applications (operating in strong inversion) this hysteresis can be useful, allowing circuit operation without need to reset, while in others, where small currents are input to the circuit, a reset of the circuit is required each time before the competition start. III. PERFORMANCE, SIMULATION AND PRELIMINARY MEASUREMENTS FROM TEST CHIP Two circuits, each designed for a different range of input currents, having eight cells were designed and fabricated in the 0.35- m, n-well, 4-metal, CMOS technology process supported by MOSIS. To verify the above analysis, the performance of both WTA circuits has been simulated with the SPECTRES Cadence simulator using TSMC 0.35- m CMOS process parameters. The preliminary measurements from a test chip were performed for the high-input currents. The supply voltage is 3.3 V. (5) (6) TABLE I SIMULATION RESULTS OF FABRICATED CIRCUITS A. Simulation Results Fig. 2 shows an example of the transient responses of voltages for two cells. For this simulation A and A, while have different values, smaller than. Because is greater than, cell is supposed to be the winner and cell 2 the loser. Two phases can be clearly identified in Fig. 2: the reset phase, where excitatory and inhibitory feedbacks are cancelled, and a computation phase, where the winner is found. It can be seen that and are constant during the reset phase. After the feedbacks are applied, rapidly falls, causing cell to be a winner. Note, that the presented voltages are the cell outputs before the inverter, thus the circuit real response time is faster than indicated in Fig. 2. Table I summarizes the main characteristics of the circuit, as were carried out by simulations. Note that mismatch problems between different devices are not taken in account in these simulation results, but can be clearly identified by measurements of a test chip, as shown in the next subsection. Monte Carlo simulations and analysis of the proposed circuit operation with a large number of inputs are subjects for further research. B. Preliminary Measurements From a Test Chip To determine the properties of the fabricated WTA circuit, a specific test board has been designed. This board is designed to provide all control signals for the circuit as well high-precision currents in very wide range. At this stage the board set-up is not finished yet, therefore only preliminary measurements for high input currents results are provided. Generally, there is a good

134 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II: EXPRESS BRIEFS, VOL. 52, NO. 3, MARCH 2005 TABLE II MAIN CHARACTERISTICS OF FABRICATED CIRCUITS WITH COMPARISON TO EXISTING WTA IMPLEMENTATIONS agreement between the measurement process and the simulation process. Two main sets of measurements where applied on the test chip to determine its properties: 1) typical delay measurements during these measurements the difference between the two highest closest input currents was about 10% (the same conditions as presented in simulation, shown in Fig. 2), and 2) precision measurements and delay measurements in case when the difference between two highest closest input currents is equal to the value of measured precision. Note, that this is the worst case for delay measurements. In order to check the influence of the device mismatch on the circuit performance, the two most far inputs were tested ( and ), while had different values, smaller than and. Following the test chip measurements, the measured mismatch is equivalent to the difference of 10 na in input currents. Table II summarizes the main characteristics of the fabricated circuit, comparing the proposed circuit with existing WTA circuits [5], [18], [32]. IV. CONCLUSION We have presented a high-speed and high-precision CM WTA circuit, described its operation and compared it to the WTA circuits previously published in the literature. A prototype chip with two WTA circuits, each for a different range of input currents, have been implemented in a standard 0.35- m CMOS process available through MOSIS. The circuit employs inhibitory and local excitatory feedbacks based on input currents average computation. The simulation results and preliminary measurements from a test chip show that the proposed circuit achieves very high speed and high precision and is suitable for a wide range of input currents. Subthreshold matching issues (including Monte Carlo simulations), circuit operation with a large number of inputs are subjects for further research. ACKNOWLEDGMENT The authors wish to thank A. Slepoy for his help with the simulations and layout for this work. REFERENCES [1] J. Lazzaro, S. Ryckebusch, M. A. Mahowald, and C. A. Mead, Winner-Take-All Networks of O(n) Complexity, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1989, vol. 1, pp. 703 711. [2] A. G. Andreou, K. A. Boahen, A. Pavasovic, P. O. Pouliquen, R. E. Jenkins, and K. Strohbehn, Current-mode subthreshold MOS circuits for analog VLSI neural systems, IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 205 213, Mar. 1991. [3] K. A. Boahen, A. G. Andreou, P. O. Pouliquen, and R. E. Jenkins, Current-mode based analog circuits for synthetic neural systems, U.S. Patent 5 206 541, Apr. 27, 1993. [4] P. O. Pouliquen, A. G. Andreou, K. Strohbehn, and R. E. Jenkins, An associative memory integrated system for character recognition, in Proc. 36th Midwest Symp. Circuits Systems, Detroit, MI, Aug. 1993, pp. 762 765. [5] J. A. Startzyk and X. Fang, CMOS current-mode winner-take-all circuit with both excitatory and inhibitory feedback, Electron. Lett., vol. 29, no. 10, pp. 908 910, 1993. [6] S. P. DeWeerth and T. G. Morris, CMOS current-mode winner-take-all circuit with distributed hysteresis, Electron. Lett., vol. 31, no. 13, pp. 1051 1053, 1995. [7] G. Indiveri, A current-mode hysteretic winner-take-all network, with excitatory and inhibitory coupling, Analog Integr. Circuits Signal Process., vol. 28, pp. 279 291, 2001.

FISH et al.: HIGH-SPEED AND HIGH-PRECISION CURRENT WTA CIRCUIT 135 [8] D. M. Wilson and S. P. DeWeerth, Winning isn t everything, in Proc. IEEE ISCAS 95, 1995, pp. 105 108. [9] R. Kalim and D. M. Wilson, Semi-parallel rank-order filtering in analog VLSI, in Proc. IEEE ISCAS 99, vol. 2, 1999, pp. 232 235. [10] T. Serrano and B. Linares-Barranco, A modular current-mode highprecision winner-take-all circuit, IEEE Trans. Circuits Syst. II, vol. 42, no. 2, pp. 132 134, Feb. 1995. [11] A. Nakada, M. Konda, T. Morimoto, T. Yonezawa, T. Shibata, and T. Ohmi, Fully-parallel VLSI implementation of vector quantization processor using neuron-mos technology, IEICE Trans. Electron., vol. E82C, no. 9, pp. 1730 1738, Sep. 1999. [12] J. L. Meador and P. D. Hylander, Pulse coded winner-take-all networks, in Silicon Implementation of Pulse Coded Neural Networks,M. E. Zaghloul, J. L. Meador, and R. W. Newcomb, Eds. Norwell, MA: Kluwer, 1994, ch. 5, pp. 79 99. [13] E. I. ElMasry, H. K. Yang, and M. A. Yakout, Implementations of artificial neural networks using current-mode pulse width modulation technique, IEEE Trans. Neural Netw., vol. 8, no. 3, pp. 532 548, May 1997. [14] J. Choi and B. J. Sheu, A high-precision VLSI winner-take-all circuit for self-organizing neural networks, IEEE J. Solid-State Circuits, vol. 28, no. 5, pp. 576 584, May 1993. [15] H. M. Yu, R. S. Miyaoka, and T. K. Lewellen, A high-speed and high-precision winner-select-output (WSO) ASIC, IEEE Trans. Nuclear Sci., vol. 45, no. 3, pp. 772 776, Jun. 1998. [16] K. T. Lau and S. T. Lee, A CMOS winner-takes-all circuit for self-organizing neural networks, Int. J. Electron., vol. 84, no. 2, pp. 131 136, 1998. [17] Y. He and E. Sanchez-Sinencio, Min-net winner-take-all CMOS implementation, Electron. Lett., vol. 29, no. 14, p. 3, 1993. [18] A. Demosthenous, S. Smedley, and J. Taylor, A CMOS analog winnertake-all network for large-scale applications, IEEE Trans. Circuits Syst. I, vol. 45, no. 3, pp. 300 304, Mar. 1998. [19] P. O. Pouliquen, A. G. Andreou, and K. Strohbehn, Winner-takes-all associative memory: a hamming distance vector quantizer, in Analog Integr. Circuits Signal Process.. Norwell, MA: Kluwer, 1997, vol. 13, pp. 211 222. [20] Y. Fang, M. A. Cohen, and T. G. Kincaid, Dynamics of a winnertake-all neural network, Neural Netw., vol. 9, no. 7, pp. 1141 1154, 1996. [21] G. Indiveri, A neuromorphic VLSI device for implementing 2D selective attention systems, IEEE Trans. Neural Netw., vol. 12, no. 6, pp. 1455 1463, Nov. 2001. [22] T. G. Morris, T. K. Horiuchi, and P. DeWeerth, Object-based selection within an analog VLSI visual attention system, IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol. 45, no. 12, pp. 1564 1572, Dec. 1998. [23] T. G. Moris and S. P. DeWeerth, A smart-scanning analog VLSI visualattention system, Analog Integr. Circuits Signal Process., vol. 21, pp. 67 78, 1999. [24], Analog VLSI excitatory feedback circuits for attentional shifts and tracking, Analog Integr. Circuits Signal Process., vol. 13, no. 1/2, pp. 79 92, 1997. [25] T. Horiuchi and E. Niebur, Conjunction search using a 1-d analog VLSI based attentional search/tracking chip, in Proc. 1999 Conf. Advanced Research VLSI, 1999, pp. 276 290. [26] T. G. Moris, S. P. DeWeerth, T. K. Horiuchi, and C. Koch, Analog VLSI circuits for attention-based, visual tracking, in Proc. Neural Information Processing System Conf., Denver, CO, Dec. 1996. [27] G. Indiveri, Neuromorphic analog VLSI sensor for visual tracking: circuits and application examples, IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol. 46, no. 11, pp. 1337 1347, Nov. 1999. [28] C. S. Wilson, T. G. Morris, and P. DeWeerth, A two-dimensional, object-based analog VLSI visual attention system, in Proc. 20th Anniversary Conference on Advanced Research in VLSI, vol. 20, Mar. 1999, pp. 291 308. [29] V. Brajovic and T. Kanade, Computational sensor for visual tracking with attention, IEEE J. Solid-State Circuits, vol. 33, no. 8, Aug. 1998. [30] N. Kumar, P. O. Pouliquen, and A. G. Andreou, Device mismatch limitations on the performance of a Hamming distance classifier, in Proc. 1993 IEEE Int. Workshop on Defect and Fault Tolerance in VLSI Systems, R. Stefanelli, Ed., 1993, pp. 327 334. [31], Device mismatch limitations on the performance of an associative memory system, in Proc. 36th Midwest Symp. Circuits Systems, Detroit, MI, Aug. 1993, pp. 570 573. [32] T. Serrano-Gotarredona and B. Linares-Barranco, A high-precision current-mode WTA-MAX circuit with multichip capability, IEEE J. Solid-State Circuits, vol. 33, pp. 280 286, Feb. 1998. [33] H. Miwa, K. Yang, P. O. Pouliquen, N. Kumar, and A. G. Andreou, Storage enhancement techniques for digital memory based, analog computational engines, in Proc. 1994 Int. Symp. Circuits Syst., vol. 5, London, U.K., Jun. 1994, pp. 45 48.