EXPERIMENTAL KOHONEN NEURAL NETWORK IMPLEMENTED IN CMOS 0.18 m TECHNOLOGY

Transcription

1 15 th Internatonal Conference MIXED DESIGN MIXDES 008 Pozna, POLAND 19-1 June 008 EXPERIMENTAL KOHONEN NEURAL NETWORK IMPLEMENTED IN CMOS 0.18m TECHNOLOGY R. DLUGOSZ 1,, T. TALASKA 3, J. DALECKI 3, R. WOJTYNA 3 1 UNIVERSITY OF NEUCHÂTEL, SWITZERLAND UNIVERSITY OF ALBERTA, CANADA 3 UNIVERSITY OF TECHNOLOGY AND LIFE SCIENCES, POLAND KEYWORDS: Kohonen neural networks, Analog CMOS crcuts, Adaptve data processng ABSTRACT: In ths paper, we present an expermental current-mode Kohonen neural network (KNN) mplemented n a CMOS 0.18 m process. The network contans four output neurons. Each neuron has three analog weghts related to three nputs. The presented KNN has been realzed usng buldng blocks proposed earler by the authors, such as bnary tree current-mode wnner takes all (WTA) crcut, Eucldean dstance calculaton crcut (), adaptve weght change mechansm (), conscence mechansm (CONS), ntal weght ntalzaton mechansm (IB). The network performance has been verfed n the way of measurements. The obtaned measurement results are n a good agreement wth theoretcal consderatons as well as HSPICE smulatons. The crcut occupes a chp area (wthout pads) equal to 0.07 mm and consumes 1 mw of power for 1.8 V supply. The nput currents are n the range between 1 and 7 A. We ntend to apply the desgned KNN to analyze ECG bomedcal sgnals. INTRODUCTION Applcaton of an artfcal ntellgence based on neural networks to varous medcal systems, e.g. cardography, s ganng popularty n last years [1, ]. Ths s due to specfc algorthms, whch n ther nature are close to the behavour of adaptve fuzzy logc systems. The neural networks are able to make a proper decson assessng nput data on the bass of ther earler dvson nto several fuzzy fles. Varous exstng network archtectures can be classfed nto two man groups usng ther learnng method as a crteron,.e. networks traned ether wth a supervson or wthout t. In ths work, we present a CMOS mplementaton of the Kohonen Neural Network, that s often called wnner takes all (WTA) network [3]. In our approach, an unsupervsed learnng algorthm s used, whch usually s faster compared to supervsed learnng algorthms. The proposed network has been realzed as a current-mode crcut to be appled n medcne for an on-lne analyss of ECG sgnals. The paper s organzed as follows. In the next secton, a general dea of the KNN s shortly presented. Then, we gve an overvew of partcular buldng blocks that have been used n the proposed crcut. Selected measurement results have been descrbed n the followng secton. Then, we provde an analyss of power dsspaton and chp area occupaton that may be useful n desgnng larger networks. Fnally, some conclusons are drawn. KOHONEN NEURAL NETWORK The network whch s n the scope of our nterest has been orgnally proposed by Teuvo Kohonen n [3]. The Kohonen neural network (KNN) features a compettve, unsupervsed learnng, whch means that neuron s weghts are modfed and adjusted wthout any feedback as ths s n case of supervsed networks. There are two general types of KNN s. One s based on the WTA and the other on WTM (Wnner Takes Most) tranng methods. In ths paper, we focus on a WTA-traned network. Networks wth WTM learnng are more popular n software mplementatons but the WTA algorthms are smpler and, therefore, easer to be mplemented n hardware. Tranng KNN s reles on presentng the network wth some learnng vectors X, n order to make the neuron s weght vectors, W, to resemble the presented tranng data. For each tranng vector X, the network frst determnes the Eucldean dstance between ths vector and the W weght vector connected wth a gven neuron, accordng to the followng formula: n d( X, W ) ( x w ) (1) l1 In the proposed network, we used a modfed form of (1), where the root operaton has been neglected. Ths approach s correct n case of our network as the exact values of (1) for partcular neurons are not mportant. Essental s, for whch neuron we get the lowest value of (1). Ths neuron becomes a wnner and gans the rght to adapt ts weghts, accordng to the followng equaton: W l ( t1) ( t) ( t) W X W () where s a learnng rate coeffcent. Weghts of other neurons n the WTA method reman unchanged. KNN PROTOTYPE IMPLEMENTED IN CMOS TECHNOLOGY A general block dagram of our neural network s shown n Fg. 1. The crcut conssts of four neurons, each contanng three analog weghts. To determne a l Copyrght 008 by Department of Mcroelectroncs & Computer Scence, Techncal Unversty of Lodz 43

2 degree of smlarty between the tranng and weght vectors, X and W, the Eucldean dstance calculaton () blocks are used. They are shown n Fg. [4]. Intalzng Block (IB) Input [X] w11(k) w1(k) w13(k) w1(k) w(k) w3(k) w31(k) w3(k) w33(k) w41(k) w4(k) w43(k) x1 x x3 w11(k) w1(k) w13(k) w 1(k) w (k) w 3(k) w 31(k) w 3(k) w 33(k) w 41(k) w 4(k) w 43(k) CONSC Iedc1(k) Icons1(k) Icount1(k) Iedc(k) Icons(k) Iedc3(k) Iedc4 (k) Icount(k) Icons3 (k) Icount3(k) Icons4 (k) Icount4 (k) W TA NE1 NE NE3 NE4 w 11(k) w 1(k) w 13(k) w 1(k) w (k) w 3(k) w 31(k) w 3(k) w 33(k) w41(k) w4(k) w43(k) w11(k+1) w1(k+1) w13(k+1) w1(k+1) w(k+1) w3(k+1) w31(k+1) w3(k+1) w33(k+1) w41(k+1) w4(k+1) w43(k+1) n calculaton of the dstance between vectors. The second output current s multpled by a programmable coeffcent, whose value s equal to the learnng rate. Ths multplcaton s realzed usng a mult-output NMOS current mrror wth bnary weghted transstors. The SQR crcut, shown n Fg. 4, s a smple onequadrant squarer, ntally proposed n [5]. In our network ths crcut has been slghtly modfed to enable controllng the squarng operaton. Output currents of partcular SQR crcuts are summed n a common output node creatng the output current used as a drect measure of the Eucldean dstance square. The output current s descrbed usng the followng equaton: I out n 1 x w A, (3) Fg. 1. Block dagram of the proposed and mplemented KNN n case of 4 neurons and 3 analog nputs Fg. 4. Classc one-quadrant currents squarer [5] used n our crcut. Fg.. Electrcal block dagram of current mode Eucldean dstance calculaton crcut () proposed n [4, 11] Fg. 5. Current mode WTA crcut: (top) general block dagram wth logc crcut, (bottom) WTA block [6, 11] (a) Fg. 3. Classcal current comparator (CMP) used n crcut and subtractng crcut (SUB) wth mplemented abs() functon. The learnng rate s controlled by 4 bts [4, 11]. In the desgned network, the block conssts of three equal channels, each contanng a comparator (CMP), subtractng crcut (SUB) and squarer (SQR). A classc current-mode comparator shown n Fg. 3a has been used n ths block. The crcut compares the nput sgnals, x, wth the sgnal representng the neuron weght, w, and generates two complementary logcal sgnals, s, that control other blocks n the network. One of them s the SUB crcut shown n Fg. 3b, whch calculates the absolute value of the dfference between currents representng the nput sgnal, x, and the neuron weght w. One of the SUB output currents s delvered to the squarer (SQR) nput and, after squarng, s used d a 1a q nq adaptaton Fg. 6. Analog adaptve weght change mechansm (): (top) general block dagram wth ndvdual clock crcut; (bottom) electrcal scheme [7, 11] EN EN 1 a 1 1a 44

3 The second mportant buldng block n our network s a WTA crcut descrbed earler n [6]. Ths crcut gets the output currents from all blocks n the network and determnes both the value of the mnmal current as well as the address of the wnnng nput. In our network we utlze only the address, but ths crcut can also be used n applcatons that requre values of mnmal or maxmal currents, e.g. n MIN/MAX nonlnear flters. A general tree structure of ths crcut s shown n Fg. 5 (top), whle a structure of the (WTA) sngle par s shown n Fg. 5 (bottom). The next mportant component of our network s the mechansm that s responsble for adaptve weght changes. The general block dagram of ths crcut s shown n Fg. 6 (top), whle the structure of a sngle par at the bottom. Each block s controlled by an ndvdual clockng crcutry, also shown n Fg 6. The adaptaton mechansm s trggered by an external clock (clk) just after fnshng the prevous stage operatons ncludng presentaton the network a new tranng vector X, calculaton the Eucldean dstance and fnally detecton of the wnner neuron. All these operatons are performed asynchronously. The external clock and the output WTA sgnals (NE) generate ndvdual EN sgnals usng AND gates. The block conssts of two current-mode S&H memory cells, whch work alternately. A new data that occurs at the nput s summed wth the prevous weght s value from one of the S&H memory cells and the result s then stored n the second memory cell. After the adaptaton, the external clock swtches over the DFF, whch makes the new weght values at the neuron s output ready to be used n the next data cycle. When a gven neuron wns a competton, ts output current may be artfcally enlarged by an addtonal current from the conscence mechansm (CONS) [8-9], whch n ths case s realzed usng analog counters [10]. SELECTED MEASUREMENT RESULTS In the tranng process of our network, three dfferent phases can be dstngushed. In the frst phase, often called the ntalzaton, neuron weghts are prelmnary programmed by an external voltage sgnal usng an analog demultplexer. The learnng phase starts just after the ntalzaton. Ths process conssts of cycles, each startng wth presentaton the network a new nput data and fnshng wth the adaptaton of weghts of the wnnng neuron. The learnng rate s ntally set up to be close to ts upper lmt 1 to enable fast network polarzaton and then s successvely decreased to the values that are fnally close to a lower lmt 0. In ths phase, the CONS mechansm can be optonally appled to actvate all neurons. In the fnal phase, that s often called the recall phase, the external clock s turned off, whch swtches off the mechansm. In ths phase, both the and WTA blocks work lke n the prevous learnng phase. The measurement experments presented n ths secton have been selected n such a way to verfy partcular buldng blocks of the network as well as performance of the entre system. Our network operates n a current mode. To obtan approprate nput current waveforms, we apply to the network nputs voltage sgnals whch are converted to currents usng external resstors of 100 k. Example nput sgnals are presented n Fgs. 7 (a) and 8 (a). (a) (c) (d) Fg. 7. The prototyped KNN operaton: (a) nput sgnals X, NE sgnals of the WTA block, (c, d) selected neuron weght. In the frst experment, nput data vary relatvely slowly wth respect to the samplng frequency whch enables observaton of a cooperaton between the WTA and the blocks. Operaton of the WTA block n ths case s llustrated n Fg. 7 the four NE sgnals shown n Fg. 5. Note that always only one of the NE sgnals s equal to logcal 1, as expected. The neuron s weghts are presented n Fg. 7 (c) (e). Fg. 7 (c) llustrates frst weghts of all neurons, whle Fg. 7 (d) the second weghts and (e) the thrd weghts of the neurons. 45

4 (a) (c) (d) (e) Fg. 8. The KNN measurement results: (a) nput sgnals, operaton of the WTA block, (c) frst (d) second and (e) thrd neuron weghts. Adaptaton n a gven neuron s allowed only when the NE sgnal, assocated wth ths neuron, equals to logcal 1. Note that the waveforms presentng neuron s weghts are nverted wth respect to the nput sgnals. Ths s because at the network s outputs, NMOS-type mrrors are used, whch are connected va external resstors (10 k) to the supply potental. When the output current ncreases, voltage across a gven resstor decreases and vce versa (phase nvertng takes place). Fg. 8 llustrates a full tranng process wthn a tme perod of.5 ms. The frst 100 s has been allocated for the ntal network polarzaton. The learnng process starts just after ths phase. Samplng frequency s equal to 500 khz n ths experment, whch means 100 tranng teratons. The learnng rate has been ntally set up to be 13/15. After 500 s from the begnnng of the learnng process, ts value has been reprogrammed to the value of 7/15, after the next 1000 s to the value of 4/15 and fnally to 3/15. Influence of the learnng rate on the adaptaton process s vsble n Fgs. 8 (c) (e). At the begnnng, the neuron weght varatons are much stronger than at the end. In the prototyped KNN, some nondealtes have been observed. One of the problems are leakages n analog memory cells restrctng, n practce, the allowed mnmum samplng frequency. The leakages have been measured after turnng off the clock generator. Ths s llustrated n Fg. 9 (a), where the clock has been turned off after 1.5 ms. After the next 1.5 ms an average lost s 1.5 A (15 mv),.e. 0 % of the maxmum range. Assumng that an acceptable error s e.g. 1 %, the lowest samplng frequency s equal to 16 khz. Ths problem s mportant as many applcatons requre a rather small f S, e.g. medcal ones. Ths problem can be solved by mprovements n the memory cells, e.g. by ncreasng ther storage capactance, that n our crcut s 00 ff. Another way s to ntroduce perodc refreshng wth small learnng rate to enable leakage compensaton, but ths method must be appled carefully, as ths affects the learnng process. We also consder a possblty of usng other memory cells wth ncreased storage tme. Nose s also a problem here. Fg. 9 shows an example neuron s weght. The nose has a peak-to-peak value equal to 0.1 A (1 mv for R = 10 k). The nput sgnal range s 7 A,.e. SNR = 37 db. The man nose source n our KNN s a very complex measurement set up wth many crossng wres, where almost 0 sgnals must be montored at the same tme. In addton, 18 nput sgnals are needed. Fortunatelly, n our network the nose problem s not crtcal. Theoretcally, ts nfluence should be vsble n the WTA block, but the appled ntegratng current comparator s, smultaneously, a low-pass flter that reduces the nose level consderably. Ths s seen from the NE sgnals shown n Fg. 7 and 8. We expect that n real systems, where our network wll be one of the used components, the nose problem wll be smaller. EVALUATION OF CHIP AREA AND POWER CONSUMPTION Chp area occupaton Layout of the prototyped KNN s shown n Fg. 10. Each neuron n the network contans three equal channels. The KNN nput currents are coped several 46

5 tmes usng the current mrrors (marked as nputs) and each neuron gets a separate copy. The modular structure, where partcular buldng blocks are grouped nto channels, enables an easy and fast mplementaton of networks wth any number of nputs and outputs, by smple placng the requred number of channels and shortng ther outputs n a proper way. a sngle channel, A CN an area of a sngle CONS block and A WT area of a sngle WTA competng par. The WTA block s a bnary tree soluton, whch contans a total number of m-1 WTA sub-blocks. The chp area as a functon of the n and m parameters s llustrated n Fg. 11. As can be seen, for n=1 nputs and m=0 outputs (values proper for advanced ECG tests wth 1 ndependent electrodes) the evaluated chp area s about 1.4 mm. (a) Fg. 11. Area occuped by the network versus the number of ts nputs and outputs 50 5 outputs 0 outputs 40 outputs 60 outputs 80 outputs 100 outputs 00 Energy [nj] Fg. 9. Influence of leakage (a) and nose on the analog memory cell propertes Inputs Fg.1. Energy consumed by the network when processng one nput tranng vector, X, versus the network nput and output numbers. Power consumpton The average power dsspaton for the supply voltage V DD = 1.8 V and the samplng frequency f S = 500 khz s equal to about 1 mw, whch means that energy per one calculaton cycle (for each new X vector) s equal to nj for 4 neurons and 167 pj per each channel. Takng these values nto account, one can show that energy consumed by a network wth n nputs and m outputs s equal to: Fg. 10. Layout of the prototyped Kohonen s network desgned n TSMC CMOS 0.18m technology Chp area occuped by the desgned network s equal to 0.07 mm (50 x 300 m). To evaluate the chp area for any number of the network outputs and nputs, we can use the formula: A nma ma ( m1 A, (4) CH CN ) where n and m are numbers of the network nputs and outputs, respectvely. A CH s a constant area occuped by WT E nme ( E (5) CH KkmECN Kkmm1) where K=1 f the CONS mechansm s operatng,.e. n the learnng phase, and K=0 n the recall phase, E CH s the energy consumed n a sngle channel, E CM energy of a sngle CONS block and E WT energy n a sngle WTA crcut. The parameter k s the network teraton number. Ths parameter must be taken nto consderaton as the appled CONS mechansm adds to the output some addtonal current whose value ncreases wth tme. For ths reason, the WTA block comparng the WT 47

6 currents consumes an ncreased amount of energy. The energy per one calculaton cycle as a functon of the n and m parameters s llustrated n Fg. 1. If, for example, the number of the network nputs s n=1 and ts outputs s m=0, and f the neurons work wth a 500 khz samplng frequency, the power consumed by the network equals to 0 mw. For each calculaton cycle, the network performs about 500 summng and 40 multplyng operatons n partcular channels and about 00 other operatons n the WTA and CONS blocks. In real medcal applcatons, as hgh samplng frequency as 500 khz s not requred and, therefore, n the next KNN prototypes we plan to ntroduce a mechansm that enables swtchng off some network blocks. As a result, for f S = 1 khz the network could, for example, stay n a power-down-mode durng 99 % of ts total operaton tme. Then, an average power dsspated by the (1x0) network could be as low as 10 W. CONCLUSIONS The paper presents an expermental Kohonen neural network mplemented n TSMC CMOS 0.18m process. The desgned network contanng 3 nputs and 4 outputs occupes an area of 0.07 mm. Operatng wth supply voltage of 1.8 V, the crcut dsspates 1 mw of power. For 1.5 V the consumed power s reduced to 700 W. The network nput data rate vares n the range from 50 to 500 ksps. Operaton of the networks can be dvded nto ntalzaton, learnng and recall phases. The crcut has been desgned n such a way to enable easy swtchng between these three phases. There exsts also a possblty of refreshng the neuron s weghts by perodcal turnng on the appled adaptve mechansm wth a small learnng rate. Ths s mportant because n the appled analog memores some current leakage appears that ntroduces a systematc error to the data held. The obtaned measurement results are n a good agreement wth postlayout smulatons as well as results obtaned n the MATLAB model. The presented crcut s a frst prototype of the Kohonen network ever reported. A next step of our studes wll be prototypng a larger network sutable for analyzng bomedcal sgnals used n dagnostc systems. THE AUTHORS Dr. Rafa Dugosz Unversty of Neuchâtel, Insttute of Mcrotechnology, Rue A.-L. Breguet, CH-000, Neuchâtel, Swtzerland; Unversty of Alberta, Department of Electrcal and Computer Engneerng, 114 St 89 Ave, Edmonton Albert-a, T6G V4, Canada, rdlugosz@ualberta.ca fellow of the MC EU Outgong Internatonal Fellowshp Prof. Ryszard Wojtyna, Jakub Daleck and Tomasz Talaska are from Unversty of Technology and Lfe Scences, Insttute of Telecommuncaton, ul. Kalskego 7, Bydgoszcz, Poland e:mal: woj@utp.edu.pl, talaska@utp.edu.pl REFERENCES [1] Ln He, Wensheng Hou, Xaoln Zhen, Chengln Peng, Recognton of ECG Patterns Usng Artfcal Neural Network, Sxth Internatonal Conference on Intellgent Systems Desgn and Applcatons, (ISDA), Vol., October 006, pp [] E.J. Tkacz, P. Kostka, K. Jonderko, B. Mka, Supervsed and Unsupervsed Learnng Systems as a Part of Hybrd Structures Appled n EGG Sgnals Classfers, IEEE Annual Conference Engneerng n Medcne and Bology, Shangha, Chna, Sept. 005 [3] Kohonen T. Self-organzng maps, Sprnger Verlag, Berln, 001 [4] T. Talaka, R. Dugosz, Current Mode Eucldean Dstance Calculaton Crcut for Kohonen's Neural Network Implemented n CMOS 0.18m Technology, IEEE Canadan Conference on Electrcal and Computer Engneerng, Vancouver, Canada, 007 [5] K. Bult, H. Wallnga, A Class of Analog CMOS Crcuts Based on the Square-Law Characterstc of an MOS Transstors n Saturaton, IEEE Journal of Sold-State Crcuts, vol. sc-, No.3, 1987, pp [6] R. Dugosz, T. Talaka, R. Wojtyna, New Bnary- Tree-Based Wnner-Takes-All Crcut for Learnng on Slcon Kohonen's Networks, Internatonal Conference On Sgnals And Electronc Systems, ICSES 006, ód 006. [7] T. Talaka, R. Dugosz, W. Pedrycz, Adaptve Weght Change Mechansm for Kohonens's Neural Network Implemented n CMOS 0.18 µm Technology, 11th European Symposum on Artfcal Neural Networks, Bruges, Belgum, Aprl 007 [8] Ahalt, S.C., Krshnamurthy, A.K., Chen, P., Melton, D.E., "Compettve learnng algorthms for vector quantzaton", Neural Networks, Vol. 3, 1990 [9] DeSeno D., Addng a conscence to compettve learnng, Proc. II IEEE, Conference Neural Network, Vol. I, 1988 [10] T. Talaka, R. Wojtyna, R. Dugosz, K. Inewsk, W. Pedrycz, Analog-Counter-Based Conscence Mechansm n Kohonen s Neural Network Implemented n CMOS 0.18 µm Technology, The IEEE 006 Workshop on Sgnal Processng Systems (SPS'06), Banff, Kanada, 006. [11] T. Talaka, R. Dugosz, R. Wojtyna, Current mode Kohonen Naural Network, Internatonal Conference Mxed Desgn of Integrated Crcuts and Systems (MIXDES), June 007, Poland Work supported partally by: Mare Cure EU Outgong Internatonal Fellowshp and partally by MNSW research project No. N /306 48