Neural-MOS Threshold Gate as a Way to Design On-Chip Learning Neuron Structures

Transcription

1 Neural-MOS Threshold Gate as a Way to Desgn On-Chp Learnng Neuron Structures RAFAIL LASHESKY Hardware Department The Unversty o Azu JAPAN Abstract: - Hardware mplementaton o artcal neuron networs (ANN) based on MOS-transstors wth loatng gates (Neuron MOS or νmos) s dscussed. Comparson o two type on-chp learnng neurons wth dgtal and analog nput weght storng s provded. The man problem n desgn the neuron wth analog nput weght memory s tolerance to devatons o crcut elements parameters and suppled voltage devaton. New neuron crcut that can compensate all nd o devatons s proposed and nvestgated. Desgn methodology o such a crcut and result o smulaton are shown. Key-words: Neural-MOS structures, νmos transstors, artcal neural networs, analog voltage weght representaton, devaton tolerance. 1 Introducton Sotware mplementaton o Artcal Neural Networs (ANN) s lexble, but advantage o hardware mplemented ANN over sotware realzaton s very ast perormance. The demand to modern ANN s hundreds o neurons and tens o thousands o synapses. The nown hardware mplemented ANN have sucently rgd lmtatons o the networs parameters. To overcome these lmtatons s the goal o many researchers. It loos possble because today s mcrochp desgn rules s less than 0.2µm and transstors dmensons mnmzaton ater ve-sx years up to 0,06µm s obvous [1]. As a result the ntegraton and densty o transstors on the chp wll ncrease n many tmes. Together wth new so called Neural-MOS structures concept t opens very eectve way or Bo-nspred systems [2] based on Intellgence Implementaton to Slcon Chp [3] such as ANN and Fuzzy Logc. I the sngle MOS-transstor s only a swtch, the Neural- MOS structure represents mult-nput threshold gate and such a gate can be used as neuron wth a large number o synapses. It s nown two ways o desgnng Neural-MOS structures: transstors wth loatng gate (νmos) [3], what s voltage-mode Neural-MOS, and β- drven threshold element [4,5], what s currentmode Neural-MOS. Transstors wth loatng gate loos better or desgn ANN because o very small statc power dsspaton, and hgher elements parameters and suppled voltage devaton tolerance. In our prevous publcaton [6] we developed the desgn methodology o two types threshold gates, based on the loatng gate νmos statc (Fg.1) and cloced (Fg.2). Fg.1 Statc νmos threshold gate The man advantage o such a methodology s tang nto account the elements - transstors and capactors parameters devaton pecular to real chp manuacturng process and n aton the suppled voltage devaton. It was shown that these devatons restrcted the number o nputs n statc threshold gates descrbed n [7] by the value not more than 10. The same reasons lead to ncreasng nput capactors values and as a result chp area occuped by the gate number o nput ncreases. The last one s because o capactance relatonal devaton became less or bgger capactors.

2 Increasng nput capactes n aton led to ncreasng dynamc part o consumed power. Fg.2 Cloced νmos threshold gate The cloced approach proposed n [8] elmnate the nluence o transstors parameter devaton, but not the devaton o capactance s and suppled voltage. As shown n [6] n cloced νmos threshold gates number o nputs could be ncreased up to many tens. But the chp area ncreased, because o atonal swtches n the each o the gate nput. The parameter devaton nluence can be substantally overcame n case o on-chp learnng ANN (LANN) by choosng n learnng process approprate nput weghts value to compensate such a devaton. As a result we can mnmze nput capactance value and dramatcally (tenths o tmes) decrease chp area, ncrease perormance and mprove the man LANN parameters, such as sum o weghts, number o nputs and weght settng accuracy. It was shown n [6], where we dscussed neuron wth dgtal way o nput weght storng. The dsadvantage o dgtal weght representaton s large capacty RAM as an nputs weght memory. Analog nput weght storng n non-volatle EEPROM memory proposed n [9]. Another way o usng n the uture non-volatle erroelectrc memory proposed n [10]. Dsadvantage o weght changng and very long learnng tme. Input weghts change taes up to several mcroseconds. We estmated the possblty o desgn LANN wth analog nput weght storng, where nputs weght represents by voltage level n the capactor based analog voltage dynamc memory [11]. Dsadvantage o usng dynamc memory s necessty o perodcally rereshng because o dschargng the dynamc memory cell capactor by parastc leaage current. Ths problem can be solved by perodcally repeatng the learnng process, the holdng tme s enough. Holdng tme s the tme beore weght represented voltage wll change on the maxmal permssble value. Ths tme depends rom memory capactor and leaage current values. We estmated such a tme as hundreds o mllseconds. Because delay tme o cloced νmos structure s tenths o nanoseconds, t s enough tme to get weght rereshng. As another way o weghts rereshng, we propose νmos based multstage voltage comparator (MSC). The dea o such an approach s perodcally comparson the value o analog voltage n dynamc memory wth voltage levels correspondng to the derent nput weght stages. Number o stages denes the weght settng accuracy. I the derence between weght voltage n the memory and voltage stage s small enough, the sgnal o weght ncrement s generated. One o the man problem n LANN desgn s how to avod the nluence o suppled voltage devaton and voltage noses on the crcut behavor, because o the possble voltage derence n learnng and worng tmes. Such a derence can lead to ncorrect neuron behavor. In case o non-stable suppled voltage we have to restrct sum o weghts, number o nputs or weght settng accuracy. As t wll be shown, the 1% suppled voltage devatons decrease maxmal sum o weghts n 4 tmes and 5% devatons n 16 tmes. Such a devatons s possble, or example, as the result o noses. Usng SPICE smulaton, the maxmal sum o weghts and number o nputs or proposed LANN was estmated together wth tme parameters, such as weght holdng tme and output calculaton tme. Desgn methodology and crcut decson whch used to get tolerance to element parameters, dmensons and suppled voltage devatons wll be especally mportant to desgn LANN based on the uture CMOS processes wth lesser element szes and suppled voltages and dstnct devatons. Suppled voltage tolerant LANN could be useul wth other types o weght memory, or example wth EEPROM or erroelectrc memory. 2 General LANN structure The neuron wth synapses calculates the threshold uncton: F 0 ω x T

3 Here x and ω - nput varables and nput weghts correspondngly, T uncton threshold. Floatng gate νmos structure wth nputs represents the threshold uncton as: out 1 0 ω, x Here ω, nput voltage represents nput weght, T threshold voltage, the loatng gate structure output voltage, out the comparator output voltage. T T Fg.4 Synapse crcut The rst means we have derent voltage n tme o calculatng derent uncton, the second n tme o calculatng one uncton and the thrd case - derent voltage n tme o derent cloc pulses. Fg.4 represents the synapse crcut wth dynamc analog voltage memory sell (Fg.5). Fg.3 Bloc dagram or the neuron structure The neuron n general consst (Fg.3) rom a set o man parts: synapse crcut (Syn) to calculate ω x, where ω s represented by ω,ι - the synapse output voltage, νmos threshold gate or calculatons (TG), threshold voltage source (TS), ampler (Amp), learnng control unt to orm out the ncrement/decrement sgnal n the learnng process, weght rereshng crcut (WRC) wth multstage voltage comparator and commutator or perodcally and consequent connecton synapses dynamc memory sells to WRC. 3 Devaton-tolerant LANN structure As t ponted n the ntroducton one o the man problem n LANN desgn s to get the tolerance to suppled voltage devatons. Such a devaton sharply reduces maxmal sum o weghts. We have to dstngush three types o devatons: very slow, slow and ast. Fg.5 Memory cell crcut The crcut based on nmos type source ollower. Snce the bul o νmos s connected to the ground, transstor threshold voltage s stable aganst devaton. As a result t gves stable synapse output weght voltage. To overcome the nluence o the devaton on the TG we proposed [11] the crcut decson shown n Fg.6. Floatng gate voltage o the TG can be calculated as: C C th th ω ω + Cth + Cω re

4 Fg.6 Neuron crcut Fg.6 shows the new neuron crcut, whch s tolerant to suppled voltage devatons. Snce TG threshold voltage th and weghts voltages w both stable aganst devatons because o usng n-type source ollowers, the equaton or gettng tolerance to devatons s: weghts or the proposed crcut n comparson wth crcut wthout compensaton mechansm s shown n Fg.7. There s no advantage o ths structure n the case o less then 1% devaton snce atonal crcut components reduce senstvty o neuron. Nevertheless the devaton s more than 1%, ecency o ths crcut s obvous. The maxmum re th The loatng gate voltage devaton n the evaluaton phase depends only on the value o the reerence voltage n ths phase. I re varatons the same as nverter threshold voltage varatons, loatng gate voltage vary n the same as nverter threshold voltage. It wll be n the case o usng as the reerence voltage source the nverter wth shorted out nput and output. It s the way to compensate all nds o devatons. Usng HSPICE smulaton we ound the maxmal sum o weghts or the proposed neuron crcut and ordnary cloced νmos neuron crcut or slow and ast devatons o suppled voltage. alue o devatons changed up to 10%. Maxmal sum o Fg.7 Comparson between proposed crcut and crcut wthout compensaton

5 sum o weghts or 5% devaton s near 800. In contrast or the crcut wthout compensaton the sum o weghts only Concluson In our research we developed the desgn methodology or statc and cloced threshold gates, whch could be used as on-chp learnng neurons, tang nto account possble devatons o elements (transstors and capactors) parameters and suppled voltage [6], [11]. Usng such a methodology t was shown that smple statc approach has very strong restrcton n number o nputs and sum o weghts. The more complcated cloced approach s stable aganst transstors parameter devaton, but not stable aganst devatons o suppled voltage. Proposed new loatng gate crcut opens the way o desgn the neuron wth sum o weghts up to 800 n the case o very hgh suppled voltage devaton up to 5%. Reerences: [1] Yan Borodovsy, Lthography ths amazng technology. Internatonal Symposum on Future o Intellectual Integrated Electroncs, Senda, Japan, March 14-17,1999, p.p [2] Ercttoz, Bo-nspred system based on analog LSI. Internatonal Symposum on Future o Intellectual Integrated Electroncs, Senda, Japan, March 14-17,1999, p.p [3] Ohm T., at al. Integratng ntellgence on slcon electronc systems, Symposum on LSI Crcut Dgest o Techncal Papers. [4] Yamada T., Iebe M., Amemya Y., A current mode nmos crcut or cellular automaton devce. Internatonal Symposum on Future o Intellectual Integrated Electroncs, Senda, Japan, March 14-17,1999, p.p [5] arshavsy., Beta-Drven Threshold Element, Proceedngs o the 8-th Great Laes Smposum on LSI, IEEE Computer Socety, Feb.19-21, 1998, p.p [6] Lashevsy R., Taaara K., Souma M. (1999). The ecency o neuron-mos transstors n threshold logc, Sot Computng 3 (1999), Sprnger-erlag, p.p [7] Shbata T., Ohm T., A unctonal MOS transstor loatng gate-level weghtng sum and threshold operatons, IEEE Transactons. Electronc Devces, ED-39(6), [8] Ozedemr H., Kepep A., Pamr B., Leblebc Y. (1996). A capactve thresholdlogc gate, IEEE Journal o Sold-State Crcuts, 312, (8). [9] Shbata T., Ish H., Ohm T., A neuron- MOS neural networ usng sel-learnngcompatble synapse crcuts. IEEE Journal o Sold-State Crcuts, vol.30, No.8, August [10] Ishwara H.,Yoon S., Toumtsu E., Fabrcaton o adaptve-learnng neurochps usng erroelectrc thn lms. Internatonal Symposum on Future o Intellectual Integrated Electroncs, Senda, Japan, March 14-17,1999, p.p [11] Lashevsy R., Sato Y., Devaton-Tolerant Floatng Gate Structures as a Way to Desgn an On-Chp Learnng Neural Networs. Sot Computng (2001), Sprnger-erlag,n prnt. [12] Montalvo A., Gyurcs R., Paulos J., Toward a General-Purpose Analog LSI Neural Networ wth On-Chp Learnng. IEEE Transectons on Neural Networs, vol.8, No.2, March 1997.