Synthesis of Bis-sclle CMOS Anlog Computtionl Circuits Using Mrgin Propgtion Ming Gu, Student memer, IEEE, Shntnu Chkrrtty, Senior Memer, IEEE Astrct Approximtion techniques re useful for implementing pttern recogniers, communiction decoders nd sensory processing lgorithms where computtionl precision is not criticl to chieve the desired system level performnce. In our previous work, we hd proposed mrgin propgtion (MP s n efficient piece-wise liner (PWL pproximtion technique to log-sum-exp function nd hd demonstrted its dvntges for implementing proilistic decoders. In this pper, we present systemtic nd generlied pproch for synthesiing nlog piecewise-liner (PWL computing circuits using the MP principle. MP circuits use only ddition, sutrction nd threshold opertions nd hence cn e implemented using universl conservtion principles like the Kirchoff s current lw. Thus, unlike the conventionl trnsliner CMOS currentmode circuits, the opertion of the MP circuits re functionlly similr in wek, moderte nd strong inversion regimes of the MOS trnsistor mking the design pproch is-sclle. This pper presents mesured results from MP circuits prototyped in.µm stndrd CMOS process verifying the is-sclle property. As n exmple, we pply the synthesis pproch towrds designing liner clssifiers nd we verify its performnce using mesured results. Index Terms Piecewise-liner (PWL circuit, mrgin propgtion (MP, nlog computtion, trnsliner, current-mode circuits, moderte-inversion circuits. I. INTRODUCTION FOR pplictions tht do not require high computtionl precision (e.g. pttern mtching, current-mode nlog signl processing (ASP offers severl dvntges over digitl signl processing techniques in terms of energy efficiency nd speed [], []. By exploiting the computtionl primitives inherent in the device physics, complicted mthemticl functions cn e implemented in nlog, using significntly lower numer of trnsistors s compred to its digitl counterprts [], [3]. The most populr pproch for synthesiing current-mode nlog computtionl circuits is sed on the trnsliner principle which exploits the exponentil current-tovoltge reltionship oserved in ipolr trnsistors [], [] nd metl-oxide-semiconductor (MOS trnsistors ised in wekinversion [6] [8]. In [9], [], the trnsliner principle hs lso een extended towrds synthesiing current-mode circuits using MOS trnsistors ised in strong-inversion. All CMOS sed trnsliner synthesis techniques shre one common property: they rely on the physicl response of the Copyright (c IEEE. Personl use of this mteril is permitted. However, permission to use this mteril for ny other purposes must e otined from the IEEE y sending n emil to pus-permissions@ieee.org. This work hs een supported y reserch grnt from the Ntionl Science Foundtion (NSF, CCF:78996. The uthors re with the Deprtment of Electricl nd Computer Engineering, Michign Stte University. All correspondence regrding this pper should e ddressed to shntnu@egr.msu.edu MOS trnsistor operting either in strong or wek inversion. Hence, trnsliner circuits designed to operte in one ising regime re generlly not portle to the other. Also, while operting in wek-inversion is eneficil for chieving superior energy-efficiency nd while operting in strong-inversion is eneficil for chieving higher-speed, it hs een rgued tht etter speed-energy trde-off is chievle when the trnsistors re ised in the moderte inversion regime [], []. Currently, no current-mode synthesis technique exists for implementing mthemticl functions using MOS trnsistors ised in moderte inversion. However, if we cn design circuits tht operte using only universl conservtion lws (e.g. current or chrge conservtion, the circuits should e portle cross different ising conditions. In this pper we propose is-sclle frmework for synthesiing nlog computtionl circuits sed on mrgin propgtion (MP principle which uses only thresholding opertions nd the Kirchoff s current conservtion principle. MP principle ws introduced in [3] s piece-wise liner (PWL pproximtion to the log-sum-exp function which is extensively used in communictions [], optimition [] nd lerning [3] lgorithms. In this pper we show tht the MP principle cn e used to implement wide-rnge of liner nd non-liner mthemticl functions. An importnt ttriute of MP sed synthesis is tht ll computtions re performed in the logrithmic domin, s result of which multiplictions mp into dditions, divisions mp into sutrctions nd power computtions mp into multiplictions. This simplifies the implementtion of mny complicted non-liner functions. We illustrte the sic synthesis procedure using simple exmple. Consider function f given y f = ( where,,, R re the opernds nd denotes multipliction opertor. The opernds cn e converted into their differentil form ccording to =, =, =, =, where,,,,,,, > re positive differentil components. Ech of these positive components cn e converted into its logrithmic form s: L = log, L = log, L = log, L = log, L = log, L = log, L = log, L = log.
Accordingly, ( cn e converted into f = e e, ( where nd re expressed s = log (e L L e L L e L L e L L = log (e L L e L L e L L e L L Thus, evlution of the function f cn e decomposed into three steps: ( input processing step which computes the loglikelihoods L x nd L x sed on the input x; ( log-sumexp computtion step given y equtions (3; nd (c output processing step which computes f sed on eqution (. We will show in this pper tht for most functions, the input nd the output processing steps re similr to equtions shown ove. The only difference in processing lies in the form of the log-sum-exp functions s shown in eqution (3. In MP sed synthesis the ojective will e to pproximte the log-sum-exp functions nd in the process generte different forms of liner nd non-liner functions. Fig. shows the rchitecture of generic MP-sed nlog signl processor. The input differentil signls =, = re converted into their logrithmic form, processed y n MP circuit nd the differentil outputs nd re used to compute the finl output ccording to (. For mny pplictions, for instnce in pttern recognition, only the sign of is importnt, so the output stge could potentilly e replced y just comprtor. Input Signls Fig.. Input Stge log log log log L L L L MP sed circuit log-likelihood domin computtion unit Generic rchitecture of n MP-sed nlog processor Output Stge e e The pper is orgnied s follows: Section II riefly descries the mthemticl concepts underlying the MP sed pproximtion followed y section III which presents exmples of MP circuits implementing the sic mthemticl functions. We juxtpose the theoreticl nd mesured results in this section to demonstrte the is-sclle property of the MP circuits. All the mesured results presented in this pper hve een otined from circuits prototyped in stndrd.µm CMOS technology. Section IV nlyes the sclility of MP circuits with respect to fundmentl noise, trnsistor mismtch nd speed. In section V, we illustrte the synthesis procedure y designing n MP sed liner pttern clssifier. In section VI, we conclude the pper y descriing some extensions of the proposed work. (3 Output Signls Fig.. II. LOG-SUM-EXP APPROXIMATION USING MARGIN L PROPAGATION Li- Illustrtion of reverse wter-filling procedure Given set of likelihood scores L i, i =,..,N, generl form of log-sum-exp function is given y ( N log = log e Li. ( In MP pproximtion, log is estimted using which is computed ccording to reverse wter-filling constrint [6] expressed s: N [L i ] = γ, ( where[ ] = mx(, denotes threshold opertion nd γ represents prmeter of the lgorithm. The solution to eqution ( cn e visulied using Fig., where the cumultive score eyond (shown y the shded re equls γ. It cn e shown tht is PWL pproximtion to log nd the sketch of the proof is summried in Appendix I. For the rest of this pper, we will denote the MP function s = M(L,...,L N,γ. (6 The MP sed PWL pproximtion is illustrted grphiclly for the following function ( e L L φ(l,l = log, (7 e L e L which is commonly used for implementing communiction decoders [7], [8]. First, similr to our exmple in section I, we express φ( in terms of the differentil components of its rguments, i.e., L = L L, L = L L ( s e L L e L L φ(l,l =log, e L L e L L ( =log e L L e L L log ( e L L e L L Thus, φ(l,l is represented s difference of two logsum-exp functions, ech of which is pproximted y its MP equivlent s: φ(l,l M(L L,L L,γ M(L L,L L,γ. (9. (8
3 Fig. 3 shows the MP-sed PWL pproximtion of φ( for fixed vlue of γ, when one of the opernds L is vried. The figure lso shows the results when different numer of opernds L,..,L N re used in eqution (8. The results visully show tht s the numer of opernds increse, more liner splines re introduced in the MP function which improves the qulity of the pproximtion. This trde-off etween the pproximtion error nd the numer of opernds lso holds for other forms of the log-sum-exp functions. In [9] we used the MP pproximtion to implement n nlog LDPC decoder nd compred its performnce with n equivlent decoder [] using min-sum pproximtion of eqution (7. We would like to point out tht MP pproximtion is more generl thn the min-sum pproximtion nd oth pproximtions re identicl when γ. φ. (L... MP with opernds MP with opernds MP with 6 opernds log sum exp. Fig. 3. Trde-off etween qulity of pproximtion nd the numer of opernds s chieved y MP function III. MP BASED COMPUTATIONAL CIRCUITS A. Bsic MP circuit A current-mode circuit implementing the sic reverse wter-filling eqution ( is shown in Fig.. The four opernds L L in (6 re represented y currents I L I L. The hyper-prmeter γ in (6 is implemented y the current I γ. The circuit in Fig. ers similrity to other prllel currentmode circuits like the winner-tke-ll circuits. However, s mentioned in the previous section, the MP implementtion is more generl nd for I γ the MP circuit implements winner-tke-ll function. The Kirchoff s current lw (KCL when pplied to node A, is equivlent to the reverse wterfilling condition [I L i I ] = I γ. I represents the MP pproximtion M(L,..,L,γ, nd [.] denotes rectify opertion which is implemented y the PMOS diodes P P. It cn e redily verified tht the PMOS diodes ensure tht V DS,Ni = V SG,Pi V DS,st, where V DS,Ni is the drinto-source voltge drop cross trnsistor N i,i =,..,, V SG,Pi is the voltge drop cross the PMOS diodes nd V DS,st is the sturtion voltge cross the current sink I γ. Thus, trnsistors N N re lwys mintined in sturtion irrespective of the opertion region (wek, moderte nd strong inversion. L I Z N Fig.. Vdd I L Vdd Input Stge IL IL I 3 L P P P3 P N N N3 N A Vdd CMOS implementtion of the core MP circuit. Vdd B I V dsst, V sg,p V dsst, This is importnt ecuse the opertion of the MP circuit relies on the ccurcy of the current mirrors formed y N N. Also, to ensure proper opertion of the core MP circuit the input currents hve to stisfy the constrint I Li I γ nd the input current rnge cn e expressed s µ nc ox ( W L ( V dd I Li. ( In the derivtion of the upper-ound of the input current rnge we hve ssumed tht the I Li >> I γ nd the PMOS nd the NMOS trnsistors hve similr drin-to-source voltge drops (implying V B V dd /. It should e noted tht the PMOS diodes in circuit shown in Fig. introduces threshold voltge drop which implies tht the supply voltge hs to e greter thn V GS V DS. However, low-voltge implementtion of the circuit in Fig. is possile y replcing the PMOS diodes y low-threshold ctive diodes [], ut t the expense of lrger silicon re, higher power dissiption nd degrded speed nd stility. Fig.. Die microgrph of the chip We hve prototyped progrmmle version of the MP circuit shown in Fig. in.µm stndrd CMOS process. The rchitecture of the prototype is shown in Fig. 6 nd its microgrph is shown in Fig.. A seril shift-register is used for selectively turning on nd off ech of different rnches of currents. In this mnner, the sme MP circuit in Fig. cn e reconfigured to implement different mthemticl functions. The rchitecture in Fig.6 uses n on-chip first-order Σ
D in SR SR SR Mesurement Circuit CLK CLK From SRs I Z V dd MP circuits I ADC current to e mesured Normlied rel numer I P N P3 N3 P N P N N I To ADC I I ref I ref Ltch Out t Normlition over time Rel numer Fig. 6. System rchitecture of the chip modultor s n nlog-to-digitl converter for mesurement nd clirtion of the output currents. Becuse the circuits used for implementing the Σ modultor hve een reported elsewhere [] we hve omitted its implementtion detils in this pper. The input currents nd hence the operting region (wek, moderte nd strong inversion of ll the trnsistors re djusted y controlling the gte voltges of the PMOS trnsistors. For ech of the operting regions, the ADC is reclirted to the mximum current nd the mesured results (presented in the next sections re normlied with respect to the mximum current. B. MP sed Addition To compute with respect to the two input vriles nd, R, R the opernds re first represented in their differentil forms = nd = with,,, R. Then, the opertion is rewritten s in Fig. = = ( ( = e log( e log(. ( All the opernds re then mpped into log-likelihood domin ccording to L = log, L = log, L = log, L = log, where L R, L R, L R, L R. Since, our circuit implementtion uses only unipolr current sources, we impose dditionl constrints,,, >. Due to the differentil representtion, this dditionl constrint will not ffect the finl output. Let denote log( nd denote log(. Then, ( cn e written s nd cn e expnded s = e e. ( = log ( = log (e log e log ( = log e L e L. (3 We now pply the MP pproximtion to the log-sum-exp functions to otin Similrly, M(L,L,γ. ( = log ( = log (e log e log ( = log e L e L M(L,L,γ ( Since the eqution ( nd the eqution ( uses only two opernds, its circuit level implementtion is identicl to Fig. except tht it hs only two input rnches insted of four. The single-ended circuit for ddition is shown in Fig. 7. I, the current through N, represents the vlue of. I L, I L nd I γ represents L, L nd γ respectively. Fig. 7. I N N I L P N I L MP circuit for implementing two opernd ddition Fig. 8 (-(c compres the output otined using floting-point implementtion of the log-sum-exp functions, using floting-point implementtion of the MP pproximtion nd from mesurements (normlied currents using the fricted prototype. P I
...... ( ( (c Fig. 8. Addition: computed ccording to ( the originl log-sum-exp function; ( MP pproximtions (simultion; nd (c MP circuits (mesurement....... ( ( (c......... (d (e (f Fig. 9. Addition: mesurement results for with trnsistors ised in (strong inversion; (moderte inversion; (cwek inversion; (d-(f computtion errors for (-(c. Fig. 9 (-(c show the mesurement results when the trnsistors re ised in ( strong inversion; ( moderte inversion; nd (c wek inversion. Fig. 9 (d-(f show the pproximtion error etween the mesured response nd the softwre model. The results vlidte our clim for the ddition opertion tht MP pproximtion is sclle cross different ising regions nd the circuit in Fig. 7 pproximtes the log-sum-exp function with resonle error. The error increses when the circuit is ised in the wek inversion nd s with ny su-threshold nlog VLSI circuit, which we ttriute to trnsistor mismtch, flicker-noise, nd errors introduced in mpping floting-point opernds into nlog opernds (currents. C. MP sed Sutrction Synthesiing n MP circuit for sutrction, with nd eing the two opernds is similr to the ddition opertion. The opertion cn e written in differentil form s = nd =, with,,, R nd stisfying,,, >. Thus, is written s = ( ( = ( ( = e log( e log( (6 which fter conversion into log-likelihood domin L = log, L = log, L = log, L = log,
6 leds to nd = log ( = log (e log e log ( = log e L e L M(L,L,γ (7 = log ( = log (e log e log ( = log e L e L M(L,L,γ (8 where nd re differentil forms of the output loglikelihoods s descried in (. The circuit implementtion of sutrction is identicl to tht of ddition, except tht the input differentil currents re now permuted. Fig. compres the mesured response with softwre sed log-sum-exp nd MP implementtion. Also, Fig. (-(c shows the mesured results when the trnsistors re ised in the three different operting regimes. Similr to the results otined for the ddition opertion, the mesured result show the is-sclle opertion of the MP circuits. D. Four qudrnt multipliction Implementing single qudrnt multiplier using MP circuit is stright-forwrd ecuse = in its logrithmic form cn e expressed s sum of likelihoods L = L L, where L x denotes the logrithm of the opernd x. Thus, single qudrnt multiplier requires only current summtion nd cn e implemented using KCL. For n MP-sed fourqudrnt multiplier the opertion is gin expressed in terms of the differentil opernds s: = ( ( = ( ( = e log( e log(. (9 After log-likelihood mpping, the following differentil forms re otined nd = log ( = log (e log( e log( M ( L L,L L,γ ( = log ( = log (e log( e log( M ( L L,L L,γ ( The circuit implementing n MP sed four-qudrnt multiplier is shown in Fig.. I denotes the vlue of. I γ representsγ. I L (I L, I L (I L represent L (L, L (L respectively. Fig.. I I L I L I L I L MP circuit for implementing four qudrnt opernd multipliction Fig. 3 compres the mesured output with the idel implementtion of the model ( nd (. Fig. (-(c shows the mesurement results when the trnsistors re ised in: ( strong; ( moderte nd (c wek inversion long with the respective pproximtion error. Agin, the mesured results show the is-scling property of MP circuits. E. Single Qudrnt Division Like the implementtion of single qudrnt multipliction, implementing single qudrnt division in logrithmic domin is stright-forwrd. For opernds, >, division in the logrithmic domin is equivlent to sutrction opertion L L which cn e implemented using KCL. To implement four qudrnt division, the opertion first requires extrcting the sign of the opernd s x = sgn(x x, where x is the solute vlue of x. A single qudrnt division cn now e used for the solute vlues nd the sign of the finl output cn e computed using only comprtors. F. Power nd Polynomil Computtion For positive opernds stisfying R, >, computing n, n Z in the logrithmic domin is equivlent to log( n = n log(, I = n L. ( which cn e implemented using simple current mirror s shown in Fig.. Thus, in MP-sed synthesis, power functions like squre-root or cue-root cn e implemented using current mirrors which significntly simplifies the implementtion compred to trnsliner sed synthesis [7]. However, for four-qudrnt implementtion of power nd polynomil function would require evluting Tylor expnsion using multipliction nd ddition opertions. Detils of the implementtion is provided in the Appendix II. As n exmple, four qudrnt squre function cn e implemented y expressing it s product of two differentil opernds s ( ( which hs n rchitecture similr to the four-qudrnt multiplier descried in section III- D. The differentil output nd corresponding to the
7.... (. (. (c Fig.. Sutrction: computed y ( originl functions; ( MP pproximtions (simultion; nd (c MP circuits (mesurement..... (. (. (c.3.3.3.. (d.. (e.. (f Fig.. Sutrction: mesurement results for with trnsistors ised in (strong inversion; (moderte inversion; (cwek inversion; nd (d-(f show respective pproximtion errors....... ( ( (c Fig. 3. Multipliction: computed ccording to the ( originl function; ( MP pproximtions (simultion; nd (c MP circuits (mesurement.
8...... ( ( (c......... (d (e (f Fig.. Multipliction: mesurement results for with trnsistors ised in (strong inversion; (moderte inversion; (cwek inversion; nd (d-(f the respective pproximtion errors. I L W L M M I Z W L. Originl function MP simultion Strong inversion Moderte inversion Wek inversion Fig.. domin. A current mirror implementing the power function in logrithmic. squre opertion cn e expressed in terms of differentil loglikelihood inputs L,L s M ( L,L,γ (3 = logl L ( Fig.6 shows the mesured results when the circuit in Fig. hs een configured to compute four-qudrnt squre function. Agin, the circuit is shown to pproximte the log-sumexp function for strong-inversion, moderte-inversion nd wek-inversion ising conditions. IV. SCALABILITY ANALYSIS OF MP CIRCUIT In Fig. 3, we showed tht the ccurcy of the PWL pproximtion improves when the numer of opernds in the mrgin function increses. In this section, we nlye the effect of hrdwre rtifcts (noise, mismtch, speed nd input/output impednce on the performnce nd sclility of MP circuits. 6 6 Fig. 6. Power: computed ccording to the log-sum-exp function, the MP pproximtion simultion nd the MP circuit mesurement. A. Noise nd Mismtch Anlysis of MP circuit The noise model for the sic MP circuit in Fig. is shown in Fig. 7. For the following nlysis we will ssume tht the numer of input current rnches in the MP circuit equls K. Due to the MP constrint ( we will ssume tht the noise in only K of the prllel rnches re uncorrelted (due to KCL constrints. The totl output noise power I n, cn therefore e expressed s: I n, = K [ I n,sourcei (K I n,n i (K In,P i (K g m R source I n,sink K I n,n ( ]
9 Vdd Vdd V dd current source in Fig. 7 ccording to I n, N I n, source I n,n I n,p N B R source P P PK A N I n, sin k Fig. 7. Noise model for MP circuit in Fig.. R sink where I n,sourcei, I n,ni, I n,pi, nd I n,sink re noise currents s shown in Fig. 7 nd R source is the output impednce of the current sources. We hve simplified the expression in ( y ssuming K to e lrge nd y ssuming tht the smll-signl prmeters g m,r source for ll the trnsistors re pproximtely equl. Eqution ( cn e written s I n,,therml = [ K g m R source K NK ] 8 3 ktg m, (6 where we hve considered only the effect of therml noise s given y I n = 8/3kTg m [3]. The effect of the flicker-noise hs een ignored in eqution (6, ecuse it exhiits similr dependency on K s the therml-noise. Eqution (6 shows tht s the numer of rnches (or opernds K increses, the output noise reduces nd in the limit K, the output noise is just equl to the noise due to trnsistor N. The reduction in the output noise with the increse in K cn e ttriuted to the ttenution nd verging effect of the K stge current mirrors formed yn N K. Also, it is importnt thtg m R source >>, not only for reducing the effect of noise ut lso to ensure tht the input impednce of the MP circuit is much lower thn the output impednce of the current sources (see Fig. 7. The mismtch nlysis for the MP circuit follows similr procedure s the noise nlysis. The MOS trnsistor mismtch models used in the nlysis re sed on the work reported in [] which uses two technology dependent constnts A VT nd A β tht determine the mismtch vrince s: σ ( V T = A V T WL ( σ( β = A β β WL (7 (8 V T denotes the threshold voltge, β = µc ox W/L, W nd L represent the width nd length of the MOS trnsistor. It is lso climed in [] tht for MOS trnsistors the mismtch in V T domintes the mismtch in β. Therefore, in this pper, our nlysis is only e sed on the V T mismtch. The error vrince σ ( V T cn e converted into n equivlent noise σ (I gmσ ( V T = gma V T (9 WL Hence, similr to the noise nlysis, the error vrince in the output current σ (I is given y: σ (I ( K gmr sourcek gma V T WL. (3 Eqution (3 shows tht for lrge K the ccurcy of the MP circuit is limited y the mismtch due to the output current mirror stge N. Thus, ll lyout nd impednce trnsformtion methods which re used for improving the ccurcy of current mirrors, like centroid lyout nd cscoding techniques, cn e used to improve the ccurcy of MP circuits. The use of cscoding t the output stge trnsistor N increses the output impednce nd mkes the core MP module sclle to implement lrge networks. B. Speed nd Bndwidth Anlysis As with other current-mode circuits, the trnsient response of the MP circuits is determined y its: ( slew-rte which determines the lrge-signl response; nd ( ndwidth which determines the smll-signl response. The nodeain Fig. hs the lrgest cpcitnce nd is dischrged y the current I γ. Therefore, the slew-rte of the MP circuit cn e expressed s: I γ SR A = (3 K(C GS,N C GS,P C DB,P where C GS,N represents the gte-to-source cpcitnce of NMOS trnsistor N through N K, nd C GS,P /C DB,P denotes the gte-to-source/drin-to-ody cpcitnce of the PMOS trnsistor P through P K. Eqution (3 shows tht the slewrte will reduce when the numer of input rnches K increse. However, the smll-signl ndwidth of the circuit remins invrint with respect K nd is determined y the frequency of the pole locted t node A nd is given y g m,p f 3dB,A = π(c GS,N C GS,P C DB,P. (3 g m,p represents the smll-signl gte referred trnsconductnce for the PMOS trnsistors P P K. The slew-rte nd ndwidth nlysis show tht scling of MP circuit is limited y the ppliction specific speed requirements which is controlled y the current I γ. However, unlike trnsliner synthesis this limittion cn e overcome in MP circuit y re-ising the circuit in strong-inversion. C. Performnce comprison with Trnsliner synthesis In this section, we summrie nd compre the performnce of CMOS circuits designed using MP sed synthesis with circuits designed using trnsliner synthesis. The comprisons mde here re qulittive in nture ecuse mny of the performnce metrics (silicon re nd power dissiption re dependent on the topology nd complexity of the trnsliner (TL circuit.
Accurcy: MP sed synthesis relies on PWL pproximtions t the lgorithmic level. However, the ccurcy of its circuit level implementtion using CMOS currentmode circuit is only limited y mismtch. Our nlysis in the previous section showed tht the techniques used for designing precision current-mirrors cn lso e used to improve the ccurcy of the MP circuits. TL sed synthesis, on the other hnd, re precise t the lgorithmic level. However, their mpping to CMOS currentmode circuits is pproximte due to the effect of suthreshold slope (unlike ipolr trnsistors nd finite drin impednce. These rtifcts lso introduce temperture dependency in TL circuits, wheres MP circuits re theoreticlly temperture invrint. Also, due to their opertion in wek-inversion, CMOS TL circuits re more prone to errors due to mismtch, wheres the MP circuits cn e re-ised in strong-inversion if higher precision is desired, ut t the expense of higher power dissiption. Dynmic rnge: The is-sclle property of the MP circuit enles it to chieve lrger dynmic rnge compred to TL circuit. Also, MP synthesis uses currents to represent log-likelihoods which cn lso chieve lrger dynmic rnge. Speed: For n MP circuit higher speed cn e chieved y re-ising the circuit in strong-inversion, without chnging the circuit topology. For CMOS trnsliner circuits higher speed cn e chieved y the use of lrge currents which implies using lrge sie trnsistors to mintin their opertion in the wek inversion. This will increse the silicon re when compred to n equivlent MP circuit. V. APPLICATION EXAMPLE: PATTERN CLASSIFIER One of the pplictions where ultr-low power nlog computing is ttrctive is pttern clssifiction [], []. Most pttern clssifiers do not require high precision nd ny nlog rtifcts (like mismtch nd non-linerity cn e clirted using n offline trining procedure. In this exmple, we use the MP circuits to design is-sclle liner clssifier. A liner clssifier implements n inner-product computtion etween weight vector w R n nd the clssifier input x R n nd is given y f = w T x (33 The weight vector w is determined y n offline trining procedure [] using leled trining dtset. The two vectors w nd x re expressed in differentil form s w = w w = [w,w,...,w N ]T [w,w,...,w N ]T x = x x = [x,x,...,x N ]T [x,x,...,x N ]T (3 with ll the w i (w i nd x i (x i R. Then eqution (33 cn e rewritten s T N ( w x = w i x i wi N ( x i w i x i wi x i = e log N (w i x i w i x i e log N (w i x i w i x i = e e. (3 when nd re expressed s = log N ( w i x i w i x i M(L w L x,l w L x,..., L wn L xn,l wn L xn,γ N ( = log w i x i w i x i M(L w L x,l w L x,..., L wn L xn,l wn L xn,γ. (36 L wi (L wi nd L xi (L xi denote the logrithms of w i (w i nd x i (x i in eqution (3. For inry clssifier, only the sign of is importnt, implying tht the output stge in the rchitecture (Fig. could e implemented using comprtor (insted of e e. The system rchitecture nd circuit digrm for the pttern clssifier is shown in Fig. 8. I Input Signls I I Lw Lx I I Lw Lx w x w N x N Input Stge logw logx logw N logx N I Lw N I Lx N Inner Product with MP Circuit I Lw N I Lx N I Comprtor Fig. 8. System rchitecture nd circuit digrm of the MP-sed pttern clssifier. In this exmple, we generted synthetic linerly seprle dtset corresponding to clsses: clss I nd clss II. We used liner support vector mchine trining lgorithm [3] to determine weight w. The prmeter vector w is then progrmmed s currents on the prototype chip. Test vectors x were selected rndomly nd were then pplied s input to the chip. For the ske of visulition, we show only results from two-dimensionl dtset. In this specil cse, the clssifier eqution (33 reduces to two-dimensionl inner product formultion s descried in the exmple in section I. The circuit implementtion is therefore identicl to tht of full qudrnt multiplier in Fig.. Fig. 9 shows the clssifiction oundry nd the clssifier scores (negtive vlues re represented y drker shdes, wheres positive vlues re represented y lighter shdes otined from softwre simultions nd from the fricted prototype under different ising conditions. Agin, the results show tht the opertion of the prototype is is-sclle nd the clssifiction performnce hs een
x..6.6 Clss I Clssifiction oundry Clss II...6.6. x (.. x..6..6..,.6.6. x (....6...6...6. x x x.6...6.6...6...6.6...6...6.6.. x x x (c (d (e Fig. 9. Clssifier: simulted y ( log-sum-exp function (simultion; ( MP pproximtion (simultion; nd mesured when the trnsistors re ised in (cstrong inversion (mesurement; (dmoderte inversion (mesurement; (ewek inversion (mesurement. verified to e similr to tht of the softwre model. VI. CONCLUSIONS AND DISCUSSIONS Unlike trnsliner circuits, current-mode circuits which rely only on universl conservtion principles like the KCL re portle cross different ising regions of MOS trnsistor. In this pper, we proposed n MP sed synthesis technique which cn e used to design nlog circuits tht re is sclle. At the core of MP synthesis is PWL pproximtion of the log-sump-exp function nd in this pper we hve shown tht mny mthemticl functions cn e derived in terms of log-sum-exp nd hence MP functions. MP circuits use only ddition, sutrction nd threshold opertions to pproximte log-sum-exp functions nd hence cn e implemented using few trnsistors. The min dvntges of the proposed MP synthesis technique re summried s follows: The circuits cn operte over wider dynmic rnge s the trnsistors cn spn different ising regions. MP synthesis provide systemtic pproch for designing MOS circuits operting in the moderte inversion region. The use of KCL nd thresholding ensures tht MP circuits re insensitive to temperture vritions. MP circuits operte in the log-likelihood domin where complex functions such s multipliction, division, squre-root opertions re mpped into simpler opertions like ddition, sutrction or scling, mking circuit design less complex. Thus, lrge-scle nlog systems which use non-liner functions (e.g. support vector mchines [] cn e esily implemented using MP circuits. The opertion of the MP circuits in the log-likelihood domin enhnces the dynmic rnge of the circuit which is importnt for nlog recursive computtions like the ones used in Hidden Mrkov Model (HMM decoding. However, there re severl open chllenges which we envision will e prt of the future work in this re. The role of the hyper-prmeter γ in the circuit synthesis hs not een fully explored. It is nticipted tht γ will ply key role in controlling the trde-off etween the pproximtion error nd the power dissiption of the MP circuit. Also, the ppliction of MP-sed synthesis cn e extended to other types of nlog circuits like chrge-mode or time-domin circuits y using other universl conservtion principles like chrge conservtion. Like trnsliner synthesis, MP synthesis could potentilly e extended towrds designing dynmic nlog circuits like filters. Also, extending this work to operte with rel nd imginry quntities would lso form prt of the future work. APPENDIX I PROOF: MP AS PWL APPROXIMATION OF THE LOG-SUM-EXP FUNCTION Proof: Given set of opernds L,...,L N, L i R,i =,...,N, the generlied form of log-sum-exp function is given y ( N log = log e Li. (37 The derivtive of log with respect to one of the opernds L k, k N is given y log L k = e L k N eli = e L k log N (eli log. (38
We introduce vrile nd define step functionu(l i, ccording to {, Li, u(l i, = i N., L i <, It cn e redily verified (shown in Fig. tht u(l i, log is lower-ound for the function e (Li log. Also, the use of step-functions to pproximte grdients will led to piecewise liner functions when integrted. Eqution (38 cn e pproximted s log L k L k = u(l k, N u(l i,. (39 Becuse the differentil eqution (39 consists of discontinuities, we will find its solution only in the continuous regions. For instnce, we will ssume tht the integrl is computed in the region where N u(l i, which is constnt. The solution to eqution (39 cn e written s i = [L iu(l i,] γ N u(l ( i, where γ is n constnt of integrtion. Eqution ( leds to N [L i ]u(l i, = γ ( N [L i ] = γ ( which is the MP function = M(L,...,L N,γ. Fig..... u(l i, e (L i.. L i The curve of e (L i nd its PWL pproximtion curve of u(l i, APPENDIX II MP SYNTHESIS OF POWER FUNCTIONS For n opernd, R, its power function is expressed s n, n Z. As illustrted in section III, is expressed in its differentil form =, ( R nd ( >. For ritrry n, n cn e written s ( n = i = i: even ( i Cn i ( n i( i C i n ( n i( i i: odd Cn i ( n i( i = e log ( i: even Ci n( n i ( i e log ( i: odd Ci n( n i ( i. (3 Designte s log( i: even Ci n( n i ( i nd to represent log( i: odd Ci n ( n i ( i in (3. Then M(L,γ. L (L is group of L i s stisfying M(L,γ. ( L i = logc i n (n ilog ilog, ( i [,n] nd i is even(odd. REFERENCES [] E. A. Vitto, Low power design: Wys to pproch the limits, in IEEE ISSCC 99 Dig. Tech. ppers, Sn Frncisco, CA, 99, pp. 8. [] S. Chkrrtty nd G. Cuwenerghs, Su-microwtt nlog VLSI trinle pttern clssifier, IEEE J. Solid-Stte Circuits, vol., no., pp. 69 79, My. 7. [3] G. E. R. Cown, R. C. Melville, nd Y. P. Tsividis, A VLSI nlog computer/digitl computer ccelertor, IEEE J. Solid-Stte Circuits, vol., no., pp. 3, Jn. 6. [] B. Gilert, Trnsliner circuits: A proposed clssifiction, Electron. Lett., vol., no., pp. 6, 97. [] E. Seevinck, R. F. Wssenr, nd H. C. K. Wong, A wide-nd technique for vector summtion nd rms-dc conversion, IEEE J. Solid- Stte Circuits, vol. 9, no. 3, pp. 3 38, 98. [6] A. G. Andreou nd K. A. Bohen, Trnsliner circuits in suthreshold MOS, Anlog Integrted Circuits nd Signl Processing, vol. 9, no., pp. 66, 996. [7] T. Serrno-Gotrredon, B. Linres-Brrnco, nd A. G. Andreou, A generl trnsliner principle for suthreshold MOS trnsistors, IEEE Trns. Circuits Syst. I, vol. 6, no., pp. 67 66, My. 999. [8] B. A. Minch, Synthesis of sttic nd dynmic multiple-input trnsliner element networks, IEEE Trns. Circuits Syst. I, vol., no., pp. 9, Fe.. [9] E. Seevinck nd R. J. Wiegerink, Generlied trnsliner circuit principle, IEEE J. Solid-Stte Circuits, vol. 6, no. 8, pp. 98, 99. [] B. A. Minch, MOS trnsliner principle for ll inversion levels, IEEE Trns. Circuits Syst. II, vol., no., pp., Fe. 8. [] C. C. En, F. Krummerncher, nd E. A. Vitto, An nlyticl MOS trnsistor model vlid in ll regions of opertion nd dedicted to lowvoltge nd low-current pplictions, Anlog Integrt. Circuits Signl Process. J., vol. 8, pp. 83, Jul. 99. [] F. Silveir, D. Flndre, nd P. G. A. Jespers, A floting-gte MOS lerning rry with loclly computed weight updtes, IEEE J. Solid- Stte Circuits, vol. 3, no. 9, pp. 3 39, Sep. 996. [3] S. Chkrrtty nd G. Cuwenerghs, Gini-support vector mchine: Qudrtic entropy sed multi-clss proility regression, Journl of Mchine Lerning Reserch, vol. 8, pp. 83 839, 7. [] M. Gu, K. Misr, H. Rdh, nd S. Chkrrtty, Sprse decoding of low density prity check codes using mrgin propgtion, in Proc. of IEEE Gloecom, Nov. 9. [] K. L. Hsiung, S. J. Kim, nd S. Boyd, Trctle pproximte roust geometric progrmming, Optimition nd Engineering, vol. 9, no., pp. 9 8, Jun. 8. [6] T. M. Cover nd J. A. Thoms, Elements of Informtion Theory, nd ed. Wiley-IEEE, 6. [7] H.-A. Loeliger, F. Lustenerger, M. Helfenstein, nd F. Trköy, Proility propgtion nd decoding in nlog VLSI, IEEE Trns. Inform. Theory, vol. 7, no., pp. 837 83, Fe..
3 [8] C. Winsted, N. Nguyen, V. Gudet, nd C. Schlegel, Low-voltge CMOS circuits for nlog itertive decoders, IEEE Trns. Circuits Syst. I, vol. 3, pp. 89 8, Apr. 6. [9] M. Gu nd S. Chkrrtty, A pj/it, (3,8 CMOS nlog lowdensity prity-check decoder sed on mrgin propgtion, IEEE J. Solid-Stte Circuits, vol. 6, no. 6, pp. 33, Jun.. [] S. Hemti, A. Bnihshemi, nd C. Plett, An 8-M/s.8-µm CMOS nlog min-sum itertive decoder for (3,8, LDPC code, IEEE J. Solid-Stte Circuits, vol., pp. 3, Nov. 6. [] Y.-H. Lm, W.-H. Ki, nd C.-Y. Tsui, Integrted low-loss CMOS ctive rectifier for wirelessly powered devices, IEEE Trns. Circuits Syst. II, vol. 3, no., pp. 378 38, Dec. 6. [] A. Gore, S. Chkrrtty, S. Pl, nd E. C. Alocilj, A multichnnel femtompere-sensitivity potentiostt rry for iosensing pplictions, IEEE Trns. Circuits Syst. I, vol. 3, no., pp. 37 363, Nov. 6. [3] B. Rvi, Design of nlog CMOS integrted circuits. McGrw-Hill Science/Engineering/Mth, Aug.. [] P. R. Kinget, Device mismtch nd trdeoffs in the design of nlog circuits, IEEE J. Solid-Stte Circuits, vol., no. 6, pp., Jun.. [] K. Kng nd T. Shit, An on-chip-trinle gussin-kernel nlog support vector mchine, IEEE Trns. Circuits Syst. I, vol. 7, no. 7, pp. 3, Jul.. Ming Gu received her B.E. nd M.E. degree in School of Automtion Science nd Electricl Engineering, Beijing University of Aeronutics nd Astronutics, Beijing, Chin, in nd 3, respectively, nd the M.E. degree in electricl nd computer engineering from Michign Stte University, Est Lnsing, MI, in 8. From 3 to 6, she worked s Reserch Engineer in the Center for Spce Science nd Applied Reserch, Chinese Acdemy of Sciences, Beijing, Chin. She is currently working towrds her Ph.D degree t Michign Stte University, Est Lnsing, MI. Her reserch interests include mixed signl VLSI circuits design, error-control coding, nd nlog signl processing. Shntnu Chkrrtty (SM 99-M -S 9 received his B.Tech degree from Indin Institute of Technology, Delhi in 996, M.S nd Ph.D in Electricl Engineering from Johns Hopkins University, Bltimore, MD in nd respectively. He is currently n ssocite professor in the deprtment of electricl nd computer engineering t Michign Stte University (MSU. From 996-999 he ws with Qulcomm Incorported, Sn Diego nd during he ws visiting resercher t The University of Tokyo. Dr. Chkrrttys work covers different spects of nlog computing, in prticulr non-voltile circuits, nd his current reserch interests include energy hrvesting sensors nd neuromorphic nd hyrid circuits nd systems. Dr. Chkrrtty ws Ctlyst foundtion fellow from 999- nd is recipient of Ntionl Science Foundtions CAREER wrd nd University Techer-Scholr Awrd from MSU. Dr. Chkrrtty is senior memer of the IEEE nd is currently serving s the ssocite editor for IEEE Trnsctions of Biomedicl Circuits nd Systems, ssocite editor for the Advnces in Artificil Neurl Systems journl nd review editor for Frontiers of Neuromorphic Engineering journl.