Diffusion leaky LMS algorithm: analysis and implementation

Transcription

1 1 Dffuson leay LMS algorthm: analyss and mplementaton Lu Lu, Haquan Zhao* School of Electrcal Engneerng, Southwest Jaotong Unversty, Chengdu, Chna. COMMEN: hs paper has been publshed n Sgnal Processng ABSRAC he dffuson least-mean square (dlms) algorthms have attracted much attenton owng to ts robustness for dstrbuted estmaton problems. However, the performance of such flters may change when they are mplemented for suppressng noses from speech sgnals. o overcome ths problem, a dffuson leay dlms algorthm s proposed n ths wor, whch s characterzed by ts numercal stablty and small msadjustment for nosy speech sgnals when the unnown system s a lowpass flter. Fnally, two mplementatons of the leay dlms are ntroduced. It s demonstrated that the leay dlms can be effectvely ntroduced nto a nose reducton networ for speech sgnals. Keywords: Dstrbuted estmaton; Dffuson LMS; Leay LMS; Nosy speech sgnal. 1. Introducton Snce the smple structure and low computatonal burden, the least-mean square (LMS) algorthm has become a wdely used adaptve flter. However, n practce, t s well nown that drect mplementaton of the conventonal LMS algorthm can be problematc. Such problems can be outlned as follows. () Numercal problem, caused by nadequacy of exctaton n the nput sgnal; () stagnaton behavour, often occur n low nput sgnal scenaros, mang the performance degradaton of the algorthm. o solve these problems, the leay LMS algorthm [1] as ts varants were proposed n dverse felds, such as actve nose control (ANC) [2], channel estmaton [3], and nonlnear acoustc echo cancellaton (NLAEC) [4]. Recently, to estmate some parameters of nterest from the data collected at nodes dstrbuted over a geographc regon, several dstrbuted estmaton algorthms were developed, ncludng ncremental [5-6], and dffuson algorthms [7-10], etc. In the ncremental strategy, a cyclc path s requred the defnton over the nodes, and ths technque s also senstve to ln falures [11]. On the other hand, the dffuson method, s wdely used because of ts ease of mplementaton. In ths strategy, each node E-mal addresses: lulu@my.swjtu.edu.cn (L. Lu), hqzhao@home.swjtu.edu.cn (H. Zhao)

2 2 communcates wth a subset of ts neghbours, whch acheves more data from ther neghbours wth moderate amount of communcatons. Partcularly, n [12], two versons of the dffuson LMS (dlms) algorthms, adapt-then-combne (AC) and combne-then-adapt (CA) were proposed, based on the dfferent orders of adaptaton and combnaton steps. Note that the AC method outperforms the CA method n all cases, and better performance can be acheved f the measurement are shared. Due to ts mert, the AC verson of the dlms algorthm (AC dlms) has been ntroduced to subband adaptve flter [13], nformaton theoretc learnng (IL) [14] to obtan mproved performance. Motvated by these consderatons, n ths paper, we proposed a leay dlms algorthm wth two combnaton strateges-ac and CA, resultng n AC leay dlms and CA leay dlms algorthms. Very recently, a dstrbuted ncremental leay LMS was proposed to surmount the drft problem of dlms algorthm [15]. Unfortunately, t s derved from the ncremental approach, whch prohbts ts practcal applcatons. Compared wth the exstng algorthms, the leay dlms algorthm s derved by mnmzng the nstantaneous leay objectve functon rather than the mean square error cost functon. Moreover, t has a superorty performance va AC strategy. Notaton: hroughout ths paper, we use {} to represent a set, denotes absolute value of a scalar, () denotes transposton, 2 E{} denotes expectaton, tr{} denotes trace operator, 2 denotes l 2-norm, and {} o s the real value of parameter. Besdes, we use boldface and normal letters to denote the random quanttes and determnstc quanttes, respectvely. 2. Dffuson LMS strateges Consder a networ of N sensor nodes dstrbuted over a geographc area. At each tme nstant, each sensor node {1,2,..., N} has access to the realzaton { d( ), u. } of some zero-mean random process { d( ), u. }, and d ( ) s a scalar and u. s an regresson vector wth length M. Suppose these measurements follow a standard model gven by: d ( ) = u w + ν ( ) (1) o, where w o M 1 C s the unnown parameter vector, and ( ) ν s the measurement nose wth varance 2 σ. Here, we assume that v, u, and ν ( ) are spatally ndependent and ndependent dentcally dstrbuted (..d.), and ν ( ) s ndependent of u,. he dlms algorthm s obtan by mnmzng a lnear combnaton of the local mean square error (MSE): J ( w) = a E e = a E d ( ) u w (2) loc 2 2 l where N s the set of nodes wth whch node shares nformaton (ncludng tself). he weghtng coeffcent { a } are rea

3 3 non-negatve, and satsfy: a = 1. N l = 1 he dlms algorthm obtans the estmaton va two steps, adaptaton and combnaton. Accordng to the order of these two steps, the dffuson LMS algorthm s classfed nto the AC dlms and CA dlms algorthms. he updaton equaton of AC dlms can be expressed as ϕ, = w, 1 + µ c u ( dl( ) u w, 1) ( adaptaton) w, = a ϕ ( combnaton) (3) where µ s the step sze (learnng rate), and ϕ s the local estmates at node. he weghtng coeffcents { c } s the rea non-negatve, satsfyng the condton c = a = 0 f l N. Smlarly, the CA dlms algorthm can be gven as ϕ, 1 = a w 1 ( combnaton) w, = ϕ, 1 + µ c u ( dl( ) u ϕ, 1) ( adaptaton). (4) 3. Proposed dffuson leay LMS algorthm of In ths secton, the leay dlms s proposed to address the two problems mentoned above. For each node, we see an estmate o w by mnmzng the followng cost functon: loc J ( w) = a E e + γw w = d ( ) u + a E l w γw w (5) where γ>0 s the leaage coeffcent, and w s the estmate of o w. Usng the steepest descent algorthm, yelds loc J ( w) = w a + γ { E e w w} w. (6) herefore, the updatng of proposed algorthm for estmatng o w at node can be derved from the steepest descent recurson { E e + γw w}, =, 1 µ w. 1 l N w w w a. (7) Under the lnear combnaton assumpton [12,14], let us defne the lnear combnaton w, at node as w = a ϕ. (8), 1 1

4 4 Introducng (8) to (7), we can obtan an teratve formula for the ntermedate estmate E e + γw w ϕ ϕ µ w E e + γw w ϕ 1 µ w = (1 γµ ) ϕ + µ u e. = 1 w. 1 1 l. wl. 1 (9) Hence, we obtan the leay dlms algorthm by transformng the steepest-descent type teraton of (7) nto a two-step teraton E e + γw w ϕ µ w w, = a ϕ.,, = w, 1 w. 1 (10) In adaptaton step of (10), the local estmate ϕ, 1 s replaced by lnear combnaton w, 1. Such substtuton s reasonable, because the lnear combnaton contans more data nformaton from neghbor nodes than ϕ, 1 [12]. hen, we extend the leay dlms algorthm to ts AC and CA forms. AC leay dlms algorthm: ϕ, = (1 µγ ) w, 1 + µ c u ( dl( ) u w, 1) ( adaptaton) w, = a ϕ ( combnaton). (11) CA leay dlms algorthm: ϕ, 1 = a w 1 ( combnaton) w, = (1 µγ) ϕ, 1 + µ c u ( dl( ) u ϕ, 1) ( adaptaton). (12) 4. Smulaton Results In the followng, we evaluate the performance of the proposed algorthm for nose reducton networ wth 20 nodes. he topology of the networ s shown n Fg. 1. he unnown vector of nterest s a lowpass flter of order M=5, whose coeffcents and normalzed frequency response are shown n Fg. 2(a-b). he regressors are zero-mean whte Gaussan dstrbuted wth covarance matrces R u, = σ I, wth 2 u 2 σ u shown n Fg. 2(c). he performance of the dfferent algorthms s measured n terms of the networ N o mean square devaton (MSD) MSD = 10log 10( wl w / N). All networ MSD curves are obtaned by ensemble averagng over 50 ndependent trals. l = 1

5 5 Fg. 1. Networ topology. 5.1 Example 1: Gaussan nput In ths example, the Gaussan sgnal wth zero mean and unt varance s employed as the nput sgna a, = 1/ n s used for l { a }, and c = 1/ n (unform,[16]), where n s the degree of node. he sgnal-to-nose rato (SNR) of each node s SNR=0dB, correspondng to the hgh bacground nose power. Ampltude Ampltude (db) Regressor varances ap Normalzed Frequency ( π rad/sample) (c) Node number (a) (b) Fg. 2. (a) Coeffcents of the flter. (b) Frequency responses of the unnown system. (c) Regressor varances. 0 baselne -5 MSD(dB) CA leay dlms AC leay dlms node 7 node 20 node Step sze Fg. 3. Networ MSD of the proposed algorthms versus dfferent step szes

6 6 MSD(dB) CA leay dlms(γ=0.001) CA leay dlms(γ=0.002) CA leay dlms(γ=0.1) AC leay dlms(γ=0.001) AC leay dlms(γ=0.002) AC leay dlms(γ=0.1) Number of teratons Fg. 4. Networ MSD of the proposed algorthms versus dfferent γ (µ=0.08). Fg. 3 shows the steady-state networ MSD of the leay dlms wth dfferent step-sze. As one can see, the steady-state MSDs of the proposed algorthms ncrease as the step szes ncrease. For µ>0.668 (node 5), the CA leay dlms fals to wor. Besdes, by usng (31), we can obtan the theory upper bounds of step szes from dfferent nodes. he calculatng data ndcate that, the upper bound of step sze from node 1 s 1.1 (maxmum upper bound among 20 nodes), and upper bound of step sze from node 7 s about 0.4 (mnmum upper bound among 20 nodes). o guarantee all the nodes wor wel µ must be µ<0.4 to ensure that the stablty. As a result, we select µ=0.08 n the followng smulatons. hen, we nvestgate the performance of the algorthm n dfferent γ, as shown n Fg. 4. It can be easly observed that the CA/AC leay dlms algorthm s not senstve to ths choce, but t turns out that the best opton s γ= Fg. 5 dsplays a comparson of MSE from the proposed algorthm and exstng algorthms. One can see that the proposed algorthm outperforms the conventonal dlms algorthms n terms of steady-state error. After the about teraton 50, all the algorthm reach steady state. In addton, t s obvously shown from these curves that the proposed algorthms reach low msadjustment ( 14dB and 18dB) n about teraton 60, whereas the CA dlms and AC dlms algorthms fnally settle at a networ MSD value of 12dB and 16dB. All the AC-based algorthms acheve mproved performance as compared wth CA-based algorthms, wth the smlar ntal convergence rate.

7 CA dlms(µ=0.1) CA leay dlms(µ=0.08,γ=0.002) AC dlms(µ=0.1) AC leay dlms(µ=0.08,γ=0.002) MSD(dB) Number of teratons Fg. 5. Networ MSD learnng curves for Gaussan nput. 5.2 Example 2: Speech nput sgnal In the second example, the nput sgnal s a speech nput (See Fg. 6). In pratcal applcaton, dfferent nodes have dfferent 2 speech sgnals. herefore, we consder a nput sgnal generated as uu, = σu speechnput. he envronment and parameters of all the algorthms are same as those of example 1. In Fg. 7, t demonstrates a comparson of networ MSD performance of dfferent algorthms. In ths cases, superor performance of the proposed algorthms are obtaned because that the leay method can stablze the system [1]. he proposed algorthm, whch adopts the AC and CA approaches, reduces the msadjustment when the speech s selected as the nput sgnal. Meanwhle, the fast convergence rate s obtaned. Furthermore, t can be obvously observed from ths fgure that the msadjustment for the AC verson s less than that of CA verson. o further demonstrate the performance of proposed algorthm, Fg. 8 depcts the waveforms of the 2 smulaton result at node 14 ( σ = 0.35). he AC leay dlms algorthm, nherts the advantages of the leay algorthm, and u acheves a stablty performance n the presence of strong dsturbances. 5 3 Ampltude Number of teratons Fg. 6. Input speech sgnals

8 CA dlms(µ=0.1) CA leay dlms(µ=0.08,γ=0.002) AC dlms(µ=0.1) AC leay dlms(µ=0.08,γ=0.002) MSD(dB) Number of teratons Fg. 7. Networ MSD learnng curves for speech nput. 3 (a) 1-1 Ampltude (b) Number of teratons Fg. 8. Waveforms of the smulaton result at node 14. (a) Nosy speech (green lne: nose sgnal wth SNR=0dB; blue lne: clean speech). (b) Fltered speech wth Algorthms the AC leay dlms algorthm. able 1 Computaton tme Computaton me(s) AC dlms AC leay dlms CA dlms CA leay dlms o evaluate the computatonal burden, we have measured the average run executon tme of the algorthm on a 2.1-GHz AMD processor wth 8GB of RAM, runnng Matlab R2013a on Wndows 7. he computaton tme of the algorthms s outlned n able. 1. One can see that the AC-dLMS algorthm s the fastest one among these dstrbuted algorthms. he proposed algorthm slghtly ncreases the executon tme, but the fnal performance s lower than other algorthms. he CA algorthms acheve the slower than the AC algorthms, stll wth an affordable computaton tme.

9 9 5. Concluson In ths paper, a leay dlms algorthm wth AC and CA strateges has been proposed for dstrbuted estmaton. he proposed algorthm updates the local estmaton based on leay method, whch can not only mantan the stablty for dstrbuted system, but also enhance the performance for nose suppresson. Smulaton results n the context of nose-reducton system show that the proposed algorthm acheves an mproved performance as compared wth exstng algorthms. Acnowledgement hs wor was partally supported by Natonal Scence Foundaton of P.R. Chna (Grant nos: , , ). he frst author would also le to acnowledge the Chna Scholarshp Councl (CSC) for provdng hm wth fnancal support to study abroad (No ). References [1] K. Mayyas,. Aboulnasr, Leay LMS algorthm: MSE analyss for Gaussan data, IEEE rans. Sgnal Process. 45 (4)(1997) [2] J. Cheer, S.J. Ellott, Actve nose control of a desel generator n a luxury yacht, Appl. Acoust. 105 (2016) [3] D.B. Bhoyar, C.G. Dethe, M.M. Mushrf, A.P. Narhede, Leay least mean square (LLMS) algorthm for channel estmaton n BPSK-QPSK-PSK MIMO-OFDM system, Internatonal Mult-Conference on Automaton, Computng, Communcaton, Control and Compressed Sensng., Kottayam, 2013, pp [4] J.M. Gl-Cacho, M. Sgnoretto,.V. Waterschoot, M. Moonen, S.H. Jensen, Nonlnear acoustc echo cancellaton based on a sldng-wndow leay ernel affne projecton algorthm, IEEE rans. Audo Speech Lang. Process. 21 (9) (2013) [5] L. L, J.A. Chambers, C.G. Lopes, A.H. Sayed, Dstrbuted estmaton over an adaptve ncremental networ based on the affne projecton algorthm, IEEE rans. Sgnal Process. 58 (1) (2010) [6] Y. Lu, W.K.S. ang, Enhanced ncremental LMS wth norm constrants for dstrbuted n-networ estmaton, Sgnal Process. 94 (2014) [7] N. aahash, I. Yamada, A.H. Sayed, Dffuson least-mean squares wth adaptve combners: Formulaton and performance analyss, IEEE rans. Sgnal Process. 58 (9) (2010) [8] J. Chen, A.H. Sayed, Dffuson adaptaton strateges for dstrbuted optmzaton and learnng over networs, IEEE rans. Sgnal Process. 60 (8) (2012) [9] R. Abdolee, B. Champagne, A.H. Sayed, Estmaton of space-tme varyng parameters usng a dffuson LMS algorthm, IEEE rans. Sgnal Process. 62 (2) (2014) [10] R. Abdolee, B. Champagne, A.H. Sayed, Dffuson adaptaton over mult-agent networs wth wreless ln mparments, IEEE rans. Moble Computng 2015 DOI: /MC

10 10 [11] R. Abdolee, B. Champagne, Dffuson LMS strateges n sensor networs wth nosy nput data, IEEE/ACM rans. Networng 24 (1)(2016) [12] F.S. Cattvel A.H. Sayed, Dffuson LMS strateges for dstrbuted estmaton, IEEE rans. Sgnal Process. 58 (3) (2010) [13] J. N, Dffuson sgn subband adaptve flterng algorthm for dstrbuted estmaton, IEEE Sgnal Process. Lett. 22 (11) (2015) [14] C. L, P. Shen, Y. Lu, Z. Zhang, Dffuson nformaton theoretc learnng for dstrbuted estmaton over networ, IEEE rans. Sgnal Process. 61 (16) (2013) [15] M. Sowjanya, A.K. Sahoo, S. Kumar, Dstrbuted ncremental Leay LMS, Internatonal Conference on Communcatons and Sgnal Processng (ICCSP), Melmaruvathur, 2015, pp [16] V.D. Blonde J.M. Hendrcx, A. Olshevsy, J.N. stsls, Convergence n multagent coordnaton, consensus, and flocng, n Proc. Jont 44th IEEE Conf. Decson Control Eur. Control Conf.(CDC-ECC), Sevlle, Span, 2005, pp