Robust Filter Design based on Generalized Maximum-Likelihood Estimation

Transcription

1 Robust Flter Desgn based on Generalzed Maxmum-Lkelhood Estmaton STEFA LEISCHER, ROBERT KLISKI, HOLGER HUTZELMA, RUDI KORR Fraunhofer Insttute for Communcaton Systems Hansastr. 3, 8686 Munch GERMAY Abstract: Maxmum-lkelhood estmaton s a commonly used technque n dgtal sgnal processng. Unfortunately most dgtal flter algorthms sho a lack of robustness n mpulsve envronments, hch s a result of heavy tals n the correspondng nose dstrbutons as explaned n [,]. In ths paper, a novel method for robust flter desgn s presented hch s based on the generalzed maxmum-lkelhood estmaton (M-Estmaton) orgnatng from the mathematcal feld of robust statstcs. In comparson to the ell knon maxmum-lkelhood technque the novel method permts the usage of a generalzed score functon ncludng negatve flter coeffcents. Thus the enhancement leads to an mproved performance and robustness of the dgtal flter hen used n envronments contanng mpulse nose. Key-Words: robust flter, mpulse nose, M-Estmaton, robust statstcs, nfluence functon Introducton Tradtonal sgnal processng technques are ell knon and provde excellent performance n numerous applcatons. But unfortunately the tradtonal sgnal processng lterature has been domnated by the assumpton of Gaussan nose as a typcal model of nterference. In many nstances ths model s ell justfed by the Gaussan Central Lmt Theorem and leads to sutable solutons for many sgnal processng problems. Unfortunately some real-orld phenomena are decdedly non-gaussan such as underater acoustc sgnals, lo-frequency atmospherc nose, stchng transents on telephone lnes or other mpulsve sgnals. The man characterstc of these probablty denstes s the fact, that the values of mpulse bursts are located far outsde the central regon of the Gaussan dstrbuton, hch causes heaver tals (.e. a sloer decay n the outer regon) of the densty curve. A good model for those heavy-taled nose dstrbutons can be found n the stable dstrbutons [, ]. Due to the typcal nonconvergence of second order moments n heavy-taled dstrbutons the Yule-Walker resp. Wener-Hopf equatons, based on second order statstcs n the correlaton matrx, are extremely ll-condtoned and thus explan the poor performance of conventonal dgtal flters n mpulsve nose. In order to overcome these dffcultes a close nvestgaton of the maxmum-lkelhood prncple as the bass of tradtonal statstcal sgnal processng structures s presented n ths paper. Startng from recently publshed results [3, 4] an enhancement n performance and robustness of statstcal sgnal processng procedures n dgtal flterng can be realzed by usng the generalzed maxmum-lkelhood prncple orgnatng form the mathematcal feld of robust statstcs. Furthermore consderatons concernng the numercal complexty of the ntroduced dgtal flters are dscussed and a comparson beteen the computatonal costs n contrast to the necessary robustness features of an M- Estmator n mpulsve nose s dran. Ths paper s organzed as follos. The basc prncples of Maxmum-Lkelhood-Estmaton n sgnal processng ncludng aspects of robustness are dscussed n chapter. In chapter 3 a method for enhanced robust dgtal flter desgn s brefly analyzed th respect to numercal complexty and robustness features folloed by a performance evaluaton of the novel dgtal flter. A concluson s gven n chapter 4. Conventonal Maxmum-Lkelhood Flter Desgn. The Maxmum-Lkelhood Prncple A ell establshed method for estmatng unknon sgnals n nosy envronments s the maxmumlkelhood technque, n hch the locaton estmate ˆ descrbes the most lkely observaton value of the nput vector x accordng to (). (x ) σ ˆ = argmax e () = " π "" σ """! lkelhood functon h{ fgaussan ( x ) } In ths defnton of the maxmum-lkelhood prncple the mportance of the Gaussan probablty densty f Gaussan (x ) = πσ e (x ) σ

2 s to be dentfed. For a smplfcaton the exact ensemble averagng method s replaced by the tme averagng method n the follong explanatons. Thus h f Gaussan x takng the natural logarthm of { ( )} ln[ h{ f ( x ) }] = Gaussan ( ) ( x - ) πσ () ln = σ and omttng the rrelevant constant expressons, equaton () can be rertten as ˆ = argmn (x ) (3) = Usng maxmum-lkelhood estmaton method for flterng purposes, a eghtng-technque to capture the temporal correlaton of the nput sgnal s mandatory. Therefore t can be assumed that the dfferent sgnal samples are ndependent, but not dentcally dstrbuted. Ths behavor results n a common locaton parameter, but varyng scale factors S. Defnng K S =, ' (4) ' a larger value of the ntroduced eght (smaller scale) makes the dstrbuton of the observaton value x more concentrated around the locaton parameter, ncreasng the relablty of the sample x. Varable K can be chosen as mutable factor for specfc purposes. Thus replacng and transformng the lkelhood-functon h f Gaussan x - n () by { ( )} x hf Gaussan S results n a eghted lkelhood-functon th nonnegatve eghts ˆ = argmn (x ) ; =. (5) = ' To solve the mnmzaton problem n (5), takng the dervatve (x ) = (x ) = (6) = = and solvng (6) by, the locaton estmate ˆ can be obtaned from x ˆ =. (7) = = For a bandpass or hghpass-type behavor of the maxmum-lkelhood estmator real-valued (.e. postve and negatve) eghts are needed. Therefore a generalzaton from the meanng of the eghts as scales of nput samples accordng equaton (4) to a broader sense s necessary and can be reached by a smple extenson of (7) to avod the denomnator to become zero [3]. x ˆ = (8) = = In equatons (7) and (8) the maxmum-lkelhood estmator, based on the Gaussan probablty densty functon, can be recognzed to be the (lnear) normalzed eghted mean of nput samples. Ths behavor equals the ell knon FIR flter structure, usng a ndo sze of samples. Unfortunately ths lo numercal complexty method s not very robust n mpulsve nose envronments, hch s to be demonstrated n the follong secton.. Analyss of the Lack of Robustness The lack of robustness of the maxmum-lkelhood estmaton can be shon n to dfferent ays. Frst some aspects of tradtonal sgnal processng methods shall be dscussed folloed by aspects hch orgnate from the feld of robust statstcs. The consderatons concernng statstcs are the theoretcal foundaton of the proposed mprovements n robust flter desgn. A ell knon method for desgnng a FIR flter s based on the Wener flterng theory. Accordng to ths desgn algorthm the optmum flter coeffcents are calculated by =R - p (9) th R=E{x(n)x H (n)} auto-correlaton of x(n), cross-correlaton p=e{x(n)d*(n)} beteen x(n) and the desred sgnal d(n) As proved n [, ] the second order moments of heavytaled dstrbutons, such as the stable dstrbuton, does not exst, hch results n nfnte elements of the second-order based correlaton matrx R. Invertng such ll-condtoned matrces leads to severe mathematcal problems. Thus the smple desgn algorthm for lnear FIR flters s not sutable for non-gaussan envronments. A second ay to prove the non-robustness of the conventonal maxmum-lkelhood method can be done va the analyss of the nfluence functon. Orgnatng from the mathematcal feld of robust statstcs, the approach based on nfluence functons provdes a ell suted technque to analyze gven maxmum-lkelhood estmators. For an exact defnton of expressons equaton (3) shall be reformulated by = ˆ = argmn ρ(r ) ; r = x ()

3 n hch r denotes the resdual error of the th datum of x and the expected value. The expresson ( ) ρ r = r denotes the score functon of the maxmum-lkelhood estmator. To solve equaton () t s necessary to calculate r ψ(r ) = () = here the dervatve ρ(r) ψ(r) = () r s called the nfluence functon of the score functon ρ(r). Thus the nfluence functon ψ(r) measures the nfluence of a datum on the value of the locaton estmate. For the least-squares estmate, hch s a drect result of maxmzng the Gaussan lkelhood functon, the nfluence functon can be derved by ψ(r) = r. Analyzng the effect of ths type of nfluence functon reveals the lnearly ncreasng nfluence of resdual errors. If ψ(r) s unbounded as a functon of r (as demonstrated n Fg. ), the value of ˆ can be nfluenced heavly by a sngle observaton component x for hch ψ(r) s large. Such a component can therefore completely overrde the eght of a possbly large number of other components n the estmaton of ˆ, hch confrms the non-robustness of the least-squares estmate [5] score functon resdual error r - nfluence functon resdual error r Fg. : Score functon and nfluence functon of the lnear flter.3 The Myrad Flter A frst proposal to overcome ths lack of robustness s ntroduced n the concept of myrad flterng [3]. Based on the negatve results acheved by takng the Gaussan probablty densty n maxmum-lkelhood estmaton t s suggested to use the Cauchy dstrbuton, hch s one of the fe stable dstrbuton to be expressed n a closed form besdes the Gaussan dstrbuton and hch provdes a better model of the real characterstca of mpulsve nose. Replacng the Gaussan probablty densty functon n the ell knon maxmum-lkelhood estmator () by the Cauchy probablty densty th scale γ γ f Cauchy (x, γ) = (3) π γ + (x ) and performng some conversons hle omttng rrelevant expressons for the maxmzaton procedure leads to equaton (4) ˆ = argmn log(γ + (x ) ). (4) = To capture the temporal correlatons of nput sgnals, a eghtng procedure as descrbed n secton. s γ mplemented th κ =. K = ˆ = argmn log(κ + (x ) ) (5) For computatonal purposes the sum of logarthms s transformed to the product of the arguments of the logarthmc expressons hch results n equaton (6) = ˆ = argmn (κ + (x ) ) (6) In ths defnton of eghted myrad flters the sample eghts are stll restrcted to non-negatve values due to mathematcal rules concernng logarthms or mnmzaton of a product hch mposes hard lmtatons on general usage of ths type of robust flter. To generalze the flter coeffcents to real-valued eghts, an analyss of negatve eghts n lnear flters s consdered to be helpful. In the lnear flter theory the effect of negatve coeffcents can be separated n the absolute value and the sgn of the coeffcent as shon n equaton (7) = = ˆ = =. (7) = x = sgn( ) x Thus the sgn of the eght s uncoupled from ts magntude and can be attached to the correspondng nput sample. A negatve eght therefore s consdered to be equvalent to eghtng the mrror sample th the postve absolute value of the eght. Transferrng ths behavor to the myrad flters and extendng the eghtng of the mrror samples to all nput samples for a better performance of the flter results n the double eghted myrad flter structure gven by ˆ = argmn (κ + g (z ) ), (8) =

4 th g =[,,...,, h, h,..., h ] and z =[x,x,...,x, -x,-x,...,-x ] here h s the eght of the mrror sample of [3]. An analyss of robustness of the myrad flter can be performed by nvestgatng the nfluence functon dscussed n secton.. For ths purpose, the eghted score functon of the myrad flter ρ(r,, κ) = log(κ + r ) ; r = x (9) as used n (5), bulds the startng pont for the dervaton of the nfluence functon, hch can be gven by ρ(r,, κ) r ψ(r,, κ) = = () r κ + r Fg. shos the score functon and the nfluence functon of the myrad flter th eghts =.5,.3 and κ= n conjuncton to the resdual error r. 4 3 score functon =.5 = resdual error r nfluence functon resdual error r Fg. : Score functon and nfluence functon of the myrad flter th eghts and κ= The loer sub-graph of Fg. demonstrates the exstence of a maxmum and a mnmum of the nfluence functon of the myrad score functon. Accordng to the results of robust statstcs the nfluence functon of a M- estmator s supposed to be bounded to provde a robust estmaton. Thus the relatve nfluence of the resdual error r, denoted at the y-axs, decreases as the error r ncreases past a defned threshold, hch expresses as a reduced nfluence of nput samples x resultng n a large dfference to the estmated value ˆ (outlers). In contrast the relatve nfluence of nput samples x located close to the estmated value ˆ (smaller resdual error) s sgnfcantly larger. Furthermore n both sub-plots of Fg. the effect of the eghts s vsualzed as a modfcaton of the score and nfluence curve accordng to the attached eghts (.e. a hgher value of the eght results n an ncreased nfluence of the correspondng nput sample x for an equvalent resdual error r). Unfortunately the mathematcal rules for processng the myrad flter algorthm merely permt the usage of nonnegatve eghts, especally concernng the sum of logarthms resp. the product of arguments of logarthmc expressons. The double eghtng procedure of the nput samples only makes sure that necessary mrror samples are taken nto account; the potental of negatve eghts s not captured ths ay. Thus the myrad flter structure stll results n a lmted performance of the desgned robust flter. To mprove the performance obtaned by the myrad flter, a generalzaton of the conventonal maxmum-lkelhood flter desgn technques s ntroduced n chapter 3. 3 Generalzed M-Flter Desgn 3. Flter desgn based on Score and Influence Functons As demonstrated n the precedng sectons, a necessary requrement of robust flter desgn s a bounded nfluence functon ψ(r). For the myrad type flter ths feature s derved by the applcaton of the Cauchy dstrbuton, mplemented n the conventonal maxmumlkelhood estmaton. Abstractng the maxmumlkelhood estmaton from the utlzaton of probablty dstrbutons and analyzng the effects of dfferent eghted score functons leads to a generalzed form of equaton (), n hch the score functon ρ(r ) can be chosen arbtrarly as an addtonal degree of freedom. For further analyss the eghted Geman-McClure score functon () and the eghted Tukey score functon () orgnatng from the feld of robust statstcs [6, 7, 8] shall be consdered as basc generalzed maxmumlkelhood estmator (M-Estmator) for robust flter desgn, hch then s referred to as M-Flter. ρ(, r, k) k = 6 k 6 ρ(, r, k) r = () k + r 3 r - - ; r k k () ; r > k In the defntons above the varables, r and k denote the eghts, the resdual error and a mutable factor to adapt the flter to dfferent scales. Form a gven score functon the correspondng nfluence functons can be calculated by solvng equaton (). Thus the Geman- McClure (3) and the Tukey (4) nfluence functons are gven by ψ(, r, k) k r = (3) ( k + r )

5 r r ψ(,r,k) = k ; ; r k r > k (4) An analyss of equatons ()-(4) reveals, that a restrcton to non-negatve eghts s not mandatory for mathematcal reasons, hch n consequence provdes enhanced flexblty n flter desgn methodology. Furthermore by utlzaton of dfferent eghted score functons an mprovement of robustness features s thn reach, dentfable n the behavor of the nfluence functon for large outlers. Fg. 3 and Fg. 4 sho a plot of the Geman-McClure and Tukey score and nfluence functon hen eghted th dfferent eghts ncludng negatve values score functon =.5 =.3 = resdual error r nfluence functon resdual error r Fg. 3: Geman-McClure score and nfluence functon th eghts and k= score functon =.5 =.3 = resdual error r nfluence functon resdual error r Fg. 4: Tukey score and nfluence functon th eghts and k=5 Fg. 4 clarfes the mprovement of robustness features by a free choce of the score functon as the nfluence of sngle outlers can be completely suppressed f the resdual errors become too large. Ths pont of complete suppresson can be controlled by the varable k, hch addtonally ncreases the flexblty of ths novel flter. For bandpass and hghpass behavor of the flter the mrror samples of the nput data have to be taken nto consderaton as explaned n secton. and.3. Snce the ntroducton of the novel score functons, hch permts negatve eghts (lke equaton () or (), does not consder the mrror samples, a double eghtng structure such as n the case of the myrad type flter structure has to be mplemented. Thus the orgnal nput samples as ell as the mrror samples are taken nto account for estmaton of the flter output. To obtan a set of eghts correspondng to the desred output sgnal of the flter a normalzed steepest descent algorthm s used, analogously to the method proposed n [3]. Ths double eght-structured type of generalzed maxmumlkelhood flter (M-Flter) s mplemented, smulated and compared to the performance of the ell knon myrad flter. 3. Smulaton Results of the novel M-Flter To verfy the capacty of the novel M-Flter a smulaton of a typcal bandpass behavor shall be executed folloed by a comparson of the obtaned results to the performance of the ell knon myrad flter. To mplement ths performance evaluaton, the eghts of the novel M-Flter have to be adapted to the desred output sgnal hle suppressng mpulsve nose. Usng the normalzed method of steepest descent the adaptaton of the eghts for the novel M-Flter types converges n about teratons each for acceptable results, usng a step-sze of µ=.. A frst smulaton has been carred out n a set of Monte-Carlo runs of the adapted M-Flters separatng the dgtal frequency of. from the test sgnal defned n Table. Frequency Ampltude Table : Dgtal frequences and ampltudes of the test sgnal In addton to the four frequences tabulated n Table the test sgnal s covered by an α-stable dstrbuted mpulsve nose (α=., γ=.). For the mutable factor k of the Geman-McClure-Flter a value of k=7 provdes good results hle k=5 s chosen for the M-Flter based on the Tukey score functon. Fg. 5 shos the frequency doman flter output of the dscussed M-Flters usng a flter ndo length = (hch results n the calculaton of 4 eghts due to the double eghtng). The ntroducton of negatve eghts n the score functons reveals that a sgnfcant mprovement up to 5

6 db n the frequency doman of the novel (Geman- McClure and Tukey) M-Flters can be reached. Furthermore a comparson of the Tukey-Flter to the Geman-McClure-Flter emphaszes the mportance of the nfluence functon as the Tukey-Flter provdes a superor fltered sgnal due to the completely vanshng nfluence of large outlers. Especally for loer frequences ths effect s recognzable. H(f) [db] Myrad Flter Geman Flter Tukey Flter Reference estmaton flter to match requrements of sgnal processng applcatons n mpulsve envronments. For ths purpose the ntroduced method offers a flexble tool to combne robustness as ell as computatonal aspects n order to obtan a problem-orented soluton n favor of the consdered task. The probable felds of applcaton of the novel M-flters may be found n all sgnal processng areas exposed to severe mpulsve nose as for example communcaton va tsted par telephone lnes. But even though the frst results of ths novel method of robust flter desgn leads to promsng flter features, a lot of research has to be done to develop a closed form of robust statstcs n sgnal processng. Of specal nterest s an optmzed algorthm to compute the eghts and other sgnfcant varables of the M-Flter. Furthermore some nvestgatons n the feld of applcatons n combnaton th consderatons of computatonal complexty have to be carred out n the future normalzed frequency f Fg. 5: Frequency doman flter output of varous M- Flters separatng a dgtal frequency of. n the test sgnal covered by α-stable nose (α=., γ=.) An addtonal mportant aspect for practcal applcatons of sgnal processng technques s ther numercal complexty. As the general condtons of the dscussed M-Flters (lke double eghtng of the ndoed nput samples etc.) are not changed n the smulaton, the computatonal costs of the dfferent score functons rses to the man focus for an evaluaton of the numercal complexty. Compared to four multplcatve and one addtve operaton n processng the Geman-McClure M- Flter, the Tukey-Flter takes up to sx multplcatve and to addtve operatons past a fall dfferentaton. Ths hgh computatonal cost results n a delay hch s about 4 tmes of the delay caused by the Geman-McClure- Flter. In contrast, the fast myrad computaton algorthm takes to multplcatons and one addton leadng to a delay tme of about a half of the tme needed for the Geman-McClure algorthm. The cost of ths lo numercal complexty algorthm s a reduced performance as demonstrated n Fg. 5. Consderng these numercal aspects the choce for the utlzed score functon can be made th respect to precseness or lo computatonal delay, hch offers an addtonal degree of freedom n robust flter desgn. 4 Conclusons In ths paper the lack of robustness of conventonal sgnal processng technques s analyzed and taken as fundament of a novel enhanced maxmum-lkelhood References: [] Shao, M., kas, C.L., Sgnal Processng th Fractonal Loer Order Moments: Stable Dstrbutons and Ther Applcatons, Proceedngs of the IEEE, Vol. 8, o. 7, July 993, pp [] Samorodntsky, G., Taquu, M., Stable on- Gaussan Random Processes: Stochastc Models th Infnte Varance, Chapman & Hall, e York, 994 [3] Kallur, S., onlnear adaptve Algorthms for Robust Sgnal Processng and Communcatons n Impulsve Envronments, Ph.D. Thess, Unversty of Delaare, 998 [4] Fredman, J., Messer, H., Cardoso, J.-F., Robust Parameter Estmaton of a Determnstc Sgnal n Impulsve ose, IEEE Trans. on Sgnal Processng, Vol. 48, o. 4, Aprl, pp [5] Kassam, S. A., Poor, H. V., Robust Technques for Sgnal Processng: A Survey, Proceedngs of the IEEE, Vol. 73, o.3, March 985, pp [6] Hampel, F., Ronchett, E. M., Rousseeu, P. J., Stahel, W. A., Robust Statstcs: The Approach Based on Influence Functons, John Wley & Sons, e York, 986 [7] Astola, J., Kousmanen, P., Fundamentals of nonlnear dgtal flterng, CRC Press, Boca Raton, 997 [8] Geman, S., McClure, D. E, Statstcal methods for tomographc mage reconstructon, Bull. Int. Statst. Inst. LII-4, 5-, 987