Proceedings of he 5h WSEAS Inernaional Conference on Signal Processing, Isanbul, urey, May 7-9, 6 (pp45-5) Laplacian Mixure Modeling for Overcomplee Mixing Marix in Wavele Pace Domain by Adapive EM-ype Algorihm and Comparisons Behzad Mozaffary Mohammad A. inai Faculy of Elecrical and Compuer Engineering Univerciy of abriz 9 Bahman Blvd., abriz, Eas Azerbaijan IRA Absrac-- Speech process has benefied a grea deal from he wavele ransforms. Wavele paces decompose signals in o broader componens using linear specral bisecing. In his paper, mixures of speech signals are decomposed using wavele paces, he phase difference beween he wo mixures are invesigaed in wavele domain. In our mehod Laplacian Mixure Model (LMM) is defined. An Expecaion Maximizaion (EM) algorihm is used for raining of he model and calculaion of model parameers which is he mixure marix. And hen we compare esimaion of mixing marix by LMM-EM wih differen wavele. herefore individual speech componens of speech mixures are separaed. Keywords: ICA, Laplacian Mixure Model, Expecaion Maximizaion, wavele paces, Blind Source Separaion, Speech Processing. Inroducion Blind source separaion echniques using independen componen analysis (ICA) have many poenial applicaions including speech recogniion sysems, elecommunicaions, and biomedical signal processing. he goal of ICA is o recover independen sources given only sensor observaion daum ha are unnown linear mixures of he unobserved independen source signals [] [6]. he sandard formulaion of ICA requires a leas as many sensors as sources. Lewici and Sejnowsi [7], [8] have proposed a generalized ICA mehod for learning overcomplee represenaions of daa ha allows more basis vecors han dimensions in he inpu. Several approaches have been invesigaed o address he overcomplee source separaion problems in he pas. Lewici [9] provided a complee Baysian approach assuming Laplacian source prior o esimaing boh he mixing marix and he source in he ime domain. Clusering soluions were inroduced by Hyvarinen [] and Bofill-Zibulesy []. Davies and Milianoudis [] employed modified discree cosine ransform (MDC) o obain a sparse represenaion. hey proposed a wo-sae Gassian mixure model (GMM) o represen he source densiies and he possible addiive noise and used an expecaion-maximizaion, (EM)-ype algorihm, o perform separaion wih reasonable performance. In his paper, we explore he case of wo-sensor seup wih no addiive noise, where he source separaion problem becomes a one-dimensional opimal deecion problem. he phase difference beween he wo-sensor daa is employed. A Laplacian mixure model (LMM) is fied o he phase difference beween he wo sensors, using an EM-ype algorihm in each wavele pace. he LMM model can be used for source separaion and source localizaion. Since in he overcomplee model of source separaion esimaion of mixure marix is very imporan in his paper, herefore we use LMM model for each wavele pace wih phase differences. oe ha wavele paces are obained from decomposiion of wo mixures.. Bacground Maerial Waveles are ransform mehods ha has received grea deal of aenion over he pas several years. he wavele ransform is a ime-scale represenaion mehod ha decomposes signals ino basis funcions of ime and scale, which maes i useful in
Proceedings of he 5h WSEAS Inernaional Conference on Signal Processing, Isanbul, urey, May 7-9, 6 (pp45-5) applicaions such as signal denoising, wave deecion, daa compression, feaure exracion, ec. here are many echniques based on wavele heory, such as wavele paces, wavele approximaion and decomposiion, discree and coninuous wavele ransform, ec. Bacbone of he waveles heory is he following wo equaions: j / j φ ( = φ( ) () j, j / j j, ( = ψ ( ψ ) () Where φ ( and ψ ( are basic scaling funcion and moher wavele funcion respecively. he wavele sysem is a se of building blocs o consruc or represen a signal or funcion. I is a wo dimensional expansion se. A linear expansion would be: + + + j ϕ j, ψ (3) = = j= f () = c ( ) + d ( ) Mos of he resuls of wavele heory are developed using filer bans. In applicaions one never has o deal direcly wih he scaling funcions or waveles, only he coefficiens of he filers in he filer bans are needed. A full wavele pace decomposiion binary ree for ree scale wavele pace ransform is shown in figure (). S ( = [ s(, s(, s3(,... s ( ] where again s i ( is he i h source. In his paper we will assume noise-less insananeous mixing model i.e. X ( = A. S( Where A denoes he mixing marix. he source separaion problems consis of esimaing he original sources S (, given he observed signals X (. In he case of an equal number of sources and sensors (=M), a number of robus approaches using independen componen analysis (ICA) have been proposed by Miianoudis [4]. In he overcomplee source separaion case (M<), he source separaion problem consiss of wo sub problems i) esimaing he mixing marix A and ii) esimaing he source signals S (. In figure () we have shown he scaer plo of he wo sensor signals, ha is, wo mixures of hree speech signals. o ge a sparser represenaion of daa, we use he wavele pace decomposiion (WPD) on he observed signals [5]-[7]. By examining of he scaer plo, we can see ha wo dimensional problem is mapped ino a one dimensional problem. he mos imporan parameer o us is he angle θ (phase difference of wo observed signal) of each poin in he plo. X..5..5 -.5 -. -.5 -. -.4 -.3 -. -....3 X Figure () scaer plo of x ( respec o x ( in wavele Domain.35 hisogram of pace in mixures.3.5 probabiliy..5. Figure () 3. Mahemaical Model Assume a se of M sensors expressed as a vecor: X ( = [ x (, x(, x3(,... x M ( ] where x i ( is he oupu of he i h sensor and also assume ha here are source signals as in vecor:.5 - -.5 - -.5.5.5 angle Figure (3) hisogram of phase difference beween wavele paces of x (, x ( If we have wo sensors and hree sources hen we can express he mixing model as:
Proceedings of he 5h WSEAS Inernaional Conference on Signal Processing, Isanbul, urey, May 7-9, 6 (pp45-5) x( = as( + as( + a3s3( x( = as( + as( + a3s3( (4) X = AS (5) For simpliciy we assume a = for all j=,,3 and hen we can wrie : X ( = b s + b s + b s (6) 3 3 Equaion (6) indicaes ha each source signal in he scaer plos will be in he b j direcion. We define phase difference of observed daa measured by sensors as follows: Pi ( x) θ = arcg[ ] (7) P( x ) i Where P i (x j ) is he i h pace wavele of j h observaion signal. In figure (3) we have ploed he hisogram of he phase difference of observed signals in wavele pace domain. j J ( α, θ, c ) = = = = = = (logα + log c log α L( θ, c c θ θ ) f ( θ ), θ ) () Where f ( θ ) represens he probabiliy of θ belonging o h Laplacian disribuion. he ieraion rules updae f ( θ ) and α. o obain he updae values for θ, c we solved derivaives of J ( α, θ, c ) wih respec o θ, c, ha is: J J =, = θ c () Using hese ieraion formulas we are able o rain he LMM and esimae he cener and oher parameers of each Laplacian disribuion. he bloc diagram of he proposed algrih is shown in figure (4). 4.Laplacian Mixure Modeling he laplacian densiy is usually expressed as: c θ θ L ( θ, c, θ ) = ce (8) Where θ represen he cener of densiy funcion and c> conrols he widh or variance of he densiy. An LMM is defined as: f ( c θ θ θ ) = α L( θ, c, θ ) = α ce (9) = = Where α,θ, c are he weighs, ceners, and widhs of each Laplacian respecively. In he nex secion we will show how he EM algorihm is used o rain he model o ge he opimum values of he model parameers. Obaining Mixures of X (, X ( Wavele Pace Decomposiion for each mixures Obaining Phase differences beween paces Calculaion of Pahse differences Hisogram LMM-EM 5. raining Process Using he EM Algorihm In [8] Bilmes proposed a procedure o find maximum lielihood mixure (MLM) densiy parameers using EM. In his secion, we use he EM algorihm o rain a LMM, based on [8]. Assuming samples for θ and Laplacian mixure densiies as in equaion (8), he log lielihood aes he following form: Calculaion of mixiure marix Figure (4) As he figure (4) shows, he wavele paces of he wo mixures of speech signal, x ( and x (, is obained. hen in every filer ban, he phase differences of he paces of x ( and x ( is calculaed. he nex sep is o manipulae he hisograms of he phase angle differences. he cener 3
Proceedings of he 5h WSEAS Inernaional Conference on Signal Processing, Isanbul, urey, May 7-9, 6 (pp45-5) of each Laplacian densiy is esimaed using he Laplacian mixure model. he raining algorihm used in his process is an EM ype. herefore, afer he convergence of he EM, he esimaion of he mixure marix is obained. 6. Experimen and simulaion We have esed our proposed scheme by choosing marix A, as presened in he following hree examples. Example : Mixing marix for wo sources: A =.5.5 Figure (5) shows scaer of wo mixing daa in wavele pace domain for all paces and also hisograms of phase difference hese paces in mixures. Figure (6-a) and (6-b) show convergence of esimaed parameers for Laplacian model. Example : Mixing marix for hree sources: A =.6.3.6 Figure (7) shows scaer of wo mixing daa in wavele pace domain for all paces and also hisograms of phase difference hese paces in mixures. Figures (8-a) and (8-b) show convergence of esimaed parameers for Laplacian model. Laplacian model componen Probabiliy.5.5 -.5-3 4 5 6 ieraion b) esimaed Laplasian Model for each source.3.. a) convergence of LMM-EM.4 hisogram of pace scaer plo of pace - -.5 - -.5.5.5 angle. - -.5 - -.5.5.5 hisogram of pace. - - -.5.5 scaer plo of pace Figure (6) a) Learning curves for convergence of LMM-EM algorihm, b) esimaed LMM of sources.5 - -.5 - -.5.5.5 - -.4 -...4.6.6.4. hisogram of pace3.5 scaer plo of pace3. hisogram of pace scaer plo of pace - -.5 - -.5.5.5 hisogram of pace4.3.. - -.5 - -.5.5.5 -.5 -. -.5 -. -.5.5..5. scaer plo of pace4. -. -.5 -. -.5.5. Figure (5) a) hisogram for phase differences, b) scaer plo of paces of mixures We can see from figure (5) ha afer 3-4 ieraions he LMM_EM converges, and he cener of each Laplacian densiy is esimaed where hey are used o esimae he enry of mixing marix. he numerical value for our example is as: A =.496.58. - - hisogram of pace.4. - - hisogram of pace3..5 - - hisogram of pace4.4. - - - - -.5.5 scaer plo of pace. -. -.5.5 scaer plo of pace3.5 -.5 -. -... scaer plo of pace4.5 -.5 -.4 -...4 Figure (7) a) hisogram for phase differences, b) scaer plo of paces of mixures We can see from figure (8-a) ha afer -3 ieraions he LMM_EM converges, and he cener of each Laplacian densiy is esimaed where hey are used o esimae he enry of mixing marix. he numerical value for our example is as: 4
Proceedings of he 5h WSEAS Inernaional Conference on Signal Processing, Isanbul, urey, May 7-9, 6 (pp45-5) A =.6.863.635 Probabiliy Laplacian Model Componen Convergence.5.5 -.5 - a) convergence of LMM-EM -.5 3 4 5 6 ieraion.4.35.3.5..5..5 b) Esimaed LMM Sources - -.5 - -.5.5.5 angle Figure (8) a) Learning curves for convergence of LMM-EM algorihm, b) Esimaed LMM of sources In he nex secion we will inspec parameer esimaion of mixing marix by differen wavele and comparison beween hem will be done. Lev6.69.934 -.77 Lev7.6938.943 -.6987 able () esimaion of mixing marix by 'dmey' Lev.767.785.6897 Lev.7384.893.695 Lev3.6965.84.77 Lev4.798.439.75 Lev5.775.37.734 Lev6.6978..74 Lev7.6989.89.736 able (3) esimaion of mixing marix by 'bior.3' Lev.755.343.6966 Lev.753.45.6969 Lev3.759.558.788 Lev4.69.377.7 Lev5.75.44.74 Lev6.684.737.6958 Lev7.74.565.6984 7. Comparison Firs we decompose phase difference of mixure signals by wavele pace in 7 levels (complee ree forma, and in each level we apply LMM-EM algorihm for any pace. hen we esimae mixing marix parameers for each pace and hen we compue average of hese marixes. We used mixing marix for his invesigaion as: A =.7..7 ables (), (), (3) show resul of esimaion in each level of wavele decomposiion. We see in hese ables, by increasing of level decomposiion, we have good esimaion. And by comparing of hese ables wih each oher we see ha good esimaion is obained when discree Meyer (dmey) wavele is used. able () esimaion of mixing marix by 'db4' Lev.794.838 -.6985 Lev.765.8676 -.73 Lev3.737. -.76 Lev4.7399. -.69 Lev5.75.9844 -.773 8. Conclusion In his invesigaion we have shown ha one can use he coheren phase informaion beween wavele paces o esimae mixing marix in a speech mixure. We have highlighed ha he EM algorihm can be used in a LMM in order o esimae he mixure parameers. When we have more sources han sensors, overcomplee case, we have shown ha he number of ieraion is abou -3 ieraions, which is much less han oher repored cases. We map wo dimensional problem o one dimensional (phase differences beween wo paces in wavele domain.) and hen we ge more accurae esimaion of mixure marix. wo examples wih wo and hree source componens in he mixure were underaen for simulaions. Resuls indicae ha we have enabled o esimae he mixing marix wih a high degree of accuracy. Finally we show ha when we use high resoluion in pace domain we obain good esimaion of mixing marix and when we use discree Meyer wavele, we obain beer resuls han oher waveles. 5
Proceedings of he 5h WSEAS Inernaional Conference on Signal Processing, Isanbul, urey, May 7-9, 6 (pp45-5) 9. References [] A. J. Bell and. J. Sejnowsi, An informaionmaximizaion approach o blind separaion and blind deconvoluion, eural Compu., vol. 7, pp. 9 59, 995. [] J.-F. Cardoso, Blind signal separaion: Saisical principles, Proc. IEEE, vol. 86, pp. 9 5, Oc. 998. [3] P. Comon, Independen componen analysis A new concep?, Signal Process., vol. 36, pp. 87 34, 994. [4] P. Comon and B. Mourrain, Decomposiion of quanics in sums of powers of linear forms, Signal Process., vol. 53, pp. 93 7, Sep. 996. [5] M. Hermann and H. Yang, Perspecives and limiaions of selforganizing maps, in Proc. ICOIP 96. [6].-W. Lee, Independen Componen Analysis: heory and Applicaions. Boson, MA: Kluwer, 998. [7] M. Lewici and. J. Sejnowsi, Learning nonlinear overcomplee represenaions for efficien coding, in Advances in eural Informaion Processing Sysems, vol.. Cambridge, MA: MI Press, 998, pp. 85 8. [8] M. S. Lewici and. J. Sejnowsi, Learning over complee represenaions, eural Compu., o be published. [] M. Davies and. Miianoudis, "A simple mixure model for sparce overcomplee ICA newors," Proc. Ins. Elec. Eng. Vision, Image,Signal Process., vol. 5, no., pp. 35-43, 4. [3] C. S. Burrus, R. A. Gopinah, H. Guo, Inroducion o Waveles and Wavele ransforms, a primer Prenice Hall ew jersey, 998. [4]. Miianoudis, "Audio source separaion using independen componen analysis," Ph.D. disseraion, Queen Mary, London, U.K, 4 [5] M.A. inai, B. Mozaffari, Comparison of ime-frequency and ime-scale analysis of speech signals using SF and DW, WSEAS ransacion on Signal Processing, Issue, Vol., pp. -6, Oc. 5 [6] M.A. inai, B. Mozaffari, A ovel Mehod for oise Cancellaion of Speech Signals Using Wavele Paces [7] M. Zibulevsy, P. Kisilev, Y. Y. Zeevi, and B. A. Pearlmuer, "Blind source separaion via mulimode sparse represenaion newors, " Adv. eural Inf. Process. Sys., vol. 4, pp. 49-56, [8] J. A. Bilmes, "A Genle uorial of he EM Algorihm and i's applicaion o parameer esimaion for Gassian Mixures and Hidden Mixure Models, " Dep. Elec. Eng. Compu. Sci., Univ. California, Bereley, California, ech. Rep., 998. [9] M. Lewici and.j. Sejnowsi, "Learning over complee represenaions newors," eural Compue., vol.,pp.337-365, [] A. Hyvarinen, "Independen componen analysis in he presence of Gassian noise by maximizing join lielihood newors," eural Compue., vol., pp.49-67,998\ [] P. Bofill and M. Zibulevsy, "Underdeermined blind source separaion using sparce represenaion newors," Signal Process., vol. 8, no., pp. 353-36, 6