SPEECH ENHANCEMENT USING A MODULATION DOMAIN KALMAN FILTER POST-PROCESSOR WITH A GAUSSIAN MIXTURE NOISE MODEL. Yu Wang and Mike Brookes

Transcription

1 14 IEEE Iteratioal Coferece o Acoustic, Speech ad Sigal Processig (ICASSP) SPEECH ENHANCEMENT USING A MODULATION DOMAIN KALMAN FILTER POST-PROCESSOR WITH A GAUSSIAN MIXTURE NOISE MODEL Yu Wag ad Mike Brookes Departmet of Electrical ad Electroic Egieerig, Exhibitio Road, Imperial College Lodo, UK {yw9, mike.brookes}@imperial.ac.uk ABSTRACT We propose a speech ehacemet algorithm that applies a Kalma filter i the modulatio domai to the output of a covetioal ehacer operatig i the time-frequecy domai. We show that the predictio residual sigal of the spectral amplitude errors at the output of the baselie ehacer do ot follow a Gaussia distributio. Accordigly, the Kalma filter used i our ehacemet algorithm combies a colored oise model with a Gaussia mixture model of the residual oise. We evaluate the performace of the speech ehacemet algorithm o the core TIMIT test set ad demostrate that it gives cosistet performace improvemets over the baselie ehacer ad over a previously proposed Kalma filter post-processor. Idex Terms speech ehacemet, post-processig, Kalma filter, Gaussia mixture model, modulatio domai 1. INTRODUCTION Over the past decades, may speech ehacemet algorithms have bee proposed i order to elimiate or reduce uwated backgroud oise. Ehacemet algorithms i the time-frequecy domai, such as [1] ad [], ca be effective i reducig the oise ad improvig the sigal-to-oise ratio (SNR) but they also distort the speech ad itroduce spurious artefacts kow as musical oise. I order to solve this problem, several post-processig methods have bee proposed that either filter the time-frequecy gai fuctio used withi the ehacer or else act directly o its output sigal. I [3], media filterig is applied to time-frequecy cells that are idetified as havig a low probability of cotaiig speech eergy i order to elimiate the isolated peaks that characterise musical oise ad i [4] musical oise i frames with low SNR is atteuated by smoothig the gai fuctio of the baselie ehacer. Other techiques, such as cepstral smoothig [5] ad Kalma filterig (KF) [6] have also bee used to post-process the miimum mea square error () spectral amplitude estimator [] to reduce the musical oise ad improve the quality of the ehaced speech. The use of a KF was itroduced i [7] for speech ehacemet assumig the oise was white ad i [8] this was later exteded to colored oise. Recet iterest i performig speech ehacemet i the modulatio domai [9, 1, 11] icludes the algorithm described i [1] which applies the KF to the short-time modulatio domai spectral amplitudes. The assumptio made i the KF is that the predictio residual sigals of both speech ad oise are Gaussia distributed with zero-mea. However, we have foud that the predictio residual sigal of the spectral amplitude errors i the ehaced speech sigal do ot follow a Gaussia distributio. Therefore, extedig the algorithm i [6], we propose i this paper a KF post-processor i the modulatio domai usig a Gaussia mixture model (GMM) of the oise which is colored due to the overlap betwee frames. The rest of the paper is orgaized as follows: Sec. gives the motivatio ad the probabilistic derivatio of the GMM Kalma filter for colored oise ad also describes the update procedure for the model parameters. I Secs 3 ad 4, we evaluate the proposed algorithm ad give our coclusios.. GMM KALMAN FILTER.1. Distributio of predictio error I the covetioal KF, the predictio residual sigal of both speech ad oise are assumed Gaussia distributed. However, after processig oisy speech by a ehacer, most of the statioary oise has bee removed leavig behid some residual oise together with musical oise artefacts especially where the iput oise power was high [13]. Because the musical oise is characterized by isolated spectral peaks i the spectrogram, it is difficult to predict i the modulatio domai. As a result, the predictio errors associated with the musical oise may be very large, ad the overall distributio of the predictio errors of the oise i the ehaced speech does ot follow a Gaussia distributio. To illustrate this, we show i Fig. 1 the distributio of the ormalized predictio error of the spectral amplitude errors i the ehaced speech i each time-frequecy bi together with a fitted sigle Gaussia distributio (i red) ad a 3-mixture GMM (i /14/$ IEEE 774

2 probability desity predictio error 3 compoets Sigle compoet oisy speech STFT ehaced speech ISTFT oise estimatio speech LPC estimatio Kalma Filter & GMM parameters update Trai iitial GMM parameters LPC estimatio ormalized predictio error Fig. 1. Distributio of the ormalized predictio error of the oise spectral amplitudes i -ehaced speech. The predictio errors are ormalized by the RMS power of the oise predictor residual i the correspodig modulatio frame. gree). The histogram shows the distributio over all timefrequecy bis usig the TIMIT core test set [14] corrupted by additive car oise at SNRs betwee 1 ad +15 db usig the framig parameters from Sec. 3. The estimated oise amplitude trajectory i each frequecy bi is represeted by a autoregressive model whose predictio error is ormalized by the root-mea-square (RMS) level of the oise predictor residual i the correspodig modulatio frame. From the figure, we see that the overall predictio residual sigal is ot zero mea ad does ot follow a Gaussia distributio. Based o the empirical predictio errors, we have exteded the covetioal colored oise KF to icorporate a GMM oise distributio. We use N (µ, ) to deote a multivariate Gaussia distributio with mea vector µ ad covariace matrix ad use N (x; µ, ) for its probability desity at x... Derivatio of GMM Kalma Filter The diagram of the proposed algorithm is show i Fig.. Followig time-frequecy domai ehacemet, the spectral amplitude of the short-time Fourier trasform (STFT) at time frame ad frequecy bi k is give by Y,k = X,k + W,k where X,k is the amplitude of the clea speech sigal ad W,k is the oise arisig from a combiatio of acoustic oise ad the ehacemet artefacts. The output from the KF ˆX,k is combied with the oisy phase spectrum,k ad passed through a iverse-stft (ISTFT) to create the output speech ˆx(t). I this ad the ext subsectio we will give the derivatio of the GMM Kalma filter (GMMKF) ad the parameter update procedure. Because each frequecy bi, k, is processed idepedetly ad for clarity, we omit the frequecy idex below. Fig.. Diagram of the proposed GMM KF algorithm Our system model is z +1 = A z + Dq (1) y +1 = c T z +1 () where z =[x x N+1 w w M+1 ] T is the N +M dimesioal state vector for both the speech, x, ad oise, w, ad q =[u v ] T cotais the correspodig predictio residuals. The (N + apple M) (N + M) trasitio matrix, A, is Ta i the form A = where T T a ad T b are the b trasitio matrices apple for speech ad oise respectively ad T a a is give by T T a = where a I is the vector of liear predictio (LPC) coefficiets for the speech. T b ad b are the correspodig quatities for the oise. The (N + M) matrix D is all zero except for d 1,1 = d N+1, =1. Likewise the colum vector c is all zero except for c 1 = c N+1 =1. We represet the predictio residuals as a -elemet vector q with a Gaussia mixture distributio of J mixtures as q N (µ, ) (3) As i a covetioal Kalma filter, we assume that the state vector at time based o observatios up to time is Gaussia distributed z s N (z, P ). Followig the time update, the distributio of z +1 becomes a Gaussia mixture P j N (z +1, P +1 ) where z +1 = Az + Dµ P +1 = AP A T + D D T Applyig the costrait c T z +1 = y +1 chages the Gaussia mixture parameters as follows [15] k +1 = P +1 c(ct P +1 c) 1 (4) z = z +1 + k +1 (y+1 ct z +1 ) (5) P = P +1 k +1 ct P +1 (6) 775

3 Fially, we collapse the GMM ito a sigle Gaussia for the estimatio of the state vector at time +1 The parameters of each model ca ow be updated adaptively as [18] +1 = P j z = P = N (y +1; c T z +1, ct P +1 c) N (y +1; c T z +1, ct P +1 c) (7) +1 z (8) +1 (P z (z )T ) z z T (9) The quatity +1 i (7) represets the posterior probability that z +1 belogs to mixture j. Thus we ca use the ew Kalma filter to process the residual oise i the ehaced speech because the GMM ca be used to model the spectral amplitude errors i the ehaced speech..3. Update of parameters The spectral amplitudes, Y,k are divided ito overlappig modulatio frames ad autocorrelatio LPC aalysis [16] is performed i each modulatio frame to obtai a vector of speech predictio coefficiets, a, ad a residual power a. To obtai the correspodig oise coefficiets, the sequece of spectral amplitudes, Y,k is passed through a oise power spectrum estimator [17] before performig LPC aalysis to obtai the oise predictor coefficiets, b, ad the residual power b. Withi the oise GMM, (3), the speech residual compoet u N(, a) is idetical i all mixture compoets but the ormalized oise residual e = v / b is modeled as a Gaussia mixture e P j N (m, j ). We model the ormalized residual rather tha the residual itself so that the GMM parameters are idepedet of the speech ad oise amplitudes. I order to update the GMM parameters we apply the oise predictor coefficiets, b, from the curret modulatio frame to the sequece of estimated oise spectral amplitudes to obtai a oise predictio error e b for each acoustic frame. The probability that e +1 comes from model j is give by p +1 = P j N (e +1; m, N (e +1; m, ) ) (1) Because ow the probability of the model give the observatio error is kow, we ca update i each acoustic frame the effective umber of observatios (O ), the sum of the observatios (S ) ad the sum of the squared observatios (T ) as O +1 = p +1 + O ad T +1 = p +1 e +1 + T, S +1 = p +1e+1 + S, where is a forgettig factor. m +1 = S +1 /O +1 (11) +1 = T +1 /O +1 m +1 (1) +1 = O +1 Pj O +1 =(1 ) O +1 (13) To iitialize the model, we trai a GMM with parameters m, ad offlie o a large amout of data ad set O = m /(1 ), S = m O ad T = ( + m )O. To esure stability of the update procedure, we impose lower bouds o p ad to prevet them becomig zero. 3. IMPLEMENTATION AND EVALUATION 3.1. Stimuli of experimets I this sectio, we compare the performace of the proposed Kalma filter post-processor based o a GMM () with the baselie ehacer from [] ad the modulatiodomai Kalma filter post-processor () from [6]. The iitial GMM parameters are traied usig a subset i the traiig set of the TIMIT database ad usig speech corrupted by white oise. The remaiig algorithm parameters were chose to optimize performace o a developmet subset of the TIMIT traiig database. The umber of mixtures used is set as J =3ad we select a acoustic frame legth 16 ms with a 4 ms icremet which gives a 5 Hz samplig frequecy i the modulatio domai. The speech ad oise LPC models are determied from a modulatio frame of 18 ms (3 acoustic frames) with a 16 ms frame icremet ad the model orders i the ad algorithms for the speech ad oise are N =3ad M =4respectively. I the experimets, we use the core test set from the TIMIT database which cotais 16 male ad 8 female speakers each readig 8 distict seteces (totallig 19 seteces) ad the speech is corrupted by the F16 oise from the RSG-1 database [19] ad street oise from the ITU-T test sigals database [] at 1, 1,, 5, 1 ad 15 db global SNR. A Hammig widow is used i the STFT aalysis ad sythesis ad the forgettig factor is set as = Performace evaluatio The performace of the algorithms is evaluated usig both segmetal SNR (segsnr) ad the perceptual evaluatio of speech quality (PESQ) measure defied i ITU-T P.86. All the measuremet values are averaged over the 19 seteces i the TIMIT core test set. The average segsnr for the corrupted speech, baselie ehacer, the algorithm ad the proposed algorithm is show for F16 776

4 segsnr (db) Fig. 3. Average segmetal SNR of ehaced speech after processig by three algorithms versus the global SNR of the iput speech corrupted by F16 aircraft oise (: proposed Kalma Filter post-processor with a Gaussia Mixture oise model; : modulatio-domai Kalma filter postprocessor from [6]; : ehacer from []). segsnr (db) Fig. 4. Average segmetal SNR of ehaced speech after processig by three algorithms versus the global SNR of the iput speech corrupted by street oise. oise i Fig. 3 as a fuctio of the global SNR of the oisy speech. We see that at 15 db global SNR all the algorithms give the same improvemet i segsnr of about 3 db. However, at db global SNR the proposed algorithm outperforms both referece algorithms by about 4 db ad 6 db respectively. The equivalet graphs for street oise are show i Fig. 4. We see that the overall tred i the results is the same ad at db the proposed algorithm gives a additioal improvemet of 1.5 db. The correspodig graphs for PESQ are show i Fig. 5 for F16 oise ad i Fig. 6 for street oise. I Figs 5 ad 6, the average PESQ scores mirror the results see for the segsnr. However, at high SNRs the proposed algorithm is also able to improve the PESQ, ad we obtai a improvemet of approximately.15 ad.5 over the algorithm ad ehacer respectively over a wide rage of SNRs. I additio, iformal listeig tests also suggest that the proposed post-processig method is able to reduce the musical oise itroduced by the ehacer. 4. CONCLUSION I this paper we propose a ew post-processor i the modulatio domai usig a GMM for modelig predictio error of the oise i the output of a covetioal spectral amplitude ehacer. We have derived a KF that icorporates a GMM oise model ad have also preseted a method for adaptively updatig the GMM parameters. We have evaluated our proposed post-processor usig segsnr ad PESQ ad show that the proposed method results i cosistetly improved performace whe compared to both the baselie ehacer ad a modulatio-domai KF postprocessor. The improvemet i segmetal SNR is over 4 db at a global SNR of db while the PESQ score is icreased by about.15 across a wide rage of iput global SNRs. PESQ Fig. 5. Average PESQ quality of ehaced speech after processig by three algorithms versus the global SNR of the iput speech corrupted by F16 aircraft oise. PESQ Fig. 6. Average PESQ quality of ehaced speech after processig by three algorithms versus the global SNR of the iput speech corrupted by street oise. 777

5 5. REFERENCES [1] S. Boll. Suppressio of acoustic oise i speech usig spectral subtractio. IEEE Tras. Acoust., Speech, Sigal Process., 7():113 1, April [] Y. Ephraim ad D. Malah. Speech ehacemet usig a miimum-mea square error short-time spectral amplitude estimator. IEEE Tras. Acoust., Speech, Sigal Process., 3(6): , December [3] Zeto Goh, Kah-Chye Ta, ad T. G. Ta. Postprocessig method for suppressig musical oise geerated by spectral subtractio. IEEE Tras. Speech Audio Process., 6(3):87 9, May [4] T. Esch ad P. Vary. Efficiet musical oise suppressio for speech ehacemet system. I Proc. IEEE Itl. Cof. o Acoustics, Speech ad Sigal Processig (ICASSP), pages , April 9. [5] C. Breithaupt, T. Gerkma, ad R. Marti. Cepstral smoothig of spectral filter gais for speech ehacemet without musical oise. Sigal Processig Letters, IEEE, 14(1): , December 7. [6] Y. Wag ad M. Brookes. Speech ehacemet usig a robust Kalma filter post-processig i the modulatio domai. I Proc. IEEE Itl. Cof. o Acoustics, Speech ad Sigal Processig (ICASSP), pages , May 13. [7] K. Paliwal ad A. Basu. A speech ehacemet method based o Kalma filterig. I Proc. IEEE Itl. Cof. o Acoustics, Speech ad Sigal Processig (ICASSP), pages , April [8] J. D. Gibso, B. Koo, ad S. D. Gray. Filterig of colored oise for speech ehacemet ad codig. IEEE Tras. Sigal Process., 39(8): , [9] K. Paliwal, K. Wojcicki, ad B. Schweri. Siglechael speech ehacemet usig spectral subtractio i the short-time modulatio domai. Speech Commuicatio, 5(5):45 475, 1. [1] T. H. Falk, S. Stadler, W. B. Kleij, ad W. Y. Cha. Noise suppressio based o extedig a speechdomiated modulatio bad. I Proc. Iterspeech Cof., pages , August 7. [11] J. G. Lyos ad K. K. Paliwal. Effect of compressig the dyamic rage of the power spectrum i modulatio filterig based speech ehacemet. I Proc. Iterspeech Cof., pages , September 8. [1] S. So ad K. K. Paliwal. Modulatio-domai Kalma filterig for sigle-chael speech ehacemet. Speech Commuicatio, 53(6):818 89, July 11. [13] O. Cappe. Elimiatio of the musical oise pheomeo with the Ephraim ad Malah oise suppressor. IEEE Tras. Speech Audio Process., (): , April [14] Joh S. Garofolo, Lori F. Lamel, William M. Fisher, Joatha G. Fiscus, David S. Pallett, Nacy L. Dahlgre, ad Victor Zue. TIMIT acoustic-phoetic cotiuous speech corpus. Corpus LDC93S1, Liguistic Data Cosortium, Philadelphia, [15] Mike Brookes. The matrix referece maual [16] J. Makhoul. Liear predictio: A tutorial review. Proceedigs of the IEEE, 63(4):561 58, April [17] T. Gerkma ad R. C. Hedriks. Ubiased based oise power estimatio with low complexity ad low trackig delay. IEEE Tras. Audio, Speech, Lag. Process., (4): , May 1. [18] D. A. Reyolds, T. F. Quatieri, ad R. Du. Speaker verificatio usig adapted Gaussia mixture models. Digital Sigal Processig, 1(1 3):19 41,. [19] H. J. M. Steeeke ad F. W. M. Geurtse. Descriptio of the RSG.1 oise data-base. Techical Report IZF , TNO Istitute for perceptio, [] ITU-T P.51. Test sigals for use i telephoometry, August