Erik Visser, Te-Won Lee. Institute for Neural Computation University of California, San Diego La Jolla, CA

Transcription

1 APPLICATION O BLIND SOURC SPARATION IN SPCH PROCSSING OR COMBIND INTRRNC RMOVAL AND ROBUST SPAKR DTCTION USING A TWO-MICROPHON STUP rik Vier, Te-Won Lee Intitute for Neural Computation Univerity of California, San Diego La Jolla, CA ABSTRACT A peech enhancement cheme i preented integrating patial and temporal ignal proceing method for blind denoiing in non tationary noie environment. In a firt tage, patially localized interferring point ource are eparated from noiy peech ignal recorded by two microphone uing a Blind Source Separation (BSS) algorithm auming no a priori knowledge about the ource involved. Spatially ditributed background noie i removed in a econd proceing tep. Here, the BSS output channel containing the deired peaker i filtered with a time-varying Wiener filter. Noie power etimate for the filter coefficient are computed from deired peaker abent time-interval identified by comparing ignal energy of eparated ource file from the BSS tage. The cheme performance i illutrated by peech recognition eperiment on real recording corrupted by babble noie and compared to conventional beamforming and ingle channel denoiing technique.. INTRODUCTION Speech enhancement in real environment remain a challenging tak and a number of approache appear in the literature. Singlemicrophone enhancement algorithm baed on temporal information about the recorded ignal are mot frequently encountered. They often ue a probabilitic framework with tatitical model of a ingle peech ignal corrupted by tationary Gauian noie []. While reaonable performance i obtained when the noie i tationary, it deteriorate rapidly when noie power varie importantly or peech miture contain ignificant reverberation. Single channel denoiing algorithm baed on minimum tatitic [, 3] and Voice Activity Detection (VAD) [4, 5] have been developed to eplicitly addre non tationary noie. Spatial information about ignal miture can be eploited by uing multiple microphone. In beamforming [7] for eample, an array of microphone with a known geometry i ued to uppre interfering ignal. Here, ource localization can be performed a well and reverberation be handled with adaptive algorithm [7]. However, thee method uually rely on a priori information about the acoutical environment and ource involved. Alo, large microphone array are required for good performance whoe implementation i difficult and cotly. The number of microphone can be dratically reduced by uing recently developed ource eparation algorithm [8, 6]. Thee algorithm eploit patial information about ignal miture recorded at different microphone location to eplicitly eparate interfering noie ignal from the deired ource ignal without auming any a priori ource model. In the following a combined patial/temporal peech enhancement approach baed on a BSS algorithm i developed.. SPCH NHANCMNT SCHM We conider an analytical framework with different microphone miture ignal compoed of point ource ignal and additive background noie "!$#% where & i the convolution order, i a (') miing matri. A key ditinction i made between patially point ource and ditributed background noie. Auming little reverberation, ignal originating from point ource can be viewed a identical when recorded at different microphone location ecept for an amplitude factor and a delay. The unmiing trategy would conit in finding thee latter parameter for each ource and umming up the realigned and caled miture ignal. However background noie originate from a large number of patially ditributed ource reulting in no defined delay and amplitude difference between ignal recorded at each microphone. Thu a background noie unmiing trategy poe a ingular problem. Thee different patial ignal characteritic are addreed in ubequent tage of the peech enhancement cheme illutrated in igure. Spatial information about interfering point ource i proceed in the blind ource eparation unit while the remaining tage remove ditributed background noie by a mied temporal/patial proceing approach. igure illutrate the correponding ignal flow on a digit utterance eample. 3. BLIND SOURC SPARATION (BSS) O INTRRING POINT SOURCS In recent year a number of algorithm have emerged implementing blind ource eparation of miture ignal into it component by decorrelating their higher-order tatitic [6]. However the econd order decorrelation approach preented in [8] yielded the mot conitent eparation performance in our eperiment. The Multiple Adaptive Decorrelation (MAD) algorithm [8] i deigned for

2 > Speaker Interferring Point Source Recorded Noiy Source Miture Ditributed Background Noie Microphone Ditributed Background Noie Recorded Noiy Source Miture Blind Source Separation (BSS) of Interfering Point Source Spatial Proceing Blind Source Separation (BSS) of Interfering Point Source Separated Noiy Speaker Signal Denoied Deired Speaker Signal Background Denoiing uing Deired Speaker Activity Detection (SAD) y Temporal/Spectral Proceing Separated Noiy Speaker Signal ig.. Propoed Speech nhancement Scheme eparating recorded miture unmiing filter matrice uch that into original ource by finding a equence of ) Q being the filter length. The unmiing filter computation i eecuted in the frequency domain where, being the pectrogram obtained by conecutively computing the Short Time ourier Tranform of length (where! " &, the convolution order), of at each time intant in an '&)( *,+ - and overlap-hift fahion [8]. If the cro correlation of the meaurement i denoted by that of the ource by &( + -, we get from 3 H G H J KJ 6LMONPRQSDQUT"TVW YX X [Z 4 / 65798;:)?A@ BDC > = Ï + H H () The firt contraint impoe that the filter length Q be much maller than to olve the frequency permutation problem [8]. Alo caling iue are olved by the econd contraint fiing the diagonal element of the filter matrice to unity. The final learning rule i \] ^ )_a`b ÏV+ Öc. It i noted that the econd contraint in problem () enure that the dominant peaker voice will be eparated at the microphone poition at which it amplitude i highet during mot of the ignal length [9], making an additional algorithmic approach to determine the deired peaker from the eparated ource unneceary. The approach ha hown robut performance in a number of application [9]. igure 3 illutrate how a deired peaker ignal y Background Denoiing uing Deired Speaker Activity Detection (SAD) Denoied Deired Speaker Signal ig.. Proceed ignal at each enhancement tage (digit utterance) (digit utterance in a noiy office environment) i eparated from an interferring point ource by applying the BSS algorithm. However, a illutrated in igure 3, both eparated ource file till contain the original baeline background noie. Thi background noie could not be eparated ince it i a comple combination of too many reverberated and diffue ource. In the following the background denoiing tage in cheme i addreed. 4. BACKGROUND DNOISING Denoiing method previouly invetigated in thi framework [9], were implicitly baed on a tationary noie aumption with noie power etimation from BSS output channel d only. Here, patial information from both eparated BSS channel i conidered to more accurately track non tationary noie. 4.. Standard Denoiing Technique Two fundamental method have emerged for determining timevarying noie power and are baed on Voice Activity Detection (VAD) [4, 5] or minimum tatitic [, 3]. VAD approache continuouly track the meaured noiy ignal power and perform a weighted noie power update depending on the probability a peech interval ha been detected. The drawback of thee method i that etenive a priori model are neceary to dicriminate peech/nonpeech interval and robutne i not guaranteed in the preence

3 $ BSS W ig. 3. Blind Source Separation (BSS) of Interfering Point Source: Input recorded noiy ource miture (left) and output eparated noiy ource (right) of peech containing diturbance like babble noie. Minimum tatitic baed denoiing algorithm eek to determine minimum noie power in each pectral ubband over a finite time horizon. Thee noie power etimate are then ued to compute the coefficient of a time-varying Wiener filter []. However, when thee receding time horizon are too hort or equivalently peech egment continuouly cover a long time period, peech ignal power i interpreted a noie and deired ignal lo and/or reverberation reult. 4.. Deired Speaker Activity Detection (SAD) Denoiing performance can be ignificantly improved if time-interval containing noiy deired peaker peech ample are differentiated from noie-only interval when etimating non tationary noie power. In the following we propoe a new robut, model-independent meaure baed on two channel information to detect deired peaker containing time-interval. If the energy of eparated BSS channel X over a time frame i given by ` X Öc X K a two-channel energy ratio factor can be defined a ` c \ ` c ` Öc ` Öc%! \ ` Kc and computed over the whole ignal length in an overlap-add fahion with hifting window of ize. The baic motivation for i given firt before dicuing the remaining term in (). In the cae of miture corrupted by ditributed background noie only, the deired peaker will be iolated into BSS channel and removed from BSS channel which will olely contain ditributed background noie. Hence, by neglecting the term in \ ( 6L ) (a well a the ma operation) and conidering ", the firt-order Taylor epanion of yield Z ` Öc ` Öc J The correponding epreion in the frequency domain with the noie variance in individual pectral ubband computed from the variance of and the noie plu peech variance from i equivalent to a Wiener filter coefficient. Similarly the correponding epreion for () i analog to a generalized Wiener filter gain function []. However, eperiment have hown that reliable filter cannot be directly etimated from the ratio of pectral ubband channel energie. In fact, although the ditributed background noie energy integrated over all pectral band in a given time-interval of i imilar to the overall background noie energy in the ame time-interval in d, thi i not true when individual pectral ubband are conidered. Intead the overall energy ratio () over a time frame can be ued to detect deired peaker activity. Indeed, ince it i aumed that background noie energie are imilar in each recorded miture when microphone are poitioned cloe enough and the overall energy i preerved from recorded to eparated ource becaue of the caling contraint in (), the denominator in () i cloe to zero and hence tend to zero when the deired peaker i abent. If it i preent, the energy in BSS channel i much larger than in channel, the quotient in () tend to zero and thu to. In practice, BSS channel may contain an interferring point ource eliminated from channel. Therefore the \ term with \ ` Kc! #" %$& K' i introduced in () to robutify the detection of deired peaker by eplicitly tracking energy change from recorded $3 channel to eparated BSS channel. actor )(+*-,/. (+*-,/. * cale the energy change in channel to a correponding energy change in * channel. The parameter allow to adjut the harpne of peech/non peech interval delimitation. The reulting i ued to provide a probability meaure for the peaker preence. The noie etimate i given by 798 O! Z ;:=> 798 O! BA O where := BA i a moothing contant and O i the autocorrelation of ` Z Öc, in time frame. The current peech plu noie power etimate i obtained from the recurrence 7 / 8 I! Z ;:=CD 7 / 8 I! OZ?:=C O where :=C i a moothing contant and i the auto-correlation of, in time frame. The noie and peech+noie power etimate are ued to compute the Wiener filter coefficient I for each frame. inally the denoied peech pectrum i obtained from with filter coefficient O G I IH Z I 798 I 7 / 8 I J It wa oberved that uing directly from () reulted in too agreive denoiing performance ince the value of i not necearily one at each local maimum and may decreae too rapidly near the edge of detected time-interval, thereby cutting off peech part. Hence i refined by replacing it by a equence of Hanning window with center determined by the local

4 maima of from () and width given by twice the ditance between ymmetric point around each maimum where reache a certain threhold. The reulting curve (ee igure 4) i mooth and ufficiently wide to avoid cutting off edge of deired peaker part. The denoied peech ignal i hown in igure 6, cae (f), where it i compared to other denoiing approache. Quantitative comparion to tandard method i preented in the net ection. 0.8 to emphaize the deired ignal amplitude by in-phae ummation. Wherea better beamforming method eit, emphai i put in thi tudy on comparion of largely blind enhancement technique. Martin ingle channel type denoiing algorithm (DN) wa preferred over model-baed VAD technique for the ame reaon. In Table and igure 5 & 6, peech recognition on recorded file (RC) i evaluated againt the conventional cheme (B+DN), BSS, BSS followed by Martin algorithm (BSS+DN), BSS+DN followed by Source Activity Detection (BSS+DN+SAD) and BSS followed by SAD (BSS+SAD) BSS+SAD B+DN BSS e() ig. 4. Separated BSS channel with correponding (dahed line) from () and after refinement (full line); threhold indicated by horizontal line (ee tet) RC 5. XPRIMNTS Recording were taken in a 3 m ' 4 m ' 6 m room with two directional microphone (SONY tereo/zoom CM-ZS90) eparated by 0 cm mounted on a dek. The deired peaker itting at a ditance of 30 cm from the microphone etup (cloer to left microphone) wa uttering continuou digit entence while 4 loud peaker poitioned in each room corner were playing an identical ound file containing a miture of babble and white noie to generate patially ditributed background noie. Additionally a loud peaker wa put at 30 cm ditance from the right ide microphone playing a prerecorded word equence to create an interferring point ource. The peech recognizer a well a a multiple noie condition databae for training the HMM model wa provided by the AURORA benchmark dataet [0]. The peech feature etraction front-end v 0 [0] wa ued for computing the 39 MCC (including energy, delta, delta-delta). The tet databae conited of file recorded at different SNR db level (from -5 to 0 db). 00 digit entence, each containing a maimum of 4 digit, were recorded for each SNR cae (i.e. _ 400 digit per cae). The SNR wa meaured from ample of clean peech ignal recorded in the office environment with loudpeaker and interferring point ource turned off and ignal recorded in the noiy environment with the ame microphone etup. The propoed cheme i compared to tandard peech enhancement method like delay-and-um beamforming and minimum tatitic type denoiing like Martin algorithm []. In the patial proceing method of beamforming(b), one miture i delayed and ummed to the other baed on the deired peaker known location 0-5 db 0 db 5 db 0 db SNR ig. 5. Word recognition accuracy for tandard (B+DN) and propoed cheme (BSS+SAD) (RC=recorded, unproceed cae) irt igure 5 clearly indicate that peech recognition on the unproceed recorded file lead to unacceptable performance even in the preence of mall diturbance. The conventional approach (DN+B) fail to raie the accuracy in an important manner. The reaon i the interferring point ource ha not been removed efficiently (ee cae (b) in igure 6). It degrading impact i largely alleviated by applying blind ource eparation (curve BSS and cae (c) in igure 6), which lift the recognition rate by at leat 0 %. inally curve BSS+SAD and cae (f) in igure 6 illutrate the BSS tep followed by SAD yielding the maimum recognition performance overall. To compare the background denoiing technique, the performance for all the pot-proceing tage involving BSS i analyzed in Table. or high SNR (5;0 db), one obtain imilar ( % difference in) performance when proceing the deired peaker ignal with conventional denoiing (BSS+DN) or SAD cheme. Indeed the ignal ha been coniderably enhanced by removing the interferring point ource and the remaining background noie level i too low to caue ignificant peech deterioration. However, at difficult SNR (-5;0 db) level, the SAD denoiing method preent ubtantial advantage over BSS+DN. Here, highly non tationary background noie component cannot be ufficiently eliminated with the minimum tatitic approach (ee cae (d) in igure 6). On the contrary, the two channel in-

5 db RC DN+B BSS BSS+DN BSS+DN+SAD BSS+SAD Table. Word recognition accuracy (%) for variou denoiing cheme (ee tet for dicuion) formation baed SAD approache achieve the neceary denoiing in noie-only interval (cae (e) and (f) in igure 6) and outperform conventional denoiing (BSS+DN) coniderably (_ 5-0 % accuracy increae, Table ). Thi how the benefit of non tationary noie etimate determined from two channel information over ingle channel, minimum noie power averaged over a long time interval. inally, the uperior performance of BSS+SAD over BSS+DN+SAD in low SNR cae ugget that le agreive denoiing in digit containing time-interval preerve more deired peech information and/or induce le artifact. The bet reference accuracy achieved wa % on 00 digit entence recorded with the ame microphone etup in the ilent office environment. Thi reflect the effect of room reverberation, peaker and recording equipment different from the one ued in the AU- RORA databae where noie wa artificially mied to TI digit. Regarding the computational requirement, the cheme work near real-time on a 550 MHz PC. The bulk of the computational load i carried out in the Blind Source Separation tep wherea the SAD enhancement unit can be eecuted fater than real-time. tage uing only two microphone and no a priori model about the peech and noie ource involved. irt the deired peaker i eparated from interferring, patially localized point ource uing a blind ource eparation algorithm. In a econd tep, patially ditributed background noie i removed uing energy information from both eparated BSS output channel to detect noie-only interval and compute a robut non tationary noie etimate. In peech recognition eperiment carried out in a noiy office environment, the cheme wa hown to yield ignificant enhancement over tandard method uch a beamforming and ingle channel Wiener filtering baed on minimum tatitic. In particular a background denoiing methodology ha been propoed which efficiently denoie highly non tationary noie interval and perform improved peech enhancement under difficult noie condition when compared to conventional ingle channel denoiing algorithm. Denoiing i independent of the background noie pectral content ince detection of deired peaker peech activity i baed on comparion of the energy between two channel only. A no a priori knowledge i ued in the BSS tage either, the cheme i uitable for environment-independent peech enhancement and recognition tak. 7. RRNCS [] phraim, Y., Statitical Model-Baed Speech nhancement Sytem, Proceeding of the I, Vol 80, No 0, October 99 [] Martin, R., Spectral Subtraction Baed on Minimum Statitic, Proceeding of the USPICO 94, 994 [3] Doblinger, G., Computationally fficient Speech nhancement by Spectral Minima Tracking in Subband, Proceeding of the UROSPCH 95, pp53, Madrid, 995 (a) (b) (c) (d) (e) (f) [4] Haigh, J.A., Maon, J.S., A Voice Activity Detector baed on Ceptral Analyi, Proceeding of the UROSPCH 93, pp , Berlin, 993 [5] Sohn, J., Kim, N.S., Sung, W., A Statitical Model-Baed Voice Activity Detection, I Signal Proceing Letter, vol 6., No, 999 [6] Bell, A.J., Sejnowki, T.J., An Information-Maimiation Approach to Blind Separation and Blind Deconvolution, Neural Computation, 7(6), pp , 995 [7] Brandtein, M., Silverman, H., A Practical Methodology for Speech Source Localization with Microphone Array, Computer, Speech and Language, vol, no, pp. 9-6, 997 ig. 6. Comparion of different denoiing trategie on a recorded digit utterance eample (trancript 000): (a) RC, (b) DN+B, (c) BSS, (d) BSS+DN, (e) BSS+DN+SAD, (f) BSS+SAD 6. CONCLUSIONS A patio-temporal peech enhancement cheme ha been preented that enhance noiy peech ignal in two ubequent proceing [8] Parra, L., Spence, C., Convolutive Blind Separation of Non- Stationary Source, I Tranaction on Speech and Audio Proceing, vol 8, pp , 000 [9] Vier,., Otuka, M., Lee, T.-W., A Spatio Temporal Speech nhancement Scheme for Robut Speech Recognition, IC- SLP00, Denver, September 00 [0] Hirch, H.G., Pearce, D., The AURORA perimental ramework. ISCA ITRW ASR000 Challenge for the New Millennium, Pari, Sept 000