SPEECH ENHANCEMENT USING ADAPTIVE FILTERS AND INDEPENDENT COMPONENT ANALYSIS APPROACH

Transcription

1 SPEECH ENHANCEMENT USING ADAPTIVE FILTERS AND INDEPENDENT COMPONENT ANALYSIS APPROACH Tomasz Rutkowski Λ, Andrzej Cichocki Λ and Allan Kardec Barros ΛΛ Λ Brain Science Institute RIKEN Wako-shi, Saitama, JAPAN ΛΛ Depto. de Engenharia Eletrica Universidade Federal do Maranhao Sao Luis - Ma - BRAZIL tomek@bsp.brain.riken.go.jp, cia@brain.riken.go.jp and allan@biomedica.org ABSTRACT In this paper we consider the problem of enhancement and extraction of one speaker speech signal corrupted by environmental acoustic noise/interferences and other speakers using array of microphones containing at least two microphones. The preprocessing unit mimic human auditory system by roughly emulating cochlea by nonuniform bandpass filter bank. We construct filter bank with center frequencies of subbands based on approximation of target speaker fundamental frequency. Then we incorporate blind signal separation method for each subband signals, to extract maximum information that represent target speaker speech. After that desired signal is reconstructed from independent components representing every subband. Experiments with office room recordings are presented to confirm the validity and good performance of the proposed method in real-world environment. 1. INTRODUCTION We propose the system that approximately emulate human auditory system usning a nonuniform bandpass filter bank with tracking the fundamental frequency of target speaker. The filter bank consists of bandpass filters with bandwidth starting from 100 Hz to 200 Hz for first filter and increasing in length by factor two for next filters. For the experiments we use telephone quality speech signals with sampling frequency 8kHz. We assume the hearing system performs a spectrographic analysis of an auditory stimulus at the cochlea, which can be regarded as a bank of nonuniform selfadaptive filters whose outputs are ordered tonotopically. At first the fundamental frequency of the target speaker in mixture of convoluted speech signals is estimated in order to properly design center frequencies of the filters. The bank of adaptive band-pass filters, processes the available microphone signals around the fundamental frequency of the target speaker and around its harmonics. In the next stage of processing we perform blind source separation BSS, blind source extraction (BSE) or independet component analysis (ICA) for each frequency sub-band (bin). The efficient learning algorithms are used to perform BSS/BSE/ICA. Finally the set switches is implemented to perform temporal masking and selection/classification tasks of one independent components with specific feature that enhances the voice of target speaker. The main problem in this stage is to decide which signal from obtained subband independent components should be discarded and which one contain essential information from target speaker. We apply spectral measure to solve this problem in every subband. After such processing we have subband signals that carry speech signals with enhanced target speaker information. Last stage of our signal processing system performs signal reconstruction. The inverted filter bank is applied to correctly reconstruct the target speaker voice, avoiding problems with aliasing of subband filtered components. Extensive computer simulations results with array of two or more microphones confirm validity and performance of the proposed approach. We present the results for real room recordings with natural reverberations from the walls and objects in the room. 2. AUDITORY SYSTEM FILTER BANK It is well known that the human auditory system can roughly be described as a nonuniform bandpass filter bank, consisting of strongly overlapping bandpass fil-

2 s 2 s 1 ~ x 1 ~ x 2 Auditory Filter Bank Auditory Filter Bank f0 Nf0 f0 Nf0 x 11 x 21 x 1N x 2N BSE 1 BSE N y 1 y N Inversed Auditory Filter Bank y ŝ 1 s 3 f0 estimation Fig. 1. Conceptual block diagram of the algorithm which roughly mimics the auditory system. First it estimates the fundamental frequency f 0 and process the microphone signals using a bank of band-pass filters (such as the inner ear). Then it process mixed/convolved signal by an BSE or blind signal deconvolution (BSD) algorithm for each frequency bin. Finally, it performs masking by set of switches and inversed filter bank. ters with bandwidth in the order of 50 to 100 Hz for signals below 500 Hz and up to 5000 Hz for signal at higher frequencies [1]. The hearing system performs a spectrographic analysis of any auditory stimulus at the cochlea, which can be regarded as a bank of nonuniform self-adaptive filters whose outputs are ordered tonotopically. Recently, many sophisticated and biologically plausible models of such auditory filters banks have been proposed. In this paper we employ the idea of speaker voice extraction/enhancement from natural mixtures recorded in real environments. Following the idea, we suggest preprocessing unit for blind signal separation signal, that transform microphone signals into subbands with center frequencies around fundamental frequency f 0 and its harmonics. The fundamental frequency of target speaker can be estimated on basis of microphone signals ex 1 and ex 2 and/or estimated signal y(t). Unvoiced speech is noisy due to the random nature of the signal generated at a narrow constriction in the vocal tract for such sounds. For both voiced and unvoiced excitation, the vocal tract, acting as a filter, amplifies certain sound frequencies while attenuating others. As a periodic signal, voiced speech has spectra consisting of harmonics of the fundamental frequency of the vocal fold vibration; this frequency, often abbreviated f 0, is the physical aspect of speech corresponding to perceived pitch. In our algorithm we use that feature to track the speaker, that fundamental frequency f 0 is known or can be estimated. Our filter bank adopt central frequencies of subband bins to enhance signal of target speaker. An important property of speech signals is that they are non-stationary, but can be regarded as locally stationary. Roughly speaking, speech can be divided in the time domain into voiced and unvoiced sounds, the first ones having more structure in the frequency domain than the later. Indeed, voiced sounds are regarded in general as quasi-periodic. In this matter, some experiments pointed that humans may be using this voiced structure to separate sounds from the background. Moreover, humans can more easily understand a voiced than a unvoiced sound. One open problem is how the auditory system segregates those sounds in a higher level (at the auditory cortex). In this work, we suggest that this can be carried out by exploiting the local statistical independence of a pair of sub-band sounds for each frequency sub-band (bin). Our final objective is to develop an efficient algorithm whose output signal y(t) is a modified version of a speech signal s i (t), i.e. signal of interest y(t) = g(s i (t)), where g( ) represents a unavoidable distortion filter and a non-linear transformation operator. Also in our algorithm we included the temporal masking characteristic of the auditory system. This can be managed by a set of switches which select from the subband components after blind separation/extraction algorithm speaker related information. Fig.1 shows the conceptual block diagram of the system. It is composed of four parts. The first one is the fundamental frequency estimation algorithm in spectral domain, which estimates the fundamental frequency from the mixed signals as well as it shows which part of the speech is voiced. It should be noted, that estimation of f 0 becomes rather difficult when all speakers are at the same distance from microphones. The second processing unit is a bank of FIR band-pass filters with center frequencies adopted to estimated f 0

3 a) x( n) y 0 H 0 H 1 y 1 v 0 v 1 Magnitude (db) Normalized Frequency ( π rad/sample) b) v 0 v 1 u 0 u 1 F 0 F 1 ^ x( n) Fig. 3. Filter bank frequency characteristic for 5 subbands. and to avoid aliasing problems we apply: Fig. 2. Filter bank analysis (a) and sections (b) for only 2 subbands. for target speaker. The filter banks process the signals around the fundamental frequency and around its harmonics. The third section is a bank of BSE (blind signal extraction) units for enhancing the desired signal for each frequency sub-band. Finally, the last section performs signal reconstruction based on subbands signal. The reconstruction section is inversed auditory filter in respect to the second unit of our system. In this way we try to reconstruct the signal from subbands avoiding aliasing and frequency distortion problems. 3. ADAPTIVE FILTER BANKS The filter bank consists of slightly overlapping bandpass filters with bandwidth starting from 100 Hz to 200 Hz for first filter and increasing in length by factor two for next filters. At first the fundamental frequency of the target speaker in mixture of convoluted speech signals is estimated in order to properly design center frequencies of the filters. The bank of adaptive band-pass filters, processes the available microphone signals around the fundamental frequency of the target speaker and around its harmonics. Following the idea of human auditory system frequency sensitivity, we decided to construct filter banks with center frequencies as follows: f 0; 4f 0; 10f 0; 22f 0;:::, making the next subbands two times wider in direction to higher frequencies, because human speech has less sound representation there. The lowpass and highpass FIR filters are implemented in cascade configuration. In order to prevent perfect reconstruction of the signal, we design analysis and synthesis filters with following constrains [2] in every section (see Fig. 2 for reference). To following constrain prevents distortion: F 0 (z)h 0 ( z) F 1 (z)h 1 ( z) =0 (2) The exemplary five section filter bank frequency characteristic is presented on the Fig. 3. The center frequency of every subband is constructed from pairs of lowpass and highpass filters is always around f 0 and its higher harmonics f 0; 4f 0; 10f 0; 22f 0;:::for every target speaker. 4. BLIND SOURCE SEPARATION In order to separate the subbands signals we can use any algorithm for BSS/ICA like Natural Gradient algorithm, SOBI, JADE, RICA etc. [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]. In this paper we use novel algorithm for blind source extraction (BSE). Due to limit of space we present here only final algorithm without the theoretical justification. Let us consider a single processing unit for extraction of voiced independent subcomponent (see Fig. 4): y i (k) = w T i x i (k) = " i (k) = y i (k) LX p=1 mx j=1 w ij x ij (k) ; (3) b ip y i (k p) = w T i x i (k) ey i ; (4) where m is the number of microphones i = 1; 2;:::;N,(N - number of frequency P bins) w i = [w i1 ;:::;w im ] T L and ey i (k) = p=1 b ipy i (k p) is the output of FIR bandpass filter with suitably chosen center frequency and bandwidth. Coefficients b ip are fixed. It can be easily shown that the weight of the BSE processing unit can be iteratively updated as follows F 0 (z)h 0 (z) F 1 (z)h 1 (z) =2z l (1) w i = ^R 1 x ix i ^R x~yi ; w iλ = w i =kw i k (5)

4 x i 1 w i1 mixture from microphone #1 x i 2 2 x im w i w im 0 f i Bandpass Filter b ip (k) y i ~ yi ( k ) mixture from microphone #2 (closer to target speaker) Fig. 4. Single processing for blind extraction of an independent voiced component (m means the number of microphones typically m =2. where MX ^R xix i = 1 x i (k)x T i (k); (6) N k=1 MX ^R xiy i = 1 x i (k)ey i (k): N k=1 It should be noted that by changing the central frequency of bandpass filter we can extract in generally different components. Moreover, using the above concept we always extract desired independent components with higher energy so masking set of switches is not necessary. Fig. 6. Result for extraction of one speaker from mixture of three speakers recorded using only two microphones (target speaker was close to microphone #2). shown in Fig. 8. For each subband i we perform the following processing with updating y 1 (k) =x i1 (k) LX p=0 b ip x i2 (k p) (7) 4b ip = y i (k)x i2 (k p): (8) 5. MULTICHANNEL BLIND DECONVOLU- TION/EQUALIZATION Instead of instantaneous blind extraction of subband signals we can apply blind deconvolution/equalization especially if bandwidth is relatively large. The simple model for multichannel deconvolution/equalization is 6. SIGNAL RECONSTRUCTION FROM IN- DEPENDENT COMPONENTS The components carrying maxima around f 0 harmonics obtained from previous section are taken into reconstruction. The signals carry speech signals with enhanced target speaker information. Last part of our signal processing system performs signal reconstruction. The inverted filter bank is applied to correctly reconstruct the target speaker voice, avoiding problems with aliasing of subband filtered components. Extensive computer simulations results presented in next section confirm validity and performance of the proposed approach. We present results for real room recordings with natural reverberations from the walls and subjects in the room. 7. EXPERIMENTS WITH SPEECH SIGNALS RECORDED IN REAL ENVIRONMENT Fig. 5. Room recording plan. The real room recordings were done in the empty experimental room, without carpet and any sound absorbing elements, with many reverberations (easy to notice even during usual conversation). We used two or three cardioid condenser boundary microphones

5 mixture from microphone #1 mixture from microphone #2 Fig. 5 Exemplary computer simulations are shown in Fig. 10 and Fig. 7. For each experiment we have obtained essential enhancement of target speaker. Due to limit of space more details will be given during workshop s presentation. For all performed experiments considerable speech enhancement has been achieved. More detailed audio experiment will be presented at conference. 8. CONCLUSIONS AND DISCUSSION mixture from microphone #3 Fig. 7. Result for extraction of one speaker from mixture of four speakers recorded using three microphones (target speaker was close to microphone #1). In this paper we have described multistage subband based system for extraction and enhancement of speech signal corrupted by other speakers and other acoustic interferences. The proposed approach can be extended to other applications like extraction of biomedical signals with reduced number of sensors. The open problems is how to extract a speaker with lower energy than others speakers or speech signal with specific features independently of this how far away is from the microphones. x i1 _ y i mixture from microphone #1 x i 2 b i FIR mixture from microphone #2 mixture from microphone #3 Fig. 8. Processing unit for blind deconvolution/equalization for m =2microphones. audio-technica PRO44, that can record sounds from half-cardiod space. Such configuration let as record sounds from many directions similarly as human being can sense using ears. Boundary microphones make the task more complicated, because they record more reverbarations from surroundings than directional ones. The microphones were connected to microphone high class line amplifier and professional 20-bit multitrack digital recording system in PC class computer. The system allows us to record up to 8 channels simultaneously with 20-bit resolution and sampling frequency 44.1kHz. The following recordings were done using natural voices and sounds from speakers: (i) 2 mixed man and woman voices talking different frazes in English; (ii) 3 man voices talking different frazes in English; (iii) mixed recordings of man and woman voices talking different frazes in English; (iv) mixed human and natural sound (rain, water fall) sounds or music. We conducted all experiments with target speaker positioned closer to microphones than other sources. The scheme of our recording conditions is presented on mixture from microphone #4 Fig. 9. Result for extraction of one speaker from mixture of five speakers recorded using four microphones (target speaker was close to microphone #3). 9. REFERENCES [1] D. O Shaughnessy, Speech Communication - Human and Machine, IEEE Press, New York, second edition, [2] G. Strang and T. Nguyen, Wavelets and Filter Banks, Wellesley - Cambridge Press, Wellesley MA USA, [3] S. Amari, ICA of temporally correlated signals - learning algorithm, in Proceedings of ICA 99:

6 mixture (speech rain) from microphone #1 mixture (speech rain) from microphone #2 Fig. 10. Result for extraction of one speaker from mixture of two speakers talking during heavy rain, recorded using only two microphones (target speaker was close to microphone #2). International workshop on blind signal separation and independent component analysis, Aussois, France, Jan. 1999, pp [4] S. Amari and A. Cichocki, Adaptive blind signal processing - neural network approaches, Proceedings IEEE, vol. 86, no. 10, pp , October 1998, (invited paper). [5] A. K. Barros and A. Cichocki, RICA - reliable and robust program for independent component analysis, Report and matlab program of RIKEN,Brain Science Institute RIKEN, 2-1 Hirosawa, Wako-shi, Saitama, JAPAN, or [6] A. Belouchrani, K.A. Meraim, and J.-F. Cardoso, A blind source separation technique using second order statistics, IEEE Transactions on Signal Processing, vol. 45, pp , February [7] A. Cichocki, R. Thawonmas, and S. Amari, Sequential blind signal extraction in order specified by stochastic properties, Electronics Letters, vol. 33, no. 1, pp , January [8] N. Delfosse and P. Loubaton, Adaptive blind separation of independent sources: a deflation approach, Signal Processing, vol. 45, pp , [9] A. Hyvärinen and E. Oja, A fast fixed-point algorithm for independent component analysis, Neural Computation, vol. 9, pp , [10] S.C. Douglas and S.-Y. Kung, Kuicnet algorithm for blind deconvolution, in Proceedings of the 1998 IEEE Workshop on Neural Networks for Signal Processing, New York, 1998, pp [11] C. Jutten and J. Herault, Blind separation of sources, part i: An adaptive algorithm based on neuromimetic architecture, Signal Processing, vol. 24, pp. 1 20, [12] L. Molgedey and H.G. Schuster, Separation of a mixture of independent signals using timedelayed correlations, Physical Review Letters, vol. 72, no. 23, pp , [13] B. A. Pearlmutter and L. C. Parra, Maximum likelihood blind source separation: A contextsensitive generalization of ica, in Proceedings of NIPS 96, 1997, vol. 9, pp [14] L. Tong, V.C. Soon, R. Liu, and Y. Huang, Amuse: a new blind identification algorithm, in Proceedings of ISCAS 90, New Orleans, LA, [15] J.K. Tugnait, Blind spatio-temporal equalization and impulse response estimation for mimo channels using a godard cost function, IEEE Transactions on Signal Processing, vol. 45, pp , January [16] S. Choi and A. Cichocki, Blind separation of nonstationary sources in noisy mixtures, Electronic Letters, vol. 36, pp , April [17] A. K. Barros and N. Ohnishi, Removal of quasi-periodic sources from physiological measurements, in Proceedings of ICA 99: International workshop on blind signal separation and independent component analysis, Aussois, France, Jan. 1999, pp [18] J. Huang, K-C. Yen, and Y. Zhao, Subbandbased adaptive decorrelation filtering for cochannel speech separation, IEEE Transactions on Speech and Audio Processing, vol. 8, no. 4, pp , July [19] A.K. Barros, H. Kawahara, A. Cichocki, S. Kojita, T. Rutkowski, M. Kawamoto, and N. Ohnishi, Enhancement of a speech signal embedded in noisy environment using two microphones, in Proceedings of the Second International Workshop on ICA and BSS, ICA 2000, Helsinki, Finland, June, 2000, pp