A Robust Feature for Speech Detection*

Transcription

1 БЪЛГАРСКА АКАДЕМИЯ НА НАУКИТЕ. BULGARIAN ACADEMY OF SCIENCES КИБЕРНЕТИКА И ИНФОРМАЦИОННИ ТЕХНОЛОГИИ Том 4, 2 CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 4, No 2 София Sofia A Robust Feature for Seech Detection* Atanas Ouzounov Institute of Information Technologies, 1113 Sofia atanas@iinf.bas.bg Abstract: A new robust feature intended for seech detection is roosed in this aer. This feature is the mean of the absolute values of the delta sectral autocorrelation function of the ower sectrum and it is named the Mean-Delta (MD) feature. For noisy seech, the trajectory variations of the MD feature are comared with those ones of the other robust feature the Energy-Entroy (EE) feature, develoed in [6]. The selected noisy seech samles from two databases (the SEAR database [8] and the BG-SRDat corus [9]) are used in the exerimental study. In all the tests the MD feature has demonstrated the same or better results than those of the EE. Keywords: seech detection, voice activity detection, sectral autocorrelation, seaker recognition. 1. Introduction The location of seech embedded in various non-seech events has many names, of which some are seech detection, endoints detection, voice (seech) activity detection, and seech/non-seech segmentation. The seech detection algorithms can be searated into two general categories. The first embraces the algorithms that analyze the time variations (trajectories) of selected arameters. These algorithms utilize a combination of a state automata and a set of thresholds (fixed or adative) in order to roduce a seech or non-seech decision for articular segment based only on the trajectories characteristics [5, 7]. The second category is comrised of algorithms based on a attern recognition * This research is suorted in art by the National Science Fund of the Bulgarian Ministry of Education and Science, Contract No И-1302/

2 technique. In this case, during the training mode the reference models for two classes (i.e., seech and non-seech) are created based on selected seech features. In classification mode, each segment is associated to one of the classes based on some kind of similarity function [1, 11]. The most frequently used features for seech detection are different energy transformations of the signal, sectral flatness, sectral entroy, autocorrelation functions, etc. The more comrehensive descrition of the seech features used for seech detection is given in [2, 5, 7, 10]. The feature selection for seech detection tasks usually is comosed of two stages. The first stage is a reliminary selection. It is based on a visual evaluation on the grahically reresented arameters. This selection is a feasible task only in cases when the arameters ossess reasonable grahical reresentation. The latter stage is the final feature selection and a recognition scheme usually is alied. The develoed seech detection algorithm is embedded as a comonent of a comlete seech or seaker recognition system. The effectiveness of different seech detection features is estimated exerimentally based on their indirect influence on the recognition erformance [1, 11]. In the resent study, a robust feature indented for trajectory-based seech detection algorithms is roosed. This feature is the mean of the absolute values of the delta sectral autocorrelation function of the ower sectrum. For different noisy seech signals the trajectory s variations of the roosed feature are comared with these ones of the other robust feature the energy-entroy based one, develoed in [6]. This is done by visual evaluation on the grahically reresented trajectories of both features. 2. The sectral autocorrelation function There are two well-known definitions of the term Sectral AutoCorrelation Function (SACF). The first one is roosed in [4] where the magnitude Fourier sectrum is flattened by bank of lifters and the sectral autocorrelation is comuted at the outut of each lifter. This autocorrelation function is used to obtain an estimation of the itch frequency for noisy seech. In the second definition, roosed in [3] SACF is defined as discrete quantities of the magnitude sectrum with sectral resolution as in the Fourier transform used to obtain the sectrum. In [3] the SACF was utilized for develoing of new linear rediction algorithm based on sectral enveloe of the seech signal. In the resent study, the second definition of the SACF will be used. An interesting feature of SACF, first demonstrated in [3], is the fact that this function is eriodic for not only voiced seech but also for unvoiced one. However, for the unvoiced seech this eriodicity is not so significant. Let x (i) be a discrete seech signal, where i 0,..., I 1, I is the number of samles and the sectrum X (k) of x(i) obtained by the discrete Fourier transform is I (1) 1 ki X ( k) x( i)ex( j2 ), i 0 I where k=0,..., K/2, K is the number of oints in the discrete Fourier transform. 4

3 According to [3] the sectral autocorrelation R B (l) is defined with the magnitude sectrum X(k) as K / 2 1 (2) R ( l) X ( k). X ( k l), B k 0 where l = 0,..., L; L is the number of correlation lags and L = K/2. In equation (2), the magnitude sectrum above Nyquist frequency is used in autocorrelation comutation. It is known that for real signals this sectrum is symmetric to this one below Nyquist frequency. We studied the sectral autocorrelation functions based not only on the magnitude sectrum but also on the ower sectrum. Some reliminary exeriments are carried out and they revealed that the ower sectrum is more suitable when the main goal is to obtain more sensitive to sectral changes autocorrelation function. If S(k) is the ower sectrum of x(i) and S(k) = X(k) 2, where k = 0,..., K/2 then the sectral autocorrelation function R (l) is defined with the ower sectrum as K / 2 1 l (3) R ( l) S( k) S( k l), k 0 where l = 0,..., L, L is the number of correlation lags and L = K/2 1. The three examles of tyical seech signals and their normalized SACFs u to lag L = 100 are shown in Fig. 1. Fig.1. Examles of SACFs for: a) voiced seech signal /a/; b) unvoiced seech signal /sh/ and c) white noise 5

4 We analyzed selected frames from three different signals. The first one is a art from a seech sound /a/, the second one is a art from a seech sound /sh/ and the third one is a white noise (generated by a rogram). The analysis arameters are samling frequency of 8 khz and frame length of 240 samles. The frame is Hammingwindowed and added with zeros for 1024-oint Fast Fourier Transform (FFT). In each column in Fig.1 are shown the waveform of the signal and normalized SACFs (R B (0) = 1, R (0) = 1) of the magnitude and ower sectrums, resectively. It is clear that the sectral autocorrelation function of the voiced sound has a eriodic roerty as is noted in [3]. The eaks in the autocorrelation functions in Fig.1 are more enhanced when functions are calculated by the ower sectrums (see Fig.1 (d) and (g), (e) and (h), (f) and (i)). It is interesting to note that for the unvoiced seech sound the ower sectrum-based autocorrelation function ossesses more clearly eaks than this one for white noise. Hereafter the term sectral autocorrelation function in text will mean the autocorrelation function that is defined with the ower sectrum. 3. The robust feature For seech frames the sectral autocorrelation function is distinguished for more eaks with higher amlitudes than for non-seech ones (as seen in Fig. 1). We suose that a single arameter, which values deend on the number of eaks and their amlitudes, can be utilized as feature for seech detection. To obtain this arameter, in the study the sectral autocorrelation function is subjected to delta rocessing. The delta rocessing (i.e. the first-order orthogonal olynomial coefficient calculation) is alied in the autocorrelation domain, not in the time domain, as it is usually done [12]. We define the delta sectral autocorrelation function (DSACF) R (n, l) for the n-th frame as (4) R P Q qr ( n, l q) q Q P ( n, l), Q 2 q q Q where l = 0,..., L; Q is tyically between 2 and 5, i.e. regions from 5 to 11 lags are analyzed in the autocorrelation domain, and n = 0,..., N 1, N is the number of frames. We suose that R (n, l) = 0 for n < 0 and l > L, i.e. the first and last few values of R (n, l) will be influenced by these border conditions. The examles of three tyical seech signals (the same as in Fig.1), their normalized SACFs and the corresonded DSACFs (Q=3) u to lag L = 100 are shown in Fig. 2. 6

5 Fig. 2. Examles of SACFs and corresonded DSACFs for: a) voiced seech signal /a/, b) unvoiced seech signal /sh/ and c) white noise. Based on the examles shown in the last row in Fig. 2 we roose as seech detection feature to be used the mean m d of the absolute values of the DSACF estimated for the art of correlation lags. For n-th frame m d (n), is comuted accordingly to the formula L 1 2 (5) m ( n) R ( n, l) d l L, L 1 where R (n, l) is the DSACF in (4) for lag l, L 1 and L 2 are the boundary lags and L = L 2 L Hereafter m d (n) will be called Mean-Delta (MD) feature. We decided to analyze the lags of the DSACF laced between its first (L 1 ) and last zero crossing (L 2 ) oints (see the Fig. 2 (g), (h) and (i)). In this way, we avoid the border effects in the DSACF due to the delta feature calculation and the influence of the close to zero lag art of the SACF. The described above ideas are included in the following algorithm for the MD feature calculation: divide the analyzed seech signal into sequence of frames with frame length of 40 ms and frame shift of 10 ms; for each frame aly Hamming window; for each frame comute the ower sectrum of windowing seech signal via FFT the FFT-oints are K=1024; 7

6 for each frame comute the non-normalized biased sectral autocorrelation function by equation (3) with L = K/2 1; for each frame comute the delta sectral autocorrelation function by equation (4) with Q = 3; erform a trajectory smoothing of delta sectral autocorrelation function (interframe rocessing). To roduce the smooth trajectory of R (n) we base on the idea roosed in [10] and used to obtain the so-called long-term sectral enveloe. Here we aly the same idea (in fact this is a trajectory s smoothing algorithm) but on the trajectory of R (n). The smoothed version of R (n, l) is R S (n, l) and it is defined as [10]: S (6) R n l R n j l j, ) max (, ) J (, where n is the current frame, l is the current lag and J=2; for each frame find first and last zero-crossing oints L 1 and L 2 on the smoothed delta sectral autocorrelation function R S (n, l); for each frame comute m d (n) using R S (n, l) in (5) for lags between oints L 1 and L 2 ; smooth m d (n) in time (inter-frame rocessing) with linear method (over five frames). j J 4. Energy-Entroy (EE) based feature A new feature for seech detection obtained by combination of the energy and the sectral entroy is roosed in [6]. The exeriments, which were carried out, revealed that this feature is more robust than the energy alone in the real-world environment as inside the driving car. This EE feature is comuted for every seech frame as follows (the frame index is omitted for simlicity): comute the energy E I 1 i 0 (7) E x( i) 2 ; as (8) estimate the robability density function P(k) for the frequency comonent k X ( k) P( k), K 2 X ( k) k 0 where k = 0,..., K/2; limit the seech sectrum frequency range ( Hz) to the range from 250 Hz to 3750 Hz and add some heuristic rules, namely if frequency comonent k is below 250 Hz and above 3750 Hz then X(k) = 0; if P(k) 0,9 then P(k) = 0; comute the negative entroy K / 2 (9) H P( k)log( P( k)) ; k 0 2 8

7 comute the EE feature as (10) M ( E C )( H C ), EE ( 1 M ), where C E and C H are the average energy and entroy of the first 10 frames resectively. In this algorithm a smoothing in time with linear method is additionally included the same rocedure as this one for the MD feature. E H 5. Exeriments We carried out series of exeriments that can be searated into two grous. The aim of these exeriments is to erform grahically and to evaluate visually the trajectories of analyzed features. These features are the currently roosed MD feature and the energy-entroy based one EE feature, roosed in [6]. During the first grou of exeriments, we used selected noise-corruted seech samles from the SEAR database [8]. In the second grou of exeriments, we utilized noisy telehone seech samles from the BG-SRDat corus [9]. In order to make a correct comarison between MD and EE features we have to comute them in the same frequency range. We selected the range acceted in [6], i.e. from 250 Hz to 3750 Hz. The obtained trajectories in all exeriments are normalized in the range from zero to one to allow direct comarison between them Exeriments No1 The SEAR database was created for erformance evaluation of different seech enhancement systems in adverse/noisy environments. The SEAR database contains samles of noise-corruted seech that have been recorded by acoustically combining clean seech and noise (with a clock-synchronous rocedure for time-alignment). We selected one record from Lombard section and one from noisy seech recordings section in the SEAR database. All records have clear reference and corresonded noisy seech samles with different Signal-to-Noise Ratio (SNR). All selected wave files are with samling frequency of 16 khz at 16 bits, PCM format and mono mode [8]. The record from the Lombard section contains seech corruted with factory noise that is recorded in a car roduction hall. The SNR of clean reference is db and for noisy recording SNR = 9.96 db; The record from the noisy section contains seech corruted with white noise. It is acquired by samling high-quality analog noise generator. The SNR of clean reference is 40 db (no noise) and for noisy recording SNR = 2.37 db; In Fig. 3 the factory noise seech examle from the Lombard section in SEAR database and the corresonded trajectories of the MD and EE features are shown. In the figure are included: (a) the clean seech samle, (b) noisy version of (a), (c) MD feature contour for noisy seech in (b) and (d) EE feature contour for noisy seech in (b). 9

8 Fig. 3. Examles from the SEAR database: a) clean seech samle, b) noisy version of a) with factory noise, c) MD feature contour for noisy seech in b) and d) EE feature contour for noisy seech in b) In Fig. 4 are shown the white noise seech examle from the noisy recording section in SEAR database and the corresonded trajectories of the MD and EE features. In the figure are included: (a) the clean seech samle, (b) noisy version of (a), (c) MD feature contour for noisy seech in (b) and (d) EE feature contour for noisy seech in (b). Fig. 4. Examles from the SEAR database: a) clean seech samle, b) noisy version of a) with white noise, c) MD feature contour for noisy seech in b) and d) EE feature contour for noisy seech in b) 1 0

9 5.2. Exeriments No 2 The BG-SRDat is a corus in Bulgarian language recorded over noisy analog telehone channels and intended for seaker recognition. To achieve more realistic real-world conditions the seech data is collected by different tyes of telehone calls (internalrouting, local and long-distance) and various acoustical environments (noisy offices, halls and streets). The corus does not rovide the clean seech reference and its noisy version. It comrises only real-world noisy seech records. The seech data included in the BG-SRDat are samled with frequency of 8 khz at 16 bits, PCM format and mono mode [9]. We selected one record, which is tyical of the BG-SRDat. It distinguishes for the resence of high-level ulse noise (due to the aaratus commutations in analog telehone line). In Fig. 5 are shown the noisy seech examle from the BG-SRDat corus in (a), its sectrogram in (b) and the corresonded trajectories of the MD feature in (c) and EE feature in (d). Fig. 5. An examle from the BG-SRDat corus: (a) noisy seech samle, (b) the sectrogram of (a), (c) MD feature contour for noisy seech in (a) and (d) EE feature contour for noisy seech in (a) 6. Discussion In series of exeriments with noisy seech data, we comared the trajectories variations of the two features mentioned above in the text. Our goal was to use these features in seech detection tasks where there is not any information about the osition of noise 1 1

10 and seech fragments in the analyzed data. This limitation gets our exeriments much closer to the real-world environment tasks. In this case, we can utilize the feature trajectory s characteristics to make a seech/non-seech decision. We decided to use the feature value (contour level) as measure for the resence of seech. This is a socalled energy-tye aroach for seech detection. It based on the assumtion that the low levels in feature s contour corresond to the non-seech frames or frames with consonants and the high levels ones mainly to the voiced or semi-voiced frames. Moreover, we can use a set of thresholds (fixed or adative) to searate the seech and non-seech events. Our visual analysis has to answer the question which one of the resented here features is more suitable for using in the mentioned above energytye aroach? The results from exeriments No 1 with noisy samles from the SEAR database are shown in Fig. 3 and Fig. 4 and they revealed interesting facts. The contours of two comared features are similar only in Fig. 4 where the added noise is white and SNR is not very low (2.37 db). In this case, the energy-tye aroach can be used in seech detection tasks for both features. In the rest of exeriments, the obtained contours of the EE feature demonstrate unsatisfactory results. Based only on the EE feature s values it is difficult to make reliable decision about the lace of the seech and non-seech arts in the analyzed data, e.g., see Fig.3 (d), around time axis tick 0.5 s and after time axis tick 3.5 s. And vice versa, the trajectory of the MD feature at these time axis ositions (see Fig. 3(c)) allows finding out the non-seech fragments in easier way. The results from exeriments No 2 are shown in Fig. 5. The main reason to include this seech data in the study was to observe the behaviour of both features for seech contaminated with a high-level ulse noise. It can be observed in Fig.5 (d), that the ulse noise has a strong effect on the EE contour. In the art of the EE feature trajectory (e.g., between time axis ticks 3.5 s and 4.5 s) where there is only a ulse noise the level is higher than this one in middle of the hrase (around time axis tick 2.5 s) where there is only a seech. The MD feature contour does not erform in the same way it is more robust to the ulse noise as seen in Fig. 5 (c). We carried out additional exeriments (with data from both databases) where the EE and MD features are comuted from the full-range ower sectrum (from 0 Hz to the Nyquist frequency). For both features, the obtained results are worse than the resented once. However, the change for the worse is more significant for the EE feature than the MD one. The using of sectral entroy as feature in seech detections tasks is based on the assumtion that the sectrum is more organized in the seech regions than in the noise ones. The entroy is maximal when the signal is white noise and minimal when it is ure tone. It is known that the entroy-based seech detection algorithms are more suitable for seech signal in white or quasi-white noise and rovide oor results for colored noise [14]. It is reasonable to combine the entroy and another seech arameter in order to comensate individual drawbacks as is done for the EE feature. However, this feature is obtained in [6] by an ad hoc rocedure and in consequence of that in some cases the individual arameters drawbacks dominate in the final feature. It is known that the SACF is an effective feature for voiced frames detection [3, 4]. In addition, as is shown in Fig. 1 and Fig. 2, for the unvoiced sounds the SACF ossesses more clearly eaks than for the white noise. The delta rocessing on the SACF enhances the eaks and eliminates the influence of the mean of autocorrelation 1 2

11 function on the final MD feature. Additional restriction of the lags range (L 1 and L 2 in (5)) reduces the close to zero lag art (this art deends mainly on the sectral enveloe shae) effect on the MD feature. As a result, we obtain a feature which is more robust than the EE feature esecially in the cases where the seech signal is corruted with ulse or colored noise. 7. Conclusions In this study, we analyzed two features intended for seech detection algorithms. The first one roosed in [6] is formed by integration both of the sectral entroy and the signal energy (EE feature). The authors in [6] utilized this feature for endoints detection in driving car noisy environment. Their algorithm uses the EE feature and a set of thresholds and durations to find the frames where the seech message starts and where it ends. The second roosed feature is the mean of the absolute values of the delta sectral autocorrelation function of the ower sectrum (named MD feature). We erformed visual evaluation on the grahical reresentations of the EE and MD features for noisy seech samles selected from two sources the SEAR database and the BG-SRDat corus. Based on exerimental results we made the following conclusions: the behaviour of both features deends on the tye of noise. This deendence is more significant for the EE feature. In our study, the best results (for both features) are obtained when the seech is corruted with white noise. The worst ones are for EE feature when the noise is ulse; the roerties of both features also deend on the frequency range of the ower sectrum used for their comutation. For noisy seech it is not recommended (for both features) to use the sectral bins above 4 khz, regardless of the fact that the MD feature is less affected by this art of sectrum; the EE feature is an integrated entroy-energy feature. The additional analysis revealed that the direct using of the seech energy as it is done in [6] is not a reliable way to create an integrated feature, esecially in the cases when the seech is corruted with ulse noise and the SNR is very low; Our further work will include the develoment of the MD feature-based seech detection algorithm. We will evaluate this algorithm in the context of seaker recognition system, in order to estimate the efficiency of this new feature as a comonent of a comlete system. R e f e r e n c e s 1. B e n d I k s e n, A., K. S t e i g l i t z. Neural networks for voiced/unvoiced seech classification. ICASSP, 1990, G a n a a t h i r a j u, A., L. W e b s t e r, J. T r i m b l e, K. B u s h, P. K o r n m a n. Comarison of energy-based endoint detectors for seech signal rocessing. In: Proceedings of the IEEE Southeastcon, Tama, Florida, USA, 1996, H o n g, K o o k K i m, H w a n g S o o L e e. Use of sectral autocorrelation in sectral enveloe linear rediction for seech recognition. In: IEEE Transactions on SAP, Vol.7, 1999, No 5,

12 4. L a h a t, M., R. J. N ie d e r j o h n, D. A. K r u b s a c k. A sectral autocorrelation method for measurement of the fundamental frequency of noise corruted seech. IEEE Trans. Acoust., Seech, Signal Processing, Vol.ASSP-35, 1987, J u n q u a, J.-C., B. M a k, B. R e a v e s. A robust algorithm for word boundary detection in the resence of noise. IEEE Transactions on SAP, vol.2, July 1994, No 3, L i a n, J. Z h e n g H u a n g, C h u n g - h o Y a n g. A novel aroach to robust seech endoint detection in car environment. ICASSP 2000, L i, Q., J. Z h e n g, A. T s a i, Q. Z h o u. Robust endoint detection and energy normalization for real-time seech and seaker recognition. IEEE Transaction on SAP, vol.10, March 2002, No 3, The SEAR Database. htt://cslu.ece.ogi.edu/nsel/data/index.html 9. O u z o u n o v, A. BG-SRDat: A corus in bulgarian language for seaker recognition over telehone channels. Cybernetics and Information Technologies, Vol. 3, 2003, No 2, R a m ir e z, J, J. C. S e g u r a, C. B e ni t e z, A. de la T o r r e, A. R u b io. Efficient voice activity detection algorithms using long-term seech information. Seech Communication, Vol. 42, Aril 2004, No 3-4, S h in, W., B. L e e, Y. L e e, J. L e e. Seech/non-seech classification using multile features for robust endoint detection. ICASSP 2000, S o o n g, F., A. E. R o s e n b e r g. On the use of instantaneous and transitional sectral information in seaker recognition. In: IEEE Transactions on Acoustics, Seech and Signal Processing, Vol. 36, 1988, No 6, Z h u, J., F. C h e n. The analysis and alication of a new endoint detection method based on distance of autocorrelated similarity. In: Euroseech 99, R e n e v e y, P., A. D r y g a j l o. Entroy based voice activity detection in very noisy conditions. In: Euroseech 01, Един параметър за детекция на говор Атанас Узунов Институт по информационни технологии, 1113 София atanas@iinf.bas.bg (Р е з ю м е) В работата е предложен нов параметър, предназначен за използване в алгоритмите за откриване (детекция) на говор. Този параметър е средната стойност на модула на делта спектралната автокорелационна функция, получена от спектъра на мощността на анализирания сигнал и е наречен Mean-Delta (MD) параметър. В работата експериментално е сравнено изменението на траекторията на предложения параметър с тази на Entroy-Energy (EE) параметърa, основаващ се на спектралната ентропия. Реализирани са експерименти със зашумен говорен материал от две бази данни SEAR и BG-SRDat. Получените резултати показват, че MD параметърът се влияе в значително по-малка степен от вида и нивото на шума в сравнение с ЕЕ параметъра. 1 4