FEATURE EXTRACTION FOR SPEECH RECOGNITON

Transcription

1 M.Tech. Credit Seminar Reort, Electronic Systems Grou, EE. Det, IIT Bombay, Submitted November2003 Abstract FEATURE EXTRACTION FOR SPEECH RECOGNITON Manish P. Kesarkar (Roll No: ) Suervisor: Prof. Preeti Rao Automatic seech recognition (ASR) has made great strides with the develoment of digital signal rocessing hardware and software. But desite of all these advances, machines can not match the erformance of their human counterarts in terms of accuracy and seed, secially in case of seaker indeendent seech recognition. So today significant ortion of seech recognition research is focussed on seaker indeendent seech recognition roblem. The reasons are its wide range of alications, and limitations of available techniques of seech recognition. In this reort we briefly discuss the signal modeling aroach for seech recognition. It is followed by overview of basic oerations involved in signal modeling. Further commonly used temoral and sectral analysis techniques of feature extraction are discussed in detail. 1. Introduction Seech recognition system erforms two fundamental oerations: signal modeling and attern matching [1]. Signal modeling reresents rocess of converting seech signal into a set of arameters. Pattern matching is the task of finding arameter set from memory which closely matches the arameter set obtained from the inut seech signal. 1.1 Motivation for signal modeling [1] 1. To obtain the ercetually meaningful arameters i.e. arameters which are analogous to those used by human auditory system. 2. To obtain the invariant arameters i.e. arameters which are robust to variations in channel, seaker and transducer. 3. To obtain arameters that cature sectral dynamics, or changes of sectrum with time. The signal modeling involves four basic oerations: sectral shaing, feature extraction, arametric transformation, and statistical modeling [1]. Sectral shaing is the rocess of converting the seech signal from sound ressure wave to a digital signal; and emhasizing imortant frequency comonents in the signal. Feature extraction is rocess of obtaining different features such as ower, itch, and vocal tract configuration from the seech signal. Parameter transformation is the rocess of converting these features into signal arameters through rocess of differentiation and concatenation. Statistical modeling involves conversion of arameters in signal observation vectors. In this reort we focus on analysis techniques used for feature extraction. Section 2 briefly discusses basic oerations involved in sectral shaing. Section 3.1 discusses sectral analysis techniques of feature extraction in detail. The temoral analysis techniques for feature extraction are discussed in section 3.2. Section 4 summarizes the reort and conclusions are drawn. 2 Sectral Shaing From Microhone Analog Anti-Aliasing Filter Analog To Digital Digital Filter (reemhasis) to Analysis Fig. 1.Basic oerations in sectral shaing "from[1]". 1

2 Sectral shaing [1] involves two basic oerations: digitisation i.e.conversion of analog seech siganl from sound ressure wave to digital signal; and digital filtering i.e.emhasizing imortant frequency comonents in the signal. This rocess is shown in Fig.1. The main urose of digitisation rocess is to roduce a samled data reresentation of seech signal with as high signal-to-noise ratio (SNR) as ossible. Once signal conversion is comlete, the last ste of digital ost filtering is most often executed using a Finite Imulse Resonse (FIR) filter given as N re H re (z) = a re (k)z -k (1) k=0 Normally, a one coefficient digital filter known as re-emhasis filter, is used H re (z) = 1 + a re z -1 (2) A tyical range of values for a re is [-1.0,-0.4]. The reemhasis filter boosts the signal sectrum aroximately 20 db er decade. Advantages of reemhasis filter 1. The voiced sections of seech signal naturally have a negative sectral sloe (attenuation of aroximately 20 db er decade due to hysiology of seech roduction system [3]). The reemhasis filter serves to offset this natural sloe before sectral analysis, thereby imroving the efficiency of the analysis [2]. 2. The hearing is more sensative above the 1-kHz region of the sectrum. The reemhasis filter amlifies this area of the sectrum. This assists the sectral analysis algorithm in modelling the ercetually imortant asects of seech sectrum [1]. 3 Feature Extraction In seaker indeendent seech recogniton, a remium is laced on extracting features that are somewhat invariant to changes in the seaker. So feture extraction involves analysis of seech siganl. Broadly the feature extraction techniques are classified as temoral analysis and sectral analysis technique. In temoral analysis the seech waveform itself is used for analysis. In sectral analysis sectral reresentation of seech signal is used for analysis. 3.1 Sectral Analysis techniques Critical Band Filter Bank Analysis It is one of the most fundamental concets in seech rocessing. It can be regarded as crude model of the initial stages of transduction in human auditory system. Motivation for filter bank reresentation [1] 1. According to "lace theory" the osition of maximum dislacement along the basilar membrane for stimuli such as ure tones is roortional to the logarithm of the frequency of the tone [8]. 2. The exeriments in human ercetion have shown that frequencies of a comlex sound within a certain bandwidth of some nominal frequency cannot be individually identified unless one of the comonents of this sound falls outside the bandwidth. This bandwidth is known as critical bandwidth [3], [8]. Combination of these two theories gave rise to the critical band filter bank analysis technique. Critical bank filter bank is simly bank of linear hase FIR bandass filters that are arranged linearly along the Bark (or mel) scale. The bandwidths are chosen to be equal to a critical bandwidth for corresonding center frequency. Bark i.e. critical band rate scale and mel scale are ercetual frequency scale defined as [3] Bark = 13atan(0.76f /1000) + 3.5atan( f 2 /(7500) 2 ) (3) mel frequency = 2595log 10 (1 + f /700) (4) 2

3 An exression for critical bandwidth is [1] BW critical = [ (f /1000) 2 ] 0.69 (5) Table 1 shows the critical filter banks based on Bark scale and mel scale. Each filter in digital filter bank is usually imlemented as a linear hase filter so that the grou delay for all filters is equal and the outut signal from the filters are synchronized in time. The filter equations for linear hase filter imlementation can be summarized as follows [1] (N i -1)/2 S i (n) = i( j ) s(n + j) (6) j=(n i -1)/2 Where i(j) denotes j th coefficient for i th critical band filter. The outut of this analysis is a vector of ower values for each frame of data. These are usually combined with other arameters, such as total ower, to form a signal measurement vector. The Filter bank attemts to decomose the signal into discrete set of sectral samles that contain information similar to what is resented to higher levels of rocessing in auditory system. Because the analysis technique is largely based on linear rocessing, it is generally robust to ambient noise [1]. Table 1. Two Critical Filter banks, source [1]. Bark Scale Mel Scale Index Center Freq. (Hz) BW (Hz) Center Freq. (Hz) BW (Hz)

4 3.1.2 Cestral Analysis This analysis technique is very useful as it rovides methodology for searating the excitation from the vocal tract shae [2]. In the linear acoustic model of seech roduction, the comosite seech sectrum, consist of excitation signal filtered by a time-varying linear filter reresenting the vocal tract shae as shown in fig.2 Excitation (sub-glottal sys) g(n) Vocal Tract Filter V(n) Seech s(n) Fig.2. Linear acoustic model of seech roduction "from [1]". The seech signal is given as s(n) = g(n)* v(n) (7) where v(n): vocal tract imulse resonse g(n): excitation signal The frequency domain reresentation S(f) = G(f). V(f) (8) Taking log on both sides log(s(f)) = log(g(f)) + log(v(f)) (9) Hence in log domain the excitation and the vocal tract shae are suerimosed, and can be searated. Cestrum is comuted by taking inverse discrete Fourier transform (IDFT) of logarithm of magnitude of discrete Fourier transform finite length inut signal as shown in fig.3. s(n) DFT S(k) LOG (k) IDFT (n) MAGNITUDE Fig. 3. System for obtaining cestrum "adated from[2]". N-1 S(k) = s(n)ex(-j2 /N)nk (10) n=0 (k) = log (S(K)) (11) N-1 (n) = (1/N) (k) ex(j2 /N)nk (12) k=0 (n) is defined as cestrum. In seech recognition cestral analysis is used for formant tracking and itch (f 0 ) detection. The samles of (n) in its first 3ms describe v(n) and can be searated from the excitation. The later is viewed as voiced if (n) exhibits shar eriodic ulses. Then the interval between these ulses is considered as itch eriod. If no such structure is visible in (n), the seech is considered unvoiced Mel Cestrum Analysis This analysis technique uses cestrum with a nonlinear frequency axis following mel scale [3]. For obtaining mel cestrum the seech waveform s(n) is first windowed with analysis window w(n) and then its DFT S(k) is comuted. The magnitude of S(k) is then weighted by a series of mel filter frequency resonses whose center frequencies and bandwidth roughly match those of auditory critical band filters. 4

5 The next ste in determining the mel cestrum is to comute the energy in this weighted sequence. If V l (k) is the frequency resonse of l th mel scale filter. The resulting energies are given for each seech frame at a time n and for the l th mel scale filter are U l E mel (n,l) =(1/A l ) V l (k) S(k) (13) k=l l Where U l and L l are uer and lower frequency indices over which each filter is nonzero and A l is the energy of filter which normalizes the filter according to their varying bandwidths so as to give equal energy for flat sectrum. The real cestrum associated with E mel (n,l) is referred as the mel-cestrum and is comuted for the seech frame at time n as N-1 C mel (n,m) = (1/N) log{e mel (n,l)}cos[2 (l +1/2)/N] (14) l=0 Such mel cestral coefficients C mel rovide alternative reresentation for seech sectra which exloits auditory rinciles as well as decorrelating roerty of cestrum Linear Predictive Coding (LPC) Analysis The basic idea behind the linear redictive coding (LPC) analysis [2] is that a seech samle can be aroximated as linear combination of ast seech samles. By minimizing the sum of the squared differences (over a finite interval) between the actual seech samles and the linearly redicted ones, a unique set of redictor coefficients is determined. Seech is modeled as the outut of linear, time-varying system excited by either quasi-eriodic ulses (during voiced seech), or random noise (during unvoiced seech). The linear rediction method rovides a robust, reliable, and accurate method for estimating the arameters that characterize the linear time-varying system reresenting vocal tract. Most recognition systems assume all ole model known as auto regressive (AR) model for seech roduction. The difference equation describing relation between seech samles s(n) and excitation u(n) for AR model is as follows s(n) = a k s(n-k) + G u(n) (15) The system function is of the form H(Z) = S(z) = G (16) U(z) 1 - a k z -k A linear redictor of order with rediction coefficients k is defined as a system whose outut is [2] (n) = k s(n-k) (17) The system function is th order olynomial P(z) = k z -k (18) The rediction error e(n) is defined as e(n) = s(n) - (n) = s(n) - k s(n-k) (19) The rediction error sequence is the outut of the system whose transfer function is 2 5

6 A(z) = 1 - k z -k (20) it can be seen by comaring eq.15 and eq.19 if k = a k,, then rediction error filter, A(z),Will be an inverse filter for the system, H(z) of eq.16 H(Z) = G (21) A(z) The basic aroach is to find set of redictor coefficients that will minimize the mean squared error over a short segment of seech waveform. The resulting arameters are then assumed to be the arameters of the system function, H(z), in the model for seech roduction. The short-time average rediction error is defined as[2] E n = (e n (m)) 2 (22) m = {s n (m)- ks n (m-k)} (23) m Where s n (m)is segment of seech in vicinity of samle n, i.e. s n (m) = s(n+ m) (24) The values of k that minimize E n are obtained by setting E n / i = 0, i = 1,2,, thereby obtaining the equations s n (m-i)s n (m) = k s n (m-i) s n (m-k) (25) m if n(i,k) = s n (m-i) s n (m-k) (26) m Then Eq. 25 is written as k n(i,k) = n(i,0) i=1,2,3. (27) There are three basic ways to solve above set of equations 1. Lattice method 2. Covariance method 3. Autocorrelation method In seech recognition the autocorrelation method is almost exclusively used because of its comutational efficiency and inherent stability. The autocorrelation method always roduces a rediction filter whose zero lies inside the circle in z-lane [2]. In autocorrelation method the seech segment is windowed as follows s n (m) = s(m+n)w(m) (28) where w(m) is finite length window i.e.zero outside interval 0 m N-1 N+-1 Then n(i,k) = s n (m-i) s n (m-k) 1 i m=0 0 k (29) n(i,k) = R n (i-k) N-1-k where R n (k) = s n (m) s n (m+k) (30) m=0 6

7 R n (k) is autocorrelation function eq. 27 is simlified as [2] k R n ( i-k ) = R n (i) 1 i (31) This results in matrix of autocorrelation values, it is toelitz matrix; i.e. symmetric and all elements along diagonal are equal. The resulting equations are solved using Durbins recursive rocedure as [2] E (0) = R(0) (32) i-1 k i ={ R(i) - j (i-1) R(i-j) }/E (i-1) (33) j =1 i (i) = k i (34) j (i) = j (i-1) - k i (j-1) (i-1) (35) E (i) = (1- k 2 i ) E (i-1) (36) Eq. 31 to 34 are solved recursively for i =1,2 and final solution is given as j = LPC coefficients = j (P) k i = PACOR coefficients For voiced regions of seech all ole model of LPC rovides a good aroximation to the vocal tract sectral enveloe. During unvoiced and nasalized regions of seech the LPC model is less effective than voiced region. The comutation involved in LPC rocessing is considerably less than cestrum analysis. Thus the imortance of method lies in ability to rovide accurate estimates of seech arameters, and in its relative seed. A very imortant LPC arameter set which is derived directly from LPC coefficients is LPC cestral coefficients c m. The recursion used for this is [6] c 0 = lng (37) m-1 c m = m + (k/m)c k a m-k 1 m (38) m-1 c m = (k/m)c k a m-k m > (39) Where G is the gain term in LPC model. This method is efficient, as it does not require exlicit cestral comutation. Hence combines decorrelating roerty of cestrum with comutational efficiency of LPC analysis Percetually Based Linear Predictive Analysis (PLP) PLP analysis [5] models ercetually motivated auditory sectrum by a low order all ole function, using the autocorrelation LP technique. Basic concet of PLP method is shown in block diagram of Fig. 4. 7

8 SPEECH CRITICAL BAND ANALYSIS DATA INTERPOLATION EQUAL LOUDNESS PRE-EMPHASIS INVERSE DFT INTENSITY LOUDNESS CONVERSION SOLUTION FOR AUTOREGRESSIVE COEFFICIENTS ALL POLE MODEL Fig 4 Block diagram of PLP seech analysis method "from [5]". It involves two major stes: obtaining auditory sectrum, aroximating the auditory sectrum by an all ole model. Auditory sectrum is derived From the seech waveform by critical-band filtering, equal loudness curve re-emhasis, and intensity loudness root comression. Eighteen critical band filter oututs with their center frequencies equally saced in bark domain, are defined as (w) = 6 ln ((w/ 1200 ) + ((w/ 1200 ) + 1) 0.5 ) (40) Center frequency of kth critical band k = 0.994k They cover the frequency range 0 f 5kHz ( Bark). Each filter is simulated by sectral weighing c k (w).the filter outut is multilied by equal loudness function E(w k ) and converted into loudness domain using Stevens ower law with exonent r = 3 as in[5] Q k = [E(w k ) c k (w).p(w) dw] 1/ r (41) 0 Critical band weighing function is [5] c k (w) = 10 ( - k + 0.5) k -0.5 = 1 k k = ( - k + 0.5) k (42) The outut thus obtained is linearly interolated to give interolated auditory sectrum. The interolated auditory sectrum is aroximated by fifth order all ole model sectrum. The IDFT of interolated auditory sectrum rovides first six terms of autocorrelation function. These are used in solution of Yule Walker equations [6] to obtain five autoregressive coefficients of all-ole filter. The PLP analysis rovides similar results as with LPC analysis but the order of PLP model is half of LP model. This allows comutational and storage saving for ASR. Also it rovides better erformance to cross seaker ASR [7]. 8

9 3.2Temoral Analysis It involves rocessing of the waveform of seech signal directly. It involves less comutation comared to sectral analysis but limited to simle seech arameters, e.g. ower and eriodicity Power Estimation The use of some sort of ower measures in seech recognition is fairly standard today. Power is rather simle to comute. It is comuted on frame by frame basis as [1] N s -1 P(n) = (1/N s ) (w(m) s(n - N s /2 + m ) ) (43) m=0 Where N s is the number of samles used to comute the ower, s(n) denotes the signal, w(m) denotes the window function, and the,and n denotes the samle index of center of the window. In most seech recognition system Hamming window is almost exclusively used. The Hamming window is secific case of Hanning window, which is given as w(n) = w- (1 - w) cos(2 n /(N s -1)) w For 0 n N s -1 w(n) = 0 elsewhere. (44) w is window constant in the range[0,1],and Ns is the window duration in samles. For Hamming window w=0.54, w is normalization constant. In ractice normalization is done so ower in signal after windowing is aroximately equal to the ower of signal before windowing. The urose of window is to weight, samles towards the center of the window this characteristic couled with overlaing analysis erforms imortant function of obtaining smoothly varying arametric estimates. The Window duration controls amount of averaging or smoothing in ower calculation. Large amount of over la results in reduction of amount of noise introduced in measurements by artifacts such as window lacement and nonstationary channel noise [2]. However excessive smoothing can obscure true variation in the signal. Rather than using ower directly in seech recognition systems use the logarithm of ower multilied by 10, defined as the ower in decibels, in an effort to emulate logarithmic resonse of human auditory system [3]. It is calculated as Power in db = 10 log 10 (P(n)) (45) The major significance of P(n) is that it rovides basis for distinguishing voiced seech segments from unvoiced seech segments. The values of P(n) for the unvoiced segments are significantly smaller than that for voiced segments. The ower can be used to locate aroximately the time at which voiced seech becomes unvoiced and vice versa Fundamental Frequency Estimation Fundamental Frequency (f 0 ) or itch is defined as the frequency at which the vocal cords vibrate during a voiced sound. Fundamental frequency has long been difficult arameter to reliably estimate from the seech signal. Previously it was neglected for number of reasons, including large comutational burden required for accurate estimation, the concern that unreliable estimation would be a barrier to achieving high erformance, and difficulty in characterizing comlex interactions between f 0 and surasegmental henomenon [3]. It is useful in seech recognition of tonal languages (e.g. Chinese) and languages that have some tonal comonents (e.g. Jaanese). Fundamental frequency is often rocessed on logarithmic scale, rather than a linear scale to match the resolution of human auditory system. There are various algorithms to estimate f 0 we will consider two widely used algorithms: Gold and Rabiner algorithm [4], cestrum based itch determination algorithm [2]. 9

10 Gold and Rabiner algorithm It is one of earliest and simlest algorithm for f 0 estimation. In this algorithm [4] the seech signal is rocessed so as to create a number of imulse trains which retain the eriodicity of the original signal and discard features which are irrelevant to the itch detection rocess. This enables use of very simle itch detectors to estimate the eriod of each imulse train. The estimates of several of these itch detectors are logically combined to infer the eriod of the seech waveform. The algorithm can be efficiently imlemented either in secial urose hardware or on general-urose comuter Cestrum based itch determination In the cestrum [2], we observe that for the voiced seech there is a eak in the cestrum at the fundamental eriod of the inut seech segment. No such a eak aears in the cesrtum for unvioced seech segment. If the cestrum eak is above the reset threshold, the inut seech is likely to be voiced, and osition of eak is good estimate of itch eriod. Its inverse rovides f 0. If the eak does not exceed the threshold, it is likely that the inut seech segment is unvoiced. 4 Conclusions The basic oerations in seech recognition system have been discussed briefly. Different temoral and sectral analysis techniques for feature extraction have been studied in detail and following conclusions are drawn 1. Temoral analysis techniques involve less comutation, ease of imlementation. But they are limited to determination simle seech arameters like ower, energy and eriodicity of seech. For finding vocal tract arameters we require sectral analysis techniques. 2. Critical band filter bank decomoses the seech signal into discrete set of sectral samles containing information, which is similar to information, resented to higher levels rocessing in auditory system. 3. Cestral analysis searates the seech signal into comonent reresenting excitation source and a comonent reresenting vocal tract imulse resonse. So it rovides information about itch and vocal tract configuration. But it is comutationally more intensive. 4. Mel cestral analysis has decorrelating roerty of cestral analysis and also includes some asects of audition. 5. LPC analysis rovides comact reresentation of vocal tract configuration by relatively simle comutation comared to cestral analysis. To minimize analysis comlexity it assumes all ole model for seech roduction system. But seech has zeros due to nasals so in these cases the result are not as good as in case of vowels but still reasonably accetable if order of model is sufficiently high. 6. LP derived cestral coefficients have decorrelating roerty of cestrum and comutational ease of LPC analysis. 7. PLP analysis uses includes certain asects of audition rovides similar sectral estimate of seech as LPC analysis but with lower order model. Also it rovides better erformance to cross seaker ASR [9]. Acknowlegdement I wish to exress my sincere gratitude to Prof. Preeti Rao for her constant guidance throughout the course of the work and many useful discussions which enabled me to know the subtleties of the subject in roer way. References [1] J. W. Picone, "Signal modelling technique in seech recognition," Proc. Of the IEEE, vol. 81, no.9, , Se [2] L. R. Rabiner and R. W. Schafer, Digital Processing of Seech Signals. Englewood Cliffs, New Jersey: Prentice-Hall, [3] D.O. Shaughnessy, Seech Communication: Human and Machine. India:University Press,2001. [4] B. Gold and L. R. Rabiner,"Parallel rocessing techniques for estimating itch eriods of seech in the time domain," J. Acoust. Soc. America, vol.46, t. 2, no. 2, , Aug

11 [5] H. Hermansky, B. A. Hanson, and H. Wakita, "Percetually based linear redictive analysis of seech," Proc. IEEE Int. Conf. on Acoustic, seech, and Signal Processing," , Aug [6] L. R. Rabiner and B. H. Juang, Fundamentals of Seech Recognition, Englewood Cliffs, New Jersey: Prentice- Hall, [7] H. Hermansky, B. A. Hanson, and H. Wakita, "Percetually based rocessing in automatic seech recognition," Proc. IEEE Int. Conf. on Acoustic, seech, and Signal Processing," , Ar [8] L. Roderer, The Physics and Psychohysics of Music: An Introduction, New York, Sringer Verlag,

12 This document was created with Win2PDF available at htt:// The unregistered version of Win2PDF is for evaluation or non-commercial use only.