Optimization of some parameters in the speech-processing module developed for the speaker independent ASR system

Transcription

1 Optmzaton of some parameters n the speeh-proessng module developed for the speaker ndependent ASR system J. V. PSUTKA and Ludek MÜLLER Department of Cybernets, Unversty of West Bohema n Plsen, Czeh Republ {psutka_j, muller@kky.zu.z} ABSTRACT Ths paper deals wth lookng for an optmum parameterzaton n automat speeh reognton systems workng wth the speeh transferred over a telephone hannel. The performed experments were supported by a large olleton of tranng data provded from telephone alls of at least one thousand speakers. MFCC and PLP epstral parameterzatons were tested wth the am to fnd the optmal number of flters and oeffents. Temporal patterns desrbng several adjaent frames of a gven frame were verfed n onneton wth tehnques ensurng feature extraton and deorelaton of pattern spae. Key words: speeh parameterzaton, optmzaton of frontend, lnear dsrmnate analyss, pattern spae normalzaton. INTRODUCTION The robust parameterzaton of speeh used n automat speeh reognton (ASR) systems s stll a great objetve of many researh teams. In tasks solved n many laboratores the software toolkt HTK [] s often used and we an see that for proessng of speeh of varous qualty unsutable default settngs of the HTK s front-end are frequently appled. Suh settngs do not sometmes agree wth the theory of human hearng, whh s norporated n the two most frequent audtory-based parameterzaton tehnques MFCC and PLP. Smultaneously we an observe under-dmensoned and/or overdmensoned front-ends, whh respet nether the applaton tasks nor the real workng ondtons as tme and memory demand. The present paper addresses ths problem and ams to ontrbute to the dsusson onernng the seleton of an optmum number of flters dstrbuted n the frequeny axs as well as the number of enumerated oeffents. In ontrast wth [2], [3], all experments usng MFCC and PLP parameterzatons were performed wth ontnuous speeh of telephone qualty employng voes of a large group of one thousand speakers for buldng HMMs. The seond am of ths paper s to ompare several tehnques of feature extraton and pattern spae deorelaton. In the performed experments we substtuted a onventonal feature vetor desrbng speeh n one frame by longer temporal vetor omposed of vetors of features of several adjaent frames. For feature extraton and pattern spae deorelaton we tested suh tehnques as the lnear dsrmnant analyss (LDA), prnpal omponent analyss appled on the between-lass satter matrx, normalzaton of pattern spae, and osne transform. Owng to the extremely tme-onsumng omputaton burden, all tests n ths ase were arred out usng tranng orpus of only one hundred speakers. The qualty of settngs of all front-ends was evaluated by the auray (A) defned as A = (N D S I) / N 00%, where N s the total number of words n the referene transrpton, S s the number of substtuton errors, D s the number of deleton and I the number of nserton errors. All experments deal wth telephone-based speaker ndependent ontnuous speeh reognton. Speeh data was taken from the Czeh telephone orpus. Ths orpus onssts of read speeh transmtted over a telephone hannel. More than one thousand speakers were asked to read varous sets of 40 sentenes. The dgtzaton of an nput analog telephone sgnal was provded by a telephone nterfae board DIALOGIC D/2D at 8 khz sample rate and onverted to the mu-law 8-bt resoluton. Speeh data was annotated usng speal annotaton software Transrber.4., whh s freely avalable from the web ste The orpus was then randomly dvded so that reordngs of one thousand speakers reated the tranng part and the remanng part (one hundred of dfferent speakers) formed the test part of the orpus. From ths tranng part 00 sentenes were randomly seleted to reate test data for all our experments. The lexon of the task ontaned 475 dfferent words. In all experments a language model based on a zero-gram was appled. 2. AUDITORY-BASED FRONT-END The MFCC and PLP-based front-ends attempt to model the audtory proessng up to atvaton of the nner har ells by the baslar membrane vbratons. Ths smulaton s usually performed through a seletve model whh s mplemented by a flter bank whose enter frequenes are spaed along the frequeny axs whh satsfes the rtal-band sale and whose partular flter wdths orrespond to the theory of rtal bandwdths [4]. The most ommon rtal-band sales are the mel-sale and the bark-sale n whh the flters are dstrbuted along the frequeny axs approxmately lnear up to about 000 Hz and logarthm above 000 Hz. The number of rtal bands depends on the whole frequeny bandwdth and/or the samplng frequeny F s. For the telephone frequeny band wth F s =8 khz approxmately from 5 to 7 rtal bands are reommended. 2. Experments wth MFCC parameterzaton The omputatonal algorthm of the MFCC parameterzaton s realzed by the bank of symmetr overlappng trangular flters spaed lnearly n a mel-frequeny axs, aordng to audtory pereptual onsderatons. The spang as well as the SYSTEMICS, CYBERNETICS AND INFORMATICS VOLUME 2 - NUMBER 3 25

2 bandwdth of the partular flters s determned by a onstant mel-frequeny nterval. In our ase the spang was approxmately 45 mels and the wdth of the trangle was 290 mels. So, for telephone frequeny band (0 246 mels) wth the samplng frequeny F s =8 khz we obtaned (usng rtal-band onept) about 5 flters dstrbuted up to the Nyqust frequeny. The MFCC parameterzaton was aomplshed by the omputaton of mel-epstral oeffents (),..., (M). In prate these oeffents are usually obtaned by applyng an nverse DCT (Dsrete Cosne Transform) to the log-energy of the flter bank outputs n order to deorelate the parameter (pattern) spae. The set of epstral oeffents s usually omplemented by the oeffent (0), whh approxmates the average log-energy of the sgnal (ths oeffent s often replaed by the energy omputed dretly from the sgnal). In ths set of experments our front-end s, n addton, omplemented by the omputaton of tme dervatves (delta plus delta-delta) of the orrespondng stat features. Ths means that our front-end provdes feature vetors wth the dmenson of 3M. In ths ase the experments were realzed for an nreasng number of flters from 4 to 2 (n the step 2) and for an nreasng number of oeffents from 4 3=2 to 8 3=54. The results of all experments are gven n Table. Sne the greatest auray (A) osllates between and %, we ndated n Table, for a learer survey better survey, by gray olor the tems wth reognton auray hgher than %. oef flters Table. Reognton results for varous numbers of flters and parameters n MFCC parameterzaton. traned on 000 voes traned on 00 voes number of flters Fgure. Dependene of the on the number of flters used n the MFCC parameterzaton. The dependene of the greatest auray for a gven number of flters s depted n Fgure. To ompare results of these experments (based on tranng orpus of 000 dfferent speakers) wth those obtaned usng voes of only 00 speakers (these experments were reported n [3]) we omplemented Fgure by a urve whh expresses orrespondng dependene (see Fgure ). The dependene of the greatest results of auray for a gven number of oeffents s depted smultaneously wth the results got for 00 speakers [3] n Fgure 2. traned on 000 voes traned on 00 voes number of oef. Fgure 2. Dependene of the on the number of oeffents used n the MFCC parameterzaton. 2.2 Experments wth PLP parameterzaton The front-end used n ths ase was based on the PLP parameterzaton desrbed n [4]. For the transformaton of a power speeh spetrum to a orrespondng audtory spetrum the PLP ombnes three omponents from psyhophyss of hearng: the rtal-band spetral seletvty, the equal-loudness urve and the ntensty-loudness power law. To arry out ths proess we have to perform followng steps: Computaton of short-term speeh spetrum Nonlnear frequeny transformaton and rtal-band spetral resoluton. The modelng of these phenomena s performed n the PLP ether by the nonlnear transformaton of frequenes from the Herz nto the Bark sale and by the onstruton of maskng urves that smulate rtal-band of hearng and are modeled by the band-pass flters. The enters of flters are spaed n the Bark doman lnearly wth the step approxmately Bark. As the speeh sgnal overs the range from 0 to 4 khz the orrespondng range n the Bark sale was Bark and we used M=7 flters spaed lnearly wth the step of Bark. The 0 th flter had a enter n the value of 0 Bark, the last (M-) st flter was entered n the value of 5.57 Bark. Crtal-bands adjustment to the urves of equal-loudness Weghted spetral summaton of power spetrum samples Enforng the ntensty-loudness power-law All-pole spetrum approxmaton Transformaton of the PLP-oeffents to the PLP-epstral representaton. The PLP-epstral oeffents (),......, (Q) are omputed by the standard approah from the Q PLP predtve oeffents. For the fnal aoust modelng we extended the orgnal PLP-epstral representaton wth derved delta and delta-delta features. In fat, the dmenson of the pattern spae n whh the aoust models of trphones were bult, nreased from Q to 3 Q. 26 SYSTEMICS, CYBERNETICS AND INFORMATICS VOLUME 2 - NUMBER 3

3 A number of experments was also performed for ths parameterzaton n whh the bank of flters was nreased from 4 to 2 (n the step 2) and the number of oeffents was enumerated from 4 3=2 to 8 3=54. The results aheved n reognton experments are gven n Table 2. oef flters Table 2. Reognton results for varous numbers of flters and parameters n PLP epstral parameterzaton Dependene of the on the number of flters and on the number of oeffents are gven n Fgure 3 and 4. Smlarly to the MFCC parameterzaton (Fgure and 2), two urves were obtaned for the ASR system traned on 00 and 000 speakers respetvely. traned on 000 voes traned on 00 voes number of flters Fgure 3. Dependene of the greates A on the number of flters used n the PLP parameterzaton. 2.3 Comparson of both parameterzaton tehnques In all experments the HMMs based on trphones were used. The number of Gaussans and states for ndvdual types of parameterzatons moves from 30k Gaussans and 3k6 states for lower number of flters and parameters up to 50k Gaussans and 6k2 states for hgher number of flters and parameters (smlarly for both MFCC and PLP), see Table and 2. Lookng at Tables and 2 we an see that there are areas of oeffents and flters wth hgh and relatvely stable reognton auray. For the MFCC and PLP parameterzatons the (A hgher than %) was aheved for 3 9 flters and oeffents (MFCC) and for 9 flters and oeffents (PLP) respetvely. Ths s a slghtly hgher number of flters, espeally for the MFCC parameterzaton, than was reported n [3] for tasks wth tranng set of 00 people. It s evdent that these optmal settngs are approahng the reommended number of flters and satsfy the theory of rtal bandwdths to a greater extent. However, the area of sutable settngs for the PLP s muh larger and the reognton results aheved are hgher by 0.5 to % on average n omparson wth the MFCC. Also the number of oeffents for the PLP does not have to be so hgh, whh ould brng omputaton savngs (useful for buldng real tme ASR systems). traned on 000 voes traned on 00 voes number of oef. Fgure 4. Dependene of the on the number of oeffents used n the PLP parameterzaton. The results depted n Fgures 4 brng expeted yet nterestng nformaton. Front-ends bult on HMMs traned wth voes of one thousand people run better by 2% on average (gven by A) than those traned on the group of one hundred people. Analyzng the results of the reognton experments, we an menton that ths mprovement was aheved by better overng several test voes, whh were norretly mathed by the old models traned on the orpus bult from speeh of 00 people. We would also lke to address another nterestng problem whh deals wth modelng speeh by monophone- and/or trphone-based HMMs. Our reent experments wth monophone-based HMMs usng the set of 00 tranng voes showed, that there are no dfferenes n reognton auray between trphone-based HMMs wth 8 mxtures and monophone-based HMMs usng at least 50 or 60 mxtures. However, our new results, obtaned wth new trphone-based HMMs wth 8 mxtures traned on the orpus of 000 speakers (see Tables and 2), exeeded the monophone struture traned on the same orpus by 2%. Fgure 5 shows the dependeny of number of mxtures Fgure 5. Dependene of the A on the number of mxtures for monophone-based HMMs wth MFCC parameterzaton. the reognton auray on the number of mxtures n monophone-based HMMs workng wth the MFCC parameterzaton (5 3=45 parameters). Apart from ths slght degradaton of the A we have to note that these relatvely outstandng results were aheved wth the set of monophone- SYSTEMICS, CYBERNETICS AND INFORMATICS VOLUME 2 - NUMBER 3 27

4 based HMMs, whh were surprsngly desrbed only by 7k Gaussans and 35 states! Further mprovements of the A ould be obtaned by the epstral mean normalzaton, ampltude normalzaton, lnear dsrmnant analyss et. The frst two mentoned tehnques were tested n [3]. Our next seton s devoted to the several feature extraton methods n the ontext of so alled temporal patterns. 3. TEMPORAL PATTERNS AND FEATURE EXTRACTION In ths seton three seres of experments wth temporal patterns [5] are desrbed. A tehnque of temporal patterns substtutes eah onventonal feature vetor by longer temporal pattern onsstng of feature vetors of several adjaent frames. Our experments were performed usng parameters of a) gven frame (the onventonal ase) v(k) = x(k), b) two adjaent frames v(k) = [x(k-), x(k), x(k+)] T, ) four adjaent frames (two from eah sde) v(k) = [x(k-2), x(k- ), x(k), x(k+), x(k+2)] T. As features for bas desrpton of patterns, the log-energes of 5 output flters of the MFCC parameterzaton were enumerated (nether delta nor delta-delta features were used), see paragraph 2.. These sets of features were subjeted to further proessng whh ams both at extraton of smaller subsets of nformatve features and deorelaton of pattern spae. Owng to extremely tme-onsumng omputaton burdens all tests n ths ase were done usng tranng orpus of only one hundred speakers. In addton, to ensure the same ondtons for all tests we used n all experments monophonebased HMMs wth 8 mxtures (only k Gaussans and 26 states). The set of 00 test sentenes stayed the same as n the last experments. Durng feature extraton and pattern spae deorelaton experments we tested suh tehnques as a lnear dsrmnant analyss (LDA), prnpal omponent analyss appled to the between-lass satter matrx (PCA), normalzaton of pattern spae (NPS), and dsrete osne transform (DCT). Now we brefly explan ndvdual tehnques. The goal of all mentoned tehnques s to fnd a transformaton matrx W T, whh transforms the gven pattern spae to the spae of lower dmenson and/or to the spae wth deorelated features. In order to arry out dsrmnant analyss t s neessary to determne ndvdual phoneme lasses and use phonetally labeled speeh orpus. For these purposes the speeh orpus of all 00 tranng speakers was phonetally labeled usng 42 Czeh phone unts. 3. Lnear dsrmnant analyss (LDA) For the -lass problem the lnear dsrmnant analyss nvolves dsrmnant funtons. Thus, the projeton s from the orgnal n-dmensonal feature spae to a m=( )-dmensonal spae. What we seek now s a transformaton matrx W T, whh n some sense maxmzes the rato of the between-lass satter matrx to the wthn-lass satter matrx. In our ase the wthnlass satter matrx S W s defned as S W = = P S where P s the a pror probablty of the lass and S s the, ovarane matrx omputed from the feature vetors belongng to the phoneme lass. S an be expressed as S = E {( v - )( v - ) T }, where s the mean vetor of the lass. Between-lass satter matrx s defned as T ( μ -μ)( μ μ), S B= P - = where s the global mean vetor μ = = It s well known [6] that the rows of an optmal transformaton matrx W T are the generalzed egenvetors that orrespond to the largest egenvalues of the matrx (S W S B ). The nput vetor v of dmenson n from the orgnal pattern spae an be then transformed to the optmum spae of dmenson m= (there are only nonzero egenvalues) n aordane wth the equaton P y = W T v Let us note that f the dmenson n of the orgnal feature spae s lower than m=, then a dmenson of a new pattern spae stays usually the same after the transformaton (equal to n). 3.2 Dsrete Cosne Transform (DCT) Dsrete osne transform s used n order to deorelate features n the pattern spae. Ths s the standard method appled to the log-energes of output flters (LogEF) durng the MFCC parameterzaton, see paragraph 2.. DFT s defned as y j n π j = v os[ ( - 0.5)], n = μ. for j = 0,,..., n where v s -th oordnate of the nput vetor v and y j s j-th oordnate of the orrespondng output vetor y. Ths transformaton an be easly expressed n the matrx notaton. 3.3 Normalzaton of pattern spae (NPS) Normalzaton of pattern spae s usually appled n order to deorelate features n the spae. The transformaton matrx G T ensurng ths transformaton should satsfy the relaton G T S W G =, where s the dentty matrx and S W the wthn-lass satter matrx. The soluton of ths equaton s G T = /2 C T, where s the dagonal n by n matrx of egenvalues and C T s n by n matrx wth rows reated by the orthonormal egenvetors that orrespond to the egenvalues of the wthnlass satter matrx S W. Ths transformaton does not hange the dmensonalty of a pattern spae. 3.4 Prnpal Component Analyss (PCA) In ths ase the transform matrx was formed from m egenvetors orrespondng to the m largest egenvalues of the between-lass satter matrx S B. There are maxmum m= nonzero egenvalues. Ths means that the transformaton an be done to the spae wth the maxmum dmenson. 28 SYSTEMICS, CYBERNETICS AND INFORMATICS VOLUME 2 - NUMBER 3

5 In Table 3 you an fnd the results of related experments. To explan ondtons of ndvdual experments we proposed the followng notaton: (LogEF_n) depts log-energes of n output flters, (dag) means that fnal ovarane matres were dagonalzed oeffents out of the man dagonal were set to zero, (full) ndates that durng experments full ovarane matres were used, (LDA_n) means the lnear dsrmnant analyss wth output vetors of dmenson n, (NPS) spefes the normalzaton of the pattern spae, (PCA) means the prnpal omponent analyss appled to the between-lass satter matrx. Auray n=5 n=45 n=75 LogEF_n dag LogEF_n DCT dag LogEF_n full n m LogEF_n LDA_m dag LogEF_n LDA_m DCT dag LogEF_n NPS+PCA_m dag LogEF_n LDA_m NPS+PCA dag LogEF_n LDA_m full Table 3. Results of experments wth several feature extraton and deorelaton tehnques appled to temporal patterns. 6. REFERENCES [] Young, S. et al.: The HTK Book. User s Manual. -In: [2] Müller, L., Psutka, J.V.: Seleton of an Optmum Speeh Parameterzaton for Contnuous Speeh Reognton System Usng a Telephone Channel. In: Pro. of SCI 200, Orlando, U.S.A., 200, pp [3] Psutka, J., Müller, L., Psutka, J.V.: Comparson of MFCC and PLP Parameterzaton n the Speaker Independent Contnuous Speeh Reognton Task. In: Pro. of EUROSPEECH 200, Denmark, Aalborg, 200, pp [4] Hermansky, H.: Pereptual lnear predtve (PLP) analyss of speeh. In J. Aoust. So. Am. 87, (990), pp [5] Hermansky, H., Sarma, S.: TRAPS Classfers of Temporal Patterns. In: Pro. of ICSLP 98, Sydney, 998. [6] Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classfaton. John Wley & Sons, In., New York, 200. [7] Veth, J., Boves, L.: Channel normalzaton tehnques for automat speeh reognton over the telephone. In: Speeh Communaton, 25 (998), pp [8] Hermansky, H., Sharma, S., Ells, D., Jan, P., Kajarekar, S.: Feature extraton usng non-lnear transformaton for robust speeh reognton on the Aurora database. -In: Pro. ICASSP 2000, Turkey, For a gven number of 8 mxtures and a monophone-based HMMs struture, we an ompare ndvdual tehnques of the feature seleton and pattern spae deorelaton: deorelate tehnques based on the DCT gve dstntly worse results than those obtaned n other tehnques; ompare (LogEF_n DCT dag) versus (LogEF_n LDA_m dag) or (LogEF_n NPS+PCA_m dag) or (LogEF_n LDA_m NPS+PCA dag) LDA slghtly mproves reognton auray, see (LogEF_n LDA_m dag) versus (LogEF_n DCT dag) or (LogEF_n NPS+PCA_m dag) full ovarane matres dstntly exeed dagonal matres (for the same number of mxtures) temporal patterns used n our experments dd not brng better reognton results n omparson wth delta+deltadelta representaton (see results depted n Fgure 5). 4. CONCLUSION The results aheved n ths paper onfrmed, onlusons desrbed n [3], but on a substantally larger porton of speeh data. Both parameterzatons (MFCC and PLP) are omparable but the PLP one provdes slghtly better and robust (stable for a larger number of oeffents and flters) results. Feature extraton tehnques and pattern spae deorelate methods tested n our experments wll also have to be estmated n the future on the trphone-based HMMs or monophone-based HMMs wth a larger number of mxtures. 5. ACKNOWLEDGEMETS Ths work was funded by the Mnstry of Eduaton of the Czeh Republ, projet No. MSM and the Grant Ageny of the Czeh Republ, projet No. 02/02/024. SYSTEMICS, CYBERNETICS AND INFORMATICS VOLUME 2 - NUMBER 3 29