A Differentiable Approximation to Speech Intelligibility Index with Applications to Listening Enhancement

Transcription

1 AES 54 TH INTERNATIONAL CONFERENCE, London, UK, 24 June 2 4 A Dfferentable Approxmaton to Speech Intellgblty Index wth Applcatons to Lstenng Enhancement Rchard Stanton, Nkolay D. Gaubtch 2, Patrck Naylor, and Mke Brookes Imperal College London, London, Unted Kngdom 2 Delft Unversty of Technology, The Netherlands Correspondence should be addressed to Rchard Stanton rchard.stanton8@mperal.ac.uk ABSTRACT The Speech Intellgblty Index s a standardsed objectve measure for estmatng the ntellgblty of speech n nose. It s, however dffcult to use t n the teratve optmsaton of speech enhancement algorthms because t s a dscontnuous functon of ts nput parameters. In ths paper, we derve an approxmaton for the Speech Intellgblty Index that s both contnuous and dfferentable, whch allows for more effcent optmsaton procedures. The use of the approxmaton s demonstrated n an applcaton to near-end speech enhancement.. INTRODUCTION The Speech Intellgblty Index [] s a standardzed objectve measure whch s correlated wth the ntellgblty of speech under a varety of adverse lstenng condtons. It s an ntrusve measure whch s a functon of the long term average spectra of the speech and nose sgnals. It s based on the Artculaton Index defned by French and Stenberg [2] and on work from Kryter [3]. It can be used to predct the performance of speech enhancements systems that nvolve lnear processng [4]. Acoustc forenscs s frequently concerned wth speech at low Sgnal-to-Nose Ratos SNRs that are at the margn of ntellgblty. hen used n court, speech can only be dependable when t s above a threshold of ntellgblty [5, 6, 7]. Thus t would be useful to use the n a speech processng algorthm as a metrc to maxmse ntellgblty. Several speech processng technques, for example nearend lstenng enhancement, use optmsaton routnes that seek to maxmse the. Near-end lstenng enhancement focuses on the presentaton of far-end speech n near-end nose that cannot be controlled. The far-end speech s lnearly fltered n order to maxmse the ntellgblty, [8, 9,, 4,, 2]. In ts present state, the has no contnuous dervatve wth respect to the speech power spectrum. Because of ths, approxmatons are commonly made n order for the to be used n optmsaton. In the contrbuton from Taal et al. [3], a convex approxmaton to the was made, whch gves a sngle stable optmsaton pont. It was an extenson of the work by Sauert et al. [4] n whch an alternatve approxmaton was used for the. In ths paper, we derve a formula that closely approxmates the. The proposed formula has several advantages. It s both contnuous and dfferentable wth respect to the speech spectral levels, whch can lead to more effcent optmsaton. e demonstrate the use of ths approxmaton n the applcaton of near-end speech enhancement as an extenson to [3], n whch, we see further postve gans n over [3]. The remander of the paper s organsed as follows. The s summarsed n Secton 2. The formula that closely approxmates the calculaton based on the ANSI standard s derved n Secton 3. Smulaton results are provded to confrm the correspondence between the two methods of calculaton. Subsequently, ths formula s used n Secton 4 to set up a cost functon for the calculaton of the equalser settngs for a near-end speech enhancement system and smulaton results are provded to demonstrate ts performance. Conclusons form ths work are drawn n Secton DESCRIPTION OF The, Ψ, s defned n []. It s a functon of the average speech spectrum power levels, E, and the average nose spectrum power levels, N, both gven n db, where represents the th subband. The s computed as the weghted sum over max subbands of the band audblty functon, A : max ΨE,N I A E [ E... E max ] N [ N... N max ], where I s the predefned band mportance functon, whch represents the relatve contrbuton of each band to the overall ntellgblty. Typcally, thrd-octave or crtcal bands are used [5]. The band audblty functon s a measure of the effectve dynamc range n each subband that contrbutes to the ntellgblty of the speech. It can be descrbed as a functon of the level dstorton factor, L, and K, a parameter that models maskng effects and that vares from to : A L K.

2 Dfferentable Approxmaton to The level dstorton factor consders the decrease n ntellgblty when speech s presented at hgh levels, thus causng dstortons. It s gven by: { L mn E } U,, 2 6 where U s the standard speech spectrum level at normal vocal effort. K represents the decrease n ntellgblty caused by maskng from nose and can be further expanded as: K mn,max,σ, 3 where σ s a functon of the equvalent speech spectrum level and the dsturbance level, D : The dsturbance level s defned as: σ E D D max{z,x + T }, 5 where X represents the nternal nose spectrum and Z s the equvalent maskng spectrum. If the lstener s mpared we can ntroduce an equvalent hearng threshold level, T, whch accounts for varous mparments and s added to the nternal nose spectrum. The equvalent maskng spectrum, Z s expanded as follows for thrd octave bands: 3.. Dfferental B, Z log N B k +3.32C k log.89 f + f k k,, Ψ where N s the nose spectrum level and f s the nomnal mdband frequency of the th sub-band. C k s the slope per octave of the spread of maskng whch, for thrd-octave bands, s gven as: 6 C 8 +.6B + log f Fnally, B s the effectve nose n the th band: The correspondng mnmum can be found usng the followng dentty: mna,b max a, b. The above functons are used to approxmate 2, 5 and 8 as follows: L ln e U E e D ln e Z + e X +T B ln e N + e V. e can approxmate 3 usng both the maxmum and mnmum approxmatons from above: ln e λ + e ln+eλσ K, λ where λ determnes the accuracy of the estmate. As 3 uses both the mnmum and maxmum functons close together, λ s ntroduced to force the approxmaton closer to the turnng pont of each functon. Hgher values for λ result n a more accurate approxmaton but a hgher peak dervatve. As 3 s tself an approxmaton based on the orgnal experments n [2], by consderng ths work a better value for λ can be chosen. In our experments we have set λ. The above approxmatons allow us to form the partal dervatve of the wth respect to each spectral level, E j, whlst holdng all other spectral levels constant, as follows: max max I L K I L K + K L The level dstorton factor s dfferentated as follows: L e 6U j E j +7 6 j e 6U j E j +7 +e. 2 j B max{n,v }, 8 where V s the self-maskng spectrum level of the speech: 3. APPROXIMATION FOR V E The above formulaton s not very easy to use n optmsaton procedures. In order to fully utlse the we need to fnd an approxmaton that s better behaved. Several of the expressons above use max and mn functons whch are not dfferentable. Thus we approxmate these terms usng the Jacoban maxmum: maxa, b ln e a + e b. Nose and maskng dfferentaton: +e λσ 2 e λσ 3 K e λ ++e λσ j < j, 3 +e λσ 2 e λσ e λ ++e λσ D > j 3 where the dstorton factor s dfferentated as: D e Z e Z + e X +T Z. Usng 3 we can smplfy the summaton part of the dervatve: Ψ L j I j K j + K max j K L j + I L.4 j+ AES 54 TH INTERNATIONAL CONFERENCE, London, UK, 24 June 2 4 Page 2 of 5

3 Dfferentable Approxmaton to e can smplfy the dervatve expressons by assumng that the effectve nose level, N, s greater than the selfspeech maskng level, V, so that 8 becomes B N. Ths assumpton has been made n several prevous approaches [3, 2, 4] and holds well for SNRs up to 35dB, where the s already hgh. As B s no longer a functon of B E,, and therefore Z, whch gves D. Thus our dervatve can be further smplfed usng the followng: Therefore 4 becomes: K, j. Ψ I j where 2 becomes: L j 6 and from 3 we obtan: L j K j + K j L j, exp 6 U j E j + 7 exp 6 U j E j exp, K j Approxmaton Valdty + e λσ 2 e λσ e λ + + e λσ. To evaluate the accuracy of the estmate we tested a range of speech and nose sgnals. The speech sgnals comprsed the TIMIT core set, [6], and we used 8 dfferent nose types from NOISEX92, [7]. The SNR was vared from 3 to 3 db, and the speech presentaton level was set to 62.5 db SPL when measured wth P.56 [8], whch s specfed as the normal speech presentaton level from Table 2 n []. The sgnals were splt nto separate bands usng a thrdoctave flter bank, where max 8, wth a samplng frequency of 6 khz: x t and n t. The equvalent spectral levels are computed as follows: E log Ê B, 5 where Ê T t x2 t s the average power n band and B s the flter bandwdth n db, from Table 3 n []. Smlarly we have N log ˆN B for the nose equvalent spectral levels. T For each par of speech and nose segments the was computed usng several methods. The mean average was taken over all the 24 speech segments and nose types. The results were compared to that of the orgnal ANSI standard and to those proposed by [3]. The approxmaton from [3] s as follows: Ψ Ê, ˆN I ξ ξ +, 6 standard devaton Taal [3] Taal [3] Fg. : The proposed estmate appled to 24 speech segments wth 8 nose types. The upper graph shows the average over all test materal. The lower graph shows the standard devaton of the approxmaton error Taal [3] Fg. 2: hen presented wth a hearng mpared lstener, the drops. The proposed approxmaton follows ths trend, however [3] does not. The upper graph n Fg. plots the versus SNR averaged over 24 speech utterances and 8 nose types together wth our proposed approxmaton and that from [3]. e see that whle [3] slghtly overestmates the true, our proposed approxmaton s extremely close. The lower graph n Fg. shows the standard devaton of the error n the two approxmatons and we see that ths too has been greatly reduced and s extremely small for our proposed approxmaton Hearng Impared Lsteners In [3], 5: D max{z,x + T }, s approxmated usng D Z. However, for mpared lsteners the nternal nose spectrum, X +T, can be much larger than Z. Thus ths approxmaton does not hold well n ths case. Fg. 2 shows the case of a moderately-severe presbycuss lstener wth hearng loss of T 6 db, [9, 2, 2]. The s poorly approxmated by [3] above db, whereas the proposed method stll remans accurate. The proposed approxmaton s therefore sutable for use n hearng ad applcatons where the accurate assessment of ntellgblty s essental. where ξ Ê ˆN s the SNR n each band. AES 54 TH INTERNATIONAL CONFERENCE, London, UK, 24 June 2 4 Page 3 of 5

4 Dfferentable Approxmaton to Far-end speech Near-end nose Analyss Analyss Equvalent Spectral Levels E N Enhancement Flterng Optmse eghts Equvalent Spectral Levels E + Synthess Fg. 3: Block dagram of a lstenng enhancement system 4. APPLICATION TO NEAR-END LISTENING EN- HANCEMENT The proposed contnuous approxmaton allows us to utlse the dervatve whch can assst applcatons n whch the am s to maxmse the. Near-end lstenng enhancement s one such applcaton whch attempts to fnd the optmal flter to mprove the of a speech segment presented n a gven nosy envronment. A block dagram of a general lstenng enhancement system s shown n Fg. 3. Intally we compute the equvalent spectral levels of the speech, E, and the nose, N, n each band, as n 5. The flterng processng multples each speech subband spectral level by a constant,. Thus the the equvalent spectral levels of the output sgnal are equal to Ẽ E +. The resultng s a functon of Ẽ and N, therefore, usng a vector notaton for a set of spectral levels E {E } etc., the optmsaton problem s: A argmaxψe +,N subject to: E + +B E +B. 7 The constrant mposes a lmtaton on the power output of the fltered sgnal. Due to lmtatons of the hardware, n real applcatons there s usually an upper bound on the sgnal power. For example, applcatons wth small speakers such as a moble phone or hearng ad. The speaker or amplfer s lkely drven at ts lmt to maxmse the output and further amplfcaton wll ntroduce dstortons. Thus we do not want to ncrease the output power beyond ths level. The fnal output sgnal can be composed by addng the subband sgnals together, to gve yt max x t /2. To solve the optmsaton problem, prevous approaches have used approxmatons of the to make the optmsaton convex. For example, 6 s convex, whch guarantees a unque soluton. Let us denote the optmsed flter weghts from [3], 6, as T : T argmax Ψ T E +,N subject to: E + +B E +B. 8 In our proposed soluton the dervatve s defned everywhere, thus the proposed approxmaton can be effectvely used n optmsaton. However as there exst multple local maxma for 7, t s possble for the optmser to converge to a sub-optmal set of weghts. For ths reason we use the soluton from 8, T, whch s globally convex, y mprovement Standard devaton n mprovement A - T - Taal [3] P A - T - Taal [3] P - Fg. 4: mprovement for dfferent algorthms averaged over the TIMIT core test set and 8 nose types, top - mean average, bottom - standard devaton as the ntal flter weghts and then optmse from there to converge on an mproved soluton, P : P argmax Ψ P E + T +,N subject to: E + + T, +B 4.. Results E +B. 9 The before and after lstenng enhancement was evaluated usng the ANSI standard, on the output equvalent speech spectral levels: ΨE +,N. The results were averaged over all speech segments and all nose types as presented n Secton 3.2. Fg. 4 shows the mprovement from the weghted speech produced usng A, T and P. The weghts were optmzed usng the fmncon nonlnear optmser wthn MATLAB [22]. Usng our proposed method we see small performance gans over 8 and large gans over the un-enhanced speech as shown n Fg. 4. The ncrease n, over the unprocessed sgnal, can correspond to an ncrease n ntellgblty of 5-2% for sentences not prevously heard by the lstener [23]. At SNRs above 5dB, the performance gans over 8 ncrease. At all SNRs the performance s postve as expected. Furthermore by usng an nequalty constrant on the nstantaneous power, we are able to acheve gans n whlst reducng the power output of the enhanced sgnal. Ths s partcularly notceable at hgh SNRs, as shown n Fg CONCLUSIONS A new approxmaton to the has been proposed. It has a globally defned dervatve whch s contnuous. Experments show that the estmate closely matches the orgnal ANSI standard. The proposed approxmaton was appled to the applcaton of near-end lstenng enhancement. Over a vared set of speech and nose segments, results show that utlsng the approxmaton gves large postve gans n AES 54 TH INTERNATIONAL CONFERENCE, London, UK, 24 June 2 4 Page 4 of 5

5 Dfferentable Approxmaton to Relatve Sgnal Power db 2 A - T - Taal [3] P - 3 Fg. 5: The relatve power dfference before and after lstenng enhancement. compared wth an unprocessed sgnal as well as an exstng method. It also allows a reducton n sgnal power wthout a loss of ntellgblty. 6. REFERENCES [] ANSI, Methods for the calculaton of the speech ntellgblty ndex, ANSI Standard S R27, Amercan Natonal Standards Insttute, 997. [2] N.R. French and J.C. Stenberg, Factors governng the ntellgblty of speech sounds, J. Acoust. Soc. Am., vol. 9, no., pp. 9 9, 947. [3] K. Kryter, Methods for the calculaton and use of the artculaton ndex, J. Acoust. Soc. Am., vol. 34, no., pp , 962. [4] B. Sauert and P. Vary, Near end lstenng enhancement: Speech ntellgblty mprovement n nosy envronments, n Proc. IEEE Intl. Conf. on Acoustcs, Speech and Sgnal Processng ICASSP. IEEE, 26, vol., pp. I I. [5] J. Tbbtts and Y. Lu, Forensc applcatons of sgnal processng, IEEE Sgnal Processng Magazne, vol. 26, no. 2, pp. 4, 29. [6] J.T. Tbbtts and.j. Myles, Crtcal nfrastructure n speech and sound processng and ts effect on the securty of Australa, Engneerng a Secure Australa, pp. 97 5, 24. [7] H.F. Hollen, Forensc voce dentfcaton, Academc Press, 22. [8] J.. Shn and N.S. Km, Perceptual renforcement of speech sgnal based on partal specfc loudness, IEEE Sgnal Process. Lett., vol. 4, no., pp , 27. [9] P. Shankar Chanda and S. Park, Speech ntellgblty enhancement usng tunable equalzaton flter, n Proc. IEEE Intl. Conf. on Acoustcs, Speech and Sgnal Processng ICASSP. IEEE, 27. [] S. Yoo, J.R. Boston, J.D. Durrant, K. Kovacyk, S. Karn, S. Shaman, A. El-Jaroud, and C.C. L, Speech enhancement based on transent speech nformaton, n Proc. IEEE orkshop on Applcatons of Sgnal Processng to Audo and Acoustcs. IEEE, 25, pp [] B. Sauert, H. Löllmann, and P. Vary, Near end lstenng enhancement by means of warped low delay flter-banks, n Proc. of ITG-Fachtagung Sprachkommunkaton, Aachen, Germany, Aachen, Oct. 28. [2] B. Sauert and P. Vary, Near end lstenng enhancement optmzed wth respect to speech ntellgblty ndex, n Proc. European Sgnal Processng Conf. EUSIPCO, Glasgow, Scotland, Aug. 29, pp [3] C.H. Taal, J. Jensen, and A. Lejon, On optmal lnear flterng of speech for near-end lstenng enhancement., IEEE Sgnal Process. Lett., vol. 2, no. 3, pp , 23. [4] B. Sauert and P. Vary, Near end lstenng enhancement optmzed wth respect to speech ntellgblty ndex and audo power lmtatons, n Proc. European Sgnal Processng Conf. EUSIPCO, Aalborg, Denmark, 2, pp [5] E. Zwcker, Subdvson of audble frequency range nto crtcal bands, J. Acoust. Soc. Am., vol. 33, no. 2, pp. 248, 96. [6] J.S. Garofolo, L.F. Lamel,.M. Fsher, J.G. Fscus, D.S. Pallett, N.L. Dahlgren, and V. Zue, TIMIT acoustc-phonetc contnuous speech corpus, Corpus LDC93S, Lngustc Data Consortum, Phladelpha, 993. [7] A. Varga and H.J.M. Steeneken, Assessment for automatc speech recognton II: NOISEX-92: a database and an experment to study the effect of addtve nose on speech recognton systems, Speech Communcaton, vol. 3, no. 3, pp , July 993. [8] ITU-T, Objectve measurement of actve speech level, Recommendaton P.56, Internatonal Telecommuncatons Unon ITU-T, Mar [9] J.M. Festen and R. Plomp, Effects of fluctuatng nose and nterferng speech on the speech-recepton threshold for mpared and normal hearng, J. Acoust. Soc. Am., vol. 88, pp. 725, 99. [2] R. Beutelmann and T. Brand, Predcton of speech ntellgblty n spatal nose and reverberaton for normal-hearng and hearng-mpared lsteners, J. Acoust. Soc. Am., vol. 2, pp. 33, 26. [2] A. Spoor, Presbycuss values n relaton to nose nduced hearng loss, Internatonal Journal of Audology, vol. 6, no., pp , 967. [22] The Mathworks Inc., MATLAB and Smulnk User s Gude, 28. [23] ANSI, Methods for the calculaton of the artculaton ndex, ANSI Standard ANSI S , Amercan Natonal Standards Insttute, New York, 969. AES 54 TH INTERNATIONAL CONFERENCE, London, UK, 24 June 2 4 Page 5 of 5