Effect of reducing slow temporal modulations on speech reception

Transcription

1 Effect of reducng slow temporal modulatons on speech recepton Rob Drullman, Joost M. Festen, and Rener Plomp Department of Otorhnolaryngology, Free Unversty Hosptal, P.O. Box 7057, 007 MB Amsterdam, The Netherlands (Receved 23 August 993; revsed 29 November 993; accepted for publcaton 4 December 993} The effect of reducng low-frequency modulatons n the temporal envelope on the speech-recepton threshold (SRT) for sentences n nose and on phoneme dentfcaton was nvestgated. For ths purpose, speech was splt up nto a seres of frequency bands (,«, or oct wde) and the ampltude envelope for each band was hgh-pass fltered at cutoff frequences of, 2, 4, 8, 6, 32, 64, or 28 Hz, or o (completely flattened). Results for 42 normal-hearng lstener show: () A dear reducton n sentence ntellgblty wth narrow-band processng for cutoff frequences above 64 Hz; and (2) no reducton of sentence ntellgblty when only ampltude varatons below 4 Hz are reduced. Based on the modulaton transfer functon of some condtons, t s concluded that fast multchannel dynamc compresson leads to an nsgnfcant change n masked SRT. Combnng these results wth prevous data on low-pass envelope flterng (temporal smearng) [Drullman et al., J. Aeoust. Soe. Am. 95, (994)] shows that at 8-0 Hz the temporal modulaton spectrum s dvded nto two equally mportant parts. Vowel and consonant dentfcaton wth nonsense syllables were studed for cutoff frequences of 2, 8, 32, 28 Hz, and oo, processed n -oct bands. Results for 2 subjects ndcate that, just as for low-pass envelope flterng, consonants are more affected than vowels. Errors n vowel dentfcaton manly consst of reduced recognton of dphthongs and of duratonal confusons. For the consonants there are no dear confuson patterns, but stops appear to suffer least. In most eases, the responses tend to fall nto the correct category (stop, frcatve, or vowel-lke). PACS numbers: 43.7.Es, Mk INTRODUCTION One way to descrbe a speech sgnal s as a summaton of a number of ampltude modulated narrow frequency bands. In ths vew, every frequency band can be consdered to consst of a carrer sgnal (fne structure) and a tme-varyng envelope. The envelope n turn contans a varety of modulaton frequences, the ampltude of whch can be llustrated by the temporal modulaton spectrum. These modulatons play an mportant role n the transmsson of nformaton n speech (of. Houtgast and Steeneken, 985). When processng and/or transmttng speech n some way, a fathful transfer of these modulaton frequences seerns necessary. In terms of percepton, we would lke to know how attenuaton of the detals (fast ampltude varatons) or of the gross movements (slow ampltude varatons) affect the understandng of everyday speech. In other words, the queston s: Wthn whch lmts can specfc ampltude modulatons be reduced before havng a detrmental effect on ntellgblt¾? In an earler study (Drullman et al., 994), the effect of temporal envelope smearng on sentence ntellgblty and phoneme recognton was nvestgated. In ths approach, the wdeband sgnal was subdvded nto a seres of frequency bands (,«, or oct wde) and the ampltude envelope of each band was low-pass fltered at a varable cutoff frequency. The results showed that preservng only modulatons up to about 6 Hz yelds almost the same speech-recepton threshold (SRT) for sentences n nose as obtaned for unprocessed speech. For lower cutoff frequen- tes the SRT ncreases, and for cutoff frequences as low as 0-2 Hz sentence ntellgblty n quet s heavly affected f envelope flterng takes place n narrow frequency bands. Consonants, especally the stops, suffer more from severe temporal smearng than vowels. In contnuaton of ths, the present study was set up to nvestgate the effect of reducng low-frequency temporal modulatons. By hgh-pass flterng the temporal envelope n a seres of frequency bands, the extent to whch ntellgblty depends on the slow ampltude varatons can be establshed. The appled sgnal processng and the expermental desgn are closely related to the low-pass flterng n the prevous study. In Drullman et al. (994), we dscussed the sgnfcance of temporal modulatons for ntellgblty, referrng to related ssues n the lterature, such as the modulaton transfer functon (MTF), voeoders, and phoneme percepton based on envelope nformaton. Apart from the mere effect of hgh-pass envelope flterng, results from the pl',cnt study can shed lght on the ssue of speech ntellgblty n multchannel compresson systems (of. Plomp, 988; Hohmann and Kollmeer, 990}. Together wth the earler low-pass flterng results, more nsght nto the contrbuton of temporal modulatons can be gven. In ths paper we wll descrbe two percepton experments. In experment, the ntellgblty for sentences n quet and the SRT for sentences n nose were measured as a functon of envelope hgh-pass cutoff frequency and processng bandwdth. In experment 2, the effects on vowel 2670 J. Acoust. Soc. Am. 95 (5), Pt., May /94/95(5)/2670//$ Acoustcal Socety of Amerca 2670

2 A FIG.. Block dagram of the speech processng. The wdeband nput speech sgnal s splt up nto several frequency bands. For each band the ampltude envelope s determned, downsampled by a factor N (6 or 64), low-pass fltered, and upsampled. The hgh-pass fltered envelope s obtaned by substractng the low-pass fltered envelope from the orgnal envelope and addng the dc component. Modulatng the fne structure accordng to the modfed envelope then yelds the new band sgnal. Each new band sgnal s low-pass fltered to elmnate undesred hgh-frequency nose. After addng all modfed bands, the wdeband sgnal s rescaled to match the rms of the nput speech. and consonant dentfcaton n nonsense syllables were studed. by the rato of the modfed and the orgnal envelope. To elmnate any spectral dstorton, the modfed sgnal s low-pass fltered wth a cutoff frequency 5% hgher than I. METHOD the upper lmt of the correspondng bandpass flter. Fnally, all modfed sgnals are added and the level of the For the sgnal processng, the analyss-resynthess new wdeband sgnal s adjusted to have the same (wdescheme used n the prevous experments wth low-pass band) rms as the nput sgnal. fltered envelopes was slghtly modfed. A block dagram All sgnal manpulatons were performed (non-realof the processng s shown n Fg.. Snce the detals of the tme) on an Olvett PCS 286 computer, usng an OROSmethod have been descrbed n Drullman etal. (994), AU2 card wth TMS320C25 sgnal processor. Fgure 2 ths secton prmarly contans nformaton on the modf- shows an example of the varous stages n the processng of catons (vz. the changes n envelope flterng). one oct band for a short sentence. The wdeband speech (samplng rate of 5625) s led through a lnear-phase FIR dgtal flter bank, and from the output of each channel the Hlbert envelope s deter- II. EXPERIMENT : SENTENCE INTELLIGIBILITY mned. In order to meet the requrements of the envelope flter's low cutoff frequences, the envelope s downsampled before flterng. The actual downsamplng factors used were 64 for envelope cutoff frequences below 80 Hz, and 6 for envelope cutoff frequences above 80 Hz. Due to downsamplng (wth precedng low-pass flterng), the upper-modulaton frequency stll present n the envelope s lowered. Therefore, drect hgh-pass flterng of the downsampled envelope would n fact be bandpass flterng wth an upper-cutoffrequency below half the samplng frequency after downsamplng. To overcome ths problem, the downsampled envelope s low-pass fltered, upsampled, and subtracted from the orgnal envelope (n the tme doman). Takng the --6-rb pont as cutoff frequency and a low-pass flter slope of about --80 db/oct, ths results n a modfed envelope that s effectvely hgh-pass fltered at the same cutoff frequency, wth a slope of approxmately +40 db/oct. In order to mantan a suffcent level, the mean level (de) of the orgnal envelope s added to the fltered verson. Parts of the modfed envelope that are stll A. Stmul, desgn Ten lsts of 3 Dutch sentences of eght to nne syllables read by a female speaker were used as basc materal (Plomp and Mmpen, 979). All sentences were dgtzed at a samplng rate of 5625 Hz and 6 bts resoluton. Ten processng condtons were nvestgated, whch the envelope n each frequency band was hgh-pass fltered at the followng cutoff frequences: 0 Hz (all modulatons ntact, control condton),, 2, 4, 8, 6, 32, 64, 28 Hz, or oo (no modulatons left). The last condton was obtaned by usng the mean ampltude level of each frequency band for the entre duraton of the utterance (ths condton s dentcal to the lowpass 0-Hz condton n Drullman eta!., 994). The possble effect of processng bandwdth was nvestgated as well; envelope flterng was performed n -, 5-, or -oct bands, coverng the range Hz. A prelmnary test revealed that senlenees presented n quet were not completely ntellgble for cutoff frequences of 64 Hz and hgher (processed n -oct bands). Therefore, negatve are set to zero (eventually resultng n a short no relable SRT n nose could be measured for cutoff freslent nterval n that partcular frequency band). For each frequency band, the modfed sgnal s obtaned by multplyng the fne structure, sample by sample, quences of 64, 28 Hz, and oo. Instead, n those condtons sentences were presented n quet and the number of correctly receved sentences was scored. Ths wll be referred 267 d. Acoust. Soc. Am., Vol. 95, No. 5, Pt., May 994 Drullman el' al: Reducng slow temporal modulatons 267

3 (a) FIG. 2. Example of the processng for one sentence (2.2-s duraton) n a sngle frequency band. (a} Orgnal wdeband sgnal; (b} -oct band sg- nal ( Hz}; (c} ampltude envelope of (b}; (d} envelope hgh-pass fltered wth 8-Hz cutoffrequency; (e) resultng modfed band sgnal, ampltude modulated accordng to (d}, after addtonalow-pass flterng at 353 Hz. to as the SIQ (sentence ntellgblty n quet) experment. For the SRT measurements a maskng nose wth the same spectrum as the long-term average of the 30 sentences was used. In order to compensate for small spectral changes n the speech materal due to processng, the maskng nose was processed separately for each condton n the same manner as the sentences. B. Subjects Subjects were 42 normal-hearng students of the Free Unversty, whose ages ranged from 8 to 27. All had puretone ar-conducton thresholds less than 5 db HL n ther preferred ear at octave frequences from 25 to 4000 Hz and at 6000 Hz. They were dvded nto three groups of 4, each group recevng the ten condtons for one of the three processng bandwdths. C. Procedure From the ten lsts of 3 sentences, seven were used n the SRT experment and three n the SIQ experment. Lsts were presented n a fxed order. The sequence of the condtons was vared accordng to a Latn square---7 X 7 for the art and 3 3 for the SIQ experment--to avod order and lst effects. Havng 4 subjects n a group, each sequence was presented to two subjects n the SRT exper- (e) (e) ment; n the SIQ expermentwo sequences were presented to fve subjects and one sequence to four subjects. For the SIQ experment, all three lsts were presented n quet, at a level of 70 db(a). Every sentence was presented once, after whch the subject had to reproduce t as accurately as possble. Subjects were encouraged to respond freely, even f they understood only fragments of the sentence. A response was scored as correct only f the complete sentence was reproduced correctly. For the SRT experment, the level of the maskng nose was fxed at 70 db(a) for each condton; the level of the sentences was changed accordng to an up-down adaptve procedure (Plomp and Mmpen, 979). The frst sentence n a lst was presented at a level below the recepton threshold. Ths sentence was repeated, each tme at a 4-rb hgher level, untl the lstener could reproduce t wthout a sngle error. The remanng 2 sentences were then presented only once, n a smple up-down procedure wth a step sze of 2 db. The average sgnal-to-nose rato for sentences 4-3 was adopted as the SRT for that partcular condton. The sentences n both the SIQ and the SRT experments were presented monaurally through a headphone (Sony MDR-CD999) to the subject's ear of preference n a soundproof room. Before the actual tests, a lst of 3 sentences pronounced by a male speaker was presented, n order to famlarze the subjects wth the procedure. For the SIQ expermenths lst conssted of sentences n the 28-Hz condton; for the SRT experment another lst n the 4-Hz condton was used. All subject started wth the SIQ experment, contnung wth the SRT experment after a short break. D. Results and dscusson The average results of the SIQ experment for the three flterng condtons n the three bandwdths are plotted n Fg. 3. For - and «-oct processng, performance vared wdely among subjects, especally at cutoff frequences of 28 Hz and oo. To evaluate the effects of bandwdth and cutoff frequency, a two-way analyss of varance (ANOVA) for repeated measures, usng aresne transformed scores (Studebaker, 985), was performed. Both factors were sgnfcant (p < 0.00 ), as was the nteracton (p<0.005}. Post hoc (Tukey HSD) tests of the smple effects (Krk, 968) showed that the mean scores for the three bandwdths were all sgnfcantly dfferent (p < 0.0 }, and that n all cases the scores n the condton wthout any modulaton ( oo ) were sgnfcantly lower (p<0.0) than n the 64- and 28-Hz condtons. Only for the -oct bandwdth was there a sgnfcant dfference (p < 0.05) between cutoff frequences of 64 and 28 Hz. The results of the 0o condtons of 7%, 32%, and 78% for processng bandwdths of -, -, and l-oct, respectvely, are n good agreement wth earler fndngs (3%, 22%, and 80%, respectvely; see low-pass 0-Hz condton n Drullman et al., 994). Varatons n the scores must be attrbuted to the earler mentoned dfference n subjects' performance. The mean SRT for sentences n nose as a functon of hgh-pass cutoff frequency and processng bandwdth s 2672 d. Acoust. Soc. Am., Vol. 95, No. 5, Pt., May 994 Drullman et al.: Reducng slow temporal modulatons 2672

4 loo 80 6o 4o 20 o -oct ' O -oct,,, ' --o //, // HP cutoff frequency (Hz) FIG. 3. Mean percentage correct for sentences n quet as a functon of hgh-pass (HP) cutoff frequency, wth processng bandwdth as pa- rarnetero plotted n Fg. 4. A smlar ANOVA as for the SIQ data revealed sgnfcant effects for bandwdth, cutoff frequency, and the nteracton between them (p<0.00). Tests for smple effects (Krk, 968) and post hoc tests showed that there was no sgnfcant bandwdth effect up to 8 Hz; n the 6-Hz condton the SRT for the l-oct bandwth dffers from the other two (p<0.0), whle n the 32-Hz condton the SRT for all three bandwdths are dfferent (p <0.0). As for the flterng condtons, there are no sg- Mleant dfferences n SRT below 4 Hz, where there appears to be a constant threshold of on average db (standard devatons.0 to.4 db). The SRT for cutoff frequences of 8 Hz and upward dffers sgnfcantly from the control {0 Hz) condton (p<0.0}. 4 a -oct o --oct o -oct -6 ' // ' ' ' ' ' ' HP cutoff frequency (Hz) Fg. 4. Mean speech-recepton threshold for sentences n nose as a functon of hgh-pass (HP) cutoff' frequency, wth processng bandwdth as parameter. The results of these experments ndcate that the ntellgblty of everyday sentences wth hgh-pass fltered envelopes remans at the same level as unprocessed speech up to a cutoff frequency of 4 Hz. For hgher cutoff frequences ntellgblty decreases progressvely, partcularly for narrow processng bands. So, t appears that envelope modulatons below 4 Hz do not ad sentence ntellgblty n nose, as long as hgher modulatons are ntact. For cutoff frequences below 8 Hz, ntellgblty does not depend on the processng bandwdth, whch ndcates that n these cases the spectral macro-nformaton,.e., the varaton n overall spectral shape, s suffcently present [see Drullman etal (994) for a dscusson on spectral mcro- and macro-nformaton]. A further dscusson of these results n relaton to those of the earler low-pass-envelope flterng experments can be found n Sec. IV. IlL EXPERIMENT 2: CONSONANT AND VOWEL IDENTIFICATION A. Stmul, desgn The am of the second experment was to assess the degree to whch ndvdual vowels and consonants n nonsense syllables are affected by hgh-pass envelope flterng. The speech materal conssted of two types of meanngless syllables. CVC syllables for vowel (V) dentfcaton, and VCV syllables for consonant (C) dentfcaton. The eve words were obtaned from 24 exstng lsts of 2 dfferent syllables each (Bosman, 989), read by the same speaker who had produced the sentences. The 2 vowels n a lst (/a,a,e,e,l,,3,o,o,u,ou,e/) were embedded n varous consonant envronments. For the dentfcaton of the consonants, we used VCV syllables spoken by the same speaker. Each syllable conssted of one of 6 consonants fo, d,f,x,h,j,k,l,m,n,p,s,t,v,w,z/surrounded by one of four vowels/a,,u,ce/, yeldng a total of 64 dfferent syllables. The four vowels were selected to nduce dfferent envelope courses just before and after the consonant. Both CVC and VCV syllables were dgtzed wth 6-bts resoluton at a samplng rate of 5625 Hz. They were normalzed for rms level. For all syllables the envelope flterng was performed n 24 -oct bands. There were sx expermental condtons: A control condton (0 Hz) and cutoff frequences of 2, 8, 32, 28 Hz, or o. The choce of the flterng condtons was based on the results of the SRT experments, vz., normal ntellgblty (control and 2 Hz), reduced ntellgblty (8 and 32 Hz), and low ntellgblty (28 Hz and o). For the sake of convenence, we wll wrte the flterng condton n parentheses followng the set of phonemes to be dentfed. For example, C (8) stands for consonants n the 8-Hz condton. From the orgnal 24 lsts of 2 CVC syllables, 36 randomzed lsts of 50 syllables were made. The frst two syllables were copes of the last two and acted as dummy trals, so that there were test stmul n a lst for the dentfcaton of V. These lsts were constructed n such a 2673 d. Acoust. Soc. Am., Vol. 95, No. 5, Pt., May 994 Drullman et al.: Reducng slow temporal modulatons 2673

5 loo 80 6O 4O TABLE I. Percentage of vowel errors due to dphthong-monophthong confusons, long-short/short-long confusons, and ncorrect / / responses. The total number of errors (out of 576 stmul) for V(8), V(32), V(28), and V(o ) are 0, 62, 224, and 240, respectvely. Condton 8 Hz 32 Hz 28 Hz o Dphthong Long-short / /responses 4 I 2 3 2O oo PIP cutoff frequency (Hz) FIG. 5. Overall vowel- and consonant-dentfcaton score as a functon of hgh-pass (HP} cutoff frequency, wth phoneme type as parameter. way that each vowel appeared four tmes n varous consonant contexts. From the orgnal four lsts of 6 VCV syllables, 36 randomzed lst of 66 syllables were constructed for the dentfcaton of C. Each lst contaned all 64 VCV syllables and agan the frst two syllables were copes of the last two. B. Subjects Subjects were 2 normal-heatng students of the Free Unversty, whose ages ranged from 8 to 29. All had puretone ar-conducton thresholds less than 5 db HL n ther preferred ear at octave frequences from 25 to 4000 Hz and at 6000 Hz. C. Procedure For both dentfcaton of C and V, the 36 lsts were assgned to the flterng condtons accordng to a 6X6 dgram-balanced Latn square to avod effects of measurement order. Each sequence of condtons was presented to two subjects. All stmul were presented n quet at a level of 70 rb(a), monaurally through a headphone (Sony MDR-CD999) to the subject's ear of preference. Subjects made ther response by means of labeled buttons on a box connected to a PC. The entre procedure s descrbed n detal n Drullman et al. (994). Half of the subjectstarted wth the vowel dentfcaton, the other half wth the consonant dentfcaton. Before each test, two lsts of 20 stmul n the 8- and 28-Hz condtons were presented to famlarze the subjects wth the expermental task. D. Results and dscusson In total, dentfcatons (2 subjects X4 utterances) were obtaned for each vowel and consonant per condton. The mean scores are plotted n Fg. 5. As an example, confuson matrces for vowels and consonants n the 32-Hz condton are gven n Appendx A. A repeated-measures analyss of varance on the arcsne transformed scores (Studebaker, 985) was performed, wth phoneme ( vowels versus consonants) and flterng condton as wthnsubjects factors. The analyss showed sgnfcant effects of phoneme (p <0.00 ), condton (p <0.00 ), and the nteracton between them (p<0.002). Post hoc (Tukey HSD) tests ndcated that for a hgh-pass cutoff frequency above 2 Hz, the decreasng scores for vowels and consonants are not sgnfcantly dfferent; for cutoff frequences of 32 Hz and hgher, however, consonant scores are sgnfcantly lower than vowel scores (p < 0.0 ). As n the prevous dentfcatons of temporally smeared syllables (Drullman et al., 994), the present results show that consonantsuffer more from envelope flterng than vowels. The scores of 58% and 33% for the extreme condtons V( m ) and C( o), respectvely, closely match earler results (56% and 25%, respectvely). Analyss of the confuson matrces for V(8), V(32), V(28), and V( o) showed that the majorty of the vowel errors are of the same type as n the prevous low-passflterng experments: Dphthong-monophthong confusons (/e/-/e/and/ou/-/a/or/ou/-/o/), long-short/ short-long confusons (/a/-/n/, /e/-/i/, and /o/-/ /, and vce versa), and ncorrect / / responses (neutral vowel). An explanaton for the former two error types s that a reducton of the modulatons (but a present 0-Hz component) causes the speech sgnal to be more steady state; therefore, dphthongs tend to sound lke monophthongs, and, due to the blurrng of the temporal structure, the percepton of vowel duraton s hampered (see also Drullman et al., 994). The percentage of the total number of errors made n the dfferent condtons that these three factors account for are lsted n Table I. Together they yeld 58% to 68% of the errors, dependng on the condton. Among the other errors one can fnd some/a/-/u/ and//-/i/confusons. For an evaluaton of the C scores, the set of consonants was dvded nto three subsets (Steeneken, 992; Drullman etal., 994): stops (/t,k,p,b,d/), frcatves (/f,s,x,v,z/), and vowel-lke consonants (/m,n,,l,w,j,h/). From the orgnal confuson matrces, the data were collapsed nto 3 X 3 matrces, whch grouped the consonants accordng to the aforementoned categores. Fgure 6 shows the dstrbuton of the responses across the three categores for C(8), C(32), C(28), and C( o). Wthn each category the percentage correct consonants s ndcated by a black dot. In accordance wth the pooled data n Fg. 5, the percentage of absolute correct responses (black dots) n d. Acoust. Soc. Am., Vol. 95, No. 5, Pt., May 994 Drullman et al.: Reducng slow tamporal modulatons 2674

6 OO c [3 stop [] fr c [] v--hke O 0 O O $28 s32 s fo0 f28 f32 t8 vco v 28v32 v8 O.00 modulaton fre< aenc, (I-Iz) FIG. 6. Dstrbuton of the responses for the consonants across the three categorestop, frcatve, and vowel-lke as a functon of stmulus category and hgh-pass cutoff frequency (s =stop/oo condton, s32----stop/ 32-Hz condton, f28 = frcatve/28 Hz condton, v$ = vowel-lke/g-hz condton, etc.). The black dots ndcate the percentage of correctly dent/fled consonants per category/condton. creases wth decreasng cutoff frequency for every category. Except for C( oo ), the score for stops s hgher than for frcatves and vowel-lkes. Ths mples that the presence of hgh-frequency modulatons s partcularly benefcal for short-duraton consonants. In all condtons the majorty of the responses fall nto the correct category. Ths s most dstnct and most consstent over condtons for the vowel-lke consonants, n whch case about 90% s perceved as vowel-lke. Wth C(32), C(28), and C( oo ) erroneous/x/and/h/responses occur relatvely often for frcatve and vowel-lke stmul, respectvely. IV. GENERAL DISCUSSION In dscussng the general mplcatons of the results of both experments we want to focus on two aspects. Frst, although the results apply to normal-heatng lsteners and the speech processng as such has no drect meanng for heatng ads, a few thngs can be sad wth respecto multchannel ampltude compresson and dynamc gan control. Second, we wll compare and combne the present results wth the data on low-pass envelope flterng-- temporal smearng- obtaned earler (Drullman eta!., 994). Before adressng the frst aspect, a closer look at the effects of the sgnal processng s needed. A. The modulaton transfer functon The effect of the present sgnal processng on the transmsson of temporal modulatons can be descrbed by the modulaton transfer functon (MTF; of. Houtgast and Steeneken, 985). The MTF s defned as the rato between the ntensty-envelope spectra after and before processng, usually appled n an octave band. A drect measurement of the MTF could thus be performed by comparng the modulaton spectra of the squared nput and output ampltude envelopes (cf. DruBman et al., 994). However, the sgnal processng we used contans a nonlnear element, vz., the elmnaton of the negatve parts (clppng) of the fltered envelope after addton of the mean ampltude (de). For FIG. 7. Normal MTF (sold lnes) and phase-locked MTFpL (dashed lnes) for the 2- and 6-Hz condtons, averaged over four oct bands wth center frequences of 0.25, I, 2, and 4 khz. the calculaton of the speech-transmsson ndex (STI), Ludvgsen et al. (990) proposed a method to deal wth nonlnear dstortons, based on the correlaton between the entre npnt and output envelopes. Because we need an estmate of the transmsson as a functon of modulaton frequency, we propose a dfferent method, closely related to the MTF. Ths method takes nto accounthe phases of the modulatons n the orgnal and modfed envelope. We wll name t the phase-locked MTF, referred to as MTFpL. Detals of the dervaton and computaton of the phaselocked MTF can be found n Appendx B. As an example, Fg. 7 shows the normal MTF (sold lnes) and the phase-locked MTFpL (dashed lnes) for the 2- and 6-Hz condtons, averaged over the ntensty envelopes of four l-oct bands wth center frequences of 0.25,, 2, and 4 khz. As can be seen n g. 7, the MTFpL for both condtons s lower than the MTF, ndcatng that the nonlneartes from the processng have been removed. It s also clear that the ndex for the lower modulatons n the 2-Hz condton s stll relatvely hgh, despte the steep slope (40 db/oct) of the hgh-pass flter used. Ths can be explaned as follows. Clppng occurs more often wth low than wth hgh envelope cutoff frequences. It can n fact be seen as a partal restoraton of the reduced low-frequency modulatons. The low-frequency modulatons that are rentroduced are not completely uncorrelated wth the orgnal modulatons and wll thus show up n the MTFpL. So, n practce t s mpossble to severely reduce low-frequency modulatons when usng low hgh-pass envelope cutoff frequences. However, f t were possble, the effect on (reduced) ntellgblty would be mnor, because other, slghtly hgher modulaton frequences are present and contan suffcent nforraaton. B. Relaton wth ampltude compresson The method used to process the speech sgnal acts drectly on the temporal envelope. In vew of the MTF concept dscussed above, t bears some relaton to multchannel ampltude compresson. As has been measured by 2675 J. Acoust. Soc. Am., VoL 95, No. 5, Pt., May 994 Drullman et al.: Reducng slow temporal modulatons 2675

7 moclulat on frequency (Hz) 0 ' FIG. 8. MTFpL for the 2-, 4-, 8-, and 6-Hz condtons, compared wth the MTFs for ampltude compresson, as measured by Plomp (988). All values are averaged over oct bands wth center frequences of 0.25,!, 2, and 4 khz. Plomp (988), short compresson tmes reduce the temporal contrasts n the speech sgnal. The MTF n case of multchannel ampltude compresson shows a (weak) hgh-pass characterstc, the slope of whch depends on the compresson rato and on the attack and release tmes. However, the reducton of the modulatons s not as severe as n most of our condtons. To llustrate ths, Fg. 8 shows the phase-locked MTFpL for the 2-, 4-, 8-, and 6-Hz condtons, averaged over -oct bands wth center frequences of 0.25,, 2, and 4 khz (dashed lnes). In the same fgure the MTFs of the two compresson systems dscussed by Plomp (988) are gven. The latter are based on a compresson threshold of 30 db below peak level, a compresson rato of 4, and attack/release tmes of 8/8 and 8/200 ms, respectvely. Roughly speakng, both compresson curves show MTFs that are n between our 2- and 6-Hz condton; most of the tme the modulaton ndex s even hgher than n our 8-Hz condton. The results of experment (-oct bandwdth) show the same SRT for the 2- and 4-Hz condtons compared to unprocessed speech, whereas n the 8- and 6-Hz condtons the SRT ncreases by.5 and 2.8 db, respectvely. These data mply that, although ampltude compresson does not mprove ntellgblty (at least not n normal hearng), the ncrease n SRT s lmted to about db, whch s less than the 5 db suggested by Plomp (988) for the 8/8-ms compresson. The dea of only lmted loss n ntellgblty s supported by data from Hohmann and Kollmeer ( 990}, who nvesrgated for normal-hearng lsteners the effect of ampltude compresson over 23 channels on CVC words. They used a fast 5/5-ms system wth a compresson threshold of 60 db below peak level and compresson ratos up to 3. For a comparatvely hgh S/N rato (- 2 db) practcally no decrease n ntellgblty was observed. C. Comparson wth temporal smearng In case of temporal smearng (Drullman et al, 994), the ampltude envelope s low-pass fltered, so that hgh- 0 ' o -oct ol-ip 0, 8 7 5, //..., //,, //. -oct ohp 0 I oo cutoff frequency O-Iz) FIG. 9. Comparson of the SRT relatve to the control condton as functon of envel.ope low-pass (LP) and hgh-pass (HP) cutoffrequency, ß I I processed m -, -, and l-oct bands. frequency modulatons are reduced. Because the hgh-pass envelope flterng n the present study s accompaned by addng of the envelope mean, the two forms of degradng the temporal envelope can be descrbed as follows. For 2676 J. Acoust. Soc. Am., Vol. 95, No. 5, Pt., May 994 Drullman el at: Reducng slow temporal modulatons 2676

8 every channel, the baselne s the mean ampltude (0-Hz component) of the entre utterance. Then other successve frequency components are added, wth full or reduced ampltude (and certan phase, but we leave that out of consderaton). Addng components means ntroducng local devatons from the mean envelope along the tme axs. Snce ampltude fluctuatons are only partly correlated over frequency bands (Houtgast and Verbage, 99), devatons from the average spee.trum are also ntroduced along the frequency axs. Addng all frequency components wth ther orgnal ampltude yelds, of course, the orgnal spectrogram. Addng only low-frequency components gves global envelope varatons. Conversely, addng only hgh-frequency components gves nformaton on the rapd changes n the envelope. Apparently, lack of fast fluctuatons can be compensated for by suffcent global nformaton (components up to 6 Hz); on the other hand, lack of global nformaton can be compensated for by gvng enough fast fluctuatons. In some way, the two types of nformaton (although they are on a contnuum) seem to be nterchangeable. It must be emphaszed that the 6-Hz lmt for low-pass flterng and the 4-Hz lmt for hgh-pass flterng do not automatcally mply that ntellgblty s unaffected f only ampltude modulatons between 4 and 6 Hz are transferred. An nterestng pont on the scale s the crossover frequency, whch dvdes the modulaton-spectrum range nto two parts that are equally mportant for ntellgblty. Fgure 9 shows the relatve SRT (.e., the measured SRT relatve to the control condton) as a functon of low- and hgh-pass cutoff frequency, separately for the three processng bandwdths. Vrtually ndependent of the bandwdth, the crossover frequency s about 8-0 Hz. At ths frequency the masked SRT for sentences has ncreased by 2 db. Ths means that, n a crtcal sgnal-to-nose rato condton, reducng ampltude modulatons below or above 8-0 Hz results n a loss n sentence ntdlgblty of 30%- 40%. Ths crossover frequency for temporal modulatons may depend on the speech materal, the measure used, and/or the talker, as s the case wth the audo frequency spectrum (reported audo crossover frequences range from about 200 to 2000 Hz; see Studebaker et al. (987) for an overvew). For the sake of completeness, crossover frequences and correspondng scores for the vowels and consonants n experment 2 are lsted n Table II. These values are generally somewhat lower, partcularly for the frcatves, ndcatng the need of low-frequency modulatons for ndvdual phoneme recognton. It should be noted that these crossover frequences are based on dentfcaton scores n quet, whereas the crossover frequences for the sentences are based on the SRTs n nose. On average, the crossover frequency for ndvdual phonemes n quet tends toward 4 Hz. As mentoned before, the vowels n the present condtons show almost the same confuson patterns as the smeared vowels. As for the consonants, the scores for the frcatves are somewhat lower, whch also accounts for the category consstency. For the vowel-lkes a hgh category consstency found n both the present and the smeared TABLE II. Crossover modulaton frequences for the three consonant categores, overall consonants, and overall vowels, together wth the correspondng dentfcaton score. Stop Fre. V-lke Cons. Vowel Frequency 8 Hz 2 Hz 6 Hz 3 Hz Score 84% 79% 76% 78% 88% condtons. Contrary to the smeared stops, the present stops n the severely degraded condtons are not as much perceved as freatves, not even n the case of 00% compresson. The latter may be due to the fact that the envelopes of all stmul contaned at least a 0-Hz component, so that lsteners got more famlar wth the speech sounds ths evokes, resultng n better performance for the "dffcult" stops. As an llustraton, Fg. 0 dsplays wdeband waveforms and correspondng spectrograms of a segment around/k/n dfferent condtons. In vew of Fg. 0, a hgh score for the 2-Hz condton [panel(b)] s plausble. For all stops together, despte the seemng correspondence n waveform between the 32-Hz and 0o condtons [panel (c) and {d}], mean recognton scores of 74% and 28% are found, respectvely. An explanaton for ths dfference can be gven by lookng at the speetrograms. One can see that the suppresson of modulatons below 32 Hz does not remove any spectral cues for the/k/burst. The lstener may be dsturbed by the "fllng" of the occluson part, by whch the typcal {temporal) character of a plosve s affected. Although ths also apples to the oo condton, spectral nformaton for the /k/ burst has practcally dsappeared n that case. V. CONCLUSIONS In summary, the most mportant conclusons of the present study are () Temporal ampltude varatons n successve, «-, or -oct frequency bands below 4 Hz can be reduced wthout reducng speech ntellgblty for normal-hearng ls- teners. (2) Sentences n quet cannot be completely understood when only fast ampltude fluctuatons above 32 Hz are present, but performance mproves for broader frequency bands. For hgh-pass envelope cutoff frequences below 8 Hz, ntellgblty s not dependent on the processng bandwdth. (3) The results suggesthat multchannel ampltude compresson wth small tme constants reduces ntellgblty, but by no more than about db n terms of the masked speech-recepton threshold. (4) Comparson of results for low- and hgh-pass envelope flterng shows that the crossover modulaton frequency for sentences s about 8-0 Hz, vrtually ndependent of the processng bandwdth. At ths frequency the masked SRT has ncreased by 2 db, correspondng to a loss n ntellgblty score of 30%--40% J. Acoust. Soc. Am., Vol. 95, No. 5, Pt., May 994 Drullrnan et al: Reducng slow temporal modulatons 2677

9 (a) 50 ( tme (ms) 50 OO 50 03) tme (ms) (c) 50 ( O tme (ms) (d) tme (ms) FIG. 0. Waveforms and spectrograms of 300-ms segments around/k/n the nonsense syllable/k/, wthgh-pass envelope cutoff frequences of (a) 0 Hz (unprocessed), (b) 2 Hz, (c) 32 Hz, and (d) oo. The processng bandwdth Is oct. ACKNOWLEDGMENTS (5) Consonants suffer more from reducng slow mod- Ths research was supported by the Lngustc Reulatons than vowels. Vowel errors are characterzed by search Foundaton, whch s funded by the Netherlands msclassfyng dphthongs as monophthongs and by short- organzaton for scentfc research, NWO. long/long-short confusons. Consonant errors manly con- APPENDIX A: SUMMED CONFUSION MATRICES sst of confusons wthn a category (stop, frcatve, or FROM THE VOWEL AND CONSONANT vowel-lke). Stops appear to suffer least; ther dentfcaton IDENTIFICATION EXPERIMENTS IN THE 32-Hz s generally better than n cases of temporal smearng. CONDITION TABLE AI. Summed confuson matrces for 2 subjects: Consonants. t k Stmulus/response HP 32 Hz t k p b d f s X v z m n q w j h sum 35 7 I l J l J l 2 I sum J. Acoust. Soc. Am., Vol. 95, No. 5, Pt., May 994 Drullman et al.: Reducng slow temporal modulatons 2678

10 TABLE All. Summed confuson matrces for 2 subjects: Vowels. Stmulus/response HP 32 Hz o a ou e e e o I a o u sum QU e e I o u O I ! I 7 3! I I sum APPENDIX B: RATIONALE OF THE PHASE-LOCKED MTF Let x(t) and y(t) be the temporal ntensty envelopes for nput and output, respectvely, wth spectra X(f ) and For the actual computaton of both the normal MTF Y(f ). The MTF s generally defned as the ampltude and the phase-locked MTFpL (cf. Fgs. 7 and 8), the longspectrum of the frequency response functon H(f ) of a term spectra of the (downsampled} squared ampltude entransmsson channel, that s velopes of an octave-band fltered 7-s speech fragment (vz., 30 concatenated sentences of experment } were ]Y(f)l [ S (f ) / used. The spectra were obtaned by computng the average MTF= I/'/(/ ) l = IX(f ' (B) of 024-pont fast Fourer transforms (0.24-Hz resoluton), usng 50% overlap and Hannng weghtng (Carter where S, (f) and Syy(f) denote the autospectra et al., 973). (autospectral densty functons) of x(t) and y(t) and a a normalzaton factor based on the mean ntenstes of x(t} We nvestgated medal phonemes only. In prevous dentfcaton experand y(t). In ths approach the phase nformaton of the ments wth low-pass envelope flterng (Drullman eta!., 994}, ntal modulaton components s not ncluded. As a consequence, and fnal consonants were also studed. The reason for not consderng ntal and fnal consonants ths tme s that the present sgnal processng modulaton frequences ntroduced by the processng that causes a sudden onset and offset of the syllable, due to the rentroducton are not n phase wth the orgnal envelope are wrongly of the dc component n the hgh-pass fltered envelope. Ths makes the consdered to be transferred, thus underestmatng the dentfcaton of fnal and ntal consonants very dffcult. One could amount of reducton. In order to measure the transfer of only the orgnal (reduced) ntensty modulatons a dfferent frequency response H l(f ) should be computed, usng the cross-spectrum (cross-spectral densty functon) Sxff ) between x(t) andy{t) (Bendat and Persol, 980), Sxy(f ) Cxff }-jqm(f ) H (f ) =a Sxx(f -- S, (f ) (B2) C,(f MTFpL=a S (f )' (B3) solve ths problem by gvng the processed syllable a smooth rse and fall, but t s unclear how that mght nfluence the percepton. Alternatvely, one could add a few hundred mllseconds of low level nose just before and after the unprocessed syllable. But snce ths nose wll be amplfed by the processng, closely attached to both ends of the syllable, one cannot speak of ntal and fnal consonants n ther proper sense any- Equaton (B2) gves the optmum lnear frequency re- Bendat, J. S., and Persol. A. G. (980). Engneerng Applcatons of sponse,.e., the frequency response whch mnmzes the Correlaton and Spectral, nalyss(wley-interscence, New York}. nonlneartes.v(t) wth respecto x(t). On the rght- Boaman, A. J. (989}. "Speech percepton by the hearng mpared," hand sde of Eq. (B2} C ff ) denotes the cospectrum, Ph.D. dssertaton, Unversty of Utrecht. Carter, G. C., Knapp, C. H., and Nuttall, A. H. 0973). "Estmaton of whch s the part where X(f ) and Y(f } are n phase, the magntude-squared coherence functon va overlapped fast Fourer whereas the magnary part Qx (f } (quadspeetrum) s the transform processng," IEEE Trans. Audo E!eetroacoust. 2, part where X(f ) and Y(f } are 9(Y out of phase. So, Drullman, R., Festen, J. M., and Plomp, R. (994). "Effect of temporal assumng that only n-phase modulatons are relevant for envelope smearng on speech receplon," J. Acoust. Soc. Am. 95, speech ntellgblty, the optmum estmate of the modula- Hohmann, V., and Kollmeer, B. (990). "Sprachvers 't:ndlchket be ton transfer wll be gven by the so-called phase-locked Dynamk-kompresson," n Fortschrtte der.4kustk D.4GA 50 ( DPGmodulaton transfer functon MTFpL, defned as Kongreas-GmbH, Bad Honef), pp J. Acoust. Soc. Am., Vol. 95, No. 5, Pt., May 994 Drullman et al.: Reducng slow temporal modulatons 2679 more.

11 Houtgast, T., and Steeneken, H. J. M. (95)."A revew of the MTF concept n room acoustcs and ts use for estmatng speech ntellgblty n audtor " J. Aeoust. Soc. Am. 77, Houtgast, T., and Verhave, J. A. (99}."A physcal approach to speech qualty assesmeat: correlaton patterns n the speech spectrogram," n Pn ceedngs of the 3rd European Conference Speech Communcaton and Technology, edted by G. Pran, Genova, September 99 (IIC, Genova, Italy), Vol., pp Krk, R. E. (968). Expermental Desgn.' Procedures for the Behavoral Scences (Brooks/Cole, Belmont, CA), st ed., Chap. 8, pp Ludvgsen, C., Elberlng, C., Kedser, G., and Poulsen, T. (990). "Predcton of ntellgblty of nonlnearly processed speech," Acta Otolaryngol. Suppl. 469, Plomp, R. (988)."The negatve effect of ampltude compresson n multchannel hearng ads n the lght of the modulaton-transfer functon," J. Acoust. Soc. Am. 83, ?. Plomp, R., and Mrapen, A.M. (979). "Improvng the relablty of testng the Speech Recepton Threshold for sentences," Audology 8, $teeneken, H. J. M. (992)."On measurng and predctng speech ntellgblty," Ph.D. dssertaton, Unversty of Amsterdam. Studebaker, G. A. (985). "A "ratonalzed" aresne transform," J. Speech Hear. Res. 28, Studebaker, G. A., Pavlovc, C. V., and $herbecoe, R. L. (987). "A frequency mportance functon for contnuous dscourse," J. Acoust. Soc. Am. 8, J. Acoust. Soc. Am., Vol. 95, No. 5, Pt., May 994 Drullman et al.: Reducng slow temporal modulatons 2680