Spectral modulation detection and vowel and consonant identification in normal hearing and cochlear implant listeners

Transcription

1 Spectral modulation detection and vowel and consonant identification in normal hearing and cochlear implant listeners Aniket A. Saoji Auditory Research and Development, Advanced Bionics Corporation, San Fernando Rd., Sylmar, CA Contact Information: Name: Aniket Saoji Tel no: Postal Address: San Fernando Rd., Sylmar, CA Leonid Litvak Advanced Bionics Corporation, San Fernando Rd., Sylmar, CA Anthony J. Spahr Department of Speech and Hearing Science, Arizona State University, Tempe, AZ David A. Eddins Department of Otolaryngology University of Rochester 2365 South Clinton Avenue, Suite 200 Rochester, NY International Center for Hearing and Speech Research Rochester Institute of Technology Rochester, NY Contact Information: Name: David A. Eddins Work phone: Work Fax: Postal Address: Department of Otolaryngology University of Rochester 2365 South Clinton Avenue, Suite 200 1

2 Rochester, NY Running title Spectral modulation detection in cochlear implants a) Portions of this data were presented at the 2005 Conference on Implantable Auditory Prosthesis at Asilomar, Pacific Grove, CA. 2

3 Abstract One factor limiting speech understanding in cochlear implant (CI) listeners may be their ability to perceive complex spectral envelopes. In this study, spectral envelope perception was characterized by measuring spectral modulation detection in normalhearing (NH) and CI listeners at spectral modulation frequencies of 0.25, 0.5, 1, and 2 cycles/octave. For CI listeners, vowel and consonant identification was measured. For NH listeners, spectral modulation detection improved with increasing modulation frequency. For CI listeners, spectral modulation declined with increasing modulation frequency. Relative to NH listeners, modulation detection thresholds for CI listeners were poorer for the highest modulation frequency (2.0 cycles/octave) and, except for a few outliers, comparable at the lowest modulation frequency (0.25 cycles/octave). Loudness DLs were estimated for 9 CI listeners. The results were inconsistent with the use of loudness-based cues in the spectral modulation detection task. For the CI listeners, modulation detection thresholds at 0.25 and 0.5 cycles/octave were strongly correlated with vowel and consonant identification scores (r 2 = 0.6 and r 2 = 0.7, respectively). Thus, spectral envelope perception, as measured by spectral modulation detection can be used to relate spectral envelope perception to vowel and consonant identification in cochlear implant listeners. ASA-PACS number: Ky, Es, Fe, Ts 3

4 I. Introduction A cochlear implant (CI) mimics the place coding of the healthy cochlea by mapping audio-frequency from high to low across an array of electrodes extending from the cochlear base towards the apex. This tonotopic mapping and the subsequent perception of place features varies considerably across CI listeners. Such features may be characterized by a pattern of intensity variations across audio-frequency (spectral envelope) corresponding to any given point in time. Since the accurate perception of speech (e.g., Bladon and Lindblom, 1981; Bladon, 1982; Zahorian and Jagharghi, 1993) and other sounds is dependent in part on spectral envelope perception, it is important to characterize spectral envelope processing in CI patients and to determine the degree to which their ability to encode and identify various spectral envelope features is related to speech perception. Spectral envelope perception by CI listeners may depend on several factors, including the front end processing in the CI speech processor, the signal processing scheme, the number of stimulated electrodes (i.e., channels), the nature and degree of channel (electrode) interaction, and the status of the peripheral and central auditory nervous systems. In the present study, the ability to detect sinusoidal spectral modulation as a function of modulation frequency was used to characterize spectral envelope perception in CI relative to normal-hearing (NH) listeners. For CI listeners, the resulting spectral modulation detection thresholds were compared to vowel and consonant identification in quiet to determine the possible relationship between spectral envelope perception and speech perception. Spectral envelope perception is based on the ability to resolve spectral details and the ability to compare those details across cochlear space or the electrode array. The 4

5 resolution of spectral detail has been studied extensively for listeners with acoustic hearing in the context of measures of frequency selectivity (for review, see Moore, 1998). The effect of reduced spectral resolution is a smearing of the spectral detail in the acoustic signal. With respect to CIs, spectral processing has been evaluated using two basic techniques. The first technique involves varying the number of stimulated electrodes for CI listeners or the number of band-pass filter channels in simulations of CIs with NH listeners. The second technique involves estimating the limits of spectral resolution in CI listeners using psychophysical techniques initially applied to NH listeners. To study the effect of reduced spectral resolution, Shannon et al. (1995) developed an acoustic simulation of CI processing that allows the experimenter to systematically vary the number of overlapping spectral bands used to represent the peakto-valley differences (spectral contrast) in the spectral envelope. By mimicking and manipulating the current spread associated with a given channel (or electrode), and the resulting overlapping bands of neural excitation in CI listeners, these CI simulations allow one to indirectly investigate spectral envelope perception thought to be characteristic of CI listeners. The results of acoustic simulations in NH listeners indicate that high levels of speech understanding can be achieved in quiet listening conditions with 4 to 12 spectral bands (Shannon et al., 1995; Turner et al., 1995; Dorman et al., 1997; Fu et al., 1998; Loizou and Poroy, 2001; Friesen et al., 2001). Loizou and Poroy (2001) compared vowel identification in quiet by CI listeners with a six-channel implant to that of NH listeners with a variable number of channels in an acoustic CI simulation. They determined the minimum peak-to-valley difference (spectral contrast) associated 5

6 with formant peaks and adjacent valleys required for vowel identification by NH and CI listeners. CI listeners and NH listeners with 6 to 8 channels in a CI simulation required a 4 to 6 db of spectral contrast for high-level vowel identification performance. NH listeners, however, required progressively less spectral contrast as the number of spectral channels increased from 6 to 12. In noisy situations, the speech perception scores of NH listeners continue to improve as CI simulations include up to as many as 16 (Fu et al., 1998) or 20 (Friesen et al., 2001) spectral channels, whereas the vowel and consonant recognition scores in speech-shaped noise for CI listeners plateau as the number of stimulated electrodes exceeded 7 to 8 electrodes. In general, acoustic simulations of CI processing indicate that relatively broad spectral patterns encoded by a fewer number of spectral bands can lead to relatively high levels of speech understanding in quiet listening situations, whereas in noisy backgrounds, listeners may depend on finer spectral details that can only be represented by a greater number of spectral bands. Rather than gauging spectral resolution from changes in performance as a function of the number of active electrodes (spectral bands), one may also measure spectral resolution directly by evaluating the ability to resolve spectral details across the electrode array. Supin et al. (1994) described a ripple phase-reversal test to measure spectral resolution and Henry et al. (2005) used a modified version of this technique to measure spectral resolution in NH, HI (hearing-impaired), and CI listeners. The ripple stimulus consists of a sinusoidal spectral modulation, characterized by successive intensity peaks and valleys distributed across audio frequency on a logarithmic frequency axis, that is applied to a broadband carrier. Henry et al. (2005) estimated spectral resolution by determining the ability of listeners to discriminate between two spectrally 6

7 modulated noise stimuli having the same spectral modulation frequency (cycles/octave) and modulation depth (peak-to-valley difference in db) but differing in modulation phase by π radians such that the peaks of the stimulus in one interval corresponded to the valleys of the stimulus in the next interval. The modulation depth was fixed at approximately 30 db and the modulation frequency was adaptively varied to determine the phase reversal threshold or the minimum discriminable spectral density in cycles/octave. The average phase-reversal threshold was 4.8 cycles/octave for NH listeners, corresponding to peaks spaced approximately 0.2 octaves apart, 1.8 cycles/octave for HI listeners, and 0.6 cycles/octave for CI listeners. The decline in spectral resolution across groups was negatively correlated with vowel (r 2 = -0.64) and consonant (r 2 = -0.66) identification. They concluded that the spectral resolution required for accurate vowel and consonant identification in quiet corresponds to phase-reversal thresholds of about 4 cycles/octave and that phase reversal thresholds poorer than 1 to 2 cycles/octave may result in highly degraded speech recognition. Henry and Turner (2003) reported that the ability to resolve spectral peaks and valleys by CI listeners improves with an increasing number of stimulated electrodes up to 4 to 6, beyond which performance plateaus. However, an acoustic simulation of the CI with normal-hearing listeners resulted in a gradual improvement in spectral resolution as the number of spectral channels increased from 1 to 16. These results are consistent with the fact that spectral resolution in CI listeners is likely reduced due to channel interaction that is not adequately modeled in the acoustic simulations, leading to broader excitation in the CI listeners as compared to listeners in the simulated CI conditions. - Insert Figure 1 about here 7

8 As described by Supin et al. (1994) and implemented by Henry et al. (2005), the ripple phase-reversal task, like the widely used notch-noise masking paradigm (e.g., Patterson et al., 1982), focuses on the failure to resolve spectral features and the resulting index of frequency resolving power is associated with estimates of auditory filter bandwidth. A single phase-reversal threshold provides an index of spectral resolution but does not capture the ability to combine and compare information across such channels. To fully assess spectral envelope perception, more comprehensive measures are required, such as extending the phase reversal task to a range of modulation depths or frequencies following the approach of Supin et al. (1999). A more common technique, however, has employed a task in which a noise modulated with a sinusoidal spectral envelope on a loglog scale, reflecting the compressive and tonotopic nature of the auditory system, is discriminated from an unmodulated (flat) spectral envelope. For a given condition, modulation frequency is fixed and modulation depth is varied adaptively to determine a modulation detection threshold (e.g., Bernstein and Green, 1987; Green et al. 1987; Hillier, 1991, Summers and Leek, 1994; Amagai et al., 1999; Eddins and Bero, 2007, Saoji and Eddins, 2007). To limit the utility of local intensity changes from interval to interval, and encourage a more global envelope comparison, several investigators have randomly varied the overall level (Green et al. 1987; Hillier, 1991, Summers and Leek, 1994; Amagai et al., 1999; Eddins and Bero, 2007) and/or the modulation phase (Eddins and Bero, 2007; Saoji and Eddins, 2007) from interval to interval. Expression of modulation detection or phase reversal threshold as a function of modulation frequency results in a spectral modulation transfer function (SMTF), similar in concept to a temporal modulation transfer function (TMTF; e.g., Viemeister, 1979). For comparison, 8

9 Figure 1 displays data obtained using the modulation detection (Eddins et al., open squares; NH listeners, current study filled circles) and phase reversal (Supin et al., open circles; Henry et al., filled triangle) tasks. Although the measurement parameters differed among the studies shown, a clear pattern emerges when thresholds are plotted in terms of spectral contrast at threshold as a function of modulation frequency. The resulting bowl-shaped SMTF reflects the limits of spectral resolution, intensity resolution, and across-channel spectral shape perception (i.e.., Eddins and Bero, 2007). The initial decrease in sensitivity with increasing modulation frequency shown indicates that NH listeners are less sensitive to broadly-spaced changes in the spectral envelope than more closely spaced changes in the spectral envelope. As shown by Eddins and Bero (2007) and Saoji and Eddins (2007), the limits of spectral resolution cannot account for this variation in sensitivity to relatively low spectral modulation frequencies. As the spectral modulation frequency continues to increase, the sensitivity to modulation is limited by the spectral resolution (i.e., frequency selectivity) of the auditory system in NH listeners. HI listeners require greater spectral modulation depth to achieve modulation detection threshold than do normal-hearing listeners (Summers and Leek, 1994), particularly at high spectral modulation frequencies (Eddins et al., 2006). These results reflect the reduction in spectral resolution in HI relative to NH listeners and are generally consistent with estimates of spectral shape perception obtained in NH and HI listeners by measuring the detection of an level increment restricted to a local frequency region against a broadband background (e.g., Lentz and Leek, 2002; 2003; Shrivastav et al., 2006). The work of Henry and colleagues indicates that spectral resolution is generally poorer in CI than NH or HI listeners. Furthermore, it is known that speech 9

10 perception declines with decreasing number of active electrodes in CI listeners (e.g, Fishman et al., 1997; Friesen et al., 2001) as well as with decreasing number of channels in CI simulations with NH listeners (e.g., Shannon et al., 1995; Fu et al., 1998). These changes are analogous to declines in speech perception as a function of the degree of spectral smearing in NH listeners (e.g., Baer and Moore, 1993; ter Keurs et al., 1992, 1993; Turner et al., 1999), all of which may be modeled as a reduction in spectral cues. Based on these results, one can speculate that SMTF should differ in CI and NH listeners. Furthermore, because reduced spectral modulation reflects a reduction in spectral contrast, and spectral contrast is related to vowel and consonant identification, there may be a relationship between differences in the SMTF and differences in vowel and consonant identification by NH and CI listeners. The current study reports the first measurements of spectral modulation detection in CI listeners. One goal was to investigate potential differences in spectral envelope perception by NH and CI listeners by measuring spectral modulation detection as a function of spectral modulation frequency. A second goal was to identify any relationship between spectral modulation detection and vowel and consonant identification in CI listeners. Ultimately, if the spectral modulation detection technique proves informative, such measures may be included in routine evaluation of CI listener performance with their everyday processors. Therefore, both spectral modulation detection and vowel and consonant identification were measured in CI listeners using their existing signal processing schema. No attempt was made to modify or normalize CI processing across listeners beyond the commonalities associated with their HiResolution strategy. 10

11 The reader should note that a companion study conducted subsequent to the collection of the data reported here was recently published by Litvak et al. (2007) while the current manuscript was under peer review. The focus of that study was to determine the accuracy with which the spectral modulation detection and speech perception performance reported here for CI listeners could be modeled by NH subjects listening to noise vocoded stimuli following parametric manipulation of the vocoder parameters. The data for NH listeners reported by Litvak et al. represented a novel data set, independent of the current study in terms of listeners and specific listening tasks. They compare their novel vocoder simulation data from NH listeners to a subset of data originally reported in the present study. II. Methods A. Subjects CI subjects included 25 post-lingually deafened adults, ranging in age from 38 to 65 years, who were using either the Advanced Bionics Clarion CII or the Advanced Bionics HiRes/90k cochlear implant. All had at least six months experience using their current HiRes implant strategy with either paired or sequential stimulation as their everyday program. Table 1 includes relevant details about each CI subject. - Insert Table 1 about here Six normal-hearing subjects ranging in age from 22 to 26 years participated in these experiments. Each subject had normal pure-tone thresholds (<20 db HL at octave frequencies from 250 to 8000 Hz; ANSI, 1996) and normal screening tympanograms (Y, 226 Hz; compensated static admittance between 0.3 and 1.7 mmho; Margolis and 11

12 Goycoolea, 1993). All implant volunteers were paid for their participation. Normal hearing subjects were affiliated with the laboratory and were not paid for their participation. This study was approved both by a private IRB board associated with Advanced Bionics and the institutional review board at Arizona State University and all participants provided their informed consent. B. Stimuli Stimuli for the spectral modulation task were generated using Matlab. Stimulus generation involved first creating the desired spectral shape and then applying that shape to a noise carrier. The desired spectral shape took the form expressed by the following equation: F ( f ) = 10 m sin 2 ( 2π ( log ( f / 350) ) fm+ ) 2 θ 0 / < f < 5600 otherwise (1) where F(f) is the amplitude of the bin with frequency f (Hz), m is the spectral modulation depth in db, f m is the spectral modulation frequency (0.25, 0.5, 1.0, or 2.0 cycles/octave), and θ 0 is the modulation starting phase selected at random from a uniform distribution (0 and 2π radians) for each interval. The low frequency edge of the audio spectrum was 350 Hz, corresponding to the corner frequency of the filter mapped to the most apical electrode according to the Advanced Bionics HiRes program. Although some patients may have used an Extended Low frequency filter with a corner frequency of 250 Hz, the nominal bandwidth remained 350 to 5600 Hz. Each stimulus was scaled to have an 12

13 equivalent overall level of 60 db SPL and each stimulus had a duration of 400 ms. The digital to analog conversion rate was Hz, yielding a bandwidth of 2.5 Hz. An inverse Fourier transform on the complex buffer pair resulted in a noise band with the desired spectral shape. Figure 2 shows a schematic representation of spectral modulation frequencies corresponding to 0.25, 0.5, 1.0, and 2.0 cycles/octave with a modulation depth of 20 db with a noise carrier from 350 to 5600 Hz. Flat spectrum noise was synthesized by setting the modulation depth m = 0. - Insert Figure 2 about here - Speech perception was assessed using vowel and consonant identification tasks. Vowel stimuli consisted of the 13 vowels created with a Klatt synthesizer (Klatt, 1980) in /bvt/ format as described by Dorman et al. (1989) and included the following tokens: bait, bart, bat, beet, bert, bet, bit, bite, boat, boot, bought, bout, but. Each vowel token was of equal duration (90 ms) to eliminate changes in vowel duration as a cue to vowel identity (Dorman et al., 1989). The consonant identification task was based on the Iowa Consonant Test published on the Iowa laser video disk (Tyler et al., 1986) and included were 16 consonant stimuli spoken by an adult male talker in the /aca/ context. Tokens included: aba, ada, afa, aga, aja, aka, ala, ama, ana, apa, asa, asha, ata, atha, ava, aza. C. CI Speech Processor Input-Output For acoustic or electrical inputs, the analog-to-digital (A/D) converter used in the Advanced Bionics body-worn speech processor is designed to take inputs from 20 to approximately 100 db SPL corresponding to a wide dynamic range of 80 db SPL. The input from the A/D converter is fed to a pre-emphasis filter which has a passband that is 13

14 approximately the inverse of the average speech spectrum over the range of 350 to 5500 Hz. The outputs are further processed through a single-channel adaptive gain control (AGC). However, for signals used in this study, which have levels at or below 60 db SPL, the AGC results in linear gain. After the AGC, the signal is passed through a set of 16 bandpass filters which are approximately logarithmically spaced from 350 to 5500 Hz. The last filter is a high-pass filter and extends to approximately 8.7 khz. A half-wave rectified, low-passed output of each channel is used to determine the electrode outputs in accordance with the mapping function which is adjusted to each patient. -Insert Figure 3 about here - Figure 3 shows the CI speech processor output corresponding to the sixteen electrodes for the spectral modulations of 0.25, 0.5, 1.0, and 2.0 cycles/octave with a spectral contrast of 20 db and starting phases of 0 or π/2 radians. The y-axis is the average envelope value for each electrode (db SPL) to the mapping function. The peakto-valley differences in the modulation spectra are faithfully represented for the modulation frequencies of 0.25 and 0.5 cycles/octave. There is a slight decrease in the spectral contrast at the modulation frequency of 1 cycles/octave. At the modulation frequency of 2 cycles/octave the spectral contrast obtained at the output of the speech processor is strongly phase dependant. For the modulation frequency of 2 cycles/octave and a starting phase of 0 radian the spectral contrast is reduced but still preserved at the output of the speech processor whereas for the starting phase of π/2 radians the modulation spectra is represented as an unmodulated spectrum at the output of the speech processor. The high frequency emphasis obtained in the modulation spectra at the output 14

15 of the speech processor is attributed to the pre-emphasis filter in the front end and due to the wider filter width in the high frequency region in the signal processing scheme. It should be noted that the speech processor output is further modified by the threshold (T), most comfortable level (M), and the input dynamic range (IDR) settings specific to each patient. Importantly, a high-frequency tone at approximately 60 db SPL will be mapped to a current level corresponding to M. The intensity level is mapped logarithmically between the T and M levels as a function of the IDR. The range of IDR varies between 20 to 80 in the HiResolution program. The IDR can influence the peak-tovalley differences or spectral contrast encoded in the cochlear implant electrode mapper, with a lower IDR representing greater peak-to-valley differences as compared to higher IDR. However, the peak-to-valley differences encoded in a CI listeners would also depend on patients dynamic range and their T and M levels. Also the overall level of the modulation spectra would be determined by the volume control settings on the speech processor. Thus there are patient specific parameters that will influence the peak-tovalley differences encoded in the electrode mapper for the CI users. D. Procedure Test environment Normal hearing listeners were tested in double-walled sound treated room. Digital stimuli were routed to a soundcard (M-Audio, Audiophile 2496) and following D/A conversion were presented to the right ear via Sennheiser HD 25-SP1 circumaural headphones. All stimuli were presented at 60 db SPL as calibrated using an artificial ear housing a microphone coupled to a preamplifier and sound level meter. 15

16 The CI listeners were seated in a quiet room. Digital stimuli were routed to a soundcard (M-Audio, Audiophile 2496) and following D/A conversion were fed to the body worn Platinum Series Processor (PSP) through the Advanced Bionics direct-connect system. Sound card output was attenuated using an in-line attenuator such that for all stimuli, the electric input to the DirectConnect system was equivalent to a 60 db SPL acoustic input to the microphone of the speech processor. The CI listeners used their everyday program in all test conditions and were asked to adjust the volume only at the beginning of the testing session so that the stimuli were presented at comfortable level. The test time excluding breaks was about 2 to 3 hours for each subject. Spectral modulation detection Spectral modulation detection thresholds were obtained using a cued, three interval, two-alternative, forced choice procedure with a three-down, one-up adaptive tracking rule that estimates 79.4 percent correct detection. Prior to data collection, subjects were familiarized with each new condition by the presentation of a few sample trials. On each trial, the three observation intervals were separated by 400-ms silent intervals. In the first interval, the standard stimulus was always presented. This cue or reminder interval is helpful in cases where listeners can hear a difference between the signal and the standard stimulus but cannot identify which is which. The signal and the second standard were presented in the remaining two intervals in random order. The NH and CI subjects were given feedback after each trial. For each 60-trial adaptive track, modulation depth was varied in a step size of 2 db for the first three reversals in the adaptive track and then in 0.5 db steps for the remainder of the track. Threshold was 16

17 based on the average modulation depth corresponding to the last even number of reversals, excluding the first three. The final threshold for each modulation frequency was defined as the average of three 60-trial tracks. Twelve thresholds were obtained for the spectral modulation frequencies of 0.25, 0.5, 1.0 and 2.0 cycles/octave in a quasirandom manner. Vowel and Consonant identification Speech perception was evaluated using vowel and consonant identification tasks. Vowel identification performance was measured in two blocks of 78 trials that included six repetitions of each of the 13 vowels in random order. Therefore, vowel identification and the resulting confusion matrix for each subjects was based on a total of 12 presentations of each vowel stimulus. Consonant identification performance was measured in two blocks of 80 trials in which each of the 16 consonants was presented 5 times in random order. Consonant identification and the resulting confusion matrix for each subject were based on a total of 10 presentations per consonant stimulus. During testing, subjects were seated in front of a computer monitor that displayed the response alternatives (13 or 16) as text boxes labeled with the English word corresponding to each stimulus in either /bvd/ or /aca/ context. Subjects responded by using a computer mouse to click on the button corresponding to their response choice. Prior to data collection, listeners were familiarized with the stimuli and the task during a practice session involving both passive and active listening. In the passive listening task, each vowel was presented twice while the text representation of the token was visually displayed on the computer screen. The listeners were not required to provide any 17

18 response or judgment. In the active listening task, listeners completed two repetitions of the actual test procedure in which feedback regarding the correct response was provided by highlighting the correct response text box. Vowel and consonant stimuli were presented at a nominal overall level of 60 db SPL. The actual level of the stimuli was randomly varied over a range of 3 db (± 1.5 db) in 0.5 db steps to overcome slight variations in loudness associated with the recorded stimuli despite having been equated in terms of RMS level. Loudness comparison procedure Although the digital amplitude of the unmodulated and modulated stimuli used in the spectral modulation detection task were scaled to equivalent levels, the complex signal processing algorithm of the CI as well as any differences in the growth of loudness across individual electrodes could potentially introduce loudness differences between the spectrally modulated signal and the unmodulated (flat spectrum) standard stimulus. Following data collection on the spectral modulation detection task, a subset of 9 CI listeners completed a loudness comparison task designed to evaluate whether or not an overall loudness cue was available to CI listeners. Using a two-interval, forced-choice procedure and a method of constant stimulus, listeners heard a series of trials in which the spectrally modulated signal and the flat-spectrum standard were randomly assigned to the two observation intervals on each trial. Listeners responded by indicating which interval they judged to be the louder of the two. Signal stimuli were presented at the same level as in the main experiment (60 db SPL) at a spectral modulation depth of 5 db above the subject s highest spectral modulation detection threshold for a given modulation 18

19 frequency (i.e., 5 db sensation level) 1. The level of the unmodulated standard was randomly chosen from a set of 11 possible levels ranging from 55 to 65 db SPL in 1 db steps. Each of the 11 comparisons was repeated 10 times for a total of 110 trials per subject at each of three spectral modulation frequencies (0.25, 0.5, and 1 cycles/octave). No difference in loudness between the signal and standard stimuli should result in a psychometric function in which the 50% point corresponds to a standard level near the reference level of 60 db SPL. Deviations from 60 db SPL greater than the typical intensity discrimination threshold for relatively broadband stimuli would indicate a potential loudness cue. III. Results For clarity of exposition, this section is divided into the following sub-sections: 1) spectral modulation detection, 2) vowel and consonant identification, and 3) loudness comparisons. Spectral modulation detection Spectral modulation detection thresholds are plotted as a function of spectral modulation frequency in Figure 4 for the six NH listeners, yielding individual SMTFs. The error bars indicate the standard error of the mean. The pattern of thresholds as a function of spectral modulation frequency differed substantially across listeners. For two listeners, the pattern was relatively flat (open symbols), while for three listeners the pattern had a negative slope (filled symbols). For the sixth listener (filled diamonds), the pattern was somewhat irregular, marked by substantial variability across the three 19

20 threshold estimates at 0.5 cycles/octave. The average thresholds across the normal hearing listeners (dashed line) shows a decrease in sensitivity to spectral modulation with increasing spectral modulation frequency. The variability across listeners was greatest for the modulation frequencies of 0.25 and 0.5 cycles/octave and ranged from 3.7 to 11.4 db. The spectral modulation detection thresholds obtained for the modulation frequencies of 1.0 and 2.0 cycles/octave showed less variability across listeners, with thresholds ranging between 2.4 to 6.1 db. A one-way repeated measures of analysis of variance (RMANOVA) indicated a significant effect of modulation frequency (F 3, 23 = 3.588, p = 0.039). Multiple-pairwise comparisons revealed a statistically significant (p < 0.05) difference between the thresholds obtained at the modulation frequencies 0.25 and 2.0 cycles/octave and no other combinations. - Insert Figure 4 about here The SMTFs obtained for the twenty-five CI listeners are shown in Figure 5 using the same format in Figure 4, although the y-axis range has been increased to accommodate the data. The overall pattern of SMTFs obtained for CI listeners indicates that modulation detection thresholds are generally lowest at 0.25 cycles/octave and progressively increase with increasing modulation frequency. For four CI listeners, modulation detection thresholds could not be obtained at 2.0 cycles/octave given a maximum possible modulation depth of 60 db. Modulation detection thresholds at 0.25 cycles/octave varied between 2.2 to 25.1 db across CI subjects with a median of 5.6 db. With the exception of three subjects, the thresholds for the modulation frequency of 0.25 cycles/octave varied between 2.2 to 9.5 db with a median of 5.0 db. The range of modulation detection thresholds at 0.5 cycles/octave was 2.3 to 15.6 with the exception of 20

21 one subject who had a threshold of 25.1 db. For the modulation frequencies of 1.0 and 2.0 cycles/octave, the thresholds obtained in CI listeners varied from 4.0 to 42.7 db and from 10.2 to > 60.0 db, respectively. A one-way RMANOVA indicated a significant effect of modulation frequency (F 3, 95 = 64.96, p < 0.001). Multiple pairwise comparisons revealed statistically significant differences (p < 0.05) among all modulation frequency combinations except 0.25 and 0.5 cycles/octave, which were not significantly different from each other. - Insert Figure 5 about here The SMTFs for the CI subjects contrast with those of the NH listeners (Figure 4) in two ways: the data for the CI listeners span a much larger range of threshold values and the form of the average SMTF for CI listeners has a positive slope rather than the negative slope. Despite these differences, the modulation detection thresholds obtained at 0.25 cycles/octave are fairly similar both in terms of the average threshold across listener groups and the range of thresholds within listener groups. The shaded region of Figure 5 shows the 95% confidence interval (mean ±2 s.d.) of the data from the NH listeners. Clearly the ability to detect spectral modulation for frequencies above 0.5 cycles/octave was substantially poorer in the CI than the normal-hearing listeners. Vowel and consonant identification There was considerable variability in the vowel and consonant identification scores across the 25 CI listeners, with performance ranging from 16 to 96% for vowel identification and 28 to 92% for consonant identification. To determine whether or not a systematic relationship exists between spectral modulation detection thresholds and 21

22 vowel or consonant identification, single- and multiple-factor linear regression analyses were undertaken. As, shown in Table 2, all single factor correlations were statistically significant. - Insert Table 2 about here Performance on both vowel and consonant identification task was negatively correlated with spectral modulation detection thresholds, consistent with the hypothesis that better spectral envelope perception, associated with lower spectral modulation detection thresholds, is related to better speech identification, associated with higher percent correct identification. Overall, correlations were higher for consonant than vowel identification and were highest for 0.5 cycles/octave, where r = (r 2 = 0.56) for vowel identification and r = (r 2 = 0.67) for consonant identification. Thus, spectral modulation detection thresholds at 0.5 cycles/octave accounts for approximately 60% of the variance associated with vowel identification in CI listeners and 70% of the variance associated with consonant identification in CI listeners. Scatter plots of the results are shown in Figure 6, where the average spectral modulation detection threshold at 0.5 cycles/octave is displayed on the abscissa and the vowel (left panel) and consonant (right panel) identification scores are displayed on the ordinate. In an attempt to better establish the relation between combinations of thresholds obtained at different spectral modulation detection thresholds and vowel and consonant identification, multi-factor forward stepwise linear regression analyses were performed. For vowel identification, threshold at 0.25 cycles/octave was the strongest predictor (r = 0.77, r 2 = 0.60), and the addition of threshold at 1.0 cycles/octave accounted for an additional 8.8 percent of the variance (r = 0.83, r 2 = 0.68). For consonant identification, 22

23 the modulation frequency of 1.0 cycles/octave was sole predictor present in the final regression equation, with r = 0.69, r 2 = Note that this factor accounted for less variance in the multiple (48%) than the simple (67%) linear regression due to missing data for four listeners in the 2.0 cycle/octave condition. Excluding the data for 2.0 cycles/octave, the final linear regression equation included 0.5 cycles/octave as the sole predictor, with r = 0.82 and r 2 = Insert Figure 6 about here - Insert Table 3 about here Loudness comparisons To better determine whether or not an overall loudness cue may have been available to or used by the CI listeners in the spectral modulation detection task, 9 (filled symbols, Figure 5) of the original 25 CI listeners performed a loudness comparison task in which the loudness of a flat-spectrum standard was compared to the loudness of a spectrally modulated signal. With the exception of three outliers in the original group of 25, the spectral modulation detection thresholds for the sub-group of 9 listeners are comparable to the larger group. The spectral modulation depth was set at 5 db above the highest spectral modulation detection threshold for each listener at each modulation frequency (Table 3). A graph showing the percentage of trials in which the standard was judged to be louder than the signal as a function of the standard level yields a psychometric function. Psychometric functions for each of the three spectral modulation frequencies, averaged across the nine listeners, are shown in Figure 7. The resulting psychometric functions are sigmoidal in shape, indicating that when the level of the standard was well below the level of the signal stimulus (e.g., 54 db SPL), CI listeners 23

24 systematically judged the standard to be softer than the signal. Conversely, when the level of the standard was well above the signal stimulus, listeners systematically judged the standard to be louder than the signal. For each individual psychometric function at each modulation frequency, the best fitting logistic function (in the least squares sense) was computed. The point corresponding to 50% correct averaged across all (27 functions) was 59.0 db SPL. One possibility is that this shift, relative to the nominal level of 60 db SPL, is due to a small but consistent loudness cue associated with spectral modulation. The 50% point is termed the point of subjective equality or PSE and is shown by the dashed line nearest the lower left corner of Figure 7. The point on the psychometric function corresponding to 75% correct is often referred to as the threshold, in this case for louder judgments (Gelfand, 1998, p 251). Since the spectral modulation detection thresholds corresponded to 79.4% correct, this point will be taken as threshold, and the difference between the 50% and 79% points on the psychometric function may be taken as the difference limen (DL) for the attribute under study (e.g., loudness or intensity). For the present data set, the loudness DL corresponds to 2.3 db ( = 2.3 db) averaged across conditions and modulation frequency. Given that the horizontal shift in the psychometric function is less than half the value of the loudness DL, it is unlikely that that shift reflects a usable loudness cue. The PSE, threshold, and resulting DL values are shown graphically in Figure 7 by the dotted lines. - Insert Figure 7 about here An alternative explanation for the shift in PSE is that the presence of a strong pitch and/or timbre attribute associated with supra-threshold spectral modulation was actually interpreted as a change in loudness. Although the subjects were instructed to 24

25 ignore the pitch/timbre differences and focus on overall loudness, several CI listeners did report that the pitch/timbre differences would sometimes interfere with their loudness judgment. Overall, given that 1) the deviation in loudness comparisons from the expectation of a PSE at 60 db SPL is less than half of the estimated loudness DL; 2) the modulated stimuli used in the loudness comparison procedure were presented at suprathreshold rather than threshold levels (where potential loudness cues presumably would be less effective); and 3) that potential pitch/timbre cues also were available, there does not appear to be clear evidence that the CI subjects relied on overall loudness differences between the flat spectrum standard and the spectrally modulated signal to perform the spectral modulation detection task, although this possibility cannot be ruled out with certainty. To further evaluate the potential relationship between performance on the loudness comparison task to either spectral modulation detection or speech identification, a series of correlations were computed based on the data from the nine listeners on each task. These analyses included the raw spectral modulation detection thresholds at 0.25, 0.5, and 1.0 cycles/octave, the averaged threshold for 0.25 and 0.5 cycles/octave, vowel identification scores, consonant identification scores, and the slope and intercept estimated from the psychometric functions for the loudness judgments at 0.25, 0.5, and 1.0 cycles/octave. None of the correlations between slope or intercept values from the fitted loudness functions and spectral modulation detection thresholds were significant (p > 0.05) nor were the correlations between slope or intercept values and vowel or consonant identification significant (p > 0.05). Thus, there does not appear to be a 25

26 systematic relation between the loudness judgments based on the presence or absence of spectral modulation and the detection of spectral modulation or speech identification. IV. Discussion In the present study, spectral modulation detection was measured in a group of NH listeners and a group of CI listeners for spectral modulation frequencies from 0.25 to 2.0 cycles/octave. For NH listeners, average spectral modulation detection thresholds decreased with increasing spectral modulation frequency over the range of 0.25 to 2.0 cycles/octave. The SMTFs obtained in this study are similar in pattern to spectral modulation detection thresholds obtained in previous studies over the same modulation frequency range (e.g., Leek et al., 1987; Bernstein and Green, 1987; Chi et al., 1999; Eddins and Bero, 2007; Saoji and Eddins, 2007). Although spectral modulation detection thresholds were similar for the NH and CI listeners at 0.25 and 0.5 cycles/octave, modulation detection worsened with further increases in spectral modulation frequency most CI listeners while it improved for most normal-hearing listeners, resulting in dramatically different spectral modulation detection thresholds for the CI than the HI listeners at the highest modulation frequencies. Thresholds at 1.0 and 2.0 cycles/octave also were marked by substantial variability across CI listeners. Indeed, a cursory inspection of the SMTFs in Figures 5 reveals CI listeners with similar thresholds at 1.0 and/or 2.0 cycles/octave could differ substantially in their ability to detect modulation frequencies at 0.25 and/or 0.5 cycles/octave. The limited spectral resolution reported in CI listeners (e.g., Henry et al., 2005) is in many ways analogous to the spectral smearing produced by broadened peripheral auditory filters in HI listeners (ter Keurs et al., 1992, 1993; Baer and Moore, 1993, 26

27 Turner et al., 1999). Broadened auditory filters will decrease the effective peak-to-valley differences or modulation depth in the internal representation of the spectral envelope. To compensate for such smearing, greater modulation depth is required in order to detect the presence of spectral envelope features. This is consistent with the notion that HI listeners needed greater modulation depth to detect spectral modulation at the higher spectral modulation frequencies (e.g., Summers and Leek, 1994). The results reported for HI listeners are similar to the elevated spectral modulation detection thresholds obtained here for CI listeners, especially for the spectral modulation frequencies of 1 and 2 cycles/octave (e.g., Eddins et al., 2006). To better capture the variability in the SMTFs obtained for CI listeners, threshold functions were fit (in the least squares sense) with an exponential function with two parameters (the of the exponent b and the y intercept A) as shown in Eq 2. (2) where f(x) represents modulation depth at threshold corresponding the modulation frequency x. Changes in the exponent b results in contraction or dilation along the x axis, altering the steepness of the function, whereas changes in the scalar A results in contraction or dilation along the y axis, changing the y intercept. Building on the logic detailed in the Introduction, changes in spectral resolution should differentially affect high relative to low modulation frequencies and in isolation would reflect an increase in the value of the exponent with little change in the y-intercept. Changes in across-channel intensity comparisons should differentially affect low relative to high modulation frequencies and in isolation would result in an increase in the y-intercept value and a decrease in the value of the exponent. A simple change in intensity resolution should 27

28 result in modulation frequency independent changes in threshold and thus would result in an increase in the y-intercept and little change in the value of the exponent. The modulation detection thresholds obtained for individual CI listeners were fit with the function defined in Eq. 2. Exponential fits to the SMTFs for the 25 CI listeners accounted for 93% of the threshold variance on average and are shown as solid curves for the individual listeners in Figure 8. All fitted functions in Figure 8 have a positive exponent as opposed to the negative exponent obtained for each of the NH listeners (not shown). Four representative patterns relative to NH listeners are observed. The first includes no change in the y-intercept and a modest change in the exponent, including a change in sign (e.g., CI10), likely reflecting a modest change in spectral resolution and no change in across-channel intensity comparison and no change in intensity resolution. The second pattern includes a modest change in both the y-intercept and exponent (e.g., CI20), likely reflecting both a change in spectral resolution and across-channel intensity comparison. The third pattern includes a modest change in the y-intercept and a large change in the exponent (e.g., CI6), reflecting larger changes in spectral resolution than pattern three. The fourth pattern includes large changes in both parameters (e.g., CI4). While no cases reflect a simple change in intensity resolution, which would be characterized by increase in the y-intercept and no change in the exponent relative to the NH listeners (i.e., a negative exponent), pattern four could also be explained by combined changes in intensity resolution, resulting in a vertical shift, and spectral resolution, resulting in an increased exponential value. A fifth possibility is that changes in spectral resolution are so dramatic that they dominate performance at all spectral modulation frequencies in which performance is worse for CI than NH listeners. However, it is 28

29 unlikely that changes in spectral resolution alone could account for the dramatically different changes in SMTF relative to normal that are observed for the listeners in CI17 and CI13, both of which show large changes at 2.0 cycles/octave whereas one shows little change and the other shows great change at 0.25 cycles/octave. Several factors contribute to the limited spectral resolution of CI listeners. For example, the electrical stimulation provided by one electrode may interact with the electrical stimulation provided by another electrode, and such interaction may vary in terms of the degree of interaction and the linearity of interaction among the various electrodes (e.g., Boex et al., 2003; Cohen et al., 2003; Abbas et al., 2004; Eddington et al., 1997). Additionally, spectral detail is limited by the number of available electrodes. With respect to spectral modulation, the impact of this chronic under sampling increases as the modulation frequency increases. For example, assuming that the tonotopic mapping of frequency to place by electrodes strictly follows a logarithmic scale, a modulation frequency of 2 cycles/octave represented by a carrier spanning four octaves would have 8 peaks and 8 valleys in the spectral envelope. Such an envelope could be represented by sixteen electrodes if the starting phase of modulation results in either a spectral peak or a spectral valley corresponding to the first electrode. Any other modulation phase would reduce the modulation depth and phases of 0 or π radians would result in a flat spectral envelope. Such undersampling is clearly a problem in representing spectral envelopes with high-modulation frequencies, as shown in Figure 3 in terms of CI processor output. In the present study, because the modulation phase was randomly selected from trial to trial, the modulation depth for the higher modulation frequencies was dependent in part on the modulation phase. Thus, the representation of 29