Common principles of across-channel processing in monaural versus binaural hearing

Transcription

1 Carl von Ossietzky Universität Oldenburg Studiengang Diplom-Physik DIPLOMARBEIT Titel: Common principles of across-channel processing in monaural versus binaural hearing vorgelegt von: Tobias Piechowiak Betreuender Gutachter: Prof. Dr. Dr. Birger Kollmeier Zweiter Gutachter: Prof. Dr. Torsten Dau Oldenburg,

2 1 CONTENTS 1 General Introduction 2 2 Psychoacoustics The Auditory Pathway Absolut and masked thresholds Binaural masking level difference Comodulation masking melease Auditory streaming Effects of auditory grouping in binaural masking Introduction Method Procedure Stimuli Subjects Results Discussion Modeling comodulation masking release Introduction The modeling concept The Equalization - Cancellation theory The modulation-filterbank model An EC model for CMR Model predictions Method Procedure Stimuli Subjects Results Discussion Summary and Conclusion 49

3 2 1 General Introduction The auditory sytem provides us with uncountable information about our surroundings that are crucial for our everday life as there are e.g. the ringing of a phone, the determination of the direction of an approaching car or understanding speech which represents information of another human beeing. There are limits to our capability to perceive sounds as the fact that one sound can be masked by another one which means that one sound s detectability can be decreased by the presence of another sound with similar spectal, spatial and timing properties. However, several phenomena have been investigated that facilitate the perception of masked sound as there are the binaural phenomenon called binaural masking level difference (BMLD) and comodulation masking release (CMR). This thesis investigates possible common principles of the mechanisms underlying binaural masking level difference (BMLD) and comodulation masking release (CMR) in two different approaches. In the first chapter, an introduction to the theoretical background of the topics of this thesis is presented. In chapter 2, the influence of auditory grouping on the amount of BMLD is measured and compared with the influence of grouping on CMR, investigated earlier in several studies e.g. Dau et al. (23). The sensitivity of BMLD and CMR to grouping paradigms could be a common property of both binaural and monaural processing of sounds and since it is assumed that auditory grouping is originating at central stages of processing it could give a hint that both BMLD and CMR occur at least partly at a higer processing stage and that they may underly similar processes. In the last chapter the nature of this possible common process is more specified by introducing a model to predict CMR on the basis of an equalizationcancellation (EC) model. This model was first mentioned to account for BMLD data and an EC approach was also used to develop a more general model that can accoount for a large variety of binaural detection data. Several exeriments are performed by human subjects as well as by the processing model as an ideal observer in a three-interval 3-alternatve choice (3-IFC) procedure to validate the performance of the EC-CMR model.

4 3 2 Psychoacoustics 2.1 The Auditory Pathway The ear is divided into three components: the outer, the middle and the inner ear. The auditory periphery transforms sound waves into neural signals. Three characteristics of this transformation are: 1. preserving the temporal structure of the acoustic signal 2. encoding changes in (rather than absolute) stimulus intensity 3. performing amplitude compression to encode stimulus intensities over 7-orders of magnitude 4. filtering frequencies onto the basilar membrane Figure 1: The Human Ear Figure 1 shows a crossection of a human ear. The pinna which is part of the outer ear bundles sound into the ear canal which travels down the ear canal in the from of compression pressure waves. These pressure waves induce vibration of the eardrum, or tympanic membrane, and the various structures attached to it. The ossicles transmit the vibrations through the middle ear to a membrane - covered opening of the cochlea which is called the oval window. The middle ear s main purpose is an impendence matching making sure that most of the

5 4 sound energy is transmitted into the cochlea rather than beeing reflected back from the oval window. The cochlea is divided into three canals which are the scala tympani, scala media and scala vestibuli. The Basilar Membrane separates the scala media from the scala vestibuli (Figure 2). Figure 2: Crosssection of the human cochlear The two scalae, which communicate at the apex of the cochlear through the helicotrema, are filled with a medium, called perilymph. A pressure change at the oval window is transmitted to the perilymph which causes then a deflection of the basilar membrane what means the basilar membrane. The basilar membrane starts to oscillate when sound reaches the oval window. The response of the basilar membrane to a sound stimulation begins at the base with a wave whose amplitude increases while continuing along the basliar membrane in form of a travelling wave towards the apex (Figure 3). Due to the decreasing stiffness of the basilar membrane down to the apical end the traveling wave reaches its maximum amplitude at different places onto the basilar membrane for different frequencies. The envelope of the travelling waves for different frequencies is shown in Figure 4. This means the basilar membrane performs a frequency selection, where higher sound frequencies are mapped towards the basal end and the lower frequencies towards the apical end. The lowest frequency that causes a deflection at the apex is 2 Hz and the highest frequency 2 Hz just at the entrance of the oval window. Other frequencies between these extremes are mapped in between the apex and the base. This mapping is called tonotopic mapping. So in effect the ear is behaving like a fourier analyser, altough not with a perfect frequency analysing power. In 194 Fletcher introduced the critical

6 5 Figure 3: Longitudinal view of the unrolled cochlear Figure 4: Envelope of the travelling wave onto the basilar membrane for three different frequencies band concept formalizing the function of the basilar membrane by a number of bandpass filters with overlapping centre frequencies, extracting the energy of sounds of different frequencies. The shape of these filters has then been determined by several experiments as e.g. Vogten(1974) or Patterson(1976). Patterson used notched noises to determine filter shape. The width of the notch is varied and then the threshold of a test signal with frequency centred at the middle of the notch is measured as a function of the notch width. Since

7 6 the noise energy that is mapped into a certain filter for a certain notch width is determined by the filter shape, measuring the treshold of the test signal delivers a way to obtain the filter shape. The movement of the basilar membrane finally is transduced to the Organ of Corti where the transformation from mechanical vibration into receptor potentials takes place. 2.2 Absolut and masked thresholds Sound is defined as density variations of an compressible medium like air. Humans can detect sound from a minimum pressure p denoted as the absolute threshold. Absolute thresholds are frequency dependent and the measurement of absolute thresholds for different frequencies is called audiogram. Humans are most sensitive to sound at a frequency of 1Hz. The sound pressure p serves as the reference pressure for the logarithmic decibel scale for sound pressure level used in acoustic sciences. ΛdB = 2 log 1 p p (1) The pressure p is set in relation to the minimum audible pressure p which has a value of 1 5 bar. It is everyday experience that sound (e.g. from the radio in the car) may be masked and become inaudible, in the presence of another sound (e.g. car s engine). Masking is formally defined by the American Standards Association as the process by which the threshold of audibility for one sound is raised by the presence of another sound. The threshold at which the one sound is becoming detectable again is called Masked Threshold. Measuring masked thresholds for different frequencies is called masked audiogram. Physilogically this can be explained by looking on the dynamic of the basilar membrane. The basilar membrane is, depending on the frequency and bandwidth of a stimuli, exited at a certain place by a tone of a certain frequency (tonotopic mapping). If a second sound event now takes place in the same frequency group it also contributes to the overal deflection of the basilar membrane at the same place and makes it generally more difficult for the auditory system to distinguish these both events. Summarized in one sentence one could say that a target sound is masked more effective by a masking sound the more similar they are in terms of timing, spectral content and spatial properties. Due to the tonotopic mapping masked thresholds are frequency dependent as

8 7 Figure 5: Masked thresholds of a pure tone as a function of frequeny. As masker serves a 9Hz wide white noise at a centre frequency of 41 Hz. Each curve shows the masked thresholds for a masker level of 2, 3, 4, 5, 6, 7 and 8 db. it is shown in Figure 5 for different masker level. A 9 Hz wide noise as the masker centred at 41 Hz is presented for different sound pressure levels. Each curve shows the pure-tone thresholds as a function of the tone frequency. As it can be seen the tones are less masked the more they are spectrally separated from the masker and the lower the masker level is. masked thresholds are measured in many psychoacoustical experiments because they can e.g. deliver information about resolution limits of auditory perception. They are very valuable to analyse effects that lead to enhenced detectability of masked sounds. Two well known effects of this kind that have been investigated in several experiments are Binaural Masing Level Differences (BMLD) and comodulation masking release (CMR) which will be explained in the following in more detail. 2.3 Binaural masking level difference The masked threshold of a signal can sometimes be lower when listening with two ears rather than one; this is demonstrated by the phenomenon of the binaural masking level differences (BMLD). This effect is illustrated in Figure 6. Noises with the identical noise waveforms are presented to both ears by headphones. A tone with the same frequency and starting phase is added to the

9 8 a) b) c) d) Figure 6: Four situations of binaural hearing. In conditions b) and d) the detectability of the tone in noise is better than in conditions a) and c) illustrated by the smile on the faces. Presented at at least one ear is a noise masker (lower panel, blue curve) and a signal (upper panel, red curve) noises. This is called a diotic hearing situation because the stimuli that reaches the both ears are identical as shown in condition a). The level of the tone is now adjusted just until it is masked by the present noise reaching its masked threshold denoted as M db. When an interaural phase shift now is applied on the tone, as it is shown in condition b), it now can be detected down to a level of M db, the newly obtained masked threshold. The difference M M db is a masking level difference because the level decreases relatively to the normal signal threshold. A similar increase in detectability occurs when comparing condtion c) with condition d). White noise is presented to only one ear in condition c), a tone is also added at one ear above threshold and its level again decreased until the masked threshold is reached. By adding the same noise at the other ear the tone is becoming more easy to detect again. Condition d)is, like b), a condition of diotic hearing meaning that the stimuli at the both ears are not the same. The BMLD can be summarised as follows; the detection of a signal in noise is improved when either the phase or level differences of the signal at the two ears are not the same as the masker. An implication of this is that the signal

10 9 and masker appear to originate from different locations in space; hence, the BMLD appears to be related to the well-known cocktail party effect. At this point it makes sense to introduce some terminology to understand different stimuli configurations. In general, we can describe a particular stimulus using the symbols S (for signal) and N (for noise), each followed by a suffix indicating the relative phase in the two ears; o for same phase (so-called homophasic) and π for a 18 degree (pi radians) phase difference (so-called antiphasic). For example, N o S o means that the noise and signal both have the same phase in each ear, and N o S π means that the noise has the same phase, but the signal is 18 degrees out of phase. Nu means that the noise is uncorrelated in the two ears, and the suffix m indicates monaural presentation (i.e. presentation to one ear only). Refering to this notation condition a) in Figure 6 would be denoted as N o S o, b) as N o S π, c) as N m S m and d) as N o S m. BMLD are in general larger for low frequencies, but level differences depend also on noise basndwidth and stimulus configuration. BMLD have been investigated in many studies e.g. by Schooneveldt and Moore (1987) or van de Par (1998). Several models have been suggested to account for BMLD. The two most important are the Webster - Jefress model (1951) and Durlach s equalization and cancellation model (1963). None of these models account for all aspects of the experimental data but they are succesfull in explaining partial aspects of BMLD. Durlach s approach will be used in chapter 4 to describe effects of across-channel processing in monaural hearing such as comodulation masking release. 2.4 Comodulation masking melease Release from masking is not only restricted to binaural listening but occurs also for monaural listening conditions. When a tone is masked by a narrow band of noise with a certain center frequency, the amount of masking can be decreased by simultaneously presenting a flanking band of noise at another center frequency than the signal and the on-frequency masker. A release of masking can be observed if the flanking band and the signal band are comodulated which means that both bands of noise have the same envelope fluctuation patterns. This is illustrated in Figure 7 for two comodulated noises at 5 Hz and 1 Hz center frequency. This release from masking has been denoted comodulation masking release (CMR) and was first observed by Hall and Haggard (1983)and Hall et al.

11 1 Amplitude Time [ms] Amplitude Time [ms] Figure 7: Comodulated 1 Hz wide gaussian noises with 5 Hz Center frequency(left panel) and 1 Hz Center frequency(right panel) (1984). One must distinguish between within channel CMR where CMR is based on information within only one auditory channel (Verhey and Dau 1996, 1997; Schooneveldt and Moore 1989) and the across channel CMR which is based on the assumption that information across two or more auditory channel is compared (Buus 1985; Hall 1986). Across channe CMR can be measured independently using narrow bands of noise spectrally widely separated or by presenting the flanking band at the ear opposite to the masker and signal ear (Schooneveldt and Moore 1987). The modulation filterbank model which was introduced by Dau et al.(1997) to describe modulation detection and modulation masking experiments can account for CMR data which has been measured in classical banwidening experiments where the masked threshold of a signal masked by a noise is measured as a function of the noise bandwidth. The measurement is performed for a totally random (R) noise where the amplitude fluctuations are not correlated to each other at different frequencies in comparison to a comodulated (M) noise where the envelopes of the noise at different frequencies are partly correlated. The difference in signal threshold for this both conditions is denoted CMR. Data for such an experiment is shown in Figure 8 and one can see that CMR adds up to 12 db at large bandwidths. The model uses only the information in one auditory channel and this information is the decrease in the effective modulation depth. Adding a signal to the modulated noise will smooth out the envelope of a signal which leads to a decrease in modulation depth, which is detected by the model in a way presumed to be similar to that in human listeners. In this aspect the modulation filterbank is an appropriate model to account for within channel CMR. The model, however, does not account for true across-channel CMR. The goal of this thesis is to develop a generaliyed model of signal processing that explains both the within and across-channel CMR.

12 11 Signal Threshold [db] Masked Threshold [khz] Figure 8: Classical bandwidening experiment by Fletcher (194). Signal threshold is measured as a function of masker bandwidth for a random noise (R) and a comodulated noise (M). The difference in thrshold defines the amount of CMR 2.5 Auditory streaming As demonstrated first by Hall and Grose (1993) comodulation across the frequency bands in CMR experiments is just a necessary prerequisite but not a sufficient one to observe CMR. It is a crucial point for the occurence of CMR that the center band of noise as well as the flanking bands are presented synchronously i.e. they are required to be grouped together over frequency. Such an effect also exists in the temporal domain and is called stream segregation or concurrent streaming. In the everyday life the human auditory system has the task of analysing complex sounds and divide them into perceptual components to make the recognition of different sound sources possible e.g. following the speech of the same speaker or distinguish between different speakers. The process where sound elements are separated into different auditory objects is known as auditory stream segregation (Figure 9), and, conversely, the process where different sound elements are assigned to a single object is known as auditory stream integration (Figure 1). The tones presented at different frequencies shown in figure 9 are heard in a single rythm they are considered as belonging to the same stream of tones, they are grouped together in time. In contrast the tones presented as illustrated in figure 1 are perceived as two separate sequences. The difference of the tones

13 12 Figure 9: An example for auditory stream integration. A sequence of two tones A and B with different frequencies is presented. The tones are heard as a single rythm (illustrated by the dotted line) Figure 1: An example for auditory stream segregation. A sequence of two tones A and B with different frequencies is presented. The tones are heard as two separate sequences (illustrated by the dotted lines) in this case are larger than in the case of the auditory stream integration. This observation was first made by van Noorden (1977). Later Bregman (1978) made the suggestion that each stream could be regarded as one auditory object and that sounds are more likely to belong to separate streams if they are separated widely in frequency. Auditory streaming also seems to have an impact on the magnitude of CMR (Hall and Grose 1993; Dau et al. 23). In the investigation of Dau et all. CMR for four flanking bands separated by one octave vanished completely when a number of noise intervals with the same level and duration interrupted by a time gap preceeded at the flanking bands. These intervals were called post - cursors. The explanation could be that each flanking band is now grouped temporally together to one stream and one auditory object destroying the comparision of the same amplitude fluctuations across frequency. This also strongly suggests that CMR is a process that may take at relatively central processing stages since some kind of feedback mechanism seems invloved that decreases CMR when the post - cursors are presented. They affect the across frequency comparison of bands that were presented in time before them

14 13 and to gather the whole auditory stream it takes at least the time for presenting the totality of noise intervals that means the signal band plus the post - cursor. In the next chapter the influence of auditory grouping on BMLD will be investigated for specific stmuli configurations.

15 3 Effects of auditory grouping in binaural masking 3.1 Introduction The major cues used in spatial hearing of sounds are interaural time differences (ITDs) and interaural intensity differences (IIDs). Since the occurrence and the amount of BMLD rely on the analysis of interaural differences stimuli properites that affect IIDs and ITDs lead to different thresholds in binaural detection experiments. Such properties are, for example the bandwidth and the statistical properties of the masker and the signal frequency. (van de Par, 1998, chaps. 5 and 6). Regarding the classical experiments using broadband noise maskers one can observe a difference in the release from masking for low signal frequencies compared to high signal frequencies. At low signal frequencies the BMLD can be as large as 1 to 15 db depending on the interaural relation of the masker and the signal. At high frequencies, however, the BMLD reduces to a few db (Durlach 1963). The explanation is that due to the lowpass filtering caused by the haircell transduction the fine structure information is lost at high frequencies and the binaural processor has to rely only on the envelope IIDs and ITDs to gain a certain amount of binaural release from masking while at low frequencies also the fine structure ITDs and IIDs contribute to binaural detection and they provide very strong cues. The factor which is responsible for the decrease of the BMLD for broadband noises at high signal frequencies is the increase of the auditory filter bandwidth towards higher frequencies. The broadening of the filter shape allows the envelope at the output of the auditory filters at high frequencies to fluctuate faster than at the output at low frequencies because the highest possible beating frequency ǫ is determined by the difference ǫ = f max f min between the lowest f min and the highest envelope frequency f max, passing through the certain filter. That means also that the ITDs and IIDs change faster at higher signal frequencies. Several experiments (Grantham, 1984) have shown that the binaural system can not follow fast fluctuations in these parameters and therefor binaural unmasking decreases at high frequencies. The only way to obtain a considerable amount of BMLD at high frequencies (> 1.5 khz) is to use narrow bands of noise as the masker. Narrow band noises as masker have been used to measure high-frequency BMLD and the BMLD was as high as 16 db for the N o S π condition (van de Par, 1998) as defined 14

16 15 in section 6. When narrow noise masker of 25 Hz bandwidth are used they contain inherent amplitude fluctuations that do not exceed 25 Hz. This means that the envelope fluctuates very slow which could make it easier for subjects to detect a comodulation across frequency bands. Although it is possible to measure a considerable amount of BMLD at high frequencies by using narrow bands of noise, the BMLD for narrow band noises at low signal frequencies is in general still higher, probably due to the loss of fine structure information that does not contribute to BMLD at high frequencies. Thus it can be assumed that high frequency BMLD use envelope correlation between the channels (ears) similar as it is in across-channel CMR between the output of different auditory filter in the same ear. The CMR data in the study of Dau et al. (23) showed that when using narrow band noises as masker in monaural unmasking experiments CMR vanishes in auditory streaming conditions that promote the perceptual segregation of the signal band with the flanker band(s). Segregation is commonly associated with higher-level processes (Hall and Grose, 1993) and hence across-channel CMR may be hypothesized to reflect a higher-level rather than peripheral process. This is in contrast to several physiologicall studies that have assumed that CMR can be associated with relatively peripheral processing (Pressnitzer, 21). Assuming that across-channel CMR, underlies effects of auditory grouping the question is whether this could be a common principle for across-channel processing of sound in more general terms? Specifically, is high-frequency BMLD also affected by auditory grouping meaning that it also would be associated with a higher level of processing? The present study adresses these issues by investigating how auditory streaming affects BMLD at low and high frequencies and how BMLD depends on signal duration. The goal is to explore similarities, common principles and fundamental differences in the processing strategies in BMLD versus CMR. 3.2 Method Procedure An adaptive signal-level adjustment 3-interval forced choice procedure was used to measure the masked threshold of a signal in the test interval. The three intervals (one test interval and two reference intervals) were separated by pauses of 3 ms. The duration of the interval length depended on the special stimuli configurations which will be described specifically in the section

17 16 A tone signal of 187 ms duration, in this case a tone, was added to one of the intervals. In an other experiment the duration of the signal was changed as a parameter within 45 and 3 ms for a special stimulus configuration. The task for the subjects now was to decide which of the intervals contained the signal. The signal level was adjusted according to a two-down one-up rule(levitt, 1971). The initial step size was 8 db and after every second reversal of the level adjustment the step size was halved until the step size of 1 db was obtained. From the level of the last 6 reversals the mean was calculated and regarded as the masked threshold value. For each stimuli configuration and subject 4 masked threshold values were measured. The mean of these values were calculated and taken as the final threshold. The experiments were performed in a sound attenuating booth Stimuli All stimuli were generated digitally by a computer that used MATLAB as signal generator and then converted to analog signals by the soundcard of the computer equipted with a 16-bit D/A converter at a sampling rate of 32 khz. The masker as well as the tone were Hanning-windowed with 2 ms raised ramps. The stimuli then were presented to the subjects over Sennheiser HD58 headphones. Each masking band had an overall sound pressure level of 6 db sound pressure level (SPL) and the same duration as the signal so that both the signal and noise masker had a synchronous onset. Gaussian noises with 25 Hz frequency bandwidth served as masker throughout all experiments and they were generated by taking gaussian noise from a digital noise generator and then converting them with a bandpass filter of rectangular shape to the desired bandwidth. BMLD was measured at a signal frequency of 5 Hz for the high-frequency BMLD and at 5 Hz signal frequency for the low-frequency BMLD for four configurations as illustrated in figure 11. In the figure the noise masker are represented by the patterned retangular boxes and the signals are presented by the solid horizontal lines contained within the boxes. The amplitude of the envelope fluctuations of the narrow band noise masker is grayscale-coded that means the darker parts of the patterned boxes are associated with higher envelope amplitudes. The purpose of presenting the involved masker in this way is that it can easily be seen what the interaural relation of the masker are. The single band condition serves as the reference from which the release of binaural masking is calculated in comparison to the following conditions. In the single band condition the masker is

18 17 Left Ear Right Ear Frequency Frequency Frequency Single Band Standard Postcursor (PC) Frequency Off-frequency Postcursor (OffPC) Time Time Figure 11: Four stimuli configurations that are used to investigate the influence of auditory grouping on high-frequency BMLD. The patterned boxes represents the masker whereas the test signal is shown as colored line within the masker. Red color line means the signal starts with a degree phase shift whereas the blue color line represents a 18 degree phase shift of the signal. The single band condition serves as the reference from which the BMLD are calculated presented only to the left ear. The standard condition refers to the N o S π condtion explained in section 2.3. A tone is added at both ears to the same masker but with 18 degree phase shift which is illustrated by the different color of the solid lines contained in the boxes. The reason for choosing this BMLD condition is that by adding the tones out of phase at both ears a very similar envelope fluctuation pattern is achieved. A subtraction of this both patterns would lead to considerable increase in the signal-to-noise ratio and thus to a higher amount of BMLD compared, for exmple, to the N o S m condtion. In this way possible changes in masking level differences caused by auditory grouping

19 18 might also have a larger dynamic range. An additional argument is given by the fact that in section 4 similar stimuli are used to measure across-channel CMR and so a direct comparison between the masking of sounds in monaural and binaural hearing is possible easily. The condition which might show an effect of auditory streaming on highfrequency BMLD is the condition in the third row of Figure 11 refered to as the postcursor condition. This denotation is due to the fact that three noise intervals with the same bandwith, duration and statistical properites (gaussian noise), but uncorrolated amongst each other, are presented after the on-frequency band at the same frequency. This was done only on one side the right ear. only to the right ear. The time gaps between the single noise bands were 67.5 ms. The left ear still receives the same stimuli as in the previous condition. These four noise intervals following the on-frequency band are called the postcursors. The masker configuartion was chosen in this special way for the purpose of creating an auditory object consisting of the on-frequency band plus the postcursors that may disturb a potential comparison of the envelope fluctuations between the on-frequency bands at both ears and which normally leads to a strong BMLD. The off-frequency postcursor condition was set up to verify that any release from BMLD in the postcursor condition is really due to auditory streaming and is not because the subjects are confused by the presence of the postcursor. The experiment was performed for low and for high frequencies. In the high-frequency condition the signal had a frequency of 5 Hz whereas the postcursor in the corresponding off-frequency condition had a frequency of 2815 Hz. In the low frequency condition the signal had a frequency of 5 Hz and the postcursor a frequency of 281 Hz in the corresponding off-frequency condition. The Postcursor and Off-frequency Postcursor condition were motivated by the effects of auditory stream integration and auditory stream segregation by van Noorden (1977) which were explained in more detail in section 2.5. The new idea here is to not only see these effects as a perceptual phenomenon but also to quantify these effects for specific masking configurations that have so far not been associated with streaming and central processing. In a second BMLD experiment the influence of stimuli duration on the BMLD was investigated. The single band and standard(n o S π ) condition were measured for 45, 9, 187 and 3 ms signal duration and the BMLD was calculated as the difference of their thresholds. The bandwidth was again 25 Hz. This experiment was only performed for high frequencies that means at 5 Hz

20 19 signal frequency. Furthermore a monaural listening experiment was performed to measure across channel CMR in an equivalent configuration as the standard BMLD (N o S π ) configuration. This was done in order to directly compare BMLD and CMR. This experimental configuration is shown in Figure 12. Left Ear Right Ear Frequency Frequency Single Band Standard Time Time Figure 12: Two stimuli configurations that are used to measure CMR in an monaural hearing experiment. The patterned boxes represents the masker whereas the test signal is shown as colored line within the masker. Red color line means the signal starts with a degree phase shift whereas the blue color line represents a 18 degree phase shift of the signal. The single band condition serves as the reference from which the BMLD are calculated The reference is the single band conditon as described above for the binaural experiment. The signal frequency was 5 Hz. In the standard condition a second signal at 2815 Hz and a 25 Hz wide gaussian noise masker centered at this frequency were added with synchronous onset as the masker at 5 Hz. The signal had a 18 degree phase shift compared to the signal at 5 Hz. The maskers were comodulated which was achieved as follows: A broadband noise at 5 Hz was generated by the noise generator. By applying a bandpass filter a bandwidth of 25 Hz was obtained. This noise was now transformed to the frequency domain by Fast-Fourier-Transformation. All frequency components were then shifted down 2185 Hz which reflects the difference frequency between the signal frequencies at 2815 Hz and 5 Hz. The inverse-fast-fouriertransformation was applied on these shifted frequencies components and the real part was separated from the result of that operation which results in a noise that has the same envelope as the noise at 5 Hz. This means both masker are comodulated. This condition is refered to as the CMR N o S π condition due to the comparable properties of the stimuli to the standard BMLD (N o S π )

21 2 condition. The CMR was measured for stimulus durations of 9, 187, 3, 6 ms as the difference between the thresholds for the single band condition and the CMR N o S π condition Subjects Three normal-hearing subjects participated in the auditory grouping experiment and four normal hearing subjects in the experiments that measured the influence of signal duration on BMLD and CMR. Subjects varied in age from 25 to 39 and had many hours of listening experience in BMLD and CMR tasks prior to the collection of data in the experiments Results The results for the auditory grouping experiments in BMLD for the three subjects are shown in Figure 13 for high frequencies and in Figure 14 for low frequencies. Results for the three subjects are represented by different symbols. measurement points of different shape as there are squares, diamonds and circles. The three conditions Standard(STA), Postcursor(PC) and Off-frequency Postcursor(OffPC) are indicated on the x-axis and the obtained BMLD is given on the y-axis. In addition the right panels show the differences in BMLD for the Postcursor condition (PC) compared to the standard condition and the Off-frequency Postcursor (OffPC) condition compared to the Standard condition i.e. the release from BMLD caused by the introduction of the postcursors. The filled squares present the mean release from the BMLD across subjects. What is very apparent for the high-frequency BMLD is that it shows a considerable variability across subjetcs in the amount of overall BMLD. The mean standard BMLD is about 13 db. One subject (diamonds) is showing very low BMLD for the standard condition and this amount is totally merged by presenting the on-frequency postcursor whereas the other two subjects show a similar amount of BMLD in the three conditions. The amount of BMLD for these two subjects is not completely eliminated but clearly reduced by the postcursor. As it can be seen in the right panel of Figure 13 the BMLD is on average reduced by the on-frequency postcursor (condition PC) by 7 db in the mean. There is no difference in the amount of the BMLD in the Off-frequency condition compared to the standard condition across subjects. In the case of low-frequency BMLD shown in Figure 14 it can be seen that the variability across subjects is reduced when compared to the high-frequency situation. The subjects also show an increased amount of BMLD in the stan-

22 21 BMLD (db) STA PC OffPC Condition Release from BMLD (db) PC OffPC Condition Figure 13: The left panel shows the high-frequency BMLD (signal frequency 5 Hz) for three subjetcs and three conditions (STA, PC, OffPC) whereas in the right panel the relase from unmasking for three subjects by auditory stream segregation for two conditions (PC, OffPC) is shown BMLD (db) STA PC OffPC Condition Release from BMLD (db) PC OffPC Condition Figure 14: The left panel shows the low-frequency BMLD (signal frequency 5 Hz) for three subjetcs and three conditions (STA, PC, OffPC) whereas in the right panel the relase from unmasking for three subjects by auditory stream segregation for two conditions (PC, OffPC) is shown dard condition which is in average about 22 db. There is again no difference in the amount of the BMLD when the Off-frequncy and the standard condition is compared. However the influence of the postcursor on the amount of the BMLD is reduced as can be seen in the right panel of figure 14. From 7 db release of BMLD in the high-frequency condition the release in the low frequency condition is only 4 db. Now the influence of signal duration of time has been investigated. Figure 15 shows the amount of BMLD measured for four different signal durations for the standard condtion (STA).

23 22 Mean BMLD (db) Signal duration (ms) Signal duration (ms) Figure 15: Standard BMLD(N o S π ) for different signal durations for four different subjects (left panel) and their mean values (right panel) Mean 1 1 CMR (db) Signal duration (ms) Signal duration (ms) Figure 16: CMR (N o S π ) condition for different signal durations for four subjects (left panel) and their mean values (right panel) As it can be seen in the left panel of Figure 15 and in the standard deviation of the mean BMLD in the right panel the variability of the BMLD across subjects is large. Interesstingly, there are two subject groups that show different behavior. One group that show higher BMLD the amount of BMLD increases with increasing signal duration whereas the group with smaller BMLD seems not to be affected by changes in signal duration. Another observation is that still a reasonable amount of BMLD (8 db) can be measured for the shortest signal duration of 45 ms. The situation is different when considering the effect of stimulus duration in the CMR N o S π shown in Figure 16. The first observation is that the variability across subjects is smaller than in BMLD. Second, no subject shows a reasonable amount of CMR up to 6 ms where CMR is as large as 7 db on

24 23 average, seen in the right panel of the Figure. One subject even shows a slighly negative amount of CMR for 187 and 3 ms signal duration that indicates reduced detactability when the comodulated band is presented in addition to the signal band. 3.3 Discussion The central question of this section is whether BMLD at high and low frequencies is sensitive to auditory streaming. The answer to this question could give a hint if envelope information is compared across the ears on a higher rather than peripheral processing level because this comparison does not take place isolated from the processes that give rise to auditory object formation and which are commonly associated with central processing. It is obvious from the results that high-frequency BMLD is affected by auditory streaming whereas low-frequency BMLD is clearly less affected by streaming. This suggests a more hard wired process probably more based on fine structure comparison that is accounting for low-frequency BMLD and which is independent of the acoustical context that is presented. Another interessting observation is that BMLD does not completely vanish in the Postcursor (PC) condition as it was the case for across channel CMR in the study of Dau et al. (23). One could assume that only a part of the observed BMLD (around 7 db) in the standard condition is due to envelope comparison across ears and can be merged by auditory grouping whereas another part could be due to lateralization effects indicating the perceived location of a sound source within the head (Moore,1983). This assumption is explained in the following. When measuring the standard BMLD with the 3-IFC procedure the sound in the reference intervals seems to be located in the center of the head because the listening condition is diotic. In the test interval where the listening condition is dichotic the signal that is presented interaurally out of phase causes the impression that the sound is running from one ear to the other within the head. This is a detection cue that will be denoted as the diffusion cue in the following. This cue which is already detectable at the onset of the sound could deliver an explanation that even for a short signal duration of 45 ms this leads to a considerable amount of BMLD of about 11 db for two subjects as illustrated in Figure 15. The additional amount of BMLD that is occuring up to 3 ms signal duration compared to the amount at 45 ms signal duration is an additional hint that more centrally based processes are involved in the detection because it needs a longer time to occur and longer time constants are

25 24 usually associated with a higher level of processing in the brain. This amount gets as large as about 6 db for the group of subjects that is sensitive to an increase in signal duration on the amount of measured BMLD and this amount could be identical with the part of the high-frequeny BMLD that is merged by auditory streaming. Another obvious conclusion that can be made from figure 15 is that the cues for high-frequency BMLD detection are not as easy detectable across subjects as the cues for low-frequency BMLD whatever they may be in detail. When it comes to the comparison of common principles in the processes underlying high-frequency BMLD and CMR one needs to consider the investigation of Dau et al. (23) where an across channel CMR of 6 db was measured for four flanking bands separated by one octave. This CMR could be completely eliminated by presenting three postcursor at each flanking band. The comparable amount of release from BMLD/CMR as well as the fact that the same masker are used in both experiments suggests that a certain part of highfrequency BMLD that uses envelope comparsion as well as across channel CMR could underlie similar machanisms. When considering at the figures 15 and 16 it can be seen that the cues that allow detection of the signal seem available at very different time scales. The standard condition delivers about 11 db of BMLD for a signal duration of 187 ms on average whereas only a very small amount of CMR (2 db) is measured for the same duration. Due to this lack of CMR the potential influence of auditoy streaming on the CMR N o S π condition was not further investigated here. For this condition with only one flanking band the results differ clearly from the amount of CMR measured in van de Par and Kohlrausch (1998) where for the CMR N o S π condition a release of 11 db was obtained for a signal duration of 3 ms. Van de Par and Kohlrausch (1998) found a similar amount of high-frequency BMLD (15 db for 3 ms signal duration) using the same masker properites as here. The reason for the apparent discrepancies between the present study and the study by van de Par and Kohlrausch (1998) are unclear. Based on the results from the present study when considering the difference in time scales it is questionable to suggest that high-frequency BMLD and CMR underlie the same meachnism. The general principle of across-channel processing, especially an EC-type process, might still be similar. However, the data suggest that, if so, they occur at a different level of processing.

26 25 4 Modeling comodulation masking release 4.1 Introduction Since the first measurements of CMR (Hall and Haggard, 1983; Hall itet al., 1984) it has been proposed that it results from across-channel comparisons of temporal envelopes. The term channel is refered to as a peripheral filter of the auditory filterbank that represents the activity of the basilar membrane. However it has been stated that one needs to distinguish between two different classes of CMR experiments depending on the type of masker used (Verhey et al. 1999, Schooneveldt and Moore, 1987). The first class of experiments is represented by the bandwidening paradigm (Fletcher, 194) where a single band of noise is centered at the frequency of the signal and signal threshold os measured as a function of the noise masker bandwidth. What can be observed is that there are in general lower thresholds when using a masker that is partly comodulated across frequency bands, typically a broadband noise multiplied by a lowpass filtered noise, in comparison to the case where the envelope fluctuations of the masker show no correlation between the auditory filter bands that are involved. This difference in the detection of thresholds is denoted within -channel CMR because this amount of CMR can already be explained in terms of changes in the activity due to the addition of the signal within one critical band the so called within-channel cue (Schooneveldt and Moore, 1989). It was argued that changes in the statistics of the envelope produced by adding the signal to the on-frequency band, which causes a decrease in modulation depth in the noises and can be detected by humans, leads to a threshold reduction even for noise bandwidth smaller than the critical band. Since the within-channel CMR is related to modulation detection the single channel modulation filterbank model introduced by Dau et al. (1997a) to model modulation detection and masking phenomen appears to provide an apropriate model to quantify the amount of CMR in the bandwidening experiment (Verhey, Dau and Kollmeier, 1999). In the second class of experimental paradigms in CMR, the masker consists of two (or more) narrow band noise bands, one at the signal frequency (onfrequency band) and one or more bands (the so called flanking bands) separated from the on-frequency band which are presented synchronously. Adding one comodulated band to the on-frequency band can lead to a release of masking of a sinusoidal signal up to 8 db (Hall et al., 1984). This release is decreasing with increasing spectral distance of the on-frequency and flanking bands

27 26 and amounts to 2-6 db at separations above one octave (Moore, 1992). This release is named across-channel or true CMR because no cues in only one channel can anymore account for it and it is assumed that this really is a consequence of a comparison across different auditory filters. Several mechanism have been proposed to account for the across-channel CMR. Two of this mechanisms are proposals of Buus (1985) from which the first one is a more detailed analysis of what can be the comparison of across-frequency modulation patterns and is based on the equalization-cancellation model for the binaural masking level difference (BMLD), see section 2.3. The envelope of a masker at the output of an auditory filter centered at the flanking bands is subtracted from the envelope plus signal at the output of the auditory filter centered at the on-frequency band leading to a noise reduction at the onfrequency band. The second idea is that subjectcs listen in the valleys of a masker at different frequency bands meaning they detect when the envelope passes through zero or in other words when the masker has a relativ low energy. This occurs much less often when the signal is added to the on-frequency masker smoothing out its envelope. It is assumed now that an across-frequency comparison tells the subjects when the masker at the flanking bands is passing zero determining the best moment to detect a signal at the on-frequency band. In energy terms the listening in the valleys mechanism is just an energy detector with a short integration time that indicates the optimal time to listen for the signal, namely, during the valleys of the masker when the signal-tomasker ratio is highest. Another group of mechanisms that has been proposed is based on crosscorrelation calculations between two frequency bands. Richards (1987) assumed that the cross-covariance between the envelopes of the masker and the flanking band are used for detecting a signal beacause the envelope crosscovariance decreases when adding a signal to the masker and this cue could be used by the human auditory system. However this model was in contradiction with data by Eddins and Wright (1994) and was rejected because it predicted CMR in a CMR paradigm where 1% sinusoidally amplitude modulated sinusoids were used as on-frequency and flanking bands. The task was to detect the addition of a sinusoid to one of the bands. It can be shown by calculation that the cross-covariance does not changed when the signal is added to the masker. Thus if changes in the cross-covariance were essential for receiving CMR, SAM stimuli should not lead to a release of masking. However the experiments clearly showed that this were the case for this kind of stimuli and so the envelope cross-covariance idea was given up as an explanation for