A biologically inspired model adding binaural aspects to soundscape analysis

Transcription

1 A bologcally nspred model addng bnaural aspects to soundscape analyss Mchel Boes, Bert De Coensel 2, Damano Oldon 3, and Dc Botteldooren 4 Acoustcs group, Department of Informaton Technology (ITEC, Ghent Unversty Snt-Petersneuwstraat 4, 9000 Gent, Belgum ABSTRACT Bnaural hearng s an essental property of the human audtory system. It s provdng the bran wth vtal nformaton about the locaton of sounds and ths nformaton contrbutes to the process of audtory stream segregaton. Thus, models for the computatonal analyss of soundscapes would beneft from the addton of a bnaural component. In ths paper, a bologcally nspred bnaural analyss model s presented. The neural pathways n the human bran calculatng nteraural tme dfference (ITD and nteraural ntensty dfference (IID, two essental bnaural cues, are modelled. Consequently, a neural map s used to ln these ITD-senstve and IID-senstve neurons to neurons representng a specfc drecton, n order to obtan an estmate of sound source drecton. Ths map turns out to be both a bologcally plausble and an elegant way to combne IID and ITD nformaton. The model s shown to ncorporate learnng and adaptaton n localzaton as well as several other features of human bnaural hearng. In addton, an analyss s presented of the model's effectveness n detectng sound source drectons, also n the presence of dstractng sounds or bacground nose. Keywords: bnaural hearng, sound localzaton. ITRODUCTIO To structure the audtory envronment, the human audtory system tres to determne what produced a perceved sound and where ths sound source s localzed. These are essental slls n human audtory scene analyss [-2]. In partcular, sound localzaton nformaton s consdered to be very mportant for the human bran n order to separate sound sources n a multple-source acoustc envronment (the so-called Coctal Party Problem [3]. Thus, also for computatonal audtory scene analyss, a sound localzaton component can provde mportant nformaton. In ths paper, a bologcally nspred bnaural analyss model s presented. In secton 2, a short overvew of current nowledge on sound source localzaton n the human bran s gven. The proposed model for computatonal sound source localzaton, nspred on the human system, s then presented n secton 3. In secton 4, the model s appled to a selecton of test cases. In secton 5, our conclusons are presented. 2. HUMA SOUD SOURCE LOCALIZATIO For sound source localzaton n the horzontal plane, the human bran maes use of two mportant cues: Interaural Tme Dfference (ITD and Interaural Intensty Dfference (IID. ITD s caused by the addtonal tme needed for the sound wave to reach the ear furthest to the sound source, whl e IID s caused by the shadng effect of the head [-4]. Whle these cues are very mportant n horzontal localzaton, they are vrtually useless n vertcal localzaton. In order to acheve vertcal localzaton, the bran maes use of monaural, spectral cues, caused by the dependence on the sound s drecton of mchel.boes@ntec.ugent.be 2 bert.decoensel@ntec.ugent.be 3 doldon@ntec.ugent.be 4 dc.botteldooren@ntec.ugent.be

2 ncdence of dffractons and reflectons on the external ears, head and shoulders [-5]. In what follows, we gve a smplfed overvew of how these cues are calculated and represented n the human audtory system. Incomng sounds are transferred va the outer- and mddle ear to the nner ear, where they are spltted up nto dfferent frequency channels by the cochlea. From ths pont, the system s tonotopcally organzed, meanng that the dfferent frequency channels are further processed by separate neural crcuts, parallel to each other, senstve to only a lmted frequency range around ther characterstc frequency (CF. Har cells convert the pressure sgnals n the cochlea nto electrcal sgnals to be transferred to the bran va the audtory nerve. The bran zone that s thought to be encodng ITD s the medal superor olve (MSO. It receves exctatory nput from both ears. Several models for explanng ITD senstvty of MSO cells exst, the classcal beng Jeffress delay lne theory [6]. In ths model, the MSO performs a calculaton somewhat smlar to a cross-correlaton between the nputs from the two ears. IID s encoded n the lateral superor olve (LSO. Ths bran zone receves exctatory nput from one ear, and nhbtory nput from the other ear, and ts output s related to the dfference n ntenstes of the two ears [].Monaural cues are thought to be encoded n the dorsal cochlear nucleus (DC [5]. Cells encodng these localzaton cues manly project to the Inferor Collculus (IC and further va the medal genculate body (MGB to the audtory cortex [7]. It s not yet completely clear what exactly happens n these zones, but a converson from the localzaton cues to a locaton estmate s necessary. 3. MODEL 3. Preprocessng Frst, tme ntervals of a fxed duraton are extracted from the ncomng sound sample. These ntervals may overlap. The obtaned fragments are then sent through a flterban, mtatng the frequency decomposton effect of the cochlea. Usng ths wndowng n both the tme doman and the frequency doman, we splt the ncomng sgnal up nto an array of sound samples wth lmted tme and frequency range. These sound samples are then analyzed and localzed ndvdually. In order to smulate the effect of the har cells, the sgnals are half-wave rectfed, and the resultng sgnal s converted to a logarthmc scale to tae nto account the effect of the audtory nerve. 3.2 Localzaton features ext, the response of MSO and LSO neurons to the ncomng sound samples s smulated. A frst approxmaton s the use of cross-correlaton to smulate the senstvty of MSO neurons to ITDs. The output of a neuron wth maxmal response at an ITD of T wll then be represented by the value of the cross-correlaton functon of the nput from both ears at a tme dfference of T. Thus, by calculatng the cross-correlaton functon of the nput sgnals at both ears, we have smulated the output of an array of MSO neurons wth maxmal response at dfferent ITDs. The response of LSO neurons can, n a frst approxmaton, be modeled as arctan( I sgnal - I neuron, wth I neuron the characterstc ntensty dfference of the smulated neuron, and I sgnal the calculated ntensty dfference (n db of the ncomng sound. It has been suggested that MSO neurons are actually senstve to nteraural phase dfference nstead of nteraural tme dfference [8]. The model of MSO neurons descrbed above can easly be adapted n order to mplement ths. Because the sound samples that are analyzed only have a lmted frequency range, t s reasonable to convert tme dfference to phase dfference usng the central frequency of each frequency band. Monaural cues are not taen nto account n ths paper, as we are manly concerned wth sound localzaton n the horzontal plane. 3.3 Mappng Subsequently, the outputs of all smulated MSO and LSO neurons are grouped nto one localzaton feature vector. Wth ths vector, a sound source drecton can be estmated, usng a mappng between localzaton feature vector-space and locaton-space. To mplement ths mappng, a smple Bayesan system s used. We defne theta as the angle n the horzontal plane where the sound orgnates, T wth an nteger between and M are ntervals nto whch the angle space ]0,360 ] s dvded, f q are localzaton feature vector element values and F q,jq are ntervals n whch the localzaton feature vector element value space s dvded. Usng Bayes' 2

3 theorem, t s then found that: P( T f,j,j, j, j P(f,j, j P( T = P(f,j, j P(f,j, j P( T = M P(f P( T ( In a next step, the assumpton that all localzaton feature vector values are statstcally ndependent s ntroduced. Although these values clearly are not statstcally ndependent, ths approxmaton wll prove to be producng satsfactory results. We get: P( T = M f P(f,j P(f,j,j... P(f... P(f, j, j, j P( T P( T (2 We can evaluate ths expresson f we have values for P(θ T and P(f q F q,jq θ T for all q, j and. P(θ T can be consdered equal for all (and thus equal to /M, or can be determned by a pror nowledge about possble locatons of the sound source. P(f q F q,jq θ T can be estmated by tranng wth recorded sounds orgnatng from drectons wth θ T. As a tranng set, an array of real broadband sounds (arplanes, cryng babes,... s used. These are recorded by an artfcal head, wth the sound source placed at a certan θ, and at a dstance of about 3m. Ths was repeated 36 tmes, samplng the horzontal plane at all nteger multples of 0. Subsequently, these sounds are analyzed as descrbed n sectons 3. and 3.2, and for every recorded value of θ, an array of localzaton feature vectors s obtaned. Usng these vectors, a Gaussan probablty dstrbuton for each vector element can be estmated and ths can be used to evaluate P(f q F q,jq θ T. In the calculaton of the parameters of the Gaussan dstrbuton, features assocated wth more ntense nput n the correspondng frequency band are gven more mportance, as features assocated wth less ntense nput n that frequency band may be domnated by nose. 4. RESULTS An nterestng way of vsualzng the results of the localzaton model descrbed n secton 3 s to dsplay detected sound source drecton as a functon of tme and frequency. Thus, for each tme-frequency wndow as descrbed n secton 3., a sound source locaton estmate s calculated as an average of sound source drectons, weghted by ther calculated probabltes. In fgure a, a spectrogram of a recordng of a polce sren s shown. In the recordng, the sound source s located at an angle of 50 (wth -90 defned to be to the left, 0 rght n front and 90 to the rght of the artfcal head. Fgure b shows the frst results of the locaton analyss. Ths does not loo very good, because the system has dffcultes dstngushng between front and bac. Humans manly use monaural cues to sense the dfference between front (θ and bac (80 - θ, as ITD and IID are nearly the same for these angles [5]. Thus, because we omtted these monaural cues n our model, t s very natural that ths problem occurs. When the color scale of the plot s changed n such a way that angles θ and 80 - θ are represented by the same color, n order to elmnate ths problem, the result of fgure c s obtaned. The source s now well localzed, as a large zone wth the color representng 50 (or 30 can be seen. 3

4 Fgure a: Spectrogram of polce sren, postoned at 50, frequency channels are n /3 octave bands, tme wndow every 0.s. b: Localzaton of polce sren. c: Localzaton of polce sren, wthout front-bac confuson. d: Smlar to b, but wthout tranng at odd multples of 0. e: Smlar to c, but wthout tranng at odd multples of 0. To smplfy comparson, a contour, defnng the most ntense zone n the spectrogram, has been added on all fgures. In the prevous example, the test sound orgnated from an exact drecton at whch the system had been traned (as t was traned at all multples of 0. In order to test whether the system stll wors wth ncomng sounds orgnatng from drectons n between traned drectons, now an analyss s presented of the same sound as before, but tang nto account only tranng data of all multples of 20. Smlar plots as n the prevous example can be found n fgures d and e. It can be seen that localzaton s stll good, but wth more front-bac confuson. In the next cases, front-bac confuson wll be elmnated, le n fgures c and e. As a second example, a recordng of an arplane at a fxed angle of 50 s consdered. The spectrogram of ths recordng can be seen n fgure 2a, and the result of the locaton analyss n fgure 2b. It can be seen n the fgure that localzaton at low frequences, below channel 5, or below about 200Hz, s not good. Ths s caused by the fact that, at these frequences, wavelengths are longer than m, whle the dstance between the two ears s only about 20cm. Because of ths, ITD s become ncreasngly dffcult to perceve as the frequency decreases. In addton, at these low frequences, 4

5 there s almost no shadng effect of the head, so IID cues also become useless. The use of dfferent features mght possbly amelorate localzaton at these low frequences, but, when tested wth nteraural phase dfference nstead of nteraural tme dfference n the localzaton feature vector (as explaned n secton 3.2, no mprovement s observed. Fgure 2 a: Spectrogram of arplane sound, postoned at 50, tme and frequency scales as n fgure. b: Localzaton of arplane sound. c: Localzaton of arplane sound, only usng ITD cues. d: Localzaton of arplane sound, only usng IID cues. As n fgure, a contour has been added for easy comparson of the fgures. In fgure 2c, the result of localzaton analyss n whch only MSO neurons, representng ITD nformaton, were taen nto account, s shown. Overall, results are stll qute good, but manly at hgher frequences the localzaton qualty deterorates. In fgure 2d, the result n whch only LSO neurons, encodng IID nformaton, are used, can be seen. Localzaton n ths case s a lot worse. These fgures show that n the model, ITD nformaton domnates localzaton, and IID nformaton becomes mportant at hgher frequences. Ths corresponds to observatons n human sound localzaton, and agrees wth the classc duplex theory of Thompson and Raylegh, whch states that ITD s used manly at lower frequences whle IID s used manly at hgher frequences [-4]. Fnally, t s shown how ths localzaton model can be appled to mprove sound source detecton and audtory stream segregaton. For ths purpose, the polce sren analyzed n fgure s supermposed on whte nose. Fgure 3a shows the sgnal to nose rato n db (SR db for each tme-frequency wndow and fgure 3b shows the results of the localzaton analyss on ths sgnal. It can be seen that tme-frequency wndows wth a hgh SR db are localzed correctly, whle wndows wth a low but postve SR db are not localzed perfectly but stll stand out of the whte nose bacground, whch s located more or less around 0. It s clear that ths localzaton nformaton can be of used to select tme-frequency wndows wth postve SR db, and thus help sound source detecton n a nosy bacground. 5

6 Fgure 3 a: Sgnal to nose rato n db of a polce sren (located at 50 n whte nose, tme and frequency scales as n fgure. b: Localzaton of the polce sren n whte nose. For easy comparson, a contour at a SR of 5dB s added. ow, the sound of the polce sren at -20 s supermposed on the sound of footsteps at 70. The rato of the frst sgnal to the second sgnal for each tme-frequency wndow s shown n fgure 4a. Fgure 4b shows the resultng angles. It s clear that n wndows where one of the two sources s domnatng, ths source s well localzed. By groupng tme-frequency wndows orgnatng from the same drecton, basc audtory stream segregaton can be accomplshed, or ths localzaton nformaton can be used to help n the process of stream segregaton. Fgure 4 a: Rato of polce sren (located at -20 to footsteps (located at 70 n db, tme and frequency scales as n fgure. b: Localzaton of the sound n 4a. For easy comparson, a contour at a rato of 5dB s added (red and one at a rato of -5dB (blue. 5. COCLUSIOS Ths paper presents a bologcally nspred model for bnaural hearng, whch s useful for localzng sounds n the horzontal plane. The model dsplays several features also seen n human bnaural hearng, ndcatng that t s a bologcally plausble model. The model also produces useful nformaton for the process of audtory stream segregaton, as bnaural hearng does for humans. The model can be extended and possbly mproved by ncludng monaural features. Ths could solve the dffcultes the model currently has wth front-bac confuson and even enable t to localze sounds n the complete space, and not just n the horzontal plane. REFERECES [] Tom C.T. Yn, eural Mechansms of Encodng Bnaural Localzaton Cues n the Audtory Branstem Chap. 4 n Integratve Functons n the Mammalan Audtory Pathway, edted by R.R. Fay and A.. Popper (Sprnger-Verlag, ew Yor, [2] Smon Hayn and Zhe Chen, The Coctal Party Problem, eural Computaton, 7,

7 (2005. [3] Masanao Ebata, Spatal unmasng and attenton related to the coctal party problem, Acoust. Sc. & Tech., 24, 5 (2003. [4] Tom C.T. Yn and Shgeyu Kuwada, Bnaural localzaton cues, Chap. 2 n The Oxford Handboo of Audtory Scence The Audtory Bran (Volume 2, edted by Adran Rees and Alan R. Palmer (Oxford Unversty Press, Oxford, 200. [5] Bradford J. May, Sound locaton: monaural cues and spectral cues for elevaton, Chap. 3 n The Oxford Handboo of Audtory Scence The Audtory Bran (Volume 2, edted by Adran Rees and Alan R. Palmer (Oxford Unversty Press, Oxford, 200. [6] Lloyd A. Jeffress, A place theory of sound localzaton, J. Comp. Physol. Psychol., 4 (948 [7] Manuel S. Malmerca and Troy A. Hacett, Structural organzaton of the ascendng audtory pathway, Chap. 2 n The Oxford Handboo of Audtory Scence The Audtory Bran (Volume 2, edted by Adran Rees and Alan R. Palmer (Oxford Unversty Press, Oxford, 200. [8] Davd McAlpne and Benedt Grothe, Sound localzaton and delay lnes do mammals ft the model?, Trends n euroscences, 26, 7 (