FEATURES FOR SPEAKER LOCALIZATION IN MULTICHANNEL BILATERAL HEARING AIDS. Joachim Thiemann, Simon Doclo, and Steven van de Par

Transcription

1 FEATURES FOR SPEAKER LOCALIZATION IN MULTICHANNEL BILATERAL HEARING AIDS Joacim Tiemann, Simon Doclo, Steven van de Par Dept. of Medical Pysics Acoustics Cluster of Excellence Hearing4All, University of Oldenburg ABSTRACT Modern earing aids often contain multiple micropones to enable te use of spatial filtering tecniques for signal enancement. To steer te spatial filtering algoritm it is necessary to localize sources of interest, wic can be intelligently acieved using computational auditory scene analysis (CASA). In tis article, we describe a CASA system using a binaural auditory processing model tat as been extended to six cannels to allow reliable localization in bot azimut elevation, tus also distinguising between front back. Te features used to estimate te direction are one level difference five inter-micropone differences of arrival (TDOA). Initial experiments are presented tat sow te localization errors tat can be expected wit tis set of features on a typical multicannel earing aid in anecoic conditions wit diffuse noise. Index Terms Computational Auditory Scene Analysis, Localization, Multicannel Hearing Aids 1. INTRODUCTION Te uman auditory system is remarkable in its ability to recognize underst sounds in very complex acoustic scenes, wit reverberation multiple interfering sources present. Tis ability is termed te cocktail party effect wic is studied in te field of Auditory Scene Analysis (ASA) [1]. Te ability of separating a sound from a mixture is greatly enanced if te listener is presented wit a binaural signal, tat is, if te listener can localize te source of interest (te target source) as well as te interfering sources, te enancement in comparison to monaural presentation usually quantified as te spatial release from masking [2]. Coupled wit researc into te underlying processes tat enable tis ability in umans is researc into mimicking tis ability by signal processing algoritms (Computational Auditory Scene Analysis, or CASA). In tis article, we focus on te sound localization aspect, based on work wic uses a probabilistic model wit a binaural auditory front-end [3]. Te model presented in [3] is able to determine te azimut of multiple sources based on interaural level differences (ILD) interaural differences (ITD) computed from a uman auditory model (consisting of a gammatone filterbank a simple neuronal transduction model). It was sown tat te model is quite robust to reverberation diffuse noise. One interesting application of CASA for sound localization would be te use in assistive earing devices (or earing aids, HA). In particular, modern HAs typically use multiple micropones to enable spatial filtering (e.g. beamforming) to increase te Signal-to-Noise Ratio (SNR) [4]. Tis filtering is usually designed to only enance sound coming from te front of te HA user. CASA-based localization could benefit HA users since te spatial filtering could be optimized based on te direction of te target te interfering sources. Te problem we are considering is te localization of multiple sound sources using a micropone array formed by a bilateral earing aid wit multiple micropones on eac earing aid. Tere are numerous well-establised algoritms for sound localization wit micropone arrays [4, ]. However, based on te results in [3], te particular geometry, variability, presence of obstructions (te ead pinnae) suggest tat a probabilistic auditory model based approac would be beneficial. In tis paper, we present investigations on extending te binaural model of [3] into a six-cannel localization algoritm. We use te 6-cannel HA described in [6], exp te location space to an upper-emisperical grid wit 1 resolution in azimut elevation. In particular, we examine ow te presence of noise affects te localization performance. 2. MODEL DESCRIPTION Te CASA localization model presented ere is based on uman audition suc tat it can be combined wit furter uman auditory based processing. Te localization model can be divided into four distinct stages. First, te acquisition of te audio signals wit te multicannel HA, ten te auditorymodel-based processing converting te audio signal into features for localization. Next, te feature analysis uses a probabilistic classifier, finally te probabilities are evaluated to make a localization decision. We assume tat te audio signal is captured by a bilateral HA wit tree micropones per side, denoting tem LF for /1/$31. 1 IEEE 1276

2 23rd European Signal Processing Conference (EUSIPCO) Left Earpiece TDOAR,FB TDOAR,FM TDOAL,FM TDOAL,FB ILD, ITD Rigt Earpiece Fig. 1. Scematic of features for localization. ITD ILD Fig. 2. One side of te binaural earing aid wit six micro- are computed from signals of te left rigt front micropones. Te TDOA features are computed for eac side. pones used for tis study. Tere are tree micropones on eac side, te locations of wic are indicated by te green arrows in te cutout. te frontmost micropone on te left side, LM for te middle micropone LB for te rearmost micropone. We assume a symmetrical arrangement of micropones is located on te opposite ear (RF, RM, RB). In te periperal auditory model processing, te six cannels of audio from te bilateral HA are first separately passed troug a F = 32 b fourt-order gammatone filterbank (GTFB). Te GTFB uses pase-compensated filters to align temporal cues across bs. Te center frequencies of te filters are equally distributed on an effective rectangular bwidt (ERB) scale [7]. Neural transduction is simulated using alf-wave rectification square-root compression. We denote te resulting signals c, were t is a frame index, f is te gammatone filter index c indicates te cannel (LF, LM,..., RB). After te periperal processing, te features X for localizing sources are computed. Building upon te features used in [3], te additional micropone cannels can provide features tat allow for determining source elevation resolve front-back confusion. As sown in Fig. 1, we use a RFM set of 6 features (ILD, ITD, TDOALFM, TDOA, LFB RFB TDOA, TDOA ) per t, f bin. ILD represents te inter-aural level difference is expressed as te energy RF difference between LF in db. Te remaining features are computed using te normalized cross-correlation between two cannels. Like te ILD, te ITD is computed RF from LF. Te TDOA features are computed only from signals witin eac side of te HA: TDOALFM from LFB LM LF LB LF, TDOA from, similar for te rigt side. In order to compute te TDOA features wit te required inter-sample accuracy, exponential interpolation [8] as been used to determine te maximum of te cross-correlation function Gaussian mixture model classifier We consider te problem of localizing a sound in a probabilistic fasion. We assign to eac direction a point on a spere 1277 centred on te ead of te HA user. For eac combination of azimut φ elevation θ direction of a sound source, denoted by λ(φ,θ) we train a Gaussian Mixture Model tat pre dicts te probability to observe te feature vector X. Tis probability is denoted as p( X λ(φ,θ)k ). Te probabilities are modelled using a Gaussian mixture model (GMM) wit V components, were eac direction λ(φ,θ)k is a class. All combinations of azimut elevation results in a total of k = 1,..., K classes. A separate GMM is trained for eac frequency b eac class Localization decision From te above described GMM, a decision for te location of a sound source can be made for eac t, f bin by likeliood maximisation, as λ (t, f )(φ,θ) = argmax p( X λ(φ,θ)k ). (1) 1 k K In some cases, it is sufficient to make a localization decision per frame only, in wic case it is possible to improve estimation by combining estimates over frequency using λ T (t)(φ,θ) = argmax 1 k K F X log p( X λ(φ,θ)k ) +, (2) f =1 were is a small constant to limit te effect of very unlikely feature combinations on te estimated probability. 3. EVALUATION In te present study, te goal is to assess te proposed localization sceme, to examine if te proposed set of features can be used for localizing sounds in te azimut elevation direction. For our experiments, we use a new database of anecoic ead-related impulse responses (HRIR) [9] recorded using te

3 same earing aid as in [6]. Te left side of tis device is sown in Fig. 2, sowing te micropones wit a distance of 1.6 mm from LF micropone to te LB micropone. Te LM micropone is approximately in te center between te front back micropones, te tree micropones forming a sallow triangle. In contrast to te database of [6], te new recordings cover elevations from -64 to 9. For tis initial study, a subset of points is used to reduce computation, covering only te upper emispere at 1 resolution in azimut elevation, for a total of 283 points. Te grid is sparser at elevations of 7 ( azimut resolution) above (3 azimut resolution at 8 elevation, single point at 9 elevation) to avoid aving a ig density of points at te pole. Speec samples are taken from te TIMIT database [1], wit a sampling frequency of 16 khz. After te periperal auditory processing, features are computed using frames of ms wit % overlap, for a frame sift of 1 ms. We use GMMs wit V = 1 components in all bs, wit diagonal covariance matrices. Te GMMs are trained by spatializing 1 romly cosen sentences from te TIMIT database at all 283 points, ten extracting te features only for frames were te energy for all micropone cannels exceeds a given tresold, to avoid training on noise. Variance normalisation is used to equalize te dimensions during training. Training used te Expectation-Maximisation algoritm [11], wit k-means clustering [12] to initialize te parameters. Testing is performed by romly selecting a male a female speec sample from te TIMIT database excluding te samples used for training te model. Te testing samples are spatialized at eac point λ φ,θ, ten multicannel rom gaussian diffuse speec-saped noise (SSN) at,, 1, 1, 2 db SNR is added. Using te same energy tresolding as used during training on te clean speec (in effect, an ideal voice activity detector, VAD), te features from active t, f bins are classified using te GMM. 4. RESULTS Te main result is sown in Fig. 3, sowing te percentage of t, f bins correctly localized in anecoic conditions wit various level of interfering SSN. Te triangles (green dased line) sow te percentage of frames were localization was correct in bot azimut elevation. By considering te azimut elevation components separately, we can see tat especially at iger SNR azimut estimation is significantly more robust tan elevation estimation. Te effect of using additional features over te ones in [3] is sown in Fig. 4. Using ILD ITD only, te performance over te entire semispere is poor (16.1% at 2 db), but adding one pair of TDOA features improves classification performance. Te dased line sows using TDOA LFB TDOA RFB (4.% at 2 db), but similar performance is seen wit TDOA LFM TDOA RFM (39.2% at 2 db) az only el only grid point Fig. 3. Localization performance at different levels of diffuse SSN. Te triangles sow percentage of frames exactly localized. Te squares crosses sow te performance if only azimut or elevation are considered Full featureset ILD, ITD, TDOA (L,R)FM ILD, ITD only Fig. 4. Grid localization performance for subsets of features. Using all TDOAs raises performance only marginally (4.9% at 2 db). In Fig. te localization errors are sown in te azimut elevation direction as well as for te central angle estimated per t, f bin. It can be seen tat localization errors are considerably smaller for elevation tan for azimut. Tis is due to te different ranges of errors; for elevation errors can be maximally 9, wile for azimut tey can be 18. Te central angle sows errors of maximally degrees at an SNR of db. Tese relatively large errors are due to te fact tat no specific provisions ave been included in te localization algoritm to accommodate for te mismatc between te training wic was done for sources witout noise, te evaluation wic was done for sources presented in diffuse background noise. In fact errors decrease to about 1 degrees for ig SNRs were te mismatc is muc smaller. Te rate of correct localization on te grid points sown in Fig. 3 appears very low, but we observe tat te localization accuracy is also strongly dependent on wic frequency b is considered. As can be seen in Fig. 6, performance is 1278

4 Angle Error (degrees) c.a. az el Fig.. Average localization error in degrees. Triangles sow te error in te central angle (or great circle) sense, squares crosses sow te average error in te azimut elevation components respectively db db 1 db 1 db db db Fig. 6. Localization performance (for combined azimut elevation estimation) as a function of frequency. very poor in low frequency bs. Tis can be explained by te fact tat speec signals are less likely to ave sufficient energy for reliable feature extraction in tose bs, tus te GMMs for tose bs are less well trained. Results of te combined azimut elevation estimation improve sligtly if te log-likeliood results are combined over frequency as described by Eq. (2). For example, at 2 db, correct classification raises from 4.9% to 2.2%, at db from 33.7% to 44.7%. At SNR 1 db below, te percentage of correct classification is about 1.4 s iger wit frequency integration. For visualizing te metod for a practical application, Fig. 7 sows te t, f maps for a segment of speec wic was rendered at azimut 4 elevation. Te signal was mixed wit diffuse noise at 1 db SNR. Panel A sows te energy in eac t, f bin, wit blank areas omitted using te VAD. Panel B sows te azimut estimation error, wile panel C sows te error in te elevation estimate. Two issues can be illustrated by panels B C. First, ow localization performance is dependent on te signal energy: were panel A sows ig energy, te localization error sown in B C tends to be low. Second, it can be seen tat in many cases te localization error is only offset by a single step in te azimut elevation grid, predominantly in te elevation estimate. Tis skews te results sown in Fig. 3 6 wic do not include a measure of estimation error.. DISCUSSION In tis study we are presenting an initial assessment of a metod to localize sound sources using a six-cannel bilateral HA. Localization is performed using a probabilistic framework, specifically a set of GMMs tat classify sixdimensional feature vectors. Te GMMs (one for eac frequency b of a auditory model analysis) compute te probabilities tat te observed features originate from any of 283 points on a grid of a emispere. It was sown previously tat te ILD ITD features work well for determining te azimut of a sound source provided te source is located in te front near te equator [3]. Te aim of tis study is to extend te metod of [3] to resolve te front-back confusion, if possible, perform localization on te vertical axis as well. Wile te results presented ere are limited in scope (significant sources of errors, suc as reverberation localizable interfering sources are not considered), we sow tat using two micropones per side near te ear te described features can discriminate directions to at least 1 resolution in elevation tus of course also solve te front-back confusion problem. As expected, we find tat it is important to ave sufficient training data. In our experiments, we find tat especially at te lower frequencies, speec energy was too sparse to properly train te GMMs from our training set. Combining estimates across frequency to get per- frame localization only provides a small benefit, but tis may also be explained by te poor quality of training at low frequencies. Furter researc is required owever to allow tis sceme to be practical wit current HA tecnology. One issue is te large number of classes to be classified caused by te twodimensional azimut-elevation grid. Reducing te number of grid points would reduce complexity during classification (important due to power constraints in HAs) as well as during training. Witout reducing te localization accuracy, tis could be acieved by eg. using a ierarcical approac, were te location of te source is first limited to te left or rigt emispere, followed by a more fine grained classifier. 1279

5 23rd European Signal Processing Conference (EUSIPCO) A B C Fig. 7. Example localization error for a segment of speec, sowing only tose /frequency (t, f) regions tat te (ideal) VAD detected. Panel A sows te energy in te t, f bins (in db), panel B sows te azimut estimation error, panel C te elevation estimation error. Tis speec sample was rendered at an azimut of 4 elevation of, mixed wit 1 db SSN REFERENCES [1] A. S. Bregman, Auditory scene analysis: Te perceptual organization of sound, MIT press, [2] M. L. Hawley, R. Y. Litovsky, J. F. Culling, Te benefit of binaural earing in a cocktail party: Effect of location type of interferer, Te Journal of te Acoustical Society of America, vol. 11, no. 2, pp , 4. [3] T. May, S. van de Par, A. Kolrausc, A probabilistic model for robust localization based on a binaural auditory front-end, IEEE Trans. Audio, Speec, Language Processing, vol. 19, no. 1, pp. 1 13, 11. [4] S. Doclo, W. Kellermann, S. Makino, S. Nordolm, Multicannel signal enancement algoritms for assisted listening devices, IEEE Signal Processing Magazine, vol. 32, no. 2, pp , Mar. 1. [] M. S. Brstein D. B. Ward, Micropone Arrays: Signal Processing Tecniques Applications, Springer, 1. [6] H. Kayser, S. D. Ewert, J. Anemüller, T. Rodenburg, V. Homann, B. Kollmeier, Database of multicannel in-ear beind-te-ear ead-related binaural room impulse responses, EURASIP Journal on Advances in Signal Processing, 9. [7] B. R. Glasberg B. J. C. Moore, Derivation of auditory filter sapes from notced-noise data, Hearing Researc, vol. 47, no. 1-2, pp , 199. [8] L. Zang X. Wu, On cross correlation based discrete delay estimation, in Proc. IEEE Int. Conf. Acous., Speec Sig. Proc. (ICASSP),, vol. 4, pp [9] J. Tiemann S. van de Par, Multiple model ig-spatial resolution HRTF measurements, in Proc. DAGA 1, Nürnberg, Germany, Mar. 1, to be publised. [1] J. S. Garofolo, L. F. Lamel, W. M. Fiser, J. G. Fiscus, D. S. Pallett, N. L. Dalgren, V. Zue, Timit acoustic-ponetic continous speec corpus, Tec. Rep. NISTIR 493, National Institute of Stards Tecnology, Gaitersburg, MD, [11] A. Dempster, N. Laird, D. Rubin, Maximum likeliood estimation from incomplete data via te EM algoritm, in Journal of te Royal Statistical Society: Series B, 1977, vol. 39, pp [12] S. P. Lloyd, Least squares quantization in PCM, IEEE Trans. Information Teory, vol. 28, no. 2, pp ,