AN UNSUPERVISED BAYESIAN CLASSIFIER FOR MULTIPLE SPEAKER DETECTION AND LOCALIZATION

Transcription

1 AN UNSUPERVISED BAYESIAN CLASSIFIER FOR MULTIPLE SPEAKER DETECTION AND LOCALIZATION Youef Oualil, Friedrich Faubel, Dietrich Klakow Spoken Language Sytem, Saarland Univerity, Saarbrücken, Germany Abtract Multiple peaker localization algorithm generally require a binary detector, which perform the ource/noie claification of the location etimate. Thi i mainly due to the unknown timevarying number of ource, and to the preence of noie and reverberation. In thi paper, we propoe an unupervied learning approach baed on a naive Bayeian claifier. The propoed approach couple two peaker location feature, namely, ) the teered repone power introduced at the location etimate, and ) the correponding maximum likelihood error, which characterize the variance of the etimate. The latter i experimentally hown to be highly correlated with the teered power at the location etimate. The propoed method i further extended to control the miclaification rate through the ue of a lo function. Thi approach i general, and can be eaily extended to integrate more peaker/peech feature. Experiment on the AV.3 corpu how the effectivene of the propoed approach. Index Term: microphone array, multiple peaker localization, ource detection, Bayeian claification.. Introduction Microphone array have become an eential tool for a large number of ignal proceing problem. Their area of application include peech eparation/enhancement, acoutic ource localization and tracking, but alo more advanced approache uch a camera teering for teleconference ytem and audio-viual tracking. Among thee application, the detection and localization of multiple concurrent peaker from a hort egment of peech remain a difficult and open tak; and that although an abundance of localization method have been propoed in the literature: multi-channel cro correlation (MCCC) [], adaptive eigenvalue decompoition (ED) [, 3, ], time difference of arrival (TDOA)-baed technique [5,, 7] and teered repone power (SRP)-baed technique [, 9], jut to name a few. A good multiple peaker localization performance cannot be achieved without a ource detector, which claifie the obtained etimate to peaker/noie. Thi i mainly due to ) the preence of noie and/or reverberation, which introduce econdary peak, and to ) the unknown time-varying number of ource per frame. Few attempt have been made to overcome thi problem, the author of [] propoed to ue the ditance eparating the etimate a a criterion to extract the number and location of the ource, wherea Do et al. [, ] propoed to combine the ignal power with a double clutering technique to etimate the number of peaker. In a more advanced approach, Lathoud et al. [3] propoed an unupervied threhold election technique to control the fale alarm rate. Following a line of thought imilar to [3], we propoe to etimate the optimal boundary between the noie and peaker clae, uing an unupervied Bayeian claifier. Contrary to the approache taken in [, 3], where a ingle power-baed feature i ued, we propoe in thi work to augment the feature pace with the Maximum Likelihood Error (), introduced at each location etimate. In doing o, the claification boundary between the two clae become more obviou. Thi property i of mot interet in low SNR/SRR environment, a well a in the multiple peaker cae, where the ignal power emerging from the econdary peaker become comparable to the noie/reverberation power. In thi framework, we firt etimate the likelihood ditribution and the prior of each cla. Thi i done by fitting a 3- component mixture to each feature pace. Then, the poterior ditribution of each cla i obtained uing a Naive Bayeian Claifier (NBC), which combine the two feature. The choice of the mixture i dependent on the ued feature. Experiment conducted on the AV.3 corpu how that ) combining the feature improve the detection performance, and that ) the propoed unupervied claification approach perform better than a upervied Support Vector Machine (SVM) claifier. We proceed in thi paper by introducing the claification feature. Then, we how how thee feature can be ued to etimate the likelihood ditribution and the prior (Section ). The unupervied Bayeian claifier i preented in Section 3. Section how the performance of the propoed approach in comparion with SVM. Finally, we conclude in Section 5.. Feature Extraction And ML Etimation In thi ection, we proceed by reviewing the multiple peaker localization approach ued to etimate the ource() location, and thereby extract the claification feature. Then, we how how the mixture ditribution can be ued to characterize each feature pace. Finally, we propoe an online algorithm to etimate the parameter of thee mixture... Multiple Speaker Localization Approach In a recent work [, 5], we have propoed a novel approach to the multiple ource localization problem. Thi framework interpret each normalized Generalized Cro Correlation function (GCC) a a probability denity function (pdf) of the Time Difference of Arrival (TDOA). Thi pdf i then approximated by a Gauian mixture (GM) ditribution uing either the Weighted Expectation Maximization (WEM) algorithm from [5] or it practical approximation in []. The reulting TDOA Gauian mixture are mapped to the location pace uing the location- TDOA mapping, given by (). The approach propoed in [] combine the GM uing a probabilitic interpretation of the Steered Repone Power (psrp), wherea the approach propoed in [5] maximize the TDOA joint pdf in the location

2 pace. The ret of thi ection preent a brief introduction to the mathematical formulation of thee two framework. Formally, let M and Q denote the number of microphone and correponding pair, repectively, and let m h denote the poition of the microphone, h =,..., M. The location-tdoa mapping between the location and the TDOA τ q(), introduced by the ource at the microphone pair q = {m g, m h } i given by τ q () = ( m h m g ) c () where c denote the peed of ound in the air. The GM approximating the normalized GCC function (interpreted a a pdf of the TDOA) of the q-th microphone pair, i given by p(τ q ) = w q k N q k (τq, µq k, (σq k ) ) () K q k= where µ q k, σq k and wq k denote the mean, tandard deviation and mixture weight of the k-th component. The probabilitic SRP SRP prob of a given location i then given by [] SRP prob () Q K q w q k N q k (τq(), µq k, (σq k ) ) (3) q= k= wherea the ML approach maximize the location likelihood ditribution given by [5] p() Q K q w q k N q k (τ q (), µ q k, (σq k ) ) () q= k= The ource location etimate e i obtained by ) extracting from each GM ditribution the Gauian component (w q e, µ q e, σ q e ) where the ource i dominant. Then, ) calculating the retriction of (3) and () on the pace region where e i dominant. Finally, 3) the optimal location etimate i obtained uing a numerical optimization algorithm. Thee two approache however ue two different detection method to claify a location etimate e to ource/noie etimate. In [], the deciion i baed on the probabilitic power coming from that particular location, that i e i a ource if SRP prob ( e) > P noie (5) where P noie i a predefined threhold, wherea [5] accomplihe thi tak by comparing the ɛ() to a predefined threhold Γ. Thi i done according to e i a ource if ɛ( e) = Q ( τ q ( e) µ q ) e < Γ () q= σ q e The difficulty with thee two detection approache lie in the choice of the deciion threhold. The latter i dependent on the environment and the ditance to the microphone array. Therefore, a tatic threhold might not be well uited to the location change, a it might poorly perform in uneen environment. In thi work, we propoe an unupervied learning approach, which improve the detection performance by combining thee two approache. Thi approach i eay to adapt to poible location change, uing an online learning proce, and provide different deciion boundarie in different environment... Cumulative Steered Repone Power Feature Similarly to the approach taken in [], we propoe to ue the teered power a the firt detection feature. Thi approach however, doe not imply conider the power coming from a ingle location, it rather conider the cumulative power emerging from the etimate region of dominance. Thi cumulative teered repone power () i calculated according to e = SRP prob () d (7) S e Q Q w q e N q e (τ q(), µ q e, (σ q e ) ) d w q e () S q= q= S e repreent the pace region where the acoutic event generating e i dominant. The equation () i obtained by mapping S to the different TDOA pace (ee [5] for more detail). Let {( i, c i)} N T i= denote the et of NT location etimate i and their correponding value c i, obtained in T frame. We propoe to eparate the ource from the noie by fitting a 3- component mixture ditribution to the data in the pace. Thi mixture i obtained by maximizing the likelihood of the etimate {c i} N T i= uing the Expectation-Maximization (EM) algorithm []. Formally, the EM algorithm etimate a mixture ditribution of the form G crp n f crp () = w crp n Gn crp (c) + w crp f crp (c) (9) (.) i a Gauian ditribution approximating the likelihood ditribution of the noie, wherea f crp (.) i a Gauian + uniform mixture ditribution approximating the likelihood ditribution of the ource (Figure -b). f crp (.) i given by f crp (c) G crp (c) + U crp (c) () (.) i a Gauian ditribution and U crp i a uniform ditribution. wn crp and w crp denote the noie and ource prior, repectively. The uniform ditribution U crp i intro- where G crp duced to model the high value, which are poorly modeled by G crp..3. Maximum Likelihood Error Feature The econd claification feature i the Maximum Likelihood Error () given by eq (). Thi feature i correlated with the nature of the acoutic ource. More preciely, we expect the to be large for diffued noie, but low for point ource. Actually, the SRP how that the diffued ource are characterized by flat peak, wherea the point ource map to harp peak. Thi property i mainly due to the nature of the GCC peak, repreenting the ame ource but in different microphone pair. For a diffued noie, the peak are generally flat, and might map to different peak in the location pace. A a reult, the variance of the etimate i expected to be large and the tend to increae, and vice vera (Figure -a). We propoe to ue the approach preented in Section. to etimate the noie and ource likelihood, with the exception of uing different ditribution. Formally, let {( i, err i)} N T i= denote the et of N T location etimate i and their correponding value err i, obtained in T frame. We propoe to ue a 3-component mixture ditribution : f mle () = w mle Γ mle (err) + wn mle fn mle (err) () where Γ mle (.) i a Gamma ditribution approximating the likelihood ditribution of the ource, wherea fn mle (.) i a

3 . Cumulative SRP Azimuth Variance of the Etimate (a) Etimate variance v power (b) Mixture fit of the feature Cla = Speaker Cla = Noie. 9 Hitogram Speaker() pdf Noie pdf Mixture pdf.9. Hitogram Noie pdf Speaker() pdf Mixture pdf (c) Mixture fit of the feature (d) Claification uing NBC Figure : The figure in (a) illutrate the high correlation between the variance of the SRP peak generating the etimate and the cumulative SRP, thi figure how clearly that the etimate map to two ditinct clae. The graph in (b) and (c) how an example of the maximum likelihood mixture ditribution approximating the ditribution and the ditribution, repectively. The graph in (d) how an example of a claification boundary obtained with the NBC. Gauian + uniform mixture ditribution approximating the likelihood ditribution of the noie. fnmle (.) i given by fnmle (err) Gnmle (err) + Unmle (err) () Similarly to eq. (), Unmle i introduced to model the high value, which are poorly modeled by Gnmle. The and the feature are combined in Section 3 to improve the detection performance... Online Parameter Etimation Acoutic ource localization application, uch a camera teering and audio-viual tracking, often require an online localization performance. Therefore, the ource/noie claification hould be alo performed online. Algorithm propoe an approach that accomplihe an online etimation of the ditribution parameter from Section. and.3. The propoed algoalgorithm : Online Parameter Etimation. Initialize the ditribution parameter randomly. Let T be the re-etimation period. for each time t multiple of T do 3. Set the initial parameter to the current parameter.. Keep the etimate from the lat N frame. 5. Re-etimate the parameter uing the EM algorithm. end for in each feature pace (Section. and.3), and then combining the reulting ditribution uing a Naive Bayeian ClaiT fier (NBC) [7]. Formally, let {Xi = (i, ci, erri )}N i= be the et of augmented etimate, and let α be the claifier deciion, α {ource,noie}. The poterior probability of the deciion α given an etimate X = (, c, err) i given by p(α X) = p(x α) p(α) p(x) (3) The NBC aume the independence of the feature [7], and expree the likelihood ditribution according to p(x α) = Y p(xk α) = p(c α) p(err α) () k= Replacing the term in (3) and () by their expreion from (9), (), () and () lead to the following unupervied claifier p(ource X) fcrp (c) Γmle (err) wcrp werr(5) p(noie X) Gncrp (c) fnmle (err) wncrp wnmle () The deciion α i independent of the probability of the etimate X. Therefore, p(x) i ignored in eq. (5) and (). X i conidered to be generated by an actual ource if p(ource X) p(noie X). 3.. Lo Function For Noie Control rithm take into account any poible change in the ditance, number of peaker and noie condition, which might affect the detection performance. Therefore, only the lat N frame are ued to re-etimate the parameter. It i worth mentioning that N hould not be too mall a well. 3. Unupervied Bayeian Claifier 3.. Naive Bayeian Claifier The detection tak can be improved by fitting a mixture ditribution to the joint -D feature pace, formed by the and the feature. Such an approach i beneficial, becaue it incorporate the correlation between the two feature, which would lead to a more realtic model. The ditribution of the -D data however narrow the poible choice of the mixture ditribution (Figure -d), which can efficiently maximize the likelihood, and thereby accurately model the etimate. Thi problem can be olved by maximizing the likelihood of the data Acoutic ource localization approache are generally combined with a large number of application, ome of which may require a reduced noie rate, uch a beamforming technique [], wherea other application, uch a the audio-viual tracking approache [7, 9], are more robut againt noie, and expect a high frequency of correct etimate, even if that lead to an increaing noie rate. The variety of thee approache require more flexibility in the acoutic ource claification. Thi idea i uccefully implemented uing the lo function [7, ]. Formally, let λ(α g), be the lo incurred for deciding α knowing that g i the true cla, with α, g {ource,noie} = {S, N }. The rik aociated with taking the deciion α given the etimate X i calculated according to R(α X) = λ(α S) p(s X) + λ(α N ) p(n X) (7) The claification according to the minimum-rik deciion rule i obtained by deciding S when R(S X) R(N X) and vie vera. Thi rule i equivalent to

4 Table : Source/Noie Claification Reult Sequence SVM + SVM + SVM + + NBC + + R P F R P F R P F R P F eq-p eq-p eq-3p eq5-3p eq37-3p λ(n S) λ(s S) p(n X) λ(s N ) λ(n N ) p(s X) () λ(s S) and λ(n N ) repreent the lo incurred for making the right deciion. Therefore, thee two parameter are generally et to. On the other hand, etting λ(n S) = λ(s N ) = lead to the NBC (Section 3.). The noie rate can be then controlled by adapting the ratio of thee two parameter.. Experiment and Reult We evaluate the propoed approach uing the AV.3 corpu [], where human peaker have been recorded in a mart meeting room (approximately 3m in ize) with a cm - channel circular microphone array. The ampling rate i khz and the real mouth poition i known with an error 5cm []. The AV.3 corpu ha a variety of cenario, uch a tationary or quickly moving peaker and varying number of imultaneou peaker. The ource localization experimental Noie (training) Noie (claified) Speaker (training) Speaker (claified) Support Vector Figure : Example of claification with SVM. etup ued in thee experiment i the imilar to that propoed in [5]. More preciely, the ignal wa divided into frame of 5 ample (3m); the GCC were calculated uing PHAT [] weighting; and a voice activity detector wa ued in order to uppre ilence frame. The multiple peaker localization approach provide etimate per frame (N max in [, 5]), wherea the number of imultaneou peaker varie between and 3. The propoed approach i compared to the claical Support Vector Machine (SVM) [7, ] approach with quadratic kernel (Figure ). Uing different kernel did not improve the reult. The SVM training data i obtained by calculating the and feature for all location given by the equence ground truth, a well a for noie location elected randomly. The reported reult were obtained with a training on the audio equence eq-p-, and then teting on the remaining multiple peaker equence from the corpu. The reult are reported in term of the Recall (R), Preciion (P) and F-meaure (F). Thee meaure are given by P = True Poitive True Poitive + Fale Poitive R = True Poitive True Poitive + Fale Negative F = R P R + P (9) () () The value of thee meaure are between and. The higher they are, the better the claification i. The recall repreent the fraction of actual ource() etimate that i correctly claified, wherea the preciion report the fraction of etimate which are correctly claified. Finally, the F-meaure i a weighted harmonic mean of preciion and recall. Thi meaure i very relevant in aeing the overall performance. Table preent the reult of the multiple ource detection tak uing the SVM approach, when it i combined with each feature eparately, a well a when the feature are jointly ued. Thee reult how that combining the and feature lead to better claification reult. More preciely, we can ee that uing the feature alone, lead to a good recall performance but very poor preciion. On the other hand, uing the feature alone reult in a good preciion performance but a poor recall. Combining thee two feature, however provide more information to the SVM claifier, which uccefully increae the F-meaure of all equence. We can alo ee that, contrary to the only and only reult, the recall and preciion performance of the joint feature experiment are balanced. We can conclude from thee reult that combining the and feature increae the detection performance. Table alo report the reult of the propoed unupervied Bayeian claifier. Thee reult how clearly that the propoed claifier perform lightly better than SVM. Thi i mainly due to the dependency of the feature on the ource location and the number of peaker. Thee two factor highly affect the ignal power level and the SNR. Therefore, uing a ingle training data to claify the different cenario propoed by the AV.3 corpu lead to a ub-optimal performance. The propoed claifier however adapt eaily to thee change. Thi i due to the elf-learning approach, which ue the data itelf to infer the bet boundary that explain the two clae. 5. Concluion We have propoed an unupervied Bayeian claifier to the multiple peaker detection tak. The propoed approach ue the maximum likelihood error and the cumulative SRP a claification feature, and ue a naive Bayeian technique to combine their ditribution. Thi approach alo provide a flexible framework to control the noie rate, and can be eaily extended to integrate more peaker/peech feature.

5 . Reference [] J. Chen, J. Benety, and Y. Huang, Robut time delay etimation exploiting redundancy among multiple microphone, IEEE Tran. Acout., Speech, Signal Proce., vol., no., pp , 3. [] J. Benety, Adaptive eigenvalue decompoition algorithm for paive acoutic ource localization, Journal of the Acoutical Society of America, vol. 7, no., pp. 3 39,. [3] J. Dmochowki, J. Benety, and S. Affe, Direction of arrival etimation uing the parameterized patial correlation matrix, IEEE Tran. Audio, Speech, and Language Proceing, vol. 5, no., pp , May 7. [], The generalization of narrowband localization method to broadband environment via parametrization of the patial correlation matrix, in Proc. EUSIPCO, Sep. 7, pp [5] J. O. Smith and J. S. Abel, Cloed-form leat-quare ource location etimation from range-difference meaurement, IEEE Tran. Acout., Speech, Signal Proce., vol. 35, no., pp. 9, Dec. 97. [] M. S. Brandtein, J. E. Adcock, and H. F. Silverman, A cloedform location etimator for ue with room environment microphone array, IEEE Tran. Acout., Speech, Signal Proce., vol. 7, no., pp. 5 5, Jan [7] Y. Oualil, F. Faubel,, and D. Klakow, A multiple hypothei Gauian mixture filter for acoutic ource localization and tracking, in 3th International Workhop on Acoutic Signal Enhancement, Sep., pp [] J. H. DiBiae, A high-accuracy, low-latency technique for talker localization in reverberant environment uing microphone array, Ph.D. diertation, Brown Univerity,. [9] J. P. Dmochowki, J. Benety, and S. Affe, Fat teered repone power ource localization uing invere mapping of relative delay, in Proc. ICASSP,, pp [] M. Nileh and M. Rainer, A calable framework for multiple peaker localization and tracking, in Proc. IWAENC,. [] H. Do and H. Silverman, A method for locating multiple ource from a frame of a large-aperture microphone array data without tracking, in Proc. ICASSP, Apr., pp [] H. Do and H. F. Silverman, SRP-PHAT method of locating imultaneou multiple talker uing a frame of microphone array data, in Proc. ICASSP,, pp. 5. [3] G. Lathoud, M. Magimai.-Do,, and B. Hervé, Threhold Selection for Unupervied Detection, with an application to Microphone array, in Proc. ICASSP, Touloue, France, May. [] Y. Oualil, M. Magimai.-Do, F. Faubel, and D. Klakow, Joint detection and localization of multiple peaker uing a probabilitic interpretation of the teered repone power, in Statitical and Perceptual Audition Workhop, Sep.. [5], A probabilitic framework for multiple peaker localization, in Proc. ICASSP, May 3. [] G. J. McLachlan and T. Krihnan, The EM Algorithm and Extenion (Wiley Serie in Probability and Statitic), nd ed. Wiley- Intercience, Mar.. [7] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Claification (nd Edition), nd ed. Wiley-Intercience, Nov.. [] H. L. Van Tree, Optimum Array Proceing (Detection, Etimation, and Modulation Theory, Part IV), t ed. Wiley- Intercience, Mar.. [9] A. Levy, S. Gannot, and A. P. Habet, Multiple-hypothei extended particle filter for acoutic ource localization in reverberant environment, IEEE Tran. Acout., Speech, Signal Proce.,. [] C. M. Bihop, Pattern Recognition and Machine Learning (Information Science and Statitic), t ed. Springer, Oct. 7. [] G. Lathoud, J.-M. Odobez, and D. Gatica-Perez, AV.3: An audio-viual corpu for peaker localization and tracking, in Proc. MLMI Workhop, May, pp. 95. [] C. H. Knapp and G. C. Carter, The generalized correlation method for etimation of time delay, IEEE Tran. Acout., Speech, Signal Proce., vol., no., pp. 3 37, 97.