Robust Estimation of Sound Direction for Robot Interface

Transcription

1 8 IEEE Intenational Confeence on Robotics and Automation asadena, CA, USA, Ma 9-3, 8 Robust Estimation of Sound Diection fo Robot Inteface Cheol-Tae Kim, Tae-Yong Choi, BongSu Choi, and Ju-Jang Lee, Senio Membe, IEEE Abstact This pape poposes a obust method to estimate sound diection fo mobile obot inteface. In contast to othe methods, the poposed method estimates sound diection at intevals of shot time to deal with deteioation b movement of sound souce o obot. The two methods, discete Kalman filte and time dela of aival (TDOA) using genealized coss-coelation (GCC), ae combined b quantifing eliabilit of TDOA. The combination pevents the deteioation b low powe of signals, noise, and evebeation because discete Kalman filte efficientl filtes out such effects. The esult of expeiments, in which mobile obot estimates azimuth angle with thee micophones, shows that the poposed method povides obustness on estimating sound diection even if the sound souce o obot is moving duing estimation pocedue. This pape poposes a obust method to estimate sound diection fo mobile obot inteface. A mobile obot equipped with thee micophones estimates diection of aival (DOA) of sound at intevals of shot time to deal with deteioation b movement of sound souce o the obot. I. INTRODUCTION ECENTLY, obots have moe commecial applications as Rit becomes moe intelligent and human-lie. Especiall, human-lie activit and inteface is moe impotant to domestic obot, and theefoe human and obot inteaction (HRI) has been widel eseached enomousl [-6]. Among the eseaches, we thin sound localization o sound diection estimation is ve helpful fo obot to inteact with human. Because obots ae designed to use multiple uses and assumed to move automaticall, the fist inteaction between human and obot will be the calling fo attention (Fig. ). Afte calling pocedue, the sound diection estimation can be used to impove the speech ecognition pocess o speae identification as a means of spatial filte. Thee possible appoaches ae mainl used to estimate sound diection. ) Maximizing the steeed esponse powe of a beamfoming [6]. ) Estimating high esolution spectal densit [7]. 3) Using time dela of aival (TDOA) [5]. The fist method is ve obust to noise and intefeence, the second is eas to intepet with pobabilit densit fo multiple sound souces, and the thid has low computational cost that means it can be opeated in eal time. Howeve, the state-of-the-at algoithms fo estimating sound diection have not consideed deteioations b movement of sound souce and obot seiousl. If a mobile obot with diffeential dives is tuning when a use calls the obot, then the movement will decease the pefomance of sound diection estimation. This wo was suppoted b the Koea Foundation fo Intenational Coopeation of Science & Technolog (KICOS) though a gant povided b the Koean Minist of Science & Technolog (MOST) in 7. The authos ae with Depatment of Compute Science and Electical Engineeing, Koea Advanced Institute of Science & Technolog, Daejeon 35-7, Koea ( ctim@odsse.aist.ac.) Fig. (a) The calling pocedue fo attention of obot (b) Afte calling pocedue, continuous tacing of speae can impove the pefomance of speech pocessing. As a esult of estimating DOA of sound at intevals of shot time, some uneliable estimation can be occued. The main causes ae low powe of signals, noise, and evebeation. acticall, the uneliable DOA estimation ma poduce unexpected behavios and it is the motivation of obust methods such as aveaging multiple esults [5] and ejecting afte outlie detection []. We get hint fom ejecting afte outlie detection to addess the obustness poblem. Two values, the maximum pea and atio of the values of the fist and second lagest pea in the GCC function, can be selectivel used to detect outlies. Based on the values, we suggest the eliabilit netwo to quantif the confidence of esult of TDOA. The gid based method detemines a most pobable DOA using thee GCC estimation esults, and finall Kalman filte pedicts DOA using the quantified eliabilit and the estimation esult fom the gid method. B modeling the measuement noise using quantified eliabilit, the Kalman filte pevents the deteioation b estimating at intevals of shot time. The emainde of this pape is oganized as follows. Section II eviews the pinciple of the DOA estimation between two micophones and discete Kalman filte to give bacgounds fo the pape. Section III povides the desciption of the poposed method. Section IV epots the expeimental esults of the poposed method which shows the obust chaacteistics in the estimation of the DOA. Finall, the concluding ema is given in section V /8/$5. 8 IEEE. 3475

2 II. BACKGROUND This section descibes two techniques to povide the bacgounds of the pape. The fist is estimating DOA of sound using two spatiall sepaated micophones. The second is discete Kalman filte which ae widel used to estimate unnown state with nois data. Moe detailed desciptions can be found in the liteatues [5], []. In the liteatue, GCC with HAT weight filte povides the best pefomance in the nois and evebeant envionment [9]. g ( f ) (5) Gˆ ss ( f) Theefoe, the DOA can be calculated b () with time dela ˆ with the maximum pea in the GCC-HAT function ˆ ag max ( ), (6) ( g ) R ss whee ( f ) is given b (5). g Fig. The elationship between DOA and time lag : The DOA can be diectl calculated b the time lag. A. DOA Estimation Fig. descibes the DOA estimation of sound with two spatiall sepaated micophones. In the case, TDOA method (estimating time dela between two micophones), which has ve long histo in signal pocessing, is geneall used and the DOA of sound is estimated b diectl calculating following tigonometical function c cos, () dm whee c is the sound velocit and d m is the distance between micophones [5], [8], [9]. The coss-coelation [8] between two signals is mainl used to detemine the estimated time dela Rss ( ) E[ s ( t) s( t )], () whee s () t and s () t ae signals emanating fom same emote souce, and E denotes expectation. Consideing the ealistic finite obsevation time, the coss coelation can be estimated b T Rˆ ss ( ) s ( t) s( t) dt T, (3) whee T epesents the obsevation inteval. Geneall, in ode to impove the accuac of the dela estimate ˆ, it is desiable to pefilte s () t and s () t pio to the integation in (3). This genealized method is called as genealized coss-coelation (GCC) and defined as follows ˆ ( g) ˆ j f Rss ( ) ( ) ( ) g f Gss f e df, (4) whee, Gˆ ss ( f ) is the estimated coss powe spectal densit and ˆ g( f ) Gs ( ) s f is the coss powe spectum between the filteed signals. B. Discete Kalman Filte The discete Kalman filte is widel used to addess the n geneal poblem of ting to estimate the state x with m the measuement z using following elationship x Ax Bu w, (7) z Hx v whee, w and v ae the pocess noise easuement noise espectivel, and assumed to have nomal pobabilit distibution N(, Q ) and N(, R ) espectivel. The matix A specifies how the state evolves in a step, matix B elates to the contol input u to the state x, atix H elates the state to the measuement. The discete Kalman filte algoithm minimizes the effect of w and v in ecusive manne and pedicts the state using (7). The detailed pocedue is ve simila to ecusive least squae pocedue as descibed in following paagaphs. Let xˆ denote a pioi estimated state at step using nowledge of the pocess pio to step, and x ˆ denote a posteioi estimated state at step given obsevation z. Then, a pioi and a posteioi estimated eo covaiance can be defined as T E[ ee ], (8) T E[ ee ], whee e ˆ x x and e ˆ x x. The Kalman filte updates a pioi state and eo covaiance based on pevious state, input, and eo covaiance xˆ Axˆ Bu A A Q T Based on a pioi state and eo covaiance, the Kalman filte updates a Kalman gain K, a posteioi eo covaiance, and computes a posteioi state xˆ (9) 3476

3 K H ( H H R) xˆ xˆ K z Hxˆ T T ( ) ( KH) () B (9) and (), the Kalman filte accomplishes minimization of a posteioi eo covaiance, and a posteioi state x ˆ is the optimal pediction fo given measuements. Note that the matix R in () is the measuement noise eo covaiance matix and is used to compute Kalman gain K. The Kalman gain K detemines the effect of the obsevation z in pediction of a posteioi state, and theefoe the matix R affects the ate of update based on the cuent obsevation z. III. ROBUST SOUND DIRECTION ESTIMATION SYSTEM Fig. 3 descibes the poposed method that estimates DOA using thee spatiall sepaated micophones. Thee signals ae extacted duing time inteval, and thee TDOAs ae estimated b GCC HAT function. Using thee TDOAs, a DOA is estimated b gid based method, and discete Kalman filte pedicts DOA based on the gid based estimation and the quantified eliabilit. We divide whole pocedue into thee majo steps. Fist step is called as Time Dela Estimation, second step is called as Diection of Aival Estimation, and thid step is called as Robust Diection of Aival Estimation. Biefl, we call the steps as step, step, and step 3. and theefoe thee micophones foms a egula tiangle. In each micophone, the aival sound is eceived in the 6bit aw data with sampling ate fs 6Hz. Fo each inteval, the estimation of time dela is pocessed when all signal powes exceed the theshold values. B. Time Dela Estimation (Step ) The time dela estimation is basicall calculated b the GCC-HAT as descibed in section II. Howeve, the method sometimes poduces uneliable esult when the DOA is estimated at intevals of shot time. Theefoe, the sstem should measue the confidence of the estimation fom the GCC-HAT. In the liteatues, seveal measues have been suggested to detect outlies to suppess the effect fom it []. We use two simple measues to quantif the eliabilit of GCC-HAT esult. Fist measue is the value of the maximum pea ( m ) in the GCC function. Second measue is the atio ( m ) of the values of the fist and second lagest pea in the GCC function. These two measues have slightl diffeent chaacteistics in the nois and evebeant envionment. * Let us define ˆ,, and e th as the estimation of GCC-HAT, eal time dela, and eo theshold value. Then, the success ate of GCC-HAT is defined as * ( ˆ e th ). Conside a database that contains signals * and eal time dela. Using the database, new taining data can be built b ecoding the conditional success ate fo given measue m (success ate fo the data of which both m, exceed the given measues) This pape suggests to use a single-hidden lae feedfowad netwo to quantif eliabilit based on both m. The netwo is stuctued with ten sigmoid functional hidden nodes, each input neuon of it has input bias, and it has a linea output lae. The taining data is newl built database that contains andomized given measues and coesponding conditional success ate. With the taining data, the neual netwo is tained using vaiable pojection method, which is modified Levenbeg-Maquadt algoithm [3]. Fig. 3 The bloc diagam of the obust sound diection estimation sstem A. Configuation of Hadwae Sstem Befoe explaining thee majo steps, we descibe the configuation of hadwae sstem. Thee micophones ae used to estimate the azimuth angle fo a single sound souce. Fo geometical advantage, the distance between each micophone pai is set to the identical value d.3m, m Fig. 4 The eliabilit estimation esult with feedfowad neual netwo. 3477

4 An example of function appoximation based on the tained neual netwo is shown in Fig. 4. This netwo is used to quantif eliabilit and named as eliabilit netwo. If the m ae both low, the success ate is unde.5 which means the estimated time dela is uneliable. While, if the m ae both high, the success ate goes to which means the estimated time dela is eliable. In the time dela estimation step, fo each micophone pai (thee pais), the estimated time dela( ˆ ) and eliabilit( ) ae computed. Using these vaiables, the DOA is estimated b geometical intepetation. C. Diection of Aival Estimation (Step ) The DOA with two micophones can be calculated b () using time dela. Howeve, the calculated DOA is not unique because it has alwas mio DOA. Theefoe, this pape suggests novel DOA estimation with thee micophones. Fig. 5 diection of aival estimation based on gid Method stochastic appoach Fig. 5 depicts the poposed stateg of DOA estimation. The objective is to estimate the diection of sound fom cente of micophones with thee time dela estimations. Let the calculated DOA between micophone - be, the DOA between micophone -3 be, and the DOA between micophone -3 be. Then, the six candidates of DOA fom the cente can be detemined, because each pai of micophones has two diections of DOA. 3, 3 3 3, 4 3 9, () As shown in Fig. 5, thee candidates ae located in eal diection of sound. We suggest the estimation based on stochastic analsis because some estimations of time dela ae uneliable. Let us assume the pobabilit of DOA of i th candidate with the eliabilit i has nomal distibution with mean i and standad deviation ( i ), whee is a use-defined penalt paamete fo uneliabilit and epesents the basic uncetaint of the sstem. Total 36 gids ae geneated to epesent the estimated DOA with the esolution. Fo the i th candidate, the pobabilities of DOA in i 3 gids ae estimated and added to the gids. The final DOA is estimated b finding the gid with maximum pobabilit value. The computation cost of this pocedue is just 366 calculations of exponential function. The eliabilit of the final DOA is just assessed b the mean value of the two lagest eliabilit values dopping the lowest eliabilit. In the diection of aival estimation step, the estimated DOA ( ˆ ) and eliabilit( ) ae computed. Using these data, the obust DOA estimation is pefomed. D. Robust Diection of Aival Estimation (Step 3) The obust DOA estimation uses the Kalman filte with the quantified eliabilit to educe the effect of uneliable DOA. Let us assume that a obot o a human moves in constant angula velocit in the fom of the Kalman state equation (7) () x x w, () w () x z v, whee x,, and ae DOA, angula velocit of DOA, and () estimation inteval espectivel. The pocess noise tems w () and w ae set to constant value, and theefoe the pocessing eo covaiance Q emains fixed. The obust DOA estimation stats with initial values of x,, and that ae set to x v, v, (3) v whee v is set to lage value because the initial eo is unnown in the stating of estimation. Fist, the sstem pocesses the pediction step (9) with the state equation descibed in (). Afte the step, the measuement eo covaiance R is set as follows R [ ( )], (4) whee is a use-defined penalt paamete fo uneliabilit. Using (4), the measuement update step () is pefomed. As noted in section II, the covaiance matix R is elated to Kalman gain K. If the matix R is updated b (4), the sstem estimates the DOA based on the pediction xˆ in (9) when the eliabilit is small, while the sstem estimates the DOA based on the measuement when the eliabilit is 3478

5 lage. If the design paametes,, and ae appopiatel set, the sstem ma estimate the DOA obustl based on the eliabilit measuement. To build up the eliabilit netwo, total 773 taining pais ae made using the signals extacted fom ms cases. The tained netwo poduces 3.54e-4 -fold coss validation eo and is used fo static and dnamic cases. Fig. 6 Mobile Robot latfom fo the Expeiment IV. EXERIMENTAL RESULTS The pefomance of the poposed obust DOA estimation sstem is descibed in this section. It is applied to small size mobile obot as shown in Fig. 6. The obot has 667MHz Via- Eden CU based hadwae and windows based softwae. The obot has DC-moto based diffeential dive and en/tilt motion sstem. The pefomance is measued in tems of Field Of View (FOV) which epesents the absolute diffeence between tue and estimated sound diection. In the obot societ, geneall fom to 5 FOV is consideed as a easonable eo. To illustate numeical esults, the poposed sstem is tested fo two cases. One is the static case and the othe is the dnamic case. The use-defined paametes,, and ae set to 4,, and 6, which ae found b intuitive tial and eo method fo all simulations. Fo convenience, the DOA estimation is called step, and the obust DOA estimation pocedue is called step 3. Because the step can be shown as a simple extension of conventional GCC-HAT, the pefomance of the poposed sstem can be compaed to the esult in the step. The vaious estimation intevals s ae checed to show the elationship between and the pefomance of the sstem. The initial paamete values of the Kalman filte ae set to as follows 8 9 4, Q 9 A. Static Case To build up the static database, a obot is fixed in the cente (,) of the oom and a human speas to the obot without movement duing shot time (about.5 second). The speae is located on ( cos( ), sin( )), whee is the distance fom the cente (,, 3, 4, and 5 mete) and is the angle fom the baseline (, 45, -45, 9, -9, 35, -35, 8) as shown in Fig. 7. The database is built up with thee speaes. Fig. 7 Static Database Acquisition Fig. 8 The DOA estimation esult fom step and step 3 in the condition ms, 3,. All speech signals ae sampled with 6 Hz sampling ate using 6 bit analog-to-digital convete. Because of existence of degeneative effects such as low powe, noise, and evebeation, the estimated time dela becomes uneliable in some inteval. Rate of Success Relationship between Estimation Inteval and Rate of Success ( All Database ) STE3 (FOV) STE3 (FOV5) STE (FOV) STE (FOV5) Estimation Inteval () Fig. 9 The elationship between the estimation inteval and the pefomance of the sstem in all databases 3479

6 Fig. 8 shows the esult in static case when ms. The poposed method (step 3) contols the outlies well. The elationship between estimation inteval and the pefomance of the sstem is illustated in Fig. 9. The figue shows that the obust DOA estimation b the Kalman filte is extemel helpful when the estimation inteval is small. To get the obust pefomance fo FOV, the estimation inteval should be lage than.. B. Dnamic Case Fo the dnamic case, the database is built up with fou diffeent motions of the obot while a human tals to the obot in a fixed point as shown in Fig.. The values fom encode sensos ae ecoded duing the motion and used fo the gound tuth of the DOA. Fig. Building up the database fo the dnamic case An example of the DOA estimation esult (move ) is shown in Fig.. The time inteval is 4ms. The obot tuns clocwise about 5 duing about 4 seconds and the DOA estimation is successful even though the gound tuth angle is continuousl changed. The pefomance of the dnamic case is summaized in Table I. The poposed sstem povides the obust pefomance fo fou diffeence motions using the shot inteval estimation stateg. Even if the DOA is estimated at intevals of shot time, the signal is not degeneated significantl b continuous delas. Estimated DOA Estimation of DOA fo Move DOA Result (Step) DOA Result (Step3) Gound Tuth Second Fig. An example of DOA estimation when the estimation inteval is 4ms (Move ) TABLE I THE ERFORMANCE OF DYNAMIC CASES ( 4ms ) STE (FOV) MOVE MOVE MOVE3 MOVE4 Step3 () Step3 (5) Step () Step (5) V. CONCLUSION A obust sound diection estimation sstem is poposed in the pape using the Kalman filte based on eliabilit measue. The poposed technolog is extemel useful when it finds and tacs a sound souce at a distance within two metes as illustated in the expeimental esult. The novelt of the pape can be found in the calculation of the measuement eo covaiance matix to filte out uneliable DOA estimations in Kalman filte pocess. The poposed DOA estimation tacs ve fast fo the eliable signals such as loud o noiseless voice wheeas tac ve slowl fo the uneliable signals such as noise. REFERENCES [] M. N. Nicolescu, M. J. Mataic, Leaning and Inteacting in human-obot domains, IEEE Tans. Sstem, Man and Cbenetics, at A., vol. 3, no. 5, pp , Sep.. [] D. Kim, J. Lee, H. S. Yoon, and E. Y. Cha, A Non-Coopeative Use Authentication Sstem in Robot Envionment, IEEE Tans. Consume Electon., vol. 53, no., pp. 84-8, Ma., 7. [3] J. Caspe and R. R. Muph, Human-Robot Inteactions Duing the Robot-Assisted Uban Seach and Rescue Response at the Wold Tade Cente, IEEE Tans. Sstem, Man and Cbenetics, at B., vol. 33, no. 3, pp , June., 3. [4] T. See, L. Wolf, S. Bileschi, M. Riesenhube, T. oggio, Robust Object Recognition with Cotex-Lie Mechanisms, IEEE Tans. atten Analsis and Machine Intelligence, vol. 9, no. 3, pp. 4-46, Mach, 7. [5] M. Omologo and. Svaize, Use of the Cosspowe-Spectum hase in Acoustic Event Location, IEEE Tans. Speech and Audio ocessing, vol. 5, no. 3, pp. 88-9, Ma, 997. [6] D. N. Notin and R. Duaiswami, Acceleated Speech Souce Localization via a Hieachical Seach of Steeed Response owe, IEEE Tans. Speech and Audio ocessing, vol., no. 5, pp , Sep., 4. [7]. Stoica and R. L. Moses, Intoduction to Spectal Analsis, entice-hall, Englewood Cliffs, NJ, 997. [8] C. H. Knapp, G. C. Cate, The Genealized Coelation Method fo Estimation of Time Dela, IEEE Tans. Acoustics, Speech, and Signal ocessing, vol. ASS-4, no. 4, Aug., 976. [9] D. Hetz, Time Dela Estimation b Combining Eficient Algoithms and Genealized Coss-Coelation Methods, IEEE Tans. Acoustics, Speech, and Signal ocessing, vol. ASS-34, no., Feb., 986. [] J. M. Valin, F. Michaud, J. Rouat, Robust 3D Localization and Tacing of Sound Souces Using Beamfoming and aticle Filteing, oc. IEEE Intl. Conf. on Acoustic, Speech, and Signal ocessing, pp , 6. [] D. Bechle, M. S. Schlosse, and K. Koschel, Sstem fo Robust 3D Speae Tacing Using Micophone Aa Measuements, oc. IEEE/RSJ, Intl. Conf. on Intelligent Robots and Sstems, pp. 7-, 4. [] G. Welch and G. Bishop, An Intoduction to the Kalman Filte, Dept. Comp. Sci. Uinv. Noth Caolina, Chapel Hill, TR95-4. [3] C. T. Kim, J. J. Lee, H. Kim, Vaiable ojection Method and Levenbeg-Maqaudt Algoithm fo Neual Netwo Taining, IEEE 3nd Annual Conf. Industial Elec., pp ,