Mobile Robot Obstacle Avoidance in a Computerized Travel Aid for the Blind

Transcription

1 1994 IEEE. Reprinted, with permissin, frm Prceedings f the 1994 IEEE Rbtics and Autmatin Cnference, San Dieg, Califrnia, May 8-13, 1994, pp Mbile Rbt Obstacle Avidance in a Cmputerized Travel Aid fr the Blind by Shraga Shval, Jhann Brenstein, and Yram Kren Department f Mechanical Engineering and Applied Mechanics The University f Michigan, Ann Arbr Abstract A blind traveler walking thrugh an unfamiliar envirnment, and a mbile rbt navigating thrugh a cluttered envirnment have an imprtant feature in cmmn: bth have the kinematic ability t perfrm the mtin, but are depended n a sensry system t detect and avid bstacles. This paper describes the use f a mbile rbt bstacle avidance system as a guidance device fr blind and visually impaired peple. Just like electrnic signals are sent t a mbile rbt's mtr cntrllers, auditry signals can guide the blind traveler arund bstacles, r alternatively, they can prvide an "acustic image" f the surrundings. The cncept has been implemented and tested in a new traveling aid fr the blind, called the Navbelt. Experimental results f subjects traveling with the Navbelt in different surrundings are presented. 1. Intrductin The mtin f a blind traveler in an unfamiliar envirnment is smewhat similar t that f a mbile rbt. Bth have the physical ability t perfrm the mtin, but are depended n a sensry system t detect bstacles in the surrundings, and t relay the infrmatin t the cntrl system (human brain r mtin cntrl cmputer). In bth cases, reliable bstacle system is mandatry fr fast and safe mtin. In rder fr a blind persn t fllw a particular rute, the persn must have sme cncept r plan f that rute. Smetimes, the traveler can learn the plan while being guided by a sighted escrt. At ther times, the traveler has nly verbal instructins t g by. Once a rute has been learned, successful travel requires the individual t be able t: (1) detect and avid bstacles and (2) fllw the rute precisely (i.e., t knw their psitin and rientatin and, if necessary, make crrectins). The perfrmance f bth tasks can be enhanced thrugh electrnic devices, called electrnic travel aids (ETAs). Amng the cmmn ETAs are the C5 Laser Cane, the Mwat Sensr, the Nttingham Obstacle detectr, the Binaural Snic Aid, the Talking Signs, and the Sna System. Existing ETAs are nt widely used, because f fur majr shrtcmings: 1. The user must actively scan the envirnment t detect bstacles. This prcedure is time-cnsuming and requires the user's cnstant activity and cnscius effrt. 2. The user must perfrm additinal measurements when an bstacle is detected, in rder t determine the dimensins f the bject. A path must then be planned arund the bstacle; again, a time cnsuming effrt that reduces the walking speed.

2 3. Only a prtin f the required mbility infrmatin is acquired r transferred t the user. The infrmatin gathered by the sensrs is inaccurate and subject t external nise. Applying a mbile rbt bstacle avidance system in a travel aid fr the blind eliminates several f the shrtcmings fund in existing devices. Using multiple ultrasnic sensrs that face in different directins frees the user frm the need t scan the surrundings manually. Furthermre, n additinal measurement is required when an bstacle is detected, since its relevant dimensins are determined simultaneusly by the multi sensr system. In additin, the bstacle avidance system can guide the blind traveler tward a target while aviding bstacles alng the path. 2. The Navbelt Based n ur experience with bstacle avidance fr mbile rbt, we have develped a new travel aid fr the blind, called the Navbelt [Brenstein, 1990; Shval et al. 1993a]. The Navbelt cnsists f a belt, a prtable cmputer, and ultrasnic sensrs. In this system, the cmputer prcesses the signals that arrive frm the sensrs, and applies the bstacle avidance algrithm. The resulting signals are relayed t the user by sterephnic headphnes, using a stere imaging technique. The similarity between this apprach and the riginal mbile rbt applicatin is illustrated in Fig. 1. The electrical signals which riginally guided the rbt arund bstacles are replaced by acustic (r tactile) signals. The Navbelt was riginally designed fr tw peratinal mdes: 1) Guidance Mde - The acustic signals actively guide the user arund bstacles in pursuit f the target directin. The signals carry infrmatin regarding the recmmended directin and speed f travel, and infrmatin abut the prximity t bstacles. Figure 1: Transfer f technlgy: mbile rbt bstacle avidance applied as a mbility aid fr the blind. 2) Image Mde - This mde presents the user with an acustic panramic image f the envirnment by using sterephnic effects: sund signals appear t sweep thrugh the user's head frm the right ear t the left. The directin t an bstacle is indicated by the perceived spatial directin f the signal, and the distance is represented by the signal's vlume. The Guidance mde is very similar t the cntrl f a mbile rbt. Hwever, this mde f peratin requires cnstantly updated infrmatin abut the traveler's psitin. In a mbile rbt this infrmatin is easily btained frm encders attached t the vehicle wheels (Odmetry). Hwever, the Navbelt is currently nt equipped with any psitining feedback device. Pssible feedback systems fr the Navbelt might cmprise f a pedmeter and an electrnic cmpass, but it is nt clear if this system wuld be accurate enugh. T vercme this prblem, we intrduced a third perating mde - the directinal guidance mde. This mde is based n the tele-autnmus mde f peratin, which was als riginally develped fr mbile rbts [Brenstein and Kren, 1990]. In this mde the system actively guides the user tward a temprary target, the lcatin f which is determined by the user. The user can prescribe the directin f the temprary target with a jystick; r when a jystick is nt used,

3 the target is assumed t be straight ahead f the user. In case an bstacle is detected, the Navbelt guides the user arund the bject with minimal deviatin frm the target directin. This mde is particularly efficient fr wall fllwing r detecting crners, dr ways etc. 3. The bstacle avidance system in the Navbelt The bstacle avidance system in the Navbelt is based n a plar map, which is divided int eight sectrs (fr the eight ultrasnic sensrs f the Navbelt). Existence f an bstacle is represented as an arc in the sectr (Fig. 2). T reduce the ccurrence f errneus readings due t nise, specular reflectin r crsstalk, we have integrated a pwerful nise reductin algrithm, knwn as EERUF [Brenstein and Kren, 1992]. EERUF allws the snars t fire at rates which are five t ten times faster than thse attained with cnventinal ultrasnic firing methds. At the same time EERUF reduces the number f errneus readings by ne t tw rders f magnitude. In additin, a lw pass filter is applied t further reduce the affect f inaccurate snar readings. Implementing the EERUF methd in the Navbelt is dne as fllws: [m] Plar Graph 1) EERUF cntrls the ultrasnic sensrs and filters errneus readings befre they are prcessed by the Figure 2: The bstacle avidance system in the bstacle avidance algrithm. Navbelt based n EERUF 2) The wrld mdel is divided int eight sectrs, each representing ne snar. The width f the sectrs is similar t the span f the ultrasnic ray fired by the snar (apprximately 15). Based n the snar's reading, the crrespnding sectr is filled with the range t an bstacle. The sectr range is updated as sn as a reliable reading is accepted by the EERUF cntrl. 3) A plar graph is than cnstructed frm the sectrial map. The value f each sectr in the graph is inversely p

4 rprtinal t the distance f the bject f the crrespnding sectr in the plar map. The recmmended travel directin is cmputed frm the plar graph. The plar graph has "peaks," which represent bstacles in that directin, and "valleys," representing bstacles far away frm the traveler, r n bstacles at all (see Figs. 2 and 3b). A threshld determines the margin f safety. Plar sectrs with values higher than the threshld are cnsidered unsafe directins as they represent bstacles in clse prximity t the traveler. Similarly, sectrs with a lwer value than the threshld are cnsidered safe directins since n bstacles are immediately threatening the traveler. Valleys cmprised f plar sectrs lwer than the threshld are called candidate valleys. Usually there are tw r mre candidate valleys, and the bstacle avidance algrithm selects the ne that mst clsely matches the directin f the target. The width f the selected valley (the number f cnsecutive sectrs in the plar graph belw the threshld) determines the required travel directin. If the valley is narrw (i.e. the number f sectrs belw the threshld is smaller than a certain value) then the required directin is at the center f the valley. In such a case, the recmmended directin is centered between clsely spaced bstacles. When the valley is wide, the algrithm selects a travel directin at a safe distance frm bstacles, but with minimal deviatin frm the target directin. Figure 3 illustrates the selectin f the candidate valleys and the required travel directin. Nte that this methd is identical t the ne that was intrduced in the VFH bstacle avidance methd fr mbile rbts [Brenstein and Kren, 1991]. The recmmended travel speed is determined accrding t the size f the valley, the distance frm the bstacles, and the user's capabilities (age, mtric skills etc.). A narrw valley r clse distance t an bstacle require mre cautius mtin, thus the recmmended speed is reduced prprtinally accrding t Eq. (1). Vmax d w V = )) [ )) + ))] (1) 2 dmax wmax C Navbelt a B A Target Threshld 180 Large valleys C 135 Selected directin b Narrw valleys Figure 3: Selectin f travel directin in guidance mde where: d max, w max - maximum distance (5 meters) and maximum valley's width (35) d, w - actual distance and valley's width V max - maximum travel speed 4. Implementatin f auditry image signals 90 B 45 A 0 Target directin The image mde prvides the user with a panramic auditry image f the surrundings. The principle is similar t the peratin f a radar system (used in air traffic cntrl, submarines etc.). An imaginary beam travels frm the right side f the user t the left thrugh the sectrs cvered by the Navbelt's snars (a span f 120 and 5 m radius). A binaural feedback system invkes the impressin f a virtual sund surce mving with the beam frm the right t the left ear in what we call a sweep. This is dne in several discrete steps, crrespnding t the discrete virtual directin steps. At each step, the amplitude f the signal is set prprtinally t the distance t the bject in that virtual directin. If n bstacles are detected by the beam, the virtual sund surce is f a lw amplitude and barely

5 audible. If, n the ther hand, bstacles are present, then the amplitude f the virtual sund surce is luder. The amplitude increases fr clse bjects, and decreases if bjects are further away. Figure 4 demnstrates the principle f the image mde. Obstacles are detected by the ultrasnic sensrs (Fig 4a), and are prjected nt the plar graph (Fig. 4b). Based n the plar graph, the binaural feedback system generates the sweep, which cmprises f 12 steps (Fig. 4c). Each step "cvers" a sectr f 15, thus the whle sweep cvers a panrama f 180. Each f the eight sectrs in the center f the panrama (cvering the sectrs between 30 and 150 ) is directly prprtinal t the crrespnding sensr. The remaining fur sectrs (tw at each side f the panrama) represent sectrs which are nt cvered by the snars. The value f these sectrs is calculated based n the values f adjining sectrs. Fr example, if the third sectr (representing the first snar) cntains an bject, than the first and secnd sectrs are autmatically assigned the same value. This way, the sides f the traveler which are nt cvered by the sensrs are blcked, therefre eliminating the pssibility f turning int an unchecked area. Each signal is mdulated by an amplitude A (indicating the distance t the bstacle in that directin), the duratin T fr which the square s wave signal is audible, and the pitch f f the square wave. The spacing time T is the length f the n interval between cnsecutive signals during a sweep. After each sweep there is a pause f duratin T, t allw the user t cmprehend the cnveyed c image. Many meaningful cmbinatins f these parameters are pssible. Fr example, because f the shrt term memry capability f the human ear, a sweep may be as shrt as 0.5 sec. Given enugh cgnitin time T, the user will cmprehend the c image. Alternatively, the sweep time may be as lng as ne secnd, cmbined with a very shrt cgnitin time. Ntice that each sweep starts with an anchr signal. This signal has a unique pitch, thus prviding the user with a cnvenient identificatin f the start f a sweep. 5. Implementatin f auditry guidance signals Anchr 180 VI VII VIII 135 Amplitude and Pitch Obstacles V IV III II Navbelt a 90 b c I 45 Ts 0 Tn Anchr Figure 4: The image mde. bstacle are detected by the ultrasnic sensrs (a), prjected nt the plar graph (b), and an acustic sweep is generated (c). Implementing the guidance mde in the Navbelt is simpler than the image mde, as the amunt f transferred infrmatin is far smaller. In the guidance mde the cmputer prvides the user nly with the recmmended travel speed and directin, based n the bstacle avidance algrithm. The travel speed and directin are relayed t the user by a single sterephnic signal. The virtual directin f the signal is the directin the bstacle avidance system has selected fr travel. The pitch and amplitude are prprtinal t the recmmended travel speed. Higher pitch and amplitude attract mre human attentin, thus the traveler intuitively reduces the walking speed and cncentrate n the sterephnic signal. A special lw pitch signal (250 Hz) is transmitted when the directin f mtin cincides (within +/- 5 ) with the required directin. This special tne serves tw purpses: Tc

6 1) The lw pitch tne is an efficient and simple feedback signal fr the user, indicating that the travel directin is crrect. 2) Lw pitch tnes cclude external sund frm the envirnment less than medium and high pitch tnes. One f the main cncerns f ptential users f the Navbelt is that the sterephnic signals may cclude the perceptin f imprtant external sunds. Hwever, when the user is traveling in the right directin, less external sund is ccluded by the lw pitch tne. The higher pitch tne is transmitted nly when the traveler needs t change the travel directin, and as sn as that directin cincides with the recmmended directin the lw pitch returns. Anther imprtant parameter is the rate in which signals are transmitted. Althugh a lw transmissin rate causes less cclusin f external sunds, it may als be t slw fr alerting the traveler t hazards. An adaptive infrmatin transfer system [Shval et. al., 1993b] adjusts the transmissin rate accrding t changes in the prcess and the user's requirements. Fr example, when the user is traveling in an unfamiliar envirnment cluttered with a large number f bstacles, the transmissin rate increases t up t 10 signals per secnd. On the ther hand, when traveling in an envirnment with little r n bstacles, the transmissin rate is reduced t ne signal every 3 secnds. 6. The Navbelt simulatr T investigate the Navbelt cncept under different situatins, a simulatr was develped. The simulatr is based n the same hardware as the Navbelt, and the same acustic signals that guide the user in the real Navbelt are used in the simulatr. The user's respnse t these signals is relayed t the cmputer by a jystick. Several maps are stred in the cmputer's memry, representing different types f envirnments with different levels f travel cmplexity. In the experiments with the simulatr, subjects "traveled" thrugh the different maps while listening t the sunds generated by the cmputer. In the experiments with the image mde, the psitin f the subject and the lcatin f the target were shwn n the cmputer screen. When subjects "cllided" with an bstacle, a verbal message infrmed the subject abut the cllisin, and n frward travel was permitted. The subject then turned r backed up t cntinue traveling tward the target. After reaching the target, the full map was superimpsed n the traveling path s that the subject's perfrmance during the run culd be evaluated. In the experiment with the guidance mde, nly the traveler's simulated psitin was shwn n the screen while the target psitin was unknwn t the subject. This is the equivalent f traveling in an unfamiliar envirnment, where the traveler depends entirely n the guiding signals frm the Navbelt. As with the image mde, when subjects "cllided" with an bstacle, a verbal message infrmed abut the cllisin, and n frward travel was permitted. 7. Experiments with the Image mde Experiments with the image mde included three stages. In the first stage subjects listened t different sweeps and drew a map f the envirnment based n the auditry infrmatin. Several cmbinatins f the sweep parameters were examined. The aim was t determine a gd cmbinatin fr perceptin and fast cmprehensin f the transmitted signals. Gd results frm these experiments were btained with the fllwing parameters: Signal transmissin time T varies between msec, where 20 msec is used fr the lngest s distance frm an bject (5 meters) and 40 msec fr very clse bjects (0.5 meter). Space time T n between signals is kept cnstant thrugh the whle sweep with T n = 10 msec.

7 The amplitude A varies inversely with the range reading frm the crrespnding snar sectr. Sixteen discrete amplitudes can be selected, where the lwest value represents n threat t the traveler and is nt audible at all, while the maximum value represents high risk frm that directin. Twelve signals per each sweep (nt including the anchring signals) are sufficient fr safe travel, and they require less than 0.5 secnds fr the transmissin f the whle sweep. Cgnitin time Tc between sweeps varies accrding the user's capabilities and the cmplexity f the sweep. The cmplexity f the sweep is calculated befre the sweep is transmitted based n the plar graph. The cmplexity is defined as the sum f the number f nn- zer signals in the plar graph, the difference between the signals within the graph, and the rate at which signals are changed in time. Eq. 2 describes the calculatin f the sweep's cmplexity C = A (t) + {A (t) - A (t)} + (A )/ t (2) 1 1 i-1 1 i=0 i=0 i=0 where A (t) is the value f sectr i in the plar graph at time t. i The secnd stage in the experiments included the Navbelt simulatr. 100 maps were selected randmly by the cmputer frm a list f 10 maps. The average travel speed was 0.52 m/sec frm all 100 runs with the image mde. An imprtant cnclusin frm this experiment was that the cmplexity f the envirnment has a majr affect n the traveling speed. A mre cmplex envirnment requires mre cnscius effrts frm the user, thus causing a reductin in the travel speed. Anther imprtant element which affects the travel speed, is the user's experience with the Navbelt. Thse with less experience r reduced auditry perceptin capabilities travel slwer than highly skilled peple. An adaptive auditry infrmatin transfer (AAIT) methd [Shval et. al., 1993b] can imprve the traveling speed by cntinuusly adjusting the infrmatin flw frm the cmputer t the persn. This adjustment is based n changes in the envirnment as well as n the persn's skills, which are represented in a perfrmance mdel. In a simulatr experiment, the AAIT methd imprved the travel speed by an average f 30% cmpared with nn adjustable infrmatin transfer. Finally, experiments with the actual Navbelt prttype were cnducted. Subjects traveled thrugh the cntrlled envirnment f the labratry, relying nly n the Navbelt's audi signals in the image mde. Obstacles were randmly psitined in the labratry, and the blindflded subject were asked t travel frm ne side f the labratry t the ther. Travel speed in these experiments was slwer than with the Navbelt simulatr, mainly because subjects were mre cautius. Hwever, after a training perid f several hurs they traveled safely thrugh the cntrlled envirnment f the labratry with an average speed f 0.4 m/sec. 8. Experiments with the Guidance mde The experiments with the guidance mde were cnducted nly with the Navbelt simulatr. A perfrmance index was intrduced fr the evaluatin f human travel in the guidance mdes. The evaluatin f the perfrmance is based n a cntinuus cmparisn f the traveler's actual trajectry t the recmmended trajectry by Eq. 3: P = [ (V - V ) dt + D - D dt]/t (3) a r a r where

8 P - Human accumulated perfrmance V r,va- Recmmended and actual traveling speed D r,da- Recmmended and actual traveling directin t - Ttal travel time V r and D r f Eq. 3 are calculated by the bstacle avidance system. a V and a D are measured accrding t the jystick psitin. Since the bstacle avidance system eliminates the pssibility f cllisin with bstacles, any deviatin f the traveler frm the recmmended directin is undesirable and results in a reductin in perfrmance evaluatin. Experiments with the Guidance mde n the Navbelt simulatr included 200 tests. Averaging the results frm all tests shws a travel speed f 0.76 m/sec, with an average directinal deviatin (deviatin frm the recmmended travel directin) f 7.7. The results naturally vary with the level f difficulty f each map, with the best results reaching a travel speed f 0.95 m/sec and directinal deviatin f 0.9, while the wrst result was 0.55 m/sec and The experiments with the directinal guidance mde were cnducted with the Navbelt prttype. In the first series f experiments, subjects were asked t travel frm ne side f the labratry t the ther side (12 meters). They were blindflded, and bstacles were psitined randmly in the rm. The average traveling speed achieved in this experiments was 0.45 m/sec. The next experiment tk place in a mre realistic envirnment. The subjects were asked t travel alng a standard ffice building crridr, with which they were familiar. The length f the path was 25 meters and included several crners. N bstacles were initially psitined alng the crridr, but passer-by were walking thrugh the crridr during the tests. Average speed in this experiment was 0.6 m/sec. Since n psitining feedback was available, n measurements f the directinal accuracy were taken. Hwever, during mst f the experiments, subjects were able t detects bstacles, and avid them safely, with n additinal detectin aid being used. 9. Cnclusins A mbile rbt bstacle avidance system has been cnverted successfully t a navigatin aid fr the blind. Instead f transmitting electrnic signals t the rbt mtin cntrllers, the bstacle avidance system relays infrmatin t the user by transmitting sterephnic signals. These signals prvide spatial infrmatin abut the lcatin f bjects in space, r guiding infrmatin fr the recmmended travel directin and speed. The methd is implemented in a new travel aid fr the blind, the Navbelt. Average traveling speed n the Navbelt simulatr in the guidance mde was 0.76 m/sec. The average traveling speed f blindflded subjects traveling with the Navbelt prttype was 0.45 m/sec, and 0.6 m/sec alng a typical ffice building crridr. Future develpments f the Navbelt will include additinal ultrasnic sensrs fr detecting verhanging bstacles, halls, steps etc. 10. References 1. Brenstein J., 1990, "The Navbelt - A Cmputerized Multi-Sensr Travel Aid fr Active Guidance f the Blind," CSUN's Fifth Annual Cnference n Technlgy and Persns with Disabilities, Ls Angeles, Califrnia, March 21-24, pp Brenstein J., Kren Y., 1991, "The Vectr Field Histgram - Fast Obstacle Avidance fr Mbile Rbts," IEEE Jurnal f Rbtics and Autmatin, Vl. 7, N. 3., June 1991, pp Brenstein J., Kren Y., 1990, "Tele-Autnmus Guidance fr Mbile Rbts," IEEE Tran. System, Man, and Cybernetics, Special Issue n Unmanned Systems and Vehicles, December, pp

9 4. Brenstein J., Kren Y., 1992, "Nise Rejectin fr Ultrasnic Sensrs in Mbile Rbt Applicatins," Prceedings f the IEEE Cnference n Rbtics and Autmatin, Nice, France, May, pp Shval S., Brenstein J., Kren Y., 1993a, "The Navbelt - A Cmputerized Travel Aid fr the Blind," Prceedings f the RESNA Cnference, June 13-18, Las Vegas, Nevada, pp Shval S., Kren Y., Brenstein J., 1993b, "Adaptive Infrmatin Transfer fr Traveling by Auditry Lcalizatin in a Navigatin Aid fr the Blind", in preparatin.

10 Here is a pht f the Navbelt. This pht was nt part f the paper. The NavBelt testing device is wrn by PhD. student (nw Dr.) Shraga Shval in 1994.