USING BACKPROPAGATION NEURAL NETWORK IN OBJECT RECOGNITION FOR HYBRID MECHANISM MOBILE ROBOT

Transcription

1 Proceedngs of the ASME 21 Internatonal Mechancal Engneerng Congress and Exposton IMECE21 November 12-18, 21, Vancouver, Canada IMECE USING BACKPROPAGATION NEURAL NETWORK IN OBJECT RECOGNITION FOR HYBRID MECHANISM MOBILE ROBOT Samer Charfa Computer Scences Corporaton Alexandra, VA, USA Pnhas Ben-Tzv Robotcs and Mechatroncs Lab The George Washngton Unversty Washngton, DC, USA ABSTRACT Navgaton of moble robots n envronments wth rregular terran s a challengng task for scentsts and engneers because t nvolves 3D envronment recognton and hgh order dynamcs of the moble robotc systems. One way to solve ths problem s to use hybrd locomoton mechansms. In ths paper, we am to establsh a novel frame work for hybrd local planner whch combnes moton prmtves wth artfcal neural networks for navgaton n rough terran. Ths artfcal neural network decson engne wll fnd the optmal modes of locomoton for the Hybrd Mechansm Moble Robot. The nput to the neural network s the measurements of the obstructon and the output s the confguratons sutable for surmountng an obstacle. Once the robot dscovers a change n the terran (such as hgh step, low step, etc.), number of measurements wll be taken usng a feature extracton method to decde the locomoton mode sutable for a partcular terran. Several measurements of the obstructon are consdered, such as heght, area, and depth of each surface n the scene. These are computed from the 3D representaton of the envronment bult usng the on-board sensors, stereo cameras, and a 3D laser range fnder. These measurements are fed nto a backpropagaton neural network n order to choose the successful canddate robot confguraton. Keywords Moble robots, Autonomous navgaton, Object recognton, Artfcal Neural Networks 1. INTRODUCTION Snce Shakey, the frst robot to navgate autonomously [1], robots have started sharng the same workspace wth humans. Clearly, t s very neffcent to restrct the locomoton of moble robots to regular wheeled locomoton and consequentally restrct ther workspace to ndoor envronments. However, many problems are nvolved n roughterran navgaton such as 3D envronment recognton and sometmes moton plannng of complex systems wth multmode locomoton. A possble approach to reduce the computatonal complexty of the moton plannng n rough terrans s based on restrctng the possble trajectores of the robot to a famly of curves/straght lnes that can be obtaned from the nterconnecton of approprately defned prmtves [2]. Moton prmtves and other types of maneuvers have been appled wdely to robotcs and dgtal anmaton. Number of general strateges has been used: a) Record and playback: Ths strategy restrcts moton to a lbrary of maneuvers. For example, humanod locomoton can be planned as a sequence of pre-computed steps [3]. Robust helcopter flght can be planned as a sequence of feed-forward control strateges to move between trm states [4-5]. The moton of peg-clmbng robots can be planned as a sequence of actons lke grab the nearest peg [7]. b) Model reducton: Ths strategy plans overall moton. For example, another way to generate natural-lookng humanod locomoton s to approxmate the robot as a cylnder, plan a 2D collson-free path of ths cylnder, and follow ths path wth a fxed gat [8]. A smlar method s used to plan the moton of nonholonomc wheeled vehcles [9]. c) Bas nverse knematc solutons: Lke model reducton, ths strategy frst plans the moton of key ponts on a robot or dgtal actor, such as the locaton of hands or feet or center of gravty. But nstead of a fxed controller, a search algorthm s used to compute a pose of the robot at each nstant that tracks these 1 Copyrght 21 by ASME

2 ponts (an nverse knematc soluton). One approach s to choose an nverse knematc soluton accordng to a probablty densty functon learned from hgh-qualty example motons [1, 11]. Recently, moton-prmtve-based algorthms have been successfully used n robotc navgaton. Hwangbo et al. [12] have used moton prmtves for local moton plannng of a sngle wng UAV. Goldberg et al. [13] have proposed the so called GESTALT navgaton algorthm. GESTALT s a set of routnes that fnd the next best drecton for a robot to move, gven the state of the world already seen, updated sensor data, and a desred waypont goal. Pvtorako et al. [14] proposed a new set of moton prmtves whch relaxed the condton that all moton prmtves are placed on constant curvature arcs. They have shown that by usng moton prmtves wth varous curvatures rather than constant curvature wll mprove the performance of the dual global and local navgaton system. For many robotc platforms, such as ndoor moble navgaton or UAV s, the choce of the best moton prmtve s mostly determned by number of factors such as mnmum dstance [12], least tme, or least energy. For a free-clmbng robot, such as the robot consdered n ths paper, many constrants play equally mportant role. These are statc equlbrum, closed-chan knematcs, collson-avodance, and torque lmts; all of whch affect the choce of the wnnng moton prmtve dfferently at each set of motons to be performed [15]. In ths paper, we propose a frame work for hybrd local planner whch combnes moton prmtves wth artfcal neural networks for navgaton n rough terran. The applcaton of artfcal ntellgence technques such as neural networks n the decson makng of the robot has been successfully done n a wde varety of robotc platforms. Antsakls [16] suggested that the use of Artfcal Neural Networks (ANN) n control systems s a natural evolutonary step to meet new challenges because they have the potental for very complcated system. Snce then, the ANNs have been used n many aspects of control systems. Robotcs moton plannng s one of the challengng problems especally for robots wth large number of degrees of freedom and eventually hgher dmensonal confguraton space. Recently, a number of researchers have appled ANN s n robot navgaton. Harb et al. [17] have used an ANN to perform object recognton and robot navgaton. They were able to recognze envronments such as corrdors, ntersectons, corners etc. However, t s not clear how the plannng part of ther algorthm navgates the robot. Gao and Han [18] have used neural networks to solve the problem of obstacle avodance and navgaton for an ndoor unmanned aeral vehcle based on mage data. Pettersson et al. [19] have used neural networks for executon montorng purposes. They were able to dentfy model-free falure predcton. Also, Hou et al. [2] have proposed neural network schemes for nformaton processng, localzaton and navgaton of moble robots. In ther approach, they have used the neural network to solve the optmzaton problem for path plannng. The path plannng algorthm s based on the mnmzaton of the dstance to an obstacle. The proposed algorthm wll use moton prmtves to plan the steps for the robot clmbng. Moreover, Backpropagaton Neural Network (BNN) wll be used to choose the wnnng moton prmtve. 2. HYBRID MECHANISM MOBILE ROBOT Wthout a loss of generalty, the proposed algorthm wll be appled to the Hybrd Mechansm Moble Robot (HMMR) shown Fgure 1. The HMMR s a mult-confguraton moble robot, whch has the ablty to utlze ts manpulator arm to clmb obstacles as well as for manpulaton purposes [21 24]. Furthermore, each lnk has the ablty to be folded nsde the prevous lnk so as to change the number of degrees of freedom dependng on the requred locomoton mode. Consequently, the HMMR can generate several modes of locomoton contngent on whether t fully or partally deploys ts manpulator arm. Ths specal property allows t to overcome regular terrans/obstacles (such as stars, dtches, and steps) and rregular terrans (such as a rubble ple). The full geometrcal symmetry of the HMMR provdes t wth the ablty to deploy ts lnks from both sdes of the platform, whch means that the robot wll have the same functonalty even f t were to flp over. Overall, the HMMR ncorporates two tracked platforms actuated ndependently and provde tracton to the robot. A central manpulator arm wth two lnks and two degrees of freedom (DOF) s cascaded n between the tracks. The actuaton of these DOF s s performed separately va motors located under the tracks. Ptch Actuated Laser Range Fnder Fgure 1: CAD drawng of the HMMR In addton to the grpper, the thrd lnk n the HMMR accommodates a servo-actuated mechansm that carres a Ptch- Actuated Laser Range Fnder (PALRF) and a stereo vson system. A sngle board computer s also housed nsde the lnk. Ths connects drectly to the camera and Ldar n order to process mages and synthesze actons accordngly. Ths robot s currently n the manufacturng process. The detaled 2 Copyrght 21 by ASME

3 descrpton of the HMMR s not n the scope of ths paper, but nterested readers can fnd more nformaton n references [21 24]. In ths paper, we are nterested n presentng the autonomous decson makng process of the robot supported by smulatons. 3. THE GENERAL FRAME OF LOCAL PLANNING The dual plannng n autonomous navgaton s a wdely adopted practce n robotcs. Most moble robot navgaton technques are developed at two levels: the local level and the global level. The global planner s assgned the msson of fndng the optmal path plan based on the lmted pror knowledge of the envronment. However, the local planner deals wth navgaton on the scale of a few meters, where the man problem s obstacle avodance. The local navgaton s manly useful for rapd responses to avod collsons. Moreover, the local planner can be of specal use for robots whch have the ablty to surmount an obstructon f needed, lke the HMMR. In ths case, once the sensors detect an obstructon, the robot should be able to classfy ths object nto classes based on the features of the obstructon. In the HMMR, these features are extracted usng the 3D laser range fnder. The classfcaton process wll not only classfy the obstructon nto surmountable or non-surmountable obstacles; t should also determne the type of surmountng process that the robot wll use. For a reconfgurable robot such as the HMMR, t uses dfferent confguratons for dfferent stuatons. In ths paper, these confguratons are lmted to four types of obstructons: Small step, Hgh step, Star, Wall. These confguratons are just examples to test the classfcatons process; however t s not lmted to the above mentoned four cases. In Fgure 2, the flow chart of the proposed local plannng algorthm s presented. The algorthm starts wth followng the global plan. Ths can be computed usng any sutable navgaton functon such as artfcal potental feld, Probablstc Road-Maps, or Rapdly-explorng Random Trees RRTs, etc. In ths paper, the harmonc artfcal potental feld s adopted. The harmonc potental feld has no local mnma. It performs very well as a global planner. However, t has been shown [25] that the harmonc potental feld performs poorly for local plannng. In ths paper, ths drawback s compensated by usng the proposed local planner. The local planner starts once the robot dscovers an obstructon usng a stereo vson system. Immedately thereafter, the robot wll start scannng the envronment usng Ptch Actuated Laser Range Fnder and a 3D cloud (X,Y,Z) representaton of the envronment wll be acqured. Ths s elaborated upon n the followng secton. The next step s to dvde the 3D cloud nto a number of moton prmtves n dfferent drectons. The moton prmtves are shown n Fgure 3, where a number of rays are llustrated. Every ray represents a possble drecton for a canddate moton prmtve. The next step s to compute metrcs of these canddates such as heght and wdth and degree of dffculty. These wll be fed nto a neural network whch wll label each wth a degree of dffculty. For now, we adopted the followng levels of dffculty: Wall = 4, star = 3, hgh step = 2, low step = 1, flat ground, =. The next step wll be to fnd the canddate moton prmtves wth the lowest dffculty score. If one s found wth score then follow t. If none s found then update the number of moton prmtves. If multple are found then check whch has the lowest potental feld values. Fgure 2: Flow chart represents the general frame of the local plannng algorthm 3 Copyrght 21 by ASME

4 Obstacles HMMR Moton prmtves Fgure 3: Moton prmtves for the HMMR 4. FEATURE EXTRACTION In Fgure 4, the Ptch Actuated Laser Range Fnder (PALRF) s shown. Every pont n the pont cloud data of the PALRF can be represented n terms of three varables (ρ, ψ, θ), where ρ s the dstance from the LRF to the poston of the pont of nterest, θ s the ptch angle, and ψ s the yaw angle. For every ptch angle θ, the values of the 2D polar varables (ρ, ψ) readngs of the LRF are projected nto a local coordnate system (x,y,z). The orgn of ths coordnate system s located at the center of the scannng level of the LRF. After movng n the ptch drecton, all ponts n the space wll be projected nto one global coordnate system. We have chosen the global coordnates system (X,Y,Z) to be located at the center of rotaton, as shown n Fgure 4. The homogenous transformaton wll transfer any pont P = [P ρ, P ψ, P θ ] T represented usng the varables (ρ, ψ, θ ), nto the global coordnates (P X, P Y, P Z ). Ths transformaton s descrbed n the followng equaton: PX 1 PY P Z 1 cos sn sn cos cos d sn sn, d cos 1 1 (1) Fgure 4: Ptch Actuated Laser Range Fnder An example of a 3D mage generated by the PALRF s shown n Fgure 6. Fgure 5 shows a 2D mage of stars taken by a dgtal camera. These stars were scanned usng the PALRF system shown n Fgure 4. All the scanned data was projected on one frame usng Equaton 2. Fgure 6 shows a 3D mage of the star usng PALRF. A segmentaton process has been appled on the 3D mage to segment each surface n order to extract useful nformaton from the 3D mage. Ths nformaton may be the heght of each step, dstance to each step, wdth of each step, etc. Ths s shown n Fgure 7. For example, the actual heght of each step was 28 cm whle the measured heght of the steps wth the PALRF was around 31 cm. It should be noted here that snce the laser scanner measures dstance to objects and not heghts, the process of projectng these dstances nto ther respectve surfaces and subtractng them to get the heghts could ntroduce error. Ths error may be attrbuted to two possble factors: the frst s the accuracy of fndng the edge of the step, and the second may be attrbuted to hardware lmtatons of the laser scanner n terms of fndng the accurate dstance. where d s the length of the rotatng arm. Eq. (1) maps every pont n the pont cloud nto a pont n the global coordnate system. Fgure 5: Scannng envronment of the PALRF 4 Copyrght 21 by ASME

5 If ncreasng a gven weght would lead to more error, that weght s reduced and vce versa After adjustng all the weghts up or down, ths process s restarted and contnued untl the weghts and the errors settle down The unqueness of general backpropagaton les n the method used to calculate the exact dervatves for all of the weghts n only one pass through the system. The GBPN works as two pass: forward evaluatons, whch use the nput events to calculate the network desred output Yˆ ; and Reverse Evaluaton Werbos [26] chan rule for ordered dervatves to calculate the error and adjust the weght, as show n Fgure Yˆ Fgure 6: 3D mage of the PALRF Yˆ Fgure 8: Forward and backward flow of GBPN Fgure 7: A segmentaton of the 3D mage nto surfaces 5. BACKPROPAGATION NEURAL NETWORK In ths secton, the neural network s brefly descrbed. After segmentng the envronment, the neural network wll nput slces of the 3D Ldar mage n order to make a decson on what moton prmtve the robot should consder, as explaned n Fgure 2. We assume that each nput event vector X has a dmenson of m and each output event vector has a dmenson of n (see Fgure 7). We also assume that the network has N+n tranable neurons. N can take any value, where N m. The backpropagaton neural network can be summarzed n the followng steps: Start wth assgnng arbtrary values for the weghts W Next, calculate the outputs Y(t) and the errors E(t) for that set of weghts Then, calculate the dervatves of E(t) wth respect to all of the weghts. In ths paper, the nput X s consdered to be the set of sub-3dmages, whch represents the moton prmtves. Each sub-3dmage s a slce of the whole 3D mage computed by the PALRF. In partcular, X s a set of 5x121 matrces. The output vector Y contans the labels of the sub-3d-mages. 5.1 Tranng Data of the Neural Network In ths secton, we show samples of tranng data for the neural network. These tranng data sets fall nto fve categores, as shown n Table 1. The HMMR has a lbrary of strateges for each of the fve sets. The robot uses dfferent locomoton technques n dealng wth each case. Ths s explaned n the followng table. Table 1: 3D mages of dfferent terrans are used as tranng set for the neural network Clear: Relatvely clear terran, traversable wthout changng confguraton from normal crusng mode 5 Copyrght 21 by ASME

6 plannng algorthm was able to ntalze 11 moton prmtves n the frontal drecton of the robot, as explaned n Fgure 3. The Moton Prmtves (MPs) are nput to the BNN and consequentally labeled by ther level of dffculty, as shown n Table 3. Low step: An obstructon that s below the heght of the tracks of the HMMR Table 3: Labelng results of the MPs shown n Fgure 9 Hgh step: Label MPs A step of suffcent heght so as to requre more sophstcated clmbng technques W 1,2,3,9,1,11 S 4 HS 5,6,7 Note that the levels of dffculty are lsted from easy to dffcult as: flat terran, small step, hgh step, star, and wall. Therefore, accordng to Table 3, the prmtves 4, 5 and 6 wll have the easest path. Consderng the confguraton of the robot and the global plannng, moton prmtve 6 would be the wnner. Stars: A common specal case for a seres of low steps Wall: An obstacle that s not traversable wth any avalable strategy from the lbrary Fgure 9: Sample scenaro to test the moton-prmtve algorthm 5.2 Testng the BNN 18 2 cases were collected for each of the above mentoned categores for tranng. Another set of 1 cases were used for testng. The neural network showed very good classfcaton. The percentages of correct classfcatons for each case are summarzed n the followng table Number of tested cases Number of mssed classfcatons Small (SS) 1 Step Star (S) 1 1 Hgh Step (HS) Table 2: Testng the recognton capablty of the BNN-based system Case Wall (W) The wnnng moton prmtve 2 Fgure 1: 3D representaton of the envronment n Fgure 9 showng the wnnng moton prmtve (dmensons n color bar are n [mm]) Note that the testng data vares n shape, dstance and orentaton from the tranng data; however, the robot was able to approxmate t nto one of the known cases. 7. CONCLUSIONS 6. SIMULATION RESULTS In ths secton, we have tested the moton prmtve algorthm on a scenaro that s shown n Fgure 9. The local In ths paper, we have establshed a novel framework for local plannng appled to clmbng robots as well as to robots workng n rregular terrans. The novel terran classfcaton 6 Copyrght 21 by ASME

7 algorthm s accomplshed by usng moton prmtves and a supervsed neural network. The feature selecton algorthm of the terran s based on a 3D mage of a Ptch Actuated Laser Range Fnder. Features of several types of terrans were used to tran a backpropagaton neural network. The neural network has been tested n a number of envronments. The smulaton results show correct classfcaton. The output generated was the correct path wth the lowest degree of dffculty. ACKNOWLEDGMENTS Ths work s supported by the Defense Advanced Research Projects Agency (DARPA) under grant number HR We also would lke to acknowledge the support provded by the DARPA Program Manager, Ms. Melane Dumas. REFERENCES [1] A Short Hstory of Robots: [2] Frazzol, E., Dahleh M.A., and Feron, E., 1999, Robust hybrd control for autonomous vehcles moton plannng, Techncal Report LIDS-P-2468, Laboratory for Informaton and Decson Systems, Massachusetts Insttute of Technology. [3] Kuffner, J.J., Nshwak, K., Kagam, S., Inaba, M., Inoue, H., 23, Moton plannng for humanod robots, In Internatonal Symposum on Robotcs Research, pp [4] Frazzol, E., Dahleh, M.A., Feron, E., 22, Maneuver-based moton plannng for nonlnear systems wth symmetres, IEEE Transacton on Robotcs, 25(1), pp [5] Frazzol, E., Dahleh, M.A., Feron, E., 22, Real-tme moton plannng for agle autonomous vehcles, AIAA Journal of Gudance, Control, and Dynamcs, 25(1), pp [6] Gavrlets, V., Frazzol, E., Mettler, B., Pedmonte, M., Feron, E., 21, Aggressve maneuverng of small autonomous helcopters: A human-centered approach, Internatonal Journal of Robotc Research, 2(1), pp [7] Bevly, D., Farrtor, S., Dubowsky, S., 2, Acton module plannng and ts applcaton to an expermental clmbng robot, In IEEE Internatonal Conference for Robotcs and Automaton, pp [8] Ng A.Y., Km H.J., Jordan M., Sastry, S., 23, Autonomous helcopter flght va renforcement learnng, Processng Advances n Neural Informaton Systems, pp [9] Laumond, J., Jacobs, P., Tax, M., Murray, R., 1994, A moton planner for nonholonomc moble robots, IEEE Transacton on Robotcs and Automaton, 1(5), pp [1] Lu, C.K., Hertzmann, A., Popov c. Z., 25, Learnng physcs-based moton style wth nonlnear nverse optmzaton, ACM Transacton on Graphcs, 24(3), pp [11] Grochow, K., Martn S.L., Hertzmann, A., Popov c, Z., 24, Style-based nverse knematcs, ACM Transacton on Graphcs, 23(3), pp [12] Hwangbo, M., Kuffner J., Kanade T., 27, Effcent Twophase 3D Moton Plannng for Small Fxed-wng UAVs, IEEE Internatonal Conference on Robotcs and Automaton, pp [13] Goldberg, S.B., Mamone M.W., Matthes, L., 22, Stereo Vson and Rover Navgaton Software for Planetary Exploraton, Proceedngs IEEE Aerospace Conference, pp [14] Pvtorako, M., Nesnas, I., Kelly, A., 29, Autonomous Robot Navgaton wth Advanced Moton Prmtves, IEEE Aerospace Conference, pp [15] Brtl, T., 26, Moton Plannng of Mult-Lmbed Robots Subject to Equlbrum Constrants: The Free-Clmbng Robot Problem, Internatonal Journal of Robotcs Research, 25(4), pp [16] Antsakls, P.J., 199, Neural Networks n Control Systems, Specal Issue on Neural Network n Control Systems, IEEE Control Systems Magazne, 1(3), pp [17] Harb, M., Abelmona, R., Naj, K., Petrul, E., 28, Neural Networks for Envronmental Recognton and Navgaton of a Moble Robot, IEEE Internatonal Instrumentaton and Measurement Technology Conference, pp [18] Gao, B., Han, L.S., 26, "Neural networks mplementaton of the vsual nformaton processng for an ntellgent aeral vehcle", IEEE Internatonal Conference on Cogntve Informatcs, pp [19] Pettersson, O., Karlsson, L., Saffott, A., 27, Model-Free Executon Montorng n Behavor-Based Robotcs, IEEE Transacton on Systems, Man, and Cybernetcs PART B, 37(4), pp [2] Hou, Z.G., Tan, M., Gupta, M.M., Nkforuk, P.N., Homma N., 25, Neural Network Methods for the Localzaton and Navgaton of Moble Robots, Canadan Conference on Electrcal and Computer Engneerng, pp [21] Ben-Tzv, P., Goldenberg, A.A., and Zu, J.W., 28, "Desgn and Analyss of a Hybrd Moble Robot Mechansm wth Compounded Locomoton and Manpulaton Capablty", Journal of Mechancal Desgn, 13, pp [22] Ben-Tzv, P., Goldenberg, A.A., and Zu, J.W., 27, "Implementaton of Sensors and Control Paradgm for a Hybrd Moble Robot Manpulator for Search and Rescue Operatons", In Proc. IEEE Internatonal Workshop on Robotc and Sensors Envronments (ROSE 27), Ontaro, Canada, pp [23] Ben-Tzv, P., Goldenberg, A.A., and Zu, J.W., 28, "Desgn, Smulatons and Optmzaton of a Tracked Moble Robot Manpulator wth Hybrd Locomoton and Manpulaton Capabltes", n Proc. IEEE Internatonal Conference on Robotcs and Automaton, ICRA 8, Pasadena, CA, pp [24] Ben-Tzv, P., 21, "Expermental Valdaton and Feld Performance Metrcs of a Hybrd Moble Robot Mechansm", Journal of Feld Robotcs, Vol. 27, Issue 3, pp [25] Tanner, H.G., Lozou, S.G., Kyrakopoulos, K.J., 23, Nonholonomc Navgaton and Control of Cooperatng Moble Manpulators, IEEE Transacton on Robotcs and Automaton, 19(1), pp [26] Werbos, P., 1994, The Roots Of Backpropagaton: From Ordered Dervatves To Neural Networks And Poltcal Forecastng, Wley-Interscence, New York, NY. 7 Copyrght 21 by ASME