Development of Outdoor Service Robots

Transcription

1 SICE-ICASE International Joint Conference 2006 Oct , 2006 in Bexco, Busan, Korea Development of Outdoor Service Robots Takesi Nisida 1, Yuji Takemura 1, Yasuiro Fucikawa 1, Suici Kurogi 1, Suji Ito 2, Masayuki Obata 2, Norio Hiratsuka 2, Hidekazu Miyagawa 2, Yasuiro Watanabe 3, Fumitaka Koga 3, Tosiri Sueiro 4, Yosiri Kawamura 5, Yosimitsu Kiara 6, Takasi Kondo 7 and Fujio Okawa 7 1 Fuculty of Engineering, Kyusu Institute of Teclogy, Fukuoka, Japan (Tel : ; nisida@cntl.kyutec.ac.jp) 2 YASKAWA INFORMATION SYSTEMS Corporation, Fukuoka, Japan 3 Mecanics & Electronics Researc Institute Fukuoka Industrial Teclogy Center, Fukuoka, Japan 4 Fukuoka Industry, Science & Teclogy Foundation, Fukuoka, Japan 5 FAIS Robotics Researc Institute, Fukuoka, Japan 6 Kiara Iron Works, Fukuoka, Japan 7 Fuculty of Computer Science and Systems Engineering, Kyusu Institute of Teclogy, Fukuoka, Japan Abstract: Te outdoor service robots wic we call OSR-01 and OSR-02 are presently under development intended for cleaning up urban areas by means of collecting discarded tras suc as plastic bottles, cans, plastic bags and so on. In tis paper, we mainly describe te arcitecture of OSR-02 wic consists of ardware suc as sensors, a manipulator, crawler structure, etc. for searcing and picking up tras. After describing te systems, we sow te result of open experiments in wic OSR-02 collected plastic bottles in te special zone for robot researc and development in Kitakyusu-city. Keywords: outdoor service robot, separated collection, laser rangefinder, camera. 1. INTRODUCTION Since November 28t of 2003, te Japanese government as designated te Fukuoka Prefecture, Fukuokacity, and Kitakyusu-city as te one of te special zones for robot researc and development. Tere researcers are allowed to operate teir robots in public areas by applying for police permission. Tus, for example, in sopping areas in te city, we can take advantages of executing realistic experiments of outdoor service robots for cleaning up streets, welcoming customers, guiding tem to a certain place, providing information about te sopping area, and so on. Under suc circumstances, we ave started developing te outdoor service robots called OSR- 01 and OSR-02 intended for cleaning up sopping streets by means of te separated collection of discarded tras, suc as plastic, glass, and steel, etc. on sopping streets. Te OSRs are designed to searc for tras wile navigating a designated route in te sopping area, and to collect it by means of te manipulators. So far, several type robots for te cleaning work ave been developed, owever, tey only vacuum on te specified route. On te oter and, since te developed robots aims to recognize te object in te sopping street etc. wit person s traffic and to collect tem wit te manipulator, te teclogy tat composes te robot and te algoritm of operation are greatly different. Altoug many operations ave been programmed for te above task, we, in tis paper, mainly sow te and te manipulation system of OSR-02 for tras detection and distance measurement of te target after giving a briefing about te ardware of te robot. Tis researc was funded by te Robotics Industry Development Council (RIDC). Te autors tank tem for teir support. 2. STRUCTURE 2.1 OSR-01 Te specification of OSR-01 is sown in Table 1 and Fig. 1 (see [1]-[3] for detail). Tis robot is first prototype robot, and te kind of te sensor, te number of arms, and te run system, etc. are different from OSR-02 described as follows. From te experiments tat used tis robot, around te entrance of te arcade were strengt of ambient ligt was greatly different and on te floor wit te value of brigtness and ue near tat of tras, te problem tat tras was t able to be recognized was clarified. Moreover, it turned out tat running in te outdoors by OSR-01 equipped wit te silicon tire was difficult. Terefore, OSR-02 was developed tat aimed at te solution of tese problems. 2.2 OSR-02 OSR-02 (Fig.2) as two manipulators and crawlers. Te lengt of te manipulators is 850[mm] and tey ave Table 1 Specification of OSR-01. D.O.F. manipulator 5 and 1 drive weel 2 dimension eigt 600[mm] widt 500[mm] dept 600[mm] manipulator lengt 750[mm] weigt 45[kg] running speed 1.8 [km/] sensor LRF 4 color CCD camera

2 top view top view touc panel display LED display and-eye camera arm camera LRF drive weel laser rangefinder camera x 2 laser rangefinder side view side view and-eye camera arm(6dof) crawler caster Fig. 1 Outdoor service robot OSR-01; entire robot; structure. five degrees of freedom and a and. Te specification of OSR-02 is sown in Table 2. Moreover, five computers for control of te crawler, te manipulators, and te sensor system are installed in OSR-02, and tey communicate via eternet. Furtermore, in order to recognize te surrounding environment, and t to arm surrounding persons, several sensors ave been installed in te robot. Table 2 Specification of OSR-02. D.O.F. manipulator 5 (x2) and 1 (x2) drive weel 2 dimension eigt 600[mm] widt 500[mm] dept 800[mm] manipulator lengt 850[mm] weigt 90[kg] running speed 2.0 [km/] sensor LRF 3 color CCD camera 4 force sensor (and) 2 magnetic sensor (and) 2 crawler servo motor 2 motor driver 2 arm 2 servo motor 6 potention meter 5 touc sensor torque sensor color CCD camera LRF 2 motor driver 6 color CCD camera LRF stepping motor motor control board motor control board pulse counter board PCI BUS USB2.0 NTSC RS232C PCI BUS RS232C PCI BUS CPU board CPU board (#2, #3) CPU boad #4 TCP/IP TCP/IP TCP/IP wireless LAN ub Fig. 2 Outdoor service robot OSR-02; entire robot; structure; electric arcitecture. 2053

3 (e) (f) Fig. 3 Equipment of OSR-02; ; and wit and-eye camera, force sensor and magnetic sensor; front LRF; back monitor and back LRF; (e) laptop PC wit touc panel; (f) publicity LED panel. Fig. 4 Magnetic sensor for tras separation; sensor; and wit magnetic sensor; sensor circuit; te circuit is built into te robot. CCD color camera 2.3 Equipment Te main equipment of OSR-02 is sown in Fig. 3. Fig. 3 sows te wit tree degrees of freedom, two CCD camera, and a LRF (Laser RangeFinder), and ten te detection of tras and te inspection of running route are done by tis device. Fig. 3 sows te and of te arm, a CCD camera is installed in te center position and te image from it are used for te and-eye system for grasping tras. Moreover, te magnetic sensor and te strain gage wit a ig resolution ave been installed into te fingernail of te rigt and te left end effector (Fig. 4). Te separated collection of te olding object can be done wit tese sensors, and it is possible to prevent te damage of te and by controlling of te olding power. Fig. 3, are LRFs for te obstacle detection of back and front of te robot, and eac LRF can measure te range of 180 [deg] in surroundings by 10 [Hz]. Te results of te image processing in te robot can be displayed on te back monitor sown in Fig. 3. It is possible to operate OSR-02 by using te laptop PC wit touc panel (Fig. 3(e)) via wireless LAN, and tis PC can be detaced. Te publicity LED panel (3(f)) displays te display of te content of te job of te robot and te advertisement of te sopping street. 2.4 Sensor system Te structure of te developed for OSR- 02 and an example of result of te object detection are sown in Fig. 5. Te can detect te object by te eigen space metod [4], tracking by CAMSHIFT algoritm [5], measure distance by LRF and estimate te roll yaw LRF pitc Fig. 5 Sensor system of OSR-02; structure; example of te camera image processing. posture of te object (see [2] for detail of te object detection and te tracking algoritms). 2.5 Running control system Te control system for OSR-02 to approac te target object was constructed as follows (Fig. 6) [1], [3]. Wen te center of te object on te left camera image plane is (x d (t),y d (t)) at time t, te yawing angle θ(t) and te pitcing angle φ(t) of te are controlled to 2054

4 turn to te front of te target object as follows, [ ] [ ][ ] [ θ k01 0 θ k02 0 = + φ 0 0 φ 0 k 12 ][ xd y d ], were!$k 01, k 02, k 12 > 0 are gain coefficients and te bias between te rotation center of te and te optical center of te cameras is disregarded. Moreover, until te robot approaces a constant distance from te target object, te input angular velocity w (w l,w r ) T is controlled as follows, were, w = R g v, (1) R g = 1 r w [ 1 l 1 l ] [ ] vg, v =, (2) w g r w is te radius of drive weel in te crawler, and l is distance between te drive weel and te center of rotation of te robot. Moreover v g and w g are te velocity and te angular velocity of te center of rotation of te robot, respectively. Te velocity input in te robot coordinate system toward te target position is decided as follows, v = K ( r p m), (3) were, r p ( r p x, r p z ) T is te vector from te center of rotation of te robot to te center of te target object in te left camera image plane, and m (0, m) T is te vector represented by using te distance m from te center of te to te center of rotation of te robot. Moreover, K is represented as follows, [ ] kr 0 K =, (4) 0 k θ were, k r is te gain coefficient about te distance from te target object and k θ is te gain coefficient about te yawing angle θ of te. Te values of k r and k θ are adjusted according to te situation of te road. Terefore, te velocity of eac weel are determined by Eqn. (1) and Eqn. (3) as follows, w = R g K ( r p m), (5) were te value of w does t exceed te maximum velocity set beforeand. 2.6 Object measurement by and-eye system Te robot raises te arm above te target object and takes initial posture for te olding of te target object after approacing to a constant distance (cf. Fig. 10). Wen te manipulator is in initial posture for te object olding, te robot views te target object by te and-eye camera and measures te olding position and te posture of te object. Te relation between and-eye camera and te target object is sown in Fig. 7. Te and-eye image coordinate system and te and coordinate system are represented as Σ i = { i x, i y} and Σ = { x, y, z}, respectively. For identifying te area of te target region in te and-eye image to old te target, te ue value obtained in a prior operation sequence is used. Namely, te area wit te specific ue value tracked in a prior sequence is searced in te and-eye camera image, and object (x d, y d ) φ r x m sy w l θ r p l s x r z vg w g w r Fig. 6 Running control system of OSR-02. only te area wit te maximum size is analyzed. Te minimum square tat suits tis area, te center position (x e,y e ) Σ i, and te posture angle θ of te target object are calculated. An example of te result of and-eye image processing is sown in Fig. 7. Terefore, te olding position (x t,y t,z t ) Σ of target object is able to be calculated from te eigt (=const.) of initial posture of te and and te eigt t of te target object estimated from te kind of tras as follows, x t x t y t z t = (x ez t )/f (y e z t )/f t, (6) were f is te focal lengt of te and-eye camera. Te and is controlled to be x t 0 and θ 0. Asmentioned above, te and and arm are controlled for olding and collection of te object by te feed forward control system. 3. TRASH COLLECTION TASK An operation sequence for te tras collection task of OSR-02 (Fig. 8) is sown as follows: 1. Te robot searces for tras on te floor by controlling te posture of te cameras installed in te sensor system. 2. Wen a piece of tras is detected, te robot approaces te target tras as follows: Te posture of te is controlled to matc te center of te target object and te center of te camera image. Te crawlers are controlled to approac constant distance from te target. 3. Te robot stops moving near te target, measures te detail position of te target object by te stereo vision and te LRF, and raises te manipulator above it. 2055

5 (x e, y e ) y ix i y x [next] start fast pattern matcing using K-L decompsition and FFT t θ z tras is continuously detected target tracking by CAMSHIFT algoritm te target is tracked te target is near eug target tracking based on te ue value center of gravity position and posture measurement of target and-eye camera Fig. 7 Hand-eye : relation of te and-eye camera image coordinate system Σ i and te and coordinate system Σ ; example of calculating result of circumscribed rectangle of minimal area, center of gravity, posture, and distance of a target object. 4. Te robot measures te position and te posture of te target by te and-eye camera, and grasps and stores it into te tras box. te robot grasped te target by te manipulator. te robot trow away te target to te tras box [next] Fig. 8 Flowcart for image processing of te tras collection task. Since te appearance of tras canges according to te position, te posture of te camera, and te kind of tras, te robustness to te condition canging (mentioned above) is necessary for te metod of detecting te object by tis robot. Tus, we ad employed a robust matcing metod compressing many template images by te Karunen-Loeve (KL) expansion and obtaining rmalized correlation approximately [4]. 4. EXPERIMENTS 4.1 Driving performance Te driving performance of OSR-02 was tested under te outdoor environment. Here, since tis test is for te evaluation of te driving performance, te robot was operated via wireless LAN. Te robot was able to get over te ditc of 180 [mm] in widt were running was difficult by te robot of te weel drive (Fig. 9). It was confirmed tat OSR-02 ad a driving performance tat was iger tan OSR-01 of te weel drive by tis driving test. Fig. 9 Driving performance test. 4.2 Tras collection Te tras collection task of OSR-02 is experimented under te outdoor environment. A plastic bottle or a can was put forward of about 2 [m] of te robot, and te collection experiment was conducted. In tis experiment, 2056

6 Fig. 11 Example of demonstration of OSR-02; at a sopping street; at a playground. (e) (f) Fig. 10 OSR-02 collects up a plastic bottle; a plastic bottle was discovered; te initial posture for olding; olding of te target object; lift; (e) separated collection of te olding object; (f) searcing for next tras. te robot was programmed to classify te object by te rigt and, and to store it into te tras box eld by te left and. It was te drizzle weater. Te results of experiment are sown in Fig. 10. A series of operation sequence was confirmed from te result of tese experiments, and it was confirmed tat te separated collection task of tras is possible by te robot. 4.3 Experiment at te public space We conducted te demonstrations of OSR-02 at a sopping arcade and a playground in Kitakyusu-city (Fig.11). Te demonstration in wic te robot patrolled te sopping arcade and te playground, and collected plastic bottles was executed several times. We confirmed tat OSR-02 was able to detect te plastic and glass bottle and cans in te route, to approac, to collect accurately, and to avoid te pedestrian. Furtermore, it was found from tese experiments tat te calculation speed of te developed vision system was fast eug for te collection of tras. 5. CONCLUDING REMARKS In tis paper, we introduced te outdoor service robot OSR-01 and OSR-02. Moreover, we confirmed from te experiments in a real sopping street and a playground tat te robots were able to detect te tras registered be- foreand and to acieve te separated collection. Now, we are advancing improvement in te reliability and processing speed of te vision system and experimenting. REFERENCES [1] M. Obata, T. Nisida, H. Miyagawa, T. Kondo and F. Okawa, Development of Outdoor Service Robot to Collect Tras on Streets, IEEJ Trans. EIS, vol. 126,. 7, pp , [2] Y. Fucikawa, T. Nisida, S. Kurogi, T. Kondo, F. Okawa, T. Sueiro, Y. Watanabe, Y. Kawamura, M. Obata, H. Miyagawa and Y. Kiara, Development of a Vision System for an Outdoor Service Robot to Collect Tras on Streets, Proc. of CGIM05, pp , [3] T. Kondo, T. Nisida, M. Obata and F. Okawa, A Researc Report about Te Outdoor Service Robot OSR-01, Proc. of Te 1st International Conference on Design Engineering and Science Vienna, pp , [4] M. Ueara and T. Kanade, Use of Fourier and Karunen-Loeve Decomposition for Fast Pattern Matcing Wit a Large Set of Templates, IEEE trans. on pattern analysis and macine intelligence, vol. 19,. 8, pp , [5] Gray R. Bradski, Computer vision face tracking as a component of a perceptual user interface, In Worksop on Applications of Computer Vision, pp ,