On-site Humanoid Navigation Through Hand-in-Hand Interface

Transcription

1 Proceedings of 0 th IEEE-RAS International Conference on Humanoid Robots On-site Humanoid Navigation Through Hand-in-Hand Interface Takashi Ogura, Atsushi Haneda, Kei Okada, Masayuki Inaba Department of Mechano-Informatics Graduate School of Information Science and Technology, The University of Tokyo Hongo, Bunkyo-ku, Tokyo , Japan {ogura, hane, k-okada, inaba}@jsk.t.u-tokyo.ac.jp Abstract In order to realize humanoid robots that support people in their daily lives, on-site humanoid behavior navigation system which a robot behaviors autonomously as well as a robot reacts to the intentions of a human. This paper proposes new on-site navigation interface method for humanoid robots which a human navigate a humanoid robot hand-in-hand. Previous humanoid researches aim to realize an autonomous intelligence, which is able to survive in a real environment without any help from a person. However to utilize a humanoid robot in a real world, it is necessary to take a robot to various locations or make a robot to manipulate and use various kinds of objects and tools, on-site navigation that a person is able to induce a humanoid behavior. In the experiment the operator takes a humanoid robot to a kitchen and teaches holding a kettle using this interaction kernel, and played back the motion sequence by planning in different environments. Index Terms Interaction, Motion Planning, Humanoid Software System, Teaching of Humanoids Fig. 1. Online teaching for humanoid robots through hand-in-hand interface. I. INTRODUCTION One of the most expected application of humanoids robots is support of daily life. What is the necessary function for that? Previous humanoid researches aim to realize an autonomous intelligence, which is able to survive in a real environment without any help from a person. However to utilize a humanoid robot in a real world, it is necessary to take a robot to various locations or make a robot to manipulate and use various kinds of objects and tools, on-site navigation that a person is able to induce a humanoid behavior like Fig.1. In order to realize humanoid robots that support people in their daily lives, on-site humanoid behavior navigation system which a robot behaviors autonomously as well as a robot reacts to the intentions of a human. This paper proposes new onsite navigation interface method for humanoid robots which a human navigate a humanoid robot hand-in-hand. The robot and the human look like a child and a parent with this interface. Previous on-site navigation systems are able to be classified into two groups, one is a direct navigation approach which a human controls a humanoid robot in a master slave manner with some devices, and an another approach is that a human teaches by demonstrating or dialogue. Our hand-in-hand navigation interface can be categorized in a direct navigation approach, however this approach is intuitive when compare to other direct approach such as the researches that uses joysticks [1], [2] and is convenient because the person does not need to use any devices. We show four experiments through proposed hand-in-hand interface. The first one shows navigation of walking on flat ground. The operator leads the robot from inside an elevator to a corridor. In the next, navigation of going upstairs and downstairs. These are direct navigation using change of hand position by impedance control as inputs. The robot changes the result of path planning by interrupt from the operator in the third experiment. This shows that he control the walking not only directly but can change the result of path planning by navigation. A teaching system through hand-in-hand interface are described in the end. The operator takes the robot to a kitchen by leading by the hand, and teaches manipulation of a kettle. II. HUMANOID NAVIGATION INTERFACE In order to realize an on-site humanoid navigation, Briefness that a human is able to navigate a robot without any training and adaptability which a robot is able to behave a memorized behavior in different places are important. This section discuses navigation methods for humanoid robots from this point of view /0/$.00 0 IEEE 17

2 input type symbolized numerical A. Navigation Level TABLE I DEVICES FOR ON-LINE HUMANOID NAVIGATION devices microphones (voice recognition, sound), buttons, switches, keyboards joysticks, cameras (vision processing), master-slave devices, bodies (direct teaching) In case of a humanoid robot that is able to behave various tasks, we discuss from what to teach first, then we are able to discuss how to teach according to a teaching method. Humanoid navigation can be classified into following five levels. direct motion teaching assistance or modification of motions instruction of targets or behaviors teaching of states or phase of the robots navigation of recognition for planning or motion Motion teaching or modification needs numerical inputs, but instruction of targets or teaching phases requires symbolized inputs. The navigation system should be able to accept numerical and symbolized inputs because these all levels are important for humanoids. B. Navigation Devices for On-site Navigation Table I shows devices for humanoid on-line navigation. These are classified into two, symbolized input methods and numerial input methods. Numerical methods such as joystick or master-slave like cockpit [3], can get analog values, and symbolized methods such as voice recognition [4] or vision processing [], [6], deal symbolic values. The methods that the operator does not have to have any devices are voice recognition, vision processing and direct teaching in these. Researches of direct teaching of humanoids using their bodies are very few, although it is intuitive. C. Navigation Methods through Bodies It is difficult to apply inputs through bodies to life size humanoids because they are large and have many degrees of freedoms. It is too dangerous and troublesome. Therefore, we use only their hands for inputs and control whole bodies. We call this interface hand-in-hand interface. If the humanoid robot has force sensors on their hands, it is able to use the forces as inputs and control the robot by interpreting them depending on their situations. However, it is a problem that it can not feed back the operation if it uses the force directly. It makes feed back of the operation possible to make its arms compliant and use the position of its hands as input. The feature of this interface is interaction as with human no devices and easy force feedback control. Force Sensors Cameras Vision Processes Color Extract Voice Recognition Impedance Control Stereo Vision of Hnads Position update position of Symbols target change of position Three-dimensional Models Robot Environments Cost Changer search cost Model Based Mtion Programs Inverse Motion Planners target Kinematics Manipulation Planner Manipulation Walking Navigator Path Planner Locomotion Sit down path walking joint angles states velocity Online Walking Pattern Generator Motion Stabilizer Motors Mic Voice Synthesis Speakers Fig. 2. Softwares that compose Humanoid Interaction Kernel and their configuration. a) interaction as with human: The operator can lead the robot somewhere leading by the hand and teach things handin-hand with this interface, as same as parents do for his/her children. Therefore, this is an intuitive interface. b) no devices and easy: This interface needs no devices and the operator can control whole body by touching only the hands. This enables on-site navigation. c) force feedback control: Impedance control realize force feedback that if a person pull the hand too strong, the robot reactive strongly. Furthermore, it is superior because it can reflect the input forces to the positions of their hands. III. IMPLEMENTATION OF NAVIGATION SYSTEM THROUGH HAND-IN-HAND INTERFACE This section shows a concrete implementation of handin-hand navigation system. Fig.2 shows the software system configuration that compose a navigation system through hand-in-and interface. Inputs are force sensors, cameras and microphone. Impedance control of hands position is basic function for this system, and Cost Changer and Walking Navigator use the output of this. The motion planners use three-dimensional models for motion generation. These models are composed on EusLisp [7], which is a lisp language that has three-dimensional modeling system and multi-thread. We used Galatea tool kit [8] for Voice Recognition and Voice Synthesis programs, and made it possible to control these from EusLisp. Vision Processes are also controlled by EusLisp. IV. WALKING NAVIGATION THROUGH HAND-IN-HAND INTERFACE A. Navigation of hands position and posture by impedance control It is a basic function that can navigate position and posture of robot hands online for teaching and navigate humanoid motions directly. Impedance control of position and posture of hands by 6 axes force sensor mounted on them realize that function. 176

3 Humanoid robots joints are often controlled by position control. eq.(1) shows the impedance control equation for position control system. M d y t 2y t 1 + y t 2 t 2 +D d y t y t 1 t +K d (y t y d )=F (1) y t is position and posture of hands at the time of t. y t 1 is y at the time of t t. y t 2 is y at the time of t 2 t. t is digitizing unit time. Solved result of eq.(1) about y t is y t = F + M y (2y t 2 t 1 y t 2 )+D t 1 d t + K d y d M d t + D 2 d t + K. (2) d B. Navigation of walking by change of position of hands This subsection shows the method for navigation of walking by leading humanoid robot s hands. Navigation of walking by leading its hands is superior to other methods in two aspects. At first it is natural because it is very similar to human. The scene that the operator takes the robot anywhere by leading its hand is like that parents lead their child s hands. The other side is that it is a device less method. For online system in diary life, the operator can not use any devices. Because he/she will be normal person without any knowledge about the robot control, the two aspects are very important. Furthermore it is important that the robot speaks its states by voice synthesis when start walking, stop walking and getting too large input. This output is useful for keeping safety. The simplest way for realizing this method is use force sensor mounted on the hands. The force can be converted to the velocity of walking directly, but this has no feed back for the operator. Even if he/she leads the hands strongly enough to upset the robot, he/she can not know that. If change of position by impedance control is converted to walking velocity instead of force, the force can be smaller and the operator can know how fast he/she is navigating. eq.(3) is an implement of decision walking velocity. V =(V x,v y,v θ )=(K x Σ i y xi,k y Σ i y yi,k θ Σ i y yi Σ i y xi ) V is walking speed. K x, K y and K θ are scalar weight. y xi means change of hand position in direction of x, arm i. i =0, 1 Fig.3 is the scene of the experiment about this navigation method. The humanoid is in the elevator at first. The operator holds the hands of the robot and pull to navigate it. He takes it out of the elevator and turns the corner. C. Navigation of walking on stairs This system can navigate going upstairs and downstairs as same as walking navigation. The input is the change of hand position by impedance control as same as walking navigation on planer grounds. (3) 7 8 Fig. 3. Walking navigation by leading robot s hand from inside of an elevator to a corridor. 1) known stairs: If the height and depth of the stairs are known, the input can use for switch. if Σ i y zi > z up then upstairs, else if Σ i y zi <z down downstairs. An experimental result is shown in Fig.4. The operator leads the hands of the robot above and the robot start walking on the stairs. And leads lower, then it walks downstairs. 2) unknown stairs: The operator must teach the height and depth of the stairs by leading the position of swing leg in unknown stairs. If he/she teaches at once, the robot can go stairs continuously as same as known stairs. V. CHANGING THE PLANNING BY INTERRUPTION This section shows the system which not only navigates motion directly but navigates autonomous motion planning by the navigation of an operator. Change of search costs enables this mechanism. This section shows the method through an experiment. Fig. shows the picture of the experimental environments and its three-dimensional geometric model. The details of the real environments are not modeled in the model. Furthermore, there are obstacles not modeled in the photograph right-hand side. The planner may generate a path via the obstacles. In such situation, if the operator can navigate the path different direction with fewer efforts, the navigation of autonomous system is effective. The robot has environmental model beforehand as shown in Fig.2, and search the path to the given target using it, as will hereinafter be described in detail. This section describes 177

4 where x is the state of the robot. K(x) is the search cost coefficient and x is moving distance in a unit step. Sum of K(x) x is the cost g. x x navi 2 means the distance from the state given navigation of the operator. Kdir navi is less value than default value initial plan: f(x) cost (θ=0) renewed plan: f(x) cost (θ=0) initial plan: f(x) cost (θ=π/2) renewed plan: f(x) cost (θ=π/2) Fig. 6. Change of cost g by on-site navigation. The areas covered with obstacles are yellow. Left two figures are f cost maps used in initial plan. This shows that clockwise path is generated. Because of human navigation, the maps are changed into right two maps, so that a counter clockwise path is created. Fig. 4. Navigation of going upstairs and downstairs start Operator s Navigation to the Left Cost Changer Start Unknow obstacles Start Goal replanning Goal Goal Fig. 7. Change of path plan by on-site navigation. The left picture is the initial plan. The right is the plan that is inducted by human navigation. Fig.. An environment of everyday life and its 3D geometric map used in the experiment. Thought the real world is similar to the 3D model, there are always modeling errors and many unpredictable things. There are humans and corrugated cartons in the circled area. the realization approach of navigation of path by search cost change. The left figure of Fig.7 shows three-dimensional model of an environment with given goal and start point. The path planner generate the clockwise path, and it started walking(figrefplanchange-exp(1)). Fig.8(2) shows the scene that the operator navigates the path. Cost Changer changes the search cost to generate a new path in navigated direction, the costs are shown in Fig.6, and replan the path(fig.7.) The results are shown in Fig.8. This method uses A* search, the search cost g is changed if the operator gives the robot navigation at the state x n avi as eq.(4). g = C K exp x x navi 2 2σ 2 Kdir navi x (4) VI. WALKING AND MANIPULATION TEACHING SYSTEM A. Locomotion Planner Locomotion Planner makes a plan of walking from the start position to the goal. Fig.9 shows the sequence of planning. This planner uses the robot and the environmental geometric model and start point and goal point. At first it makes bounding cylinder of the robot and build up grid map the robot can go through (figure of (2)). Then search the nearest path by A* search method for the cylinder and generates the path of walking of the robot. B. Manipulation Planner Manipulation Planner makes whole body motion plan from environmental geometric model and target position and posture. This planner generates using whole body inverse kinematics with various types of constraints [9]. Statically-stable full body posture and collision-free motion can be generated by this planner[]. Fig. shows the motion sequence of grasping the target (the handle of a red kettle). 178

5 (1) (2) (3) (4) Fig. 8. The scene of a real world navigation experiment. The system planned a global path and the robot started walking toward the left direction of that. Because a human operator found that there are cardboard boxes on the path, he navigated the robot toward the opposite direction. This changed cost coefficient vectors of this state and its surrounding states, and this follows that the robot changed the planning and changed the walking direction. C. Evaluation experiment of walking and manipulation teaching system This section shows an experimental result using a humanoid HRP-2[11] and verify the system discussed above. In the kitchen as we use in daily life, the operator takes the robot in front of a kettle, teaches pouring out tea. The robot replays the whole body motions using path planner and motion planner after the teaching phase. Fig.11 explains the software system for the experiment. Motion Navigators are composed of Head Navigator, Walking Navigator and Arm Navigator. Walking Navigator enables the navigation of walking by leading the robot hand. The operator teaches a target point for locomotion by leading the hand of the humanoid robot, and then teaches manipulation and grasping directly. After this teaching phase, the robot replays the motion using the planners explained in this paper. Fig.14 shows the experimental environment and results. Though the initial position of robot is changed and an obstacle, which is a table, appeared, it can go to the target point that taught in the teaching phase. However environment map and the start position are given. The robot finds the kettle as shown in Fig.13 and grasp it in the next. After find it by extracting kettle color, stereo vision calculates the 3D position of that and motion planner generates grasping motion using inverse kinematics. VII. CONCLUSION We have described on-site humanoid navigation interface that requires no devices and has intuitive interfaces. This interface enabled us to navigate walking of robots by leading () (6) Fig. 9. Walking path planner. (1)Start, goal and geometric environmental models, the robot model are given. (2)Then make grid map collision free space. (3),(4) Search the path to the goal using simplified cylinder robot model. The path and motion sequence is shown in () and (6). Head Legs Motion Navigators Head Walking Navigator Navigator Vision Voice Teaching Phase on/off Fig.. Arms Arm Navigator Arms Force Fig. 11. Manipulation Planner Purpose Interpreter Memory Environment Model Target Object Target Position Experimental system. Head Legs Arms Motion Planner Walking Planner Manipulation Planner Vision Playback Phase 179

6 Fig. 12. Teaching phase: the operator leads the robot s hand for navigating walking and manipulation. Fig. 14. Play back phase: the robot uses planner for replay the taught motions Fig. 13. Image Processing for manipulation. The stereo vision calculates the position of target object. its hands, going upstairs, and manipulation of a kettle. Not only controlling the motions directly, the robot has some autonomy, and the operator can guide the path. Then we showed it is possible to compose an online teaching system for humanoids by body interaction that can play back taught motions even if the environments changes by the hand-in-hand interface. This illustrated that you can teach many things with this interface if the robot knows the situation. However combination of other inteface, like voice recognition, is needed for recognition of change of situations and this is essential for teaching of humanoid robots. Hand-in-hand interface will be one of the most important interface comparable with voice, which human does not feel that it is artificial one. REFERENCES [1] Neo, Ee, Sian., Yokoi, K., Kajita, S., Kanehiro, F., Tanie, and K. Whole Body Teleoperation of a Humanoid Robot -Development of a Simple Master Device using Joysticks-. In Proc. Int. Conf. on Intelligent Robotics and Systems(IROS), pages , 02. [2] Koichi Nishiwaki, Satoshi Kagami, Yasuo Kuniyoshi, Masayuki Inaba, and Hirochika Inoue. Online Generation of Humanoid Walking Motion based on a Fast Generation Method of Motion Pattern that Follows Desired ZMP. In Proc. Int. Conf. on Robotics and Automation (ICRA), pages , 02. [3] Hasunuma H., Kobayashi M., Moriyama H., Itoko T., Yanagihara Y., Ueno T., Ohya K., and Yokoi K. A Tele-operated Humanoid Robot Drives a Lift Truck. In Proc. Int. Conf. on Robotics and Automation(ICRA), pages , 02. [4] Kazuhiro Nakadai, Hiroshi G. Okuno, and Hiroaki Kitano. Robot Recognizes Three Simultaneous Speech By Active Audition. In Proceedings of the 03 IEEE International Conference on Robotics and Automation (ICRA03), pages , 03. [] Kei Okada, Yasuyuki Kino, Masayuki Inaba, and Hirochika Inoue. Visually-based Humanoid Remote Control System under Operator s Assistance and it s Application to Object Manipulation. In Proceedings of Third IEEE International Conference on Humanoid Robots (Humanoid 03), 03. [6] Yoichi Sato, Makiko Saito, and Hideki Koike. Real-time input of 3D pose and gestures of a user s hand and its applications for HCI. In Proceedings of 01 IEEE Virtual Reality Conference (IEEE VR01), pages 79 86, 01. [7] T. Matsui. Multithread object-oriented language euslisp for parallel and asynchronous programming in robotics. In In Workshop on Concurrent Object-based Systems, IEEE 6th Symposium on Parallel and Distributed Processing, [8] Shin ichi Kawamoto, Hiroshi Shimodaira, Tsuneo Nitta, Takuya Nishimoto, Satoshi Nakamura, Katsunobu Itou, Shigeo Morishima, Tatsuo Yotsukura, Atsuhiko Kai, Akinobu Lee, Yoichi Yamashita, Takao Kobayashi, Keiichi Tokuda, Keikichi Hirose, Nobuaki Minematsu, Atsushi Yamada, Yasuharu Den, Takehito Utsuro, and Shigeki Sagayama. Galatea: Open-Source Software for Developing Anthropomorphic Spoken Dialog Agents. Life-Like Characters, Tools, Affective Functions, and Applications, pages , 03. [9] Katsu Yamane and Yoshihiko Nakamura. Natural Motion Animation through Constraining and Deconstraining at Will. IEEE TRANSAC- TIONS ON VISUALIZATION AND COMPUTER GRAPHICS, 9(3), [] Kei Okada, Takashi Ogura, Atushi Haneda, Daisuke Kousaka, Hiroyuki Nakai, Masayuki Inaba, and Hirochika Inoue. Integrated System Software for HRP2 Humanoid. In Proc. of International Conference on Robotics and Automation (ICRA 04), pages , 04. [11] K.Kaneko, F.Kanehiro, S.Kajita, H.Hirukawa, T.Kawasaki, M.Hirata, K.Akachi, and T.Isozumi. Humanoid Robot HRP-2. In Proc. Int. Conf. on Robotics and Automation(ICRA), pages 83 90,