A Co-worker Robot PaDY" for Automobile Assembly Line

A Co-worker Robot PaDY" for Automobile Assembly Line Kazuhiro Kosuge Department of Bioengineering and Robotics Graduate School of Engineering Tohoku University Sendai 980-8579, JAPAN http://www.irs.mech.tohoku.ac.jp

Outline Human-Robot Coordination and Interaction Examples of human-robot cooperation Dance Partner Robots MS DanceR PBDR PaDY Concept PaDY in a factory B-PaDY D-PaDY IVIS Conclusions

Human Power Augmentation Human Power Augmentation [1] K. Kosuge, Y. Fujisawa, T. Fukuda, Mechanical System Control with Man-Machine-Environment Interactions, [Proceedings of the 1993 IEEE International Conference on Robotics and Automation (1993) 239-244]. [2] 小菅一弘, 藤沢佳生, 福田敏男, 環境との相互作用が生じるマンマシン系の制御, [ 日本機械学会論文集 (C 編 ) 59 (562) (1993) 1751-1756].

Human Power Augmentation F h Q 1 M v F e D v K v Operator Tool Environment Use of a Tool

Human Power Augmentation F h Q 1 M v F e D v K v M v!!+ x D x! v + K v x = QF h F e Virtual Tool Dynamics

Robot Helpers Human-Robot Cooperation (Kosuge, 1993)

Robot Helpers Robot j Robot i Object Human l Human m Robot k Passive Dynamics Stability Issues

Robot Helpers MR Helper (Mobile Robot Helper, 1997~) [1] K. Kosuge, M. Sato, Mobile Robot Helper, [Proceedings of the 2000 IEEE International Conference on Robotics and Automation (2000) 583-588]. [2] 小菅一弘, 須田理央, 風村典秀, 佐藤学, 角谷啓, 人と双腕型移動ロボット MR Helper による物体の協調搬送, [ 日本機械学会論文集 (C 編 ) 69 (685) (2003) 84-90].

Impedance-based Control Algorithm I F ext Object M obj F base D obj K obj x obj Mobile Base M base x base D base World Frame

Robot Helpers Aggressive maneuver Robustness in 2004 MR Helper (Mobile Robot Helper, 1997~)

Impedance-based Control Algorithm II Object M obj F ext K base Object M obj F base D base d x base M base x base D obj x obj Mobile Base World Frame World Frame

Robot Helpers MR Helper Dual Mode Coordination

Robot Helpers DR Helpers (Distributed Robot Helpers) Y. Hirata, K. Kosuge, Distributed Robot Helpers Handling a Single Object in Cooperation with a Human, [Proceedings of the 2000 IEEE International Conference on Robotics and Automations (2000) 458-463]. 平田泰久, 初雁卓郎, 小菅一弘, 淺間一, 嘉悦早人, 川端邦明, 人間と複数の分散型ロボットヘルパ - との協調による単一物体の搬送, [ 日本機械学会論文集 (C 編 ) 68 (668) (2002) 181-188].

Robot Helpers DR Helpers (Distributed Robot Helpers) Y. Hirata, Y. Kume, Z. D. Wang, K. Kosuge, Decentralized Control of Multiple Mobile Manipulators Based on Virtual 3-D Caster Motion for Handling an Object in Cooperation with a Human, [Proceedings of the 2003 IEEE International Conference on Robotics and Automation (2003) 938-943].

for Human Environments In a Science Museum

for Human Environments Priority-on-Orientation Control Priority-on-Position Control Singular Configuration Control K. Nakai, K. Kosuge, Y. Hirata, Control of Robot in Singular Configurations for Human-Robot Coordination, [Proceedings of the 2002 IEEE International Workshop on Robot and Human Interactive Communication (2002) 356-361].

for Human Environments Model Based Adaptive Collision Detection [1] S. Morinaga, K. Kosuge, Compliant Motion Control of Manipulator s Redundant DOF Based on Model-based Collision Detection System, [Proceedings of the 2004 IEEE International Conference on Robotics and Automation (2004) 5212-5217]. [2] 小菅一弘, 松本大志, 盛永真也, 適応制御を利用したマニピュレータの衝突検出, [ 計測自動制御学会論文集 39 (6) (2003) 552-558]. [3] T. Matsumoto, K. Kosuge, Dynamic Collision Detection Method Using Adaptive Control Law, [Proceedings of IEEE International Conference on Industrial Electronics Control and Instrument (2000) 2243-2248].

Model Based Adaptive Collision Detection Model Based Adaptive Collision Detection [1] S. Morinaga, K. Kosuge, Compliant Motion Control of Manipulator s Redundant DOF Based on Model-based Collision Detection System, [Proceedings of the 2004 IEEE International Conference on Robotics and Automation (2004) 5212-5217]. [2] 小菅一弘, 松本大志, 盛永真也, 適応制御を利用したマニピュレータの衝突検出 [3] 小菅, 松本, 盛永, 適応制御を利用したマニピュレータの衝突検出, [ 計測自動制御学会論文集 39 (6) (2003) 552-558]., [3] T. Matsumoto, K. Kosuge, Dynamic Collision Detection 計測自動制御学会論文集 Method Using Adaptive Control, 第 39 巻 Law,, 第 6 号 [Proceedings,pp.552-558, of 2003 IEEE International Conference on Industrial Electronics Control and Instrument (2000) 2243-2248].

Self-Collision Detection Based on RoBE [1] F. Seto, K. Kosuge, Y. Hirata, Self-collision Avoidance Motion Control for Human Robot Cooperation System using RoBE, [Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (2005) 50-55]. [2] F. Seto, K. Kosuge, Y. Hirata, Real-time Self-collision Avoidance System for Robots using RoBE, [International Journal of Humanoid Robotics 1 (3) (2004) 533-550].

Self-Collision Detection Based on RoBE B S SHOULDER S E ELBOW E H BODY HAND [1] F. Seto, K. Kosuge, Y. Hirata, Self-collision Avoidance Motion Control for Human Robot Cooperation System using RoBE, [Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (2005) 50-55]. [2] F. Seto, K. Kosuge, Y. Hirata, Real-time Self-collision Avoidance System for Robots using RoBE, [International Journal of Humanoid Robotics 1 (3) (2004) 533-550].

Self-Collision Detection Based on RoBE Self-Collision Detection Based on RoBE [1] F. Seto, K. Kosuge, Y. Hirata, Self-collision Avoidance Motion Control for Human Robot Cooperation System using RoBE, [Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (2005) 50-55]. [2] F. Seto, K. Kosuge, Y. Hirata, Real-time Self-collision Avoidance System for Robots using RoBE, [International Journal of Humanoid Robotics 1 (3) (2004) 533-550].

Outline Human-Robot Coordination and Interaction Examples of human-robot cooperation Dance Partner Robots MS DanceR PBDR PaDY Concept PaDY in a factory B-PaDY D-PaDY IVIS Conclusions

Lessons Learned Lessons learned from Robot Helpers Some simple tasks, which could not be done by a human/humans, could be done with a robot helper(s). General tasks could not be done easily even with the assistive robot system(s), because the robot does not know how to collaborate with the human.

Lessons Learned In order to collaborate with the user, the robot has to know the task, its user s intention, and how the user wants to be assisted

Dance Partner Robot To develop a mechanism for closer humanrobot coordination/interaction

Design of Estimator Reference Data Dance Step A Transition Dance Step B Force/Moment T effective Time Primitive Methods: Instantaneous Data at Transitions of Steps

Control Architecture CAST (Control Architecture based-on Step Transition) Blues Step Transition Step Motion Knowledge Waltz Step Transition Step Motion Tango Step Transition Step Motion Knowledge of ballroom dances Step Estimator Estimated step Motion Generator Basic step trajectory Sensory information Sensory information Desired motion Robot Hardware Sensors Actuators

Experiment

Aichi Expo (March 24 ~ September 25, 2005)

Sensory Data Used for Estimation Reference Data Dance Step A Transition Dance Step B Force/Moment T effective Time Time series data include uncertainty such as time-lag and variation because a dancer cannot always apply the same force/moment for each step transition.

Sensory Data Used for Estimation Next Dance Step HMM-based Step Estimator Dance Step A Transition Dance Step B Force/Moment T effective Time

Dance Partner Robots PBDR as a Research Platform for Human-robot interaction

Challenges for Dance Partner Robots Stable physical interaction between a human and a robot Female Dance Partner Robot Human behavior/intention estimation How to read the its partner s lead Male Dance Partner Robot How to convey robot s intention to its human partner Motion Entrainment Based on Human modeling

Outline Human-Robot Coordination and Interaction Examples of human-robot cooperation Dance Partner Robots MS DanceR PBDR PaDY Concept PaDY in a factory B-PaDY D-PaDY IVIS Conclusions

Automobile Factories Welding Process Industrial robots have played important roles in manufacturing industries, especially in automobile factories.

Automobile Assembly Line If a robot could provide the worker with necessary parts and tools when he/ she needs them, the worker could concentrate on the assembly tasks. What the worker has to do is to assemble delivered parts using delivered tools to accomplish the work. [1] 衣川潤, 川合雄太, 菅原雄介, 小菅一弘, 組立作業支援パートナロボット PaDY( 第 1 報, コンセプトモデルの開発とその制御 ),[ 日本機械学会論文集,C 編,77(783),(2011),4204-4217] [2] J. Kinugawa, Y. Kawaai, Y. Sugahara and K. Kosuge, PaDY : Human-Friendly/Cooperative Working Support Robot for Production Site, [The 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems Proceedings,(2010), 5472-5479].

Working Partner Robot PaDY PaDY is a robot which delivers necessary parts and tools to a worker when he/she needs them. to reduce the worker s load to improve efficiency of the work to prevent mistakes of the work etc. PaDY [1] 衣川潤, 川合雄太, 菅原雄介, 小菅一弘, 組立作業支援パートナロボット PaDY( 第 1 報, コンセプトモデルの開発とその制御 ),[ 日本機械学会論文集,C 編,77(783),(2011), 4204-4217] [2] J. Kinugawa, Y. Kawaai, Y. Sugahara and K. Kosuge, PaDY : Human-Friendly/Cooperative Working Support Robot for Production Site, [The 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems Proceedings,(2010),5472-5479]. in-time Parts/tools Delivery to You robot

Working Partner Robot PaDY How the work is done completely depends on the worker. The work could not be done as scheduled. The work may be done earlier or delayed

Working Partner Robot PaDY Even if robot arm motion is generated by the work schedule given to the worker, the cooperative work dose not go well.

Working Partner Robot PaDY In order to deliver parts/tools to a place, where the worker needs them, when the worker needs them without disturbing the worker s work. the robot needs to know the task, its user s intention, and how the user want to be assisted

The First Prototype of PaDY (P1) Size:(W)1370 (D)590 (H)1035[mm] Link Mechanism :Horizontal ArEculated Robot Maximum Reach:2.0 [m](1 st Link Length:1168[mm], 2 nd Link Length:982[mm]) Weight of Working Parts:11.5[kg] Maximum Load:3 [kg] Range of Movement: 1 st Joint: 200[deg], 2 nd Joint: 360[deg] Actuator:1 st Joint & 2 nd Joint:DC Servo Motor 80[W], 3 rd Joint: DC Servo Motor 15[W]

Outline Human-Robot Coordination and Interaction Examples of human-robot cooperation Dance Partner Robots MS DanceR PBDR PaDY Concept PaDY in a factory B-PaDY D-PaDY IVIS Conclusions

B-PaDY

Outline Human-Robot Coordination and Interaction Examples of human-robot cooperation Dance Partner Robots MS DanceR PBDR PaDY Concept PaDY in a factory B-PaDY D-PaDY IVIS Conclusions

49/40 D-PaDY

Outline Human-Robot Coordination and Interaction Examples of human-robot cooperation Dance Partner Robots MS DanceR PBDR PaDY Concept PaDY in a factory B-PaDY D-PaDY IVIS Conclusions

Integration of Vision and Force ivis Integrated Visual and Impedance Servo Takashi Nanmoto,Kazuhiro Kosuge,Koichi Hashimoto, Model-Based Compliant Motion Control Scheme for Assembly Tasks Using Vision and Force Information, [Proceedings of 2013 IEEE International Conference on Automation Science and Engineering,(2013)]

Control Architecture Force Control Coordinate system Attached to the CAD model Trajectory GeneraEon + + + + Inverse Kinematics Impedance Model Impedance Control CAD model based Impedance control Motion Controller Forward Kinematics External Force Estimation Force Control Coordinate System Estimation Manipulator Force Sensor Goal Image Generated by a CAD model Estimation of Motion deviation Pose Estimation Visual Tracking Servo Image Sensor Visual Servo 52

Experiments Camera image of an object A and its CAD model based image Goal Image of the assembled objects Camera image of an object B and its CAD model based image 53

Assembly of two parts using multipurpose robot hands by ivis 54

Assembly of Two Parts by Demonstration

Robotics Research Unit Technologies Technical Issues Required Services Domain 1 Elderly Care Domain 2 Agriculture Domain 3 Medicine Applications/Services New Services Service/Applicationoriented Robotics Robotics Foundations Robotics Foundations Current Robot Function JST, CRDS, 2009, Modified by Kosuge, August, 2011