Human-Robot Interaction from Dance Partner Robot to Co-worker Robot

Similar documents
A Co-worker Robot PaDY" for Automobile Assembly Line

HMM-based Error Recovery of Dance Step Selection for Dance Partner Robot

IN MOST human robot coordination systems that have

製品系列統合化設計とそのタスク構造 日本機械学会論文集 C 編. 65(629) P.416-P

ジェスチャ併用型 Voice-to-MIDI システムの提案 第五回知識創造支援システムシンポジウム報告書 : 本著作物の著作権は著者に帰属します

Cognitive robots and emotional intelligence Cloud robotics Ethical, legal and social issues of robotic Construction robots Human activities in many

Birth of An Intelligent Humanoid Robot in Singapore

NCCT IEEE PROJECTS ADVANCED ROBOTICS SOLUTIONS. Latest Projects, in various Domains. Promise for the Best Projects

* Intelli Robotic Wheel Chair for Specialty Operations & Physically Challenged

[1] 大橋和也, 森拓哉, 古関隆章 運転整理時における乗車率に応じた旅客行動の変化のモデル化 電気学会論文誌 D,J-Rail 2013 特集,2015,pp

Dr. Ashish Dutta. Professor, Dept. of Mechanical Engineering Indian Institute of Technology Kanpur, INDIA

HAND-SHAPED INTERFACE FOR INTUITIVE HUMAN- ROBOT COMMUNICATION THROUGH HAPTIC MEDIA

Humanoid robot. Honda's ASIMO, an example of a humanoid robot

Nagoya University Center of Innovation (COI)

Design and Control of the BUAA Four-Fingered Hand

Available theses in robotics (March 2018) Prof. Paolo Rocco Prof. Andrea Maria Zanchettin

Robotics: Evolution, Technology and Applications

Available theses in robotics (November 2017) Prof. Paolo Rocco Prof. Andrea Maria Zanchettin

ROBOTICS ENG YOUSEF A. SHATNAWI INTRODUCTION

Laser-Assisted Telerobotic Control for Enhancing Manipulation Capabilities of Persons with Disabilities

Robotics Evolution: From Production Rate to Human Productivity

4R and 5R Parallel Mechanism Mobile Robots

On Observer-based Passive Robust Impedance Control of a Robot Manipulator

Glossary of terms. Short explanation

Ensuring the Safety of an Autonomous Robot in Interaction with Children

Human-like Assembly Robots in Factories

A NEW ROBOTIC MANIPULATOR IN CONSTRUCTION BASED ON MAN-ROBOT COOPERATION WORK

Information and Program

LASER ASSISTED COMBINED TELEOPERATION AND AUTONOMOUS CONTROL

Available theses (October 2011) MERLIN Group

23 May 2018, Galveson, TX Science of Team Science 2018 Conference Ge WANG 1,3 and Ken-ichi SATO 2,3

Jane Li. Assistant Professor Mechanical Engineering Department, Robotic Engineering Program Worcester Polytechnic Institute

Optic Flow Based Skill Learning for A Humanoid to Trap, Approach to, and Pass a Ball

Self-learning Assistive Exoskeleton with Sliding Mode Admittance Control

FUNDAMENTALS ROBOT TECHNOLOGY. An Introduction to Industrial Robots, T eleoperators and Robot Vehicles. D J Todd. Kogan Page

Service Research and Innovation in Japan

Man-Machine Synergy Effector

Robots Learning from Robots: A proof of Concept Study for Co-Manipulation Tasks. Luka Peternel and Arash Ajoudani Presented by Halishia Chugani

Impacts and Risks Caused by AI Networking, and Future Challenges

INDUSTRY 4.0. Modern massive Data Analysis for Industry 4.0 Industry 4.0 at VŠB-TUO

Booklet of teaching units

Cognitive Systems and Robotics: opportunities in FP7

Motion Control of a Three Active Wheeled Mobile Robot and Collision-Free Human Following Navigation in Outdoor Environment

Autonomous and Mobile Robotics Prof. Giuseppe Oriolo. Introduction: Applications, Problems, Architectures

Robotics for Children

A robot which operates semi- or fully autonomously to perform services useful to the well-being of humans

CS494/594: Software for Intelligent Robotics

EE631 Cooperating Autonomous Mobile Robots. Lecture 1: Introduction. Prof. Yi Guo ECE Department

Chapter 1 Introduction

(Osaka Industrial Technology - Platform)

Robotics: Applications

ARTIFICIAL INTELLIGENCE - ROBOTICS

Multi-bit Sigma-Delta TDC Architecture for Digital Signal Timing Measurement

What is a robot. Robots (seen as artificial beings) appeared in books and movies long before real applications. Basilio Bona ROBOTICS 01PEEQW

Summary of the meeting with Professor Stewart Freeman

Introduction to Robotics

Henry Lin, Department of Electrical and Computer Engineering, California State University, Bakersfield Lecture 8 (Robotics) July 25 th, 2012

情Propagation Characteristics of 700MHz Band V2X Wireless Communication*

Magellan Systems Japan, Inc.

Automation and Mechatronics Engineering Program. Your Path Towards Success

Stabilize humanoid robot teleoperated by a RGB-D sensor

Human Robot Interaction (HRI)

Development of a telepresence agent

Haptic Virtual Fixtures for Robot-Assisted Manipulation

Summary of the Report by Study Group for Higher Quality of Life through Utilization of IoT and Other Digital Tools Introduced into Lifestyle Products

Cognitive Robotics 2017/2018

Cognitive Robotics 2016/2017

A Feasibility Study of Time-Domain Passivity Approach for Bilateral Teleoperation of Mobile Manipulator

Applied Robotics for Installations and Base Operations (ARIBO)

Available theses (October 2012) MERLIN Group

Computer Assisted Medical Interventions

SECOND YEAR PROJECT SUMMARY

Contents. Mental Commit Robot (Mental Calming Robot) Industrial Robots. In What Way are These Robots Intelligent. Video: Mental Commit Robots

Image processing of the weld pool and tracking of the welding line in pulsed MAG welding *

Android (Child android)

* Institute of Electrical and Electronics Engineers (IEEE), WORLD HAPTICS CONGRESS 2011 held in Istanbul, Turkey, in June 2011

The seven pillars of Data Science

Development of Drum CVT for a Wire-Driven Robot Hand

ROBOTICS, Jump to the next generation

FP7 ICT Call 6: Cognitive Systems and Robotics

Intermediate Conversation Material #10

Front Digital page Strategy and Leadership

Prospective Teleautonomy For EOD Operations

Industry 4.0. Advanced and integrated SAFETY tools for tecnhical plants

CG Image Generation of Four-Dimensional Origami 4 次元折り紙の CG 画像生成

Chapter 1. Robot and Robotics PP

Traffic Control for a Swarm of Robots: Avoiding Group Conflicts

Department of Robotics Ritsumeikan University

Proactive Behavior of a Humanoid Robot in a Haptic Transportation Task with a Human Partner

System of Recognizing Human Action by Mining in Time-Series Motion Logs and Applications

Overview Agents, environments, typical components

ENHANCED HUMAN-AGENT INTERACTION: AUGMENTING INTERACTION MODELS WITH EMBODIED AGENTS BY SERAFIN BENTO. MASTER OF SCIENCE in INFORMATION SYSTEMS

Cooperative Transportation by Humanoid Robots Learning to Correct Positioning

Robotics Introduction Matteo Matteucci

Digitizing European Industry

Performance Issues in Collaborative Haptic Training

2. Introduction to Computer Haptics

Digitalisation as day-to-day-business

Ubiquitous Network Robots for Life Support

Easy Robot Programming for Industrial Manipulators by Manual Volume Sweeping

Transcription:

Human-Robot Interaction from Dance Partner Robot to Co-worker Robot Kazuhiro Kosuge Systems Robotics Lab. Department of Robotics Graduate School of Engineering Tohoku University Sendai 980-8579, JAPAN http://www.irs.mech.tohoku.ac.jp

Outline Challenges and Opportunities of Robotics Overview of Robotics (by JST in 2009) Examples of Physical Human-Robot Interaction Dance Partner Robots MS DanceR and PBDR RoboDANTE PaDY PaDY in a factory Conclusions

Outline Challenges and Opportunities of Robotics Overview of Robotics (by JST in 2009) Examples of Physical Human-Robot Interaction Dance Partner Robots MS DanceR and PBDR RoboDANTE PaDY PaDY in a factory Conclusion

Robotics Societal Level Societal Values Service Level Service enablers Services Fundamental Technologies Level Foundations CRDS, JST, 2009, Modified by Kosuge, August, 2011

Societal Values For Individuals For Communities For Families For Industries For Local Government For Nations For the Globe Quality of Life Industrial Competitiveness Global Issues CRDS, JST, 2009, Modified by Kosuge, August, 2011

Challenges and Opportunities of Robotics Social Value Services Emerging Technology Fundamental Global Level Community Level Quality of Life Environmental Monitoring Natural Resources Exploration and Development Space Exploration Deep Undersea and Underground Exploration Anti-terrorism Rescue Operation Prevention of Infectious Diseases Robot Systems Integration Human Robot Interaction Real-world Real-time Intelligence Spatio-temporal System Design Sensing and Machine Cognition Government Service Agriculture Forestry Fishery Mining Manufacturing Construction Wastes Treatment/ Management Cyborg (Cybernetic organism) Stochasticity in Robotics Performance evaluation and Benchmarking Ambient intelligence Autonomous Robots Teleoperation Robotic Emotion (artificial emotion) Utilities Retailer/Wholesaler Transportation Communication Service Industries Medicine Education Research and Development Medicine Therapy Daily Life Assist Healthcare Rehabilitation Mental care Learning Child care Housekeeping Software framework Social Concerns Functional Safety Nano-micro Robotics Human Modeling Wearable Technology Service Contents Design Robot Kinematics and Dynamics Manipulation Mobility Actuation Physics-based Control Security Mobility Shopping Hobby Entertainment Sports Comfort Life Watch Communication Service/Application-oriented Robotics Robotics Foundations CRDS, JST, 2009, Modified by Kosuge, August, 2011

Robotics Research Unit Technologies Technical Issues Required Services Domain 1 Elderly Care Domain 2 Agriculture Domain 3 Medicine Applications/Services New Services Service/Applicationoriented Robotics Robotics Foundations Robotics Foundations Current Robot Function CRDS, JST, 2009, Modified by Kosuge, August, 2011

Robotics Research Unit Technologies Technical Issues Required Services New Services Domain 1 Elderly Care Domain Design 2a service/services Domain 3 necessary Applications/Services Agriculture Medicine for the application as a sustainable business. Service/Applicationoriented Robotics Robotics Foundations Robotics Foundations Current Robot Function CRDS, JST, 2009, Modified by Kosuge, August, 2011

Robotics Research Unit Technologies Technical Issues Required Services New Services Domain 1 Elderly Care Domain Design 2a service/services Domain 3 necessary Applications/Services Agriculture Medicine for the application as a sustainable business. Service/Applicationoriented Robotics Enhance Robotics unit technologies to meet Foundations the requirements for the service/services. Current Robot Function Robotics Foundations CRDS, JST, 2009, Modified by Kosuge, August, 2011

Robotics Research Unit Technologies Technical Issues Required Services New Services Domain 1 Elderly Care Domain Design 2a service/services Domain 3 necessary Applications/Services Agriculture Medicine for the application as a sustainable business. Develop new fundamentals necessary for the service/services. Service/Applicationoriented Robotics Enhance Robotics unit technologies to meet Foundations the requirements for the service/services. Current Robot Function Robotics Foundations CRDS, JST, 2009, Modified by Kosuge, August, 2011

Robotics Research Unit Technologies Technical Issues Required Services New Services Domain 1 Elderly Care Domain Design 2a service/services Domain 3 necessary Applications/Services Agriculture Medicine for the application as a sustainable business. Develop new fundamentals necessary for the service/services. Service/Applicationoriented Robotics Enhance Robotics unit technologies to meet Foundations the requirements for the service/services. Current Robot Function Robotics Foundations Enrich robotics foundations through application-oriented research CRDS, JST, 2009, Modified by Kosuge, August, 2011

Systems Robotics System robotics is a new field of robotics dealing with robot-related issues in real environments. Several prototypes of real world robots have been designed and developed based on robot technologies developed in our laboratory. Walking Helper Assistive Robotics Power Assisted Chair Cycle Intention Recognition/Transfer Intelligent Car Transportation Robot icart icart and icart Concept II Intelligent Car Autonomous-Robot- Transporters Robot Co-worker PaDY (in-time Parts/tools Delivery robot) Universal Robot Hand ugripp with Two-degrees of Freedom Assembly and Manipulauion by Dual Manipulators Stable Power Augmentation Human Robot Coordination Integration of Visual and Impedance Servo Mobile Manipulators Human-Robot Interaction Universal Manipulation Multiple Robots Coordination

Systems Robotics System robotics is a new field of robotics dealing with robot-related issues in real environments. Several prototypes of real world robots have been designed and developed based on robot technologies developed in our laboratory. Walking Helper Assistive Robotics Power Assisted Chair Cycle Intention Recognition/Transfer Intelligent Car Transportation Robot icart icart and icart Concept II Intelligent Car Autonomous-Robot- Transporters Robot Co-worker PaDY (in-time Parts/tools Delivery robot) Universal Robot Hand ugripp with Two-degrees of Freedom Assembly and Manipulauion by Dual Manipulators Stable Power Augmentation Human Robot Coordination Integration of Visual and Impedance Servo Mobile Manipulators Human-Robot Interaction Universal Manipulation Multiple Robots Coordination

Outline Challenges and Opportunities of Robotics Overview of Robotics (by JST in 2009) Examples of Physical Human-Robot Interaction Dance Partner Robots MS DanceR and PBDR RoboDANTE PaDY PaDY in a factory Conclusion

Human Power Augmentation Operator Robot Environment Human Power Augmentation [1] K. Kosuge, Y. Fujisawa, T. Fukuda, Mechanical System Control with Man-Machine-Environment Interactions, [Proceedings of the 1993 IEEE International Conference on Robotics and Automation (1993) 239-244]. [2] 小菅一弘, 藤沢佳生, 福田敏男, 環境との相互作用が生じるマン マシン系の制御, [ 日本機械学会論文集 (C 編 ) 59 (562) (1993) 1751-1756].

Robot Helpers Human-Robot Cooperation (Kosuge, 1993)

Robot Helpers Robot j Robot i Object Human l Human m Robot k Passive Dynamics Stability Issues

Robot Helpers MR Helper (Mobile Robot Helper, 1997~) [1] K. Kosuge, M. Sato, Mobile Robot Helper, [Proceedings of the 2000 IEEE International Conference on Robotics and Automation (2000) 583-588]. [2] 小菅一弘, 須田理央, 風村典秀, 佐藤学, 角谷啓, 人と双腕型移動ロボット MR Helper による物体の協調搬送, [ 日本機械学会論文集 (C 編 ) 69 (685) (2003) 84-90].

Robot Helpers DR Helpers (Distributed Robot Helpers) Y. Hirata, K. Kosuge, Distributed Robot Helpers Handling a Single Object in Cooperation with a Human, [Proceedings of the 2000 IEEE International Conference on Robotics and Automations (2000) 458-463]. 平田泰久, 初雁卓郎, 小菅一弘, 淺間一, 嘉悦早人, 川端邦明, 人間と複数の分散型ロボットヘルパ - との協調による単一物体の搬送, [ 日本機械学会論文集 (C 編 ) 68 (668) (2002) 181-188].

Robot Helpers DR Helpers (Distributed Robot Helpers) Y. Hirata, Y. Kume, Z. D. Wang, K. Kosuge, Decentralized Control of Multiple Mobile Manipulators Based on Virtual 3-D Caster Motion for Handling an Object in Cooperation with a Human, [Proceedings of the 2003 IEEE International Conference on Robotics and Automation (2003) 938-943].

Outline Challenges and Opportunities of Robotics Overview of Robotics (by JST in 2009) Examples of Physical Human-Robot Interaction Dance Partner Robots MS DanceR and PBDR RoboDANTE PaDY PaDY in a factory Conclusions

Lessons Learned DR Helper MR Helper

Lessons Learned from Robot Helpers Some simple tasks, which could not be done by a human/humans, could be done with a robot helper(s). General tasks could not be done easily even with the assistive robot system(s), because the robot does not know how to collaborate with the human.

Lessons Learned from Robot Helpers In order to collaborate with the user, the robot has to know the task, its user s intention, how the user wants to be assisted

Dance Partner Robot To develop a mechanism for closer human-robot coordination/interaction

Ballroom Dances A ballroom dance is performed by a dance couple, a male dancer and a female dancer. A dance consists of a set of figures of the dance.

Ballroom Dances The dance is lead by a male dancer. Which figure is coming next is controlled by the male dancer based on the rule of the dance and their surroundings.

Ballroom Dances The female dancer estimates the following figure at each figure transition through the interaction with the male dancer based on the knowledge of the dance.

Ballroom Dances The female dancer has to know the dance in order to dance with her partner: The set of the dance figures. The figure transition rule. How to be lead by the male dancer or how to estimate the partner s figure.

Dance Partner Robot MS DanceR

Aichi Expo (March 24 ~ September 25, 2005)

Design of Robot Mechanism DoF: Neck : 1 Waist : 3 Arms : 4 x 2 Omni-directional mobile base : 3 Designed by Nomura Unison Co. Ltd. in cooperation with Tohoku University

Force/Moment Sensory Data Used for Estimation Reference Data Dance Figure A Transition Dance Figure B T effective Time Time series data include uncertainty such as time-lag and variation because a dancer cannot always apply the same force/moment for each figure transition. T. Takeda, K. Kosuge, Y. Hirata, HMM-based Dance Step Estimation for Dance Partner Robot -MS DanceR-, [Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (2005) 1602-1607].

Force/Moment Sensory Data Used for Estimation HMM-based Figure Estimator Reference Data Dance Figure A Transition Dance Figure B T effective Time T. Takeda, K. Kosuge, Y. Hirata, HMM-based Dance Step Estimation for Dance Partner Robot -MS DanceR-, [Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (2005) 1602-1607].

Dance Partner Robot PBDR PBDR as a Research Platform for Human-robot interaction

Dance Partner Robots PBDR as a Research Platform for Physical Human-Robot Interaction

Dance Partner Robots PBDR as a Research Platform for Physical Human-Robot Interaction

Dance Partner Robot

Dance Partner Robot

Dance Partner Robot

Dance Partner Robot

PBDR in Korea

Outline Challenges and Opportunities of Robotics Overview of Robotics (by JST in 2009) Examples of Physical Human-Robot Interaction Dance Partner Robots MS DanceR and PBDR RoboDANTE PaDY PaDY in a factory Conclusions

Challenges for Dance Partner Robots Stable physical interaction between a human and a robot Female Dance Partner Robot Human behavior/intention estimation How to read the its partner s lead Male Dance Partner Robot How to convey robot s intention to its human partner Motion Entrainment Based on Human modeling

Challenges for Dance Partner Robots Stable physical interaction between a human and a robot Female Dance Partner Robot Human behavior/intention estimation How to read the its partner s lead Male Dance Partner Robot Dance Teaching Robot How to convey robot s intention to its human partner Motion Entrainment Based on Human modeling

Robot Design Upper Body Mechanical Assembly Whole Body Assembly Interactive Screen F/T Sensor LRF-1 PC Electronics Mobile Base LRF-2

Motion Tests

Motion Analysis of Dancers Analysis of the motions in dance using motion capture system: COM cycle within a dance figure. An example of captured motions during the dance figure: closed change using commercially available software [1]. [1] Motion Analysis Corporation. (n.d.). Motion Analysis. Retrieved March 27, 2015, from http://www.motionanalysis.com/html/industrial/industrial.html

Motion Analysis of Dancers Example of four dance figures COM motions in 3D. COM cycle within a dance figure.

Z [m] 0.1 0 Motion Analysis of Dancers -0.2-0.1-0.2-0.3 0.8 0.2 0.6 0.4 0.4 0.2 0.6 0 0.8-0.2 X [m] Y [m] An example of COM Motions in 3D space 0 Z [m] 0.1 0-0.1-0.2-0.3 0-0.5 X [m] -1 0 Z [m] 0.2 0.1 0-0.1-0.2 Natural Turn 3D Trajectory Directional Change XY Trajectory Motion Initiation Z [m] 0.2 0.1 0-0.1-0.2 ReverseTurn 3D Trajectory XY Trajectory -0.3 1.5 1 0.5 X [m] 0 1.5 1 Y [m] 0.5 0-0.3 1.5 1 0.5 X [m] 0 1.5 1 3D Trajectory XY Trajectory During a dance figure in Waltz, we observed that COM motions is related to Motion initiation. Directional changes. Back W hisk W hisk and Chasse 0.2 0.2 3D Trajectory XY Trajecto

Motion Analysis of Dancers An example of COM Motions in 3D space Transition Motion Initiation During the transitions of dance figures in Waltz, we also observed the similar pattern in COM motions relating to motion initiation and figure transition.

Leading the Partner by COM Motion Guidance using COM motions through an unstable equilibrium point (G-UEP): Bringing the partner s COM to an unstable equilibrium point (UEP) by elevating the robot s COM during the physical interaction in the dance. Exerting a force in the desired motion direction. Unstable equilibrium point example. Example of guidance

Evaluation of the Proposed Method Cycle Type 1 1Cycle Type 2 Cycle 23 Cycle Type 4 3Desired Type Traject. 4 0.7 0.6 47.71 46.91 0.5 Y [m] 0.4 0.3 0.2 0.1 F 12 F 34 29.44 28.25 0-0.8-0.7-0.6-0.5-0.4-0.3-0.2-0.1 0 X [m] Example of trajectories achieved by a subject during the 4 types of test. Example video of experiments with subjects.

Evaluation of the Proposed Method 100,00% Motion Understanding 80,00% 89,17% 90,42% 60,00% 71,92% 70,00% 40,00% 20,00% 0,00% 1. G-UEP with voice-over 2. Pure force with voice-over 3. G-UEP without voice-over 4. Pure force without voice-over Understanding of robot s intention of motion was evaluated with the percentage of success in the motion (correct direction of motion). G-UEP shows approximately 20% more motion direction understanding than pure force guidance. Voice-over on interactions did not show difference in the motion understanding rate.

Evaluation of the Proposed Method F 12 F 34 F 12 F 34 Significant reduction in interactions force was obtained by using the G-UEP.

Subjective Evaluation Comfort: Emotional states which people have positive feelings during interactions such as relief, ease, and comfortable. Peace of Mind: Emotional states that reflects the opposite to negative feelings during interaction such as uneasiness and stress with the robots. Performance: Evaluative judgment about the robot s ability to interact. Controllability: Evaluative judgment about negative aspect regarding the interaction with the robot such as performing unanticipated actions, and harming humans. Human-likeness: perceived similarity with humans in interactions. Enjoyability: self-evaluated level of enjoyment during the dance. Personal Growth: perceived personal improvement in the dance. Kamide, H., Kawabe, K., Shigemi, S., & Arai, T. (2015). Anshin as a concept of subjective well-being between humans and robots in Japan. Advanced Robotics, 29(24), 1624 1636.

Subjective Evaluation Significant difference was obtained in factors of Comfort, Performance, and Humanlikeness. 100,00% 90,00% 88,00% 88,00% 80,00% 80,00% 86,00% 84,00% 72,00% 60,00% 60,00% 40,00% 20,00% 0,00% 92,00% 90,00% 90,00% 84,00% 94,00% 86,00% 74,00% 84,00% 78,00% 74,00% 76,00% 74,00% 68,80% 58,00% 59,20% 60,00% 60,00% 59,20% 44,00% 48,00% [G-UEP & Voice-over] [G-UEP without Voice-over] [Pure force with voice-over] [Pure force without voice-over]

Progressive Teaching The progressive teaching (PT) methodology is proposed based on Piagets theory of cognitive development states. Knowledge is constructed based on experiences related to mental, biological and physical stage of the development. [1] [1] L.-D. Hammond, K. Austin, and S. Orcutt, How People Learn, Stanford University, Tech. Rep., 2001. Proposed methodology: 1. Assessment of the skill based on the current stage of physical development. 2. Feedback of the assessed skill performance for enhancing skill model formation. 3. Physical feedback based on skill performance. Diego Felipe Paez Granados, Breno A. Yamamoto, Hiroko Kamide, Jun Kinugawa, and Kazuhiro Kosuge, Dance Teaching by a Robot: Combining Cognitive and Physical Human Robot Interaction for Supporting the Skill Learning Process, IEEE Robotics and Automation Letters, Voo.2, pp.1452-1459, 2017

Progressive Teaching The progressive teaching (PT) methodology is proposed based on Piagets theory of cognitive development states. Knowledge is constructed based on experiences related to mental, biological and physical stage of the development. [1] Proposed methodology: 1. Assessment of the skill based on the current stage of physical development. 2. Feedback of the assessed skill performance for enhancing skill model formation. 3. Physical feedback based on skill performance. [1] L.-D. Hammond, K. Austin, and S. Orcutt, How People Learn, Stanford University, Tech. Rep., 2001.

Skill Assessment of Dance Actual Desired Analysis of the skill in the velocity profile permits a better assessment for any user. Skill is assessed based on mean squared error of velocity calculated as follows: Where e n = X dn (kτ) X n (kτ) 2 represents the error and W n represent a n x n weighing matrix. Diego Felipe Paez Granados, Breno A. Yamamoto, Hiroko Kamide, Jun Kinugawa, and Kazuhiro Kosuge, Dance Teaching by a Robot: Combining Cognitive and Physical Human Robot Interaction for Supporting the Skill Learning Process, IEEE Robotics and Automation Letters, Voo.2, pp.1452-1459, 2017

Skill Assessment of Dance Skill assessment is carried out based on score zones. Current stage is assumed proportional to practices count: Generous scoring for novice dancers. Severe scoring for experienced dancers. Diego Felipe Paez Granados, Breno A. Yamamoto, Hiroko Kamide, Jun Kinugawa, and Kazuhiro Kosuge, Dance Teaching by a Robot: Combining Cognitive and Physical Human Robot Interaction for Supporting the Skill Learning Process, IEEE Robotics and Automation Letters, Voo.2, pp.1452-1459, 2017

Progressive Teaching The progressive teaching (PT) methodology is proposed based on Piagets theory of cognitive development states. Knowledge is constructed based on experiences related to mental, biological and physical stage of the development. [1] [1] L.-D. Hammond, K. Austin, and S. Orcutt, How People Learn, Stanford University, Tech. Rep., 2001. Proposed methodology: 1. Assessment of the skill based on the current stage of physical development. 2. Feedback of the assessed skill performance for enhancing skill model formation. 3. Physical feedback based on skill performance. Diego Felipe Paez Granados, Breno A. Yamamoto, Hiroko Kamide, Jun Kinugawa, and Kazuhiro Kosuge, Dance Teaching by a Robot: Combining Cognitive and Physical Human Robot Interaction for Supporting the Skill Learning Process, IEEE Robotics and Automation Letters, Voo.2, pp.1452-1459, 2017

Progressive Teaching Combining cognitive and physical interaction for enhancing the skill learning. Diego Felipe Paez Granados, Breno A. Yamamoto, Hiroko Kamide, Jun Kinugawa, and Kazuhiro Kosuge, Dance Teaching by a Robot: Combining Cognitive and Physical Human Robot Interaction for Supporting the Skill Learning Process, IEEE Robotics and Automation Letters, Voo.2, pp.1452-1459, 2017

Progressive Teaching The robot gives feedback of the current state through CPS in color scales. Knowledge Performance Novice Expert Diego Felipe Paez Granados, Breno A. Yamamoto, Hiroko Kamide, Jun Kinugawa, and Kazuhiro Kosuge, Dance Teaching by a Robot: Combining Cognitive and Physical Human Robot Interaction for Supporting the Skill Learning Process, IEEE Robotics and Automation Letters, Voo.2, pp.1452-1459, 2017

Diego Felipe Paez Granados, Breno A. Yamamoto, Hiroko Kamide, Jun Kinugawa, and Kazuhiro Kosuge, Dance Teaching by a Robot: Combining Cognitive and Physical Human Robot Interaction for Supporting the Skill Learning Process, IEEE Robotics and Automation Letters, Voo.2, pp.1452-1459, 2017 Progressive Teaching The progressive teaching (PT) methodology is proposed based on Piagets theory of cognitive development states. Knowledge is constructed based on experiences related to mental, biological and physical stage of the development. [1] [1] L.-D. Hammond, K. Austin, and S. Orcutt, How People Learn, Stanford University, Tech. Rep., 2001. Proposed methodology: 1. Assessment of the skill based on the current stage of physical development. 2. Feedback of the assessed skill performance for enhancing skill model formation. 3. Physical feedback based on skill performance.

Adaptive Interaction Control Low-level controller model Robot Impedance Human Partner K f F d F i F r K h K d M d M h. X d. X c D h X c = X d + M d 1 ( K d X d X K f F d F i ) K d Damping Gain K f Interaction ForceError gain F i Estimated Interation Force M d Desired Impedance Inertial Matrix K h Human Stifness D h Human Damping Modifying the robot s impedance, so that, for low CPS values would produce higher damping and guidance force and high CPS would decrease both damping and force. Diego Felipe Paez Granados, Breno A. Yamamoto, Hiroko Kamide, Jun Kinugawa, and Kazuhiro Kosuge, Dance Teaching by a Robot: Combining Cognitive and Physical Human Robot Interaction for Supporting the Skill Learning Process, IEEE Robotics and Automation Letters, Voo.2, pp.1452-1459, 2017

Adaptive Interaction Control Enhancing the perceived safety and trust in the robot during interactions through the usage of a limiting interaction force. For beginners level the maximum f d has been settled to 60 N. Diego Felipe Paez Granados, Breno A. Yamamoto, Hiroko Kamide, Jun Kinugawa, and Kazuhiro Kosuge, Dance Teaching by a Robot: Combining Cognitive and Physical Human Robot Interaction for Supporting the Skill Learning Process, IEEE Robotics and Automation Letters, Voo.2, pp.1452-1459, 2017

Evaluation of Progressive Teaching User-based Configurations: 12 Users: 6 Constant Dynamics (non-adaptive impedance control) and 6 users with the proposed adaptive PT controller. 50% female and 50% male. 6 types of dance figures:

Evaluation of Progressive Teaching Subjective evaluation [1] among trainings: 12 subjects Significant difference at the p <.01level for the two training conditions was found for Comfort, Peace of mind, and Performance factors. CPS: Cumulative Performance Score [1] Kamide, H., Kawabe, K., Shigemi, S., & Arai, T. (2015). Anshin as a concept of subjective well-being between humans and robots in Japan. Advanced Robotics, 29(24), 1624 1636.

RoboDANTE (Robot DANce TEacher) http://spectrum.ieee.org/automaton/robotics/robotics-hardware/video-friday-robot-dance-teacher-transformer-drone-pneumatic-reel-actuator

RoboDANTE (Robot DANce TEacher)

Outline Challenges and Opportunities of Robotics Overview of Robotics (by JST in 2009) Examples of Physical Human-Robot Interaction Dance Partner Robots MS DanceR and PBDR RoboDANTE PaDY PaDY in a factory Conclusions

Automobile Factories Welding Process Industrial robots have played important roles in manufacturing industries, especially in automobile factories.

Automobile Factories Assembly Process http://response.jp/issue/2004/0120/article57131_1.images/61053.html There are many non-automated processes. Industrial robots are not suitable for tasks that require dexterous human skills, tasks in unstructured environment, etc.

Automobile Assembly Line A sequence of the tasks, necessary parts/tools for each task, and when and where each task is performed are scheduled a priori for each type of the car produced. During the work, the worker needs to return to a work bench with parts and tools several times to pick up necessary parts/tools.

Automobile Assembly Line If a robot could provide the worker with necessary parts and tools when he/she needs them, the worker could concentrate on the assembly tasks. [1] 衣川潤, 川合雄太, 菅原雄介, 小菅一弘, 組立作業支援パートナロボット PaDY( 第 1 報, コンセプトモデルの開発とその制御 ),[ 日本機械学会論文集,C 編,77(783),(2011),4204-4217] [2] J. Kinugawa, Y. Kawaai, Y. Sugahara and K. Kosuge, PaDY : Human-Friendly/Cooperative Working Support Robot for Production Site, [The 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems Proceedings,(2010),5472-5479].

Co-worker Robot PaDY PaDY is a robot which delivers necessary parts and tools to a worker when he/she needs them. to reduce the worker s load to improve efficiency of the work to prevent mistakes of the work etc. PaDY [1] 衣川潤, 川合雄太, 菅原雄介, 小菅一弘, 組立作業支援パートナロボット PaDY( 第 1 報, コンセプトモデルの開発とその制御 ),[ 日本機械学会論文集,C 編,77(783),(2011),4204-4217] [2] J. Kinugawa, Y. Kawaai, Y. Sugahara and K. Kosuge, PaDY : Human- Friendly/Cooperative Working Support Robot for Production Site, [The 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems Proceedings,(2010),5472-5479]. in-time Parts/tools Delivery to You robot

Co-worker Robot PaDY In order to deliver parts/tools to a place, where the worker needs them, when the worker needs them without disturbing the worker s work, the robot needs to know the task, its user s intention, and how the user want to be assisted in-time Parts/tools Delivery to You robot

The First Prototype of PaDY (P1) Size:(W)1370 (D)590 (H)1035[mm] Link Mechanism :Horizontal Articulated Robot Maximum Reach:2.0 [m](1 st Link Length:1168[mm], 2 nd Link Length:982[mm]) Weight of Working Parts:11.5[kg] Maximum Load:3 [kg] Range of Movement: 1 st Joint: 200[deg], 2 nd Joint: 360[deg] Actuator:1 st Joint & 2 nd Joint:DC Servo Motor 80[W], 3 rd Joint: DC Servo Motor 15[W]

Evaluation Experiment +:LRF1 :LRF2 Measured data The worker s motion necessary for picking parts/tools has been reduced. The worker could finish his tasks earlier than the work schedule. Effect of PaDY Estimated result

Factory Installation

1.2 m Improved Worker s Behavior Estimation PaDY Vehicle Body Task1 2 Task2 3 Task3 1 2.0 m Start Goal X LRF LRF Y LRF X PaDY LRF Y PaDY PaDY A subject is requested to do three tasks. After the three tasks were finished, the probabilistic model is updated. Jun Kinugawa, Akira Kanazawa, Shogo Arai, and Kazuhiro Kosuge, Adaptive Task Scheduling for an Assembly Task Coworker Robot Based on Incremental Learning of Human s Motion Patterns, IEEE Robotics and Automation letters, Vol. 2, No. 2, pp.856-863, 2017

Improved Worker s Behavior Estimation Ordinary Motion (10 th trial by Subject A) Variance of Estimated Trajectory Task1 Estimated Trajetory Task2 Observed Workers Position Task3 Start and Goal Position Jun Kinugawa, Akira Kanazawa, Shogo Arai, and Kazuhiro Kosuge, Adaptive Task Scheduling for an Assembly Task Coworker Robot Based on Incremental Learning of Human s Motion Patterns, IEEE Robotics and Automation letters, Vol. 2, No. 2, pp.856-863, 2017 PaDY

Improved Worker s Behavior Estimation trial 1 (without estimation) trial 10 (with estimation) Start Task1 Task1 Task2 Comparison of the work by subject B Jun Kinugawa, Akira Kanazawa, Shogo Arai, and Kazuhiro Kosuge, Adaptive Task Scheduling for an Assembly Task Coworker Robot Based on Incremental Learning of Human s Motion Patterns, IEEE Robotics and Automation letters, Vol. 2, No. 2, pp.856-863, 2017

Subject C, Irregular 1 Improved Worker s Behavior Estimation Subject B, Irregular 3

Improved Worker s Behavior Estimation Subject D, Irregular 3 Subject E, Irregular 1

B-PaDY

B-PaDY

B-PaDY

B-PaDY

D-PaDY

D-PaDY

D-PaDY D-PaDY

D-PaDY 94

Conclusions Overview of Robotics has been introduced. Dance Partner Robots, PBDR and RoboDANTE have been introduced as research platforms for phri (Physical Human-Robot Interaction). The platforms have given us opportunities to reconsider issues relating to phri. PaDY has been introduced as examples of applications of phri. HRI will open new applications in many manufacturing processes.

Robotics Research Unit Technologies Technical Issues Required Services New Services Domain 1 Elderly Care Domain Design 2a service/services Domain 3 necessary Applications/Services Agriculture Medicine for the application as a sustainable business. Develop new fundamentals necessary for the service/services. Service/Applicationoriented Robotics Enhance Robotics unit technologies to meet Foundations the requirements for the service/services. Current Robot Function Robotics Foundations Enrich robotics foundations through application-oriented research CRDS, JST, 2009, Modified by Kosuge, August, 2011