1 Studies on Interactive Robots Hiroshi ISHIGURO Distinguished Professor of Osaka University Visiting Director & Fellow of ATR Hiroshi Ishiguro Laboratories Research Director of JST ERATO Ishiguro Symbiotic HRI Project ishiguro@sys.es.osaka-u.ac.jp www.is.sys.es.osaka-u.ac.jp, www.geminoid.jp. 4 What is identity? Who has the Ishiguro s identity? Is it Ishiguro or the Geminoid? Robot Society 1
Human-Robot Symbiotic Society 7 Android Human Humanoid robots and Android robots Android Information/sensor network 10 Why Do We Need Humanoid Robots? Humans have a brain that recognizes humans. The ideal interface for humans is a human. Therefore, information media devices, e.g., talking rice cookers, should be at least partially humanlike. On the other hand, the very humanlike robots, android robots, are necessary for understanding humans. Personal Robots Interactive Communicative Robots The Future Society Supported by Humanoids & Androids 11 Humanoids Public places Learning Robot and Science Elderly care Department stores Stations 2
Constructive Science with Intelligent Systems Past brain studies have found that human intelligence comprises elements of memory, computation, inference, learning, etc. The challenge is to identify how these elements can be put together for realizing human intelligence. The constructive approach will reconstruct robots and androids to model humans, and use such robots and androids to study human macro-level functions. Intelligent robotics Developing humanlike robots based on knowledge in science Understanding humans by using the robots Cognitive & Neuroscience 13 Progress of Robot Technology and Human Science Cognitive Science with Robots Social relationship Voice recognition Consciousness Image understanding Intention/Desire Sensor Multi-modal Integration Manipulator Embodiment Mechanics Intelligence 14 Robot Technology Human Android Science Human Fundamental functions of robots Meta-level Cognitive functions Robot Development 15 Language teacher robots Order-taking robots at restaurants 16 Companion robots Counseling robots Conversation training robots Interpreter/Guide robots Receptionist/Concierge robots Shop keeper robots Conversational robots for elderly and children Conversational robots for the elderly Receptionist/Concierge robots Language teacher robots Oder-taking robots at restaurants Intelligent sensor network Self-organizing sensor network Science and Robot Development Total Turing Test 17 Total Turing Test (TTT) as a scientific and engineering goal 18 Science Intelligence Embodiment Multi-modal Integration Intention/Desire Consciousness Social Relationship Interpreter/Guide robots Receptionist/Concierge robots Shop keeper robots Counseling robots Companion robots Conversation training robots TTT is to compare between a robot manipulated by human operators and an autonomous robot controlled by developed technology. One of the important challenges in intelligent robotics is to pass TTT. It evaluates the total humanlikeness through all of the modalities. It evaluates the social acceptance as a member of our society. Tele-operated Robot Compare Autonomous Robot Conversational robots for the elderly and children Language teacher robots Oder-taking robots at restaurants 2017 2022 Robot development 2027 Interlocutor Social Relationship Consciousness Intention/Desire Multi-modal Integration Embodiment Intelligence Interlocutor 3
19 20 Studies on Interactive Robots Interaction in Robotics Engineering Science Mechanical Engineering Systems Engineering Neuro science Robotics Pattern Recognition Cognitive science Psychology Hypothesis & verification Ishiguro has built a new field in robotics: Interaction. Before that, navigation and manipulation were major topics in robotics. Recently, 10-20% papers at major conferences focus on interaction. Japanese government expects that the market of interactive robots/service robots, will be bigger than manipulators and it will reach 50 Billion USD in 2035. 100 80 60 40 20 50 Billion USD Interaction Navigation Manipulation Year 2020 2035 Applications of the Androids Androids for Extending Human Presence Ishiguro s Android Geminoid Android of a Living National Treasure 4
Android of a Great Literary Figure Androids and Human Presence 27 28 What is human presence? What is human? Androids for Representing Humanlikeness in Artistic Areas House Mannequin Android Android as an Idol Singer 5
Android Theater with Oriza Hirata Movie Sayonara (Goodbye) 33 34 What is heart/mind? We feel the humanlike hearts /minds of the android, but it is a simple computer program. Androids with Conversational Functions - Toward Total Turing Test - Autonomous Conversational Android 35 36 Internet Operator Big-data approach Collection of simple conversational patterns through the Internet for unlimited topics Chatbots Story-based approach Stories presented in graphical models Complex conversation for limited topics Computer 6
37 38 Conversational Autonomous Androids NTT Laboratories Osaka University and ATR Microphone array by NTT Voice recognition by NTT Chatbot by NTT Android by Osak Univ. Lip and head movements by ATR Chat-Oriented Dialogue System 39 Androidol U 40 with NTT 2015 2016 SXSW, LLC Internet TV 41 with NTT 42 ロボット社会 7
Casual Discussion Dialogue System with NTT 43 Deep Discussion Dialogue System with NTT 44 Mechanism of Deep Discussion Voice Recognition (Words, Positive/Negative) Answer Next topic 45 Toulmin Model of Argument Harry must be a British subject. Harry was born in Bermuda. 46 Claim Ground (or Data) Human Question Answer Back Channel Claim Ground Warrant Warrant A man born in Bermuda will be a British subject. 47 48 Big-data approach Collection of simple conversational patterns through the Internet for unlimited topics Chatbots Story-based approach Stories presented in graphical models Complex conversation for limited topics JST ERATO ISHIGURO Symbiotic Human Robot Interaction Project 2014-2020 mainly with Prof. Kawahara (Kyoto University) Dr. Minato, Dr. Ishii, and NTT 8
Android with Intensions and Desires Speech and gesture recognition Hierarchical models of Intention utterance and Desire behavior generation Robot Utterance and behavior generation Intention Intention Intention Intention Intention Intention 49 Implementation of Desire Model The intention binds the desires and actions to satisfy them. The action that mostly satisfies the current desires is chosen. Ex. Sequence beginning with question which shop to go to buy cloths Self The other Self Me, too. Cheap Feedback of action is best. Talk contents Which shop do That s great Action(Sequential organization topic, focus) The other s state you go? Usually UNIQLO. such as emotion Usually department stores Sequential organization Topic, focus Cloths Convenience store (Pattern of utterance sequence) Sequence Sequence Sequence beginning beginning beginning Greeting with with with question statement monologue Focus Closin Intention g talk Weights depending on how the action satisfies the intention Talking Talking about Letting the about something the other other talk myself likes Greeting Talk contents The other s state Brand-name goods Topic Values, Opinion Preference 50 Desire Desire Desire Desire Weights depending on how the intention satisfies the desire Self-esteem Relationship-building Update desire ISHIKI System with Prof. Kawahara (Kyoto Univ.) 51 Interaction Composer 52 ERICA with Prof. Kawahara (Kyoto Univ.) 53 ERICA 54 9
55 56 ERICA 57 58 Fundamental Issues of Conversational Robots 59 60 How to represent the feeling of presence? mainly with Prof. Ogawa, Dr. Nishio and Dr. Sumioka What is conversation? mainly with Prof. Yoshikawa Feeling of presence Sonzaikan 10
Geminoid: Teleoperated Android to Transfer Human Presence 61 Meeting with the Geminoid 62 Internet Operator 63 Recognition Based on Observation 64 How do we recognize the robot/android? Recognition Based on Imagination 65 66 Robot that Activates User s Imagination 11
Telenoid: Teleoperated Robot Based on User s Imagination With Nishio 67 68 The Minimum Media to Transfer the Human Presence 69 The minimum condition to feel human presence It requires two modalities. Hugvie: the minimum media. Talk by phones 70 With Sumioka Smartphone holder Blood & saliva test Relaxing Effect of Hugvie Talking to an unknown person for 15 min. 5 min. rest Phone 被験者 Hugvie With Sumioka Blood & saliva test 71 Cortisol, the Stress Hormone, Is Reduced with Hugvie. Scientific Reports, 2013 p<0.05 p<0.05 72 with Sumioka Hugvie Phone Hugvie Phone Blood Saliva 12
Representation with two modalities for activating the imagination 73 1 st grade children in an elementary school With Sumioka 74 Voice + Tactile sensation Appearance + Tactile sensation Smell + Tactile sensation Voice + Smell Strength of feeling 0 1 2 3 4 5 # of modalities With Hugvie 75 With Sumioka 76 Conversation CommU: Communication with Two Robots with Yoshikawa 77 Conversation without voice recognition With If there are two robots, robots do not need to Yoshikawa understand the person s utterance. The robots talk to each other and bring the person to their conversation. 78 13
How to Ignore the Person s Utterance with Yoshikawa 79 80 Ask the person a question just before switching the subject. Ask the same question to both another robot and the person. What Is Conversation? Robots do not need to recognize the voice. 81 82 Shopkeeper android Conversation by Using Touch-Panel with Ogawa with Ogawa Selecting an option is much easier than speaking. The visitor coming to the shop does not have many stories. Android voice Story Selection by touching Computer voice 83 84 Touch-Panel with Ogawa What Is Conversation? Positive Positive Positive Negative Robots do not need to recognize the voice. People do not need to pronounce. (Choice from options is enough) 14
Touch-panel conversation The computer controls the stories. with Ogawa People can adapt to stories by selecting options and hearing the voice. 85 People Start Liking Each Other by Touch-Panel Conversation With Ogawa 86 Story controlled by the computer Computer voice Selection by touching Selection by touching How to Connect People Human-Robot Symbiotic Society 88 Information/sensor network 15