1. Introduction Conversational Informatics

Transcription

1 Conversational Informatics (E), October 5, Introduction Conversational Informatics Toyoaki Nishida Kyoto University Copyright 2016, Toyoaki Nishida, Atsushi Nakazawa, Yoshimasa Ohmoto, Yasser Mohammad, At,Inc. All Rights Reserved.

2 Our living space today Google Translate Pokémon GO Line Takara Hello!MiP

3 Our living space in the near future Cloud People / things / information / intelligence Prosthesis for mind Transportation Prosthesis for body Daily work Industry

4 ~ History of AI and C&C 1936 Turing machines 1951 Commercial computer UNIVAC 1956 Dartmouth Conferences Dawn 1966 ELIZA / ALPAC report 1969 ARPA-net 1 st Winter 1973 Lighthill report 1974 MYCIN 1975 Microsoft Grow up 1976 Ethernet 1982 Version space / Fifth generation project 1984 First Machintosh 1985 AARON 1986 Back propagation 1989 ALVINN 1991 WWW 1994 Internet shopping mall 1995 Windows 95 2 nd Winter 1997 Google Search / Deep Blue won G. Kasparov 3 ½ - 2 ½ 1999 SONY AIBO 2000 Honda ASIMO 2004 Facebook 2006 Twitter / Deep learning by Hinton 2007 iphone 2008 Android OS / Nao Blossom 2011 IBM Watson / Siri / Google Self Driving Car / Kinect 2015 Pepper 2016 AlphaGo won 4-1 against Lee Sedol / Pokémon GO

5 ~ History of AI and C&C 1936 Turing machines 1951 Commercial computer UNIVAC Heuristic programming: 1956 Dartmouth Conferences Dawn - intelligence as clever search 1966 ELIZA / ALPAC report 1969 ARPA-net 1 st Winter 1973 Lighthill report 1974 MYCIN 1975 Microsoft Grow up 1976 Ethernet 1982 Version space / Fifth generation project Symbolic computing: 1984 First Machintosh - knowledge representation and inference 1985 AARON 1986 Back propagation 1989 ALVINN 1991 WWW 2 nd Winter 1994 Internet shopping mall 1995 Windows Google Search / Deep Blue won G. Kasparov 3 ½ - 2 ½ 1999 SONY AIBO 2000 Honda ASIMO 2004 Facebook 2006 Twitter / Deep learning by Hinton 2007 iphone 2008 Android OS / Nao Blossom 2011 IBM Watson / Siri / Google Self Driving Car / Kinect 2015 Pepper 2016 AlphaGo won 4-1 against Lee Sedol / Pokémon GO Machine learning and data mining: - discovering patterns from data, potentially to improve its behavior

6 History of human-agent interaction H: Men are all alike. E: IN WHAT WAY H: They're always bugging us about something or other. E: CAN YOU THINK OF A SPECIFIC EXAMPLE H: Well, my boyfriend made me come here. E: YOUR BOYFRIEND MADE YOU COME HERE H: He says I'm depressed much of the time. E: I AM SORRY TO HEAR YOU ARE DEPRESSED ELIZA: natural language dialogue (1966) The Knowledge Navigator: AI battler (concept, 1987) Façade: interactive drama (2004) USER: Pick up a big red block SHRDLU: OK simulated robot arm SHRDLU: natural language understanding system(1971) ALIVE: artificial life (1995) PUT-THAT-THERE: multi-modal interaction (1980) PEEDY: assistive interface (1997) t [Nishida, T., Nakazawa, A., Ohmoto, Y., Mohammad, Y. Conversational Informatics A Data-Intensive Approach with Emphasis on Nonverbal Communication, Springer 2014]

7 AI in present day Problem solving / decision making Language Heuristics Learning Emotion Reproduction by computer Living space Perception / Embodiment

8 Humanity and technology AI Human society Data Service Common ground

9 AI in present day Competence Embodiment Emotion Language Heuristics PS, DM Learning Application Perception Game play Conversation partner Information assistant Diagnosis / Advise Watchdog Fintech Receptionist Human care Autonomous car Guardian Cyber security Logistics Manufacturing Science Education Art :Primary, :Value adding, : Expected [Nishida 2016 (J)]

10 Social impacts Service copying Hybrid computing t

11 Level 1: Machinas by natural power In a larger picture Before industrial evolution (~ mid-18 th century): thrusted by water, wind,.... Machina (Latin): machine Level 2: Mechanical machinas Industrial evolution and after (late-18 th century ~): driven by large power source using coal, gasoline,. Level 3: Electric machinas Late-19 th century: Relatively complex and flexible control becomes available using electricity. Telephone was invented. Level 4: Computer-controlled machinas Late-20 th century: Globally connected universal computers direct repetitive and interactive tasks with ultra-high speed and reliability Level 5: Perceptual machinas 21 st century: Capable of recognizing complex environment and conduct skillful tasks. Level 6: Autonomous machinas Make decisions by themselves to comply with ethical principles. Level 7: Conscious machinas Behave based on the sense of self.

12 Social concerns with AI 1. Safety and controllability 2. Increased risk 3. Unemployment 4. Autonomous weapon 5. Vulnerability of humanity Short/mid-term view service Human interface Monitor Drive Supervisor Developers Manufacture AI R & D Researchers Consumer Participation / governance Human society

13 Human potential Human and social potential The power of an individual that enables her/him to actively sustain an endeavor to achieve a goal in maintaining a social relationship with other people. It involves vision, activity, sustainability, empathy, ethics, humor, and aesthetic sense. Social potential The power that a society of people possesses as a whole. It encompasses generosity, supportivity, conviviality, diversity, connectedness, and innovativeness. [Nishida, 2016]

14 Future life supported by Digital companion Role of digital companion Digital Companion - 2 Protect boss Help boss Monitoring boss s health Entertain boss Negotiate on behalf of boss Make a diary for boss Learn from boss Digital Companion - 1 Social interaction Digital Companion - 3 Social platform

15 Companion Agent Ten do's and don'ts for artificial companionship 1. do something useful! 2. don t just do one thing! 3. do understand the situation (as much as possible!) 4. do what I want! 5. be a good conversationalist! 6. do revisit conversations appropriately! 7. do understand what you are saying! 8. don t be repetitive! 9. do learn and customize! 10.do maintain immersion! [Traum 2015]

16 Conversation in the wild In the wild In the lab setting

17 Conversation is a complex business Trust Conviviality Social networks Asking Eye gaze Proposing Negotiating Facial expression Hand gesture Posture [A marketplace in Chiang Mai; courtesy of Sutasinee Thovuttikul]

18 Modalities of Interaction Conversation scene Verbal Pragmatics Discourse Semantics Syntax Morphology Word Phoneme Paralinguistic loudness, rate, pitch, pitch contour, Nonverbal Embodiment Behaviors Eye gaze and facial expressions Silence Touch Space Time Color Social signals [Nishida 1999 (J)]

19 Verbal expressions A: Here is a flower for you. B: Thank you so much! Where did you find it? Locutionary acts: action of making an utterance. The speaker makes sound + facial expressions +... Illocutionary acts: a social action performed by an utterance. The speaker asks for the place the flower was found. Perlocutionary acts: social effects brought about by the utterance. The speaker shows an interest to maintain friendship. [Austin 1962; Searle 1969]

20 Nonverbal expressions Symbol Illustrators this big Nop Regulators Well.. Emotional expressions Oh Oh Oh Deictic that Synchrony [Nishida 1999 (J)]

21 Common Ground Communal background including the cultural and biological aspects Common ground Imaginary scene shared by participants Physical configuration

22 Studies on conversational interactions (1) Narratives and content flow (2) Social discourse (3) Verbal communication (4) Nonverbal communication (5) Cognitive process

23 Approach 1: Making environment playful Daily living space Augmentation by MR (VR AR) Smart conversation space that encompasses participants and referents of conversation. -> Engaged conversational interactions -> More insights about the common ground

24 Projecting the real world into the virtual world [Ohmoto 2011]

25 The interactive dome Appearance Architecture Projection Inside view

26 Approach 2: Understanding by measurement Analyze the behaviors of participants by integrating audiovisual and physiological.measurement

27 Approach 3: Robotic Apprentice Human instructor Robotic assistant

28 Agenda Credits Will be awarded based on a report on subjects given at the class. Due date (January 31st, 2017) Agenda (planned) 1. Introduction (October 5) Nishida 2. Methodologies for Conversational System Development -1-(October 12) Nishida 3. Methodologies for Conversational System Development -2-(October 19) Nishida 4. Affective Computing (October 26) Nishida 5. Theory of Mind (November 2) Nishida 6. Smart Conversation Space (November 9) Ohmoto 7. Measurement, Analysis and Modeling (November 16) Ohmoto 8. Cognitive Design (November 30) Ohmoto 9. Learning by Imitation (December 7) Nishida 10. Aspects of Conversation -1- (December 14) Nishida 11. Aspects of Conversation -2- (December 21) Nishida 12. Turn-taking (December 28) Nishida 13. Storytelling, Games, and Conversation (January 11) Nishida 14. Synergy and Wrap up (January 18) Nishida Course materials available from: