1. Introduction. Toyoaki Nishida Kyoto University

Transcription

1 Artificial Intelligence Adv., April 9, Introduction Toyoaki Nishida Kyoto University Copyright 2014, Toyoaki Nishida, Atsushi Nakazawa, Yoshimasa Ohmoto, Yasser Mohammad, At,Inc. All Rights Reserved.

2 Prologue Conversational Informatics: a field of study that centers on understanding and augmenting conversation Super Intelligence People Communicative Intelligence

3 Long-term goal Challenge: A robot that can participate in conversation

4 Conversation is a complex business Trust Conviviality Social networks Asking Eye gaze Proposing Negotiating Facial expression Hand gesture Posture

5 Long-term goal: primordial soup of conversation [Nishida-Nakazawa-Ohmoto-Mohammad 2014]

6 Understanding and Designing Interaction Application Conversational interactions Content production Model building Measuremen t Platform Evaluation Analysis Theory [Nishida Nakazawa Ohmoto Mohammad 2014]

7 What is AI? Artificial intelligence Artificial mind Example Enthiran (The Robot), 2010

8 AI in the movies Year Title Who Wikipedia : A Space Odyssey HAL Star Wars C 3PO, R2 D Blade Runner Replicants 1984 The Terminator The terminator E3%83%BC%E3%82%BA%E3%83%BB%E3%82%B7%E3%83%AA%E3%83%BC%E3%82%BA E3%83%8A%E3%83%BC E3%83%BC_(%E6%98%A0%E7%94%BB) 1987 RoboCop RoboCop (cyborg) War Games WOPR: War Operation Plan Response E3%83%A0_(%E6%98%A0%E7%94%BB) 1994 Disclosure Angel E3%82%B8%E3%83%A3%E3%83%BC_(%E6%98%A0%E7%94%BB) 1998 Bicentennial Man Andrew, a new NDR 114 robot DR The Matrix Computer 2001 A.I. Artificial Intelligence David, achild Mecha Minority Report Insect robots (user interface is interesting, too) %E6%98%A0%E7%94%BB) E3%83%BB%E3%83%AA%E3%83%9D%E3%83%BC%E3%83% I, ROBOT V.I.K.I., Sonny ATOM Astro boy Astro boy Avatar (tele existence) Surrogate (tele existence) 2014 Transcendence (Technical singularity) %E7%94%BB) E3%83%B3%E3%82%B9 [Nishida 2012b]

9 Year AI ICT 1940~ 1950~ 1960~ 1970~ 1980~ 1990~ 2000~ 2010~ : Checker program by A.Samuel 1956: Dartmouth Conference 1957: FORTRAN by J.Backus 1936: Turing Machine, 1947: von Neumann Computer, 1948: Information Theory, by C. Shannon and W. Weaver, 1948: Cybernetics by Wiener 1961: Symbolic Integration program SAINT by J.Slagle 1960 Prof. Toshiyuki 1961: Mathematical Sakai s group theory at of Kyoto Packet Networks by L. Kleinrock 1962: Perceptron by F.Rosenblatt 1961 Sonotype: 1963: phonetic Interactive typewriter Computer by Graphics Shuji Doshita by I.Sutherland 1966: The ALPAC report against Machine Translation by R. Pierce 1967: Formula Manipulation System Macsyma by J.Moses 1960 s Image 1968: Understanding Mouse Bitmap and NLP display by for Makoto on Line Nagao System (NLS) by D.C.Engelbart 1967: Dendral for Mass Spectrum Analysis by E.Feigenbaum 1969: ARPA net 1971: Natural Language Dialogue System SHRDLU, by T.Winograd 1970 The world s 1970: first ALOHAnet face recognition system by Takeo Kanade 1973: Combinatorial Explosion problem pointed out in The Lighthill report 1970: Relational Database Theory by E.F.Codd 1974: MYCIN by T.Shortliffe 1970 s Research 1972: groups Theory of for NP completeness AI were formed by in S.Cook Tokyo, and Osaka, R.Karp Kyushu Mid 1970 s: Prial Sketch and Visual Perceptron by D.Marr Mid 1970 s: Alto Machine by A.Kay and A.Goldberg 1976: Automated Mathematician (AM) by D.Lenat 1979 IJCAI : (Tokyo) Ethernet 1979: Autonomous Vehicle Stanford Cart by H.Moravec 1979: Spreadsheet Program Visicalc by D.Bricklin 1979 Japanese Word Processor JW-10 by Toshiba 1982: Fifth Generation Computer Project 1982:TCP/IP Protocol by B.Kahn and V.Cerf 1984: The CYC Project by D.Lenat Mid 1980 s: First Wireless Tag Products Mid 1980 s: Back propagation algorithm was widely used 1987: UUNET started the Commercial UUCP Network Connection Service 1985: the Cybernetic Artist Aaron by H.Cohen 1988: Internet worm (Morris Worm) 1986: Subsumption Architecture by R.Brooks 1989: World Wide Web by T.Berners Lee 1989: An Autonomous Vehicle ALVINN by D.Pomerleau Fifth 1989: Generation The number Project of hosts on the Internet has exceeded 100, JSAI (Japanese Society for AI) 1990: Genetic Programming by J.R.Koza 1992: The number of hosts on the Internet has exceeded 1,000,000. Early1990 s: TD Gammon by G.Tesauro 1994: Shopping malls on the Internet Mid 1990 s: Data Mining Technology st PRICAI (Nagoya, Japan) 1994: W3C was founded by T. Berners Lee 1997: DeepBlue defeated the World Chess Champion G.Kasparov 1997: Google Search 1997 IJCAI 97 (Nagoya) 1997: The First Robocup by H.Kitano 1998: XML1.0(eXtensible Markup Language) by W3C 1999: Robot pets became commercially available st Robocup 1998: games PayPal and conference at Nagoya, Japan 2000: Honda Asimo 2004: The Mars Exploration Rovers (Spirit & Opportunity) 2010: Google Driverless Car / Kinect 2011: IBM Watson Jeopardy defeated two of the greatest champions 2012: Siri History of AI research in contrast with ICT 2001: Wikipedia. 2003: Skype / itunes store 2004: Facebook 2005: YouTube / Google Earth 2006: Twitter 2007: Google Street View

10 Successful Topics of AI Large-scale Search Knowledge-based Systems Language, Speech, Vision Planning Machine Learning and Data Mining Using AI in Creating Works of Art DeepBlue (1997) IBM Watson (2011) The Mars Exploration Rovers (2003) ALVINN (An Autonomous Land Vehicle in a Neural Network) on Navlab (1989) Experiments in Musical Intelligence (1987) AARON (1985)

11 Recent Trends Landmarks 1997: Deep Blue defeated G. Kasparov Source:IBM Deep Blue, wikipedia 1997: Official opening of Robocup (H. Kitano) Source: robocup.org, robocup.or.jp, Wikipedia 1997: Mars Pathfinder Source:NASA Mars Pathfinfer, Wikipedia 1999: Robot pet SONY AIBO Source: SONY AIBO, Wikipedia 1999: OpenCV Project Source:OpenCVWiki,wikipedia 2000: Honda ASIMO Source: Honda ASIMO, Wikipedia 2004: Mars Exploration Rovers Source:NASA JPL 2010: Google Driverless Car Source:Wikipedia, Sebastian Thrun s home page, 3P 2010: Kinect Source:xbox.com, Wikipedia 2011: IBM Watson defeated two Jeopardy! champions Source:IBM Watson, Wikipedia 2011: iphone Siri Source:Apple, Wikipedia 2011: Google Voice Search Source: Google 2012: Zen Takemiya Masaki with 4 stones Source: Sig ECS, UEC 2012: Google Glass Project Source: Project Glass 2012: NTT DOCOMO s Shabette Concier NTT DOCOMO Other: Smarter than You Think (New York Times) From philosophy to science technology Toy problems to the real world Powerful tools Grand challenges and competitions

12 Traditional AI [Nishida-Nakazawa-Ohmoto-Mohammad 2014]

13 Dark side of super intelligence Technology abuse Responsibility flaw Moral in crisis Over dependency on technology

14 (Technological) singularity The day when machine intelligence surpasses human intelligence. IEEE Spectrum June 2008 issue of the singularity The AI Scenario: We create superhuman artificial intelligence (AI) in computers. The IA Scenario: We enhance human intelligence through human to computer interfaces that is, we achieve intelligence amplification (IA). The fear of utopia

15 So what? Dismiss as nonsense. Neo Luddism. Invent a better solution. Endow AI with empathic capabilities.

16 Communicative Intelligence for Bridging People and CI Super Intelligence People Communicative Intelligence [Nishida Nakazawa Ohmoto Mohammad 2014]

17 Towards Empathic Agents [Nishida Nakazawa Ohmoto Mohammad 2014]

18 Traditional AI: High competence Future AI: High empathy DeepBlue Entertain with a game IBM Watson Entertain with a game Siri Conversation partner (AI that can pass entrance exam) Effective and affective tutor

19 Empathy The ability to understand others emotions and/or perspectives and, often, to resonate with others emotional states. or An affective response that is identical, or very similar, to what the other person is feeling or might be expected to feel given the context: a response stemming from an understanding of another s emotional state or condition. [Eisenberg 2010]

20 Sharing hypothesis The more common ground is shared, the more empathy will be gained.... the universe of discourse, first person view, knowledge and skills, the communication style and rituals, the value system,... [Nishida 2013c]

21 A road to empathic agents Empathy Sharing hypothesis Primordial soup of conversation Common ground Conversational intelligence Engagement [Nishida Nakazawa Ohmoto Mohammad 2014]

22 Research platform (a) IMADE A real-world interaction measurement, analysis and design environment [Sumi 2010a; Nishida-Nakazawa-Ohmoto-Mohammad 2014]

23 Research platform (b) ICIE immersive collaborative interaction environment [Nishida-Nakazawa-Ohmoto-Mohammad 2014]

24 Synthesized Real World Google Street View Navigation on Immersive 360 Display [1] Youtube, KyotoUniNishidaLab 9SKpcMrzk [1] C. Nitschke, 2013.

25 Synthesized Real World Remote Meeting and Interaction in Immersive Shared Environment [1] Youtube, KyotoUniNishidaLab [1] C. Nitschke, D. Lala, 2013.

26 Virtual Basketball Virtual Basketball [1] Youtube, KyotoUniNishidaLab [1] D. Lala, Y.F.O. Mohammad, T. Nishida. Unsupervised gesture recognition system for learning manipulative actions in virtual basketball. Proc. ICHAI (2013).

27 Immersive WOZ environment [Nishida-Nakazawa-Ohmoto-Mohammad 2014]

28 Immersive WOZ environment [Nishida-Nakazawa-Ohmoto-Mohammad 2014]

29 Learning by imitation [Mohammad 2009]

30 Learning by imitation [Mohammad 2009]

31 Learning by imitation Constrained Motif Discovery: Given a time series X(t) find recurring patterns of length between L 1 and L 2 using distance function D subject to the constraint P(t), where P(t) is an estimation of the probability that a motif occurrence exists near time step t. likely P(t) Change point discovery unlikely [Mohammad 2009]

32 Future Change angle G H Past Future t ;...; 1 H t seq t n seq t 1 ;...; G t seq t seq t n ˆ f f T i i i l i i i l i i s t t t cs x c t T t V t S t U t H ) ( ) ( ) ( ) ( Find optimal l P g g T u u t G t G ) ( ) ( Find optimal l F 1 1 and, ) ( j j j F g i i l i u t f T l l T l l i l i t U U t U U t, ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ˆ( ) ( ~ t t t t t x t x P F P F Learning by imitation Robust Singular Spectrum Transform [Mohammad 2009]

33 3D conversation capture [Yano 2012]

34 Collaborative annotation system [Nishida-Nakazawa-Ohmoto-Mohammad 2014]

35 Corneal Imaging The cornea of the human eye acts like a mirror that partially reflects incident light from a person s environment over a large field of view (>200 degree), which can be well noticed when looking at a person s eye. These corneal reflections can be extracted from an image of the eye by modeling the eye camera geometry as a catadioptric (mirror + lens) imaging system. Corneal reflections. (a) The cornea. (b) The corneal limbus is the surface shape discontinuity, where the cornea dissolves into the sclera. (c) A reflected office environment. (d) Focus on iris texture instead of corneal reflections. (e),(f) Examples of corneal images and corresponding scene images. [Nakawaza-Nitschke 2013]

36 Corneal Imaging (c) First prototype of a corneal imaging camera. [Nakawaza-Nitschke 2013]

37 Experimental settings for evaluating the facilitative agent [Nishida-Nakazawa-Ohmoto-Mohammad 2014]

38 Conversation quantization as conceptual foundation B: I know. That is a volcano. A: That s Diamond Head. C: I went there with my family in [Nishida Nakazawa Ohmoto Mohammad 2014]

39 Conversation quantization as conceptual foundation Discourse Preceding / Succeeding / Related Interaction A: <pointing M>That s Diamond Head B: <nod> I know. That s is a volcano. C: <gaze at M>I wend there with my family in 1985 Ground A: participant M: mountain B: participant C: participant (a) Conversation scene (b) Conversation quantum (c) (Hypothetical) observer [Nishida Nakazawa Ohmoto Mohammad 2014]

40 Producing a conversation quantum Schemata dictionary Schemata based recognition Segmentation and transcription Discourse Preceding / Succeeding / Related Interaction A: <pointing M>That s Diamond Head B: <nod> I know. That s is a volcano. C: <gaze at M>I wend there with my family in 1985 Ground A: participant M: mountain A: That s Diamond Head. B: I know. That is a volcano. C: I went there with my family in B: participant C: participant Conversation quantum Schema [Nishida Nakazawa Ohmoto Mohammad 2014]

41 Consuming a conversation quantum Schemata dictionary Schemata based recognition Discourse Preceding / Succeeding / Related Interaction A: <pointing M>That s Diamond Head B: <nod> I know. That s is a volcano. C: <gaze at M>I wend there with my family in 1985 Ground P: That s Diamond Head. M: mountain A: participant Schema B: participant C: participant Conversation quantum Q: I know. That is a volcano. P: That s Diamond Head. Dialogue manager Agent controller [Nishida Nakazawa Ohmoto Mohammad 2014]

42 Summary 1. This course centers on conversations. 2. Why conversations? > Foundation of thought and communication 3. Why do people converse with each other? > As a part of social interaction, for creating and maintaining stories, for fun. 4. The complexity of conversations: > Multi level, multi layered, polysemy, polymorphism, coordination of multi modality. 5. People are not only proficient in expressing ideas but also skillful in interpreting utterances, thereby they learn from each other. 6. Building empathic agents helps people effectively communicate with each other by conversations. 7. Conversational informatics addresses science and engineering of communicative intelligence. 8. Conversational intelligence can be counted as an important branch of artificial intelligence. 9. We will draw on a data intensive approach. 10. Conversation quantization as a foundation of data intensive approach to conversational informatics.

43 Agenda Credits: Will be awarded based on one or more reports on subjects given at the class. Calendar (tentative) 1. Introduction (April 9th) 2. History of Conversational System Development (April 16th) 3. Methodologies for Conversational System Development (April 23rd) 4. Smart Conversation Space (April 30th) 5. Measuring, Analysis and Modeling (May 7th) 6. From Observation to Interaction (May 14th) 7. Application of Simulation and Imitation for Interaction Learning (May 21st) 8. Cognitive Design for Discussion Support (May 28th) 9. Computer Vision Techniques for Conversational Interaction 1 (June 4th) 10. Computer Vision Techniques for Conversational Interaction 2 (June 11th) 11. Affective Computing (June 25th) 12. Language Use 1(July 2nd) 13. Language Use 2 (July 9th) 14. Speaking Turn Taking System (July 16th)

44 References [Eisenberg 2010] Eisenberg, N., Eggum, N., & Di Giunta, L. Empathy-related responding: Associations with prosocial behavior, aggression, and intergroup relations. Social Issues and Policy Review, 4(1), , [Nishida 2012a] Toyoaki Nishida: The Best of AI in Japan - Prologue. AI Magazine 33(2): , [Nishida 2012b] Toyoaki Nishida: Artificial intelligence research in the second half century, Journal of Information Processing and Management 55(7): , 2012 (in Japanese) [Nishida 2013a] Toyoaki Nishida: What s AI, Journal of Japanese Society for Artificial Intelligence 28(2): , 2013 (in Japanese) [Nishida 2013b] Toyoaki Nishida: Conversation Quantization as a Foundation of Conversational Intelligence. DNIS 2013: , [Nishida 2013c] Nishida, T. Toward mutual dependency between empathy and technology. AI SOC, 28(3), , [Nishida-Nakazawa-Ohmoto-Mohammad 2014] Toyoaki Nishida, Atsushi Nakazawa, Yoshimasa Ohmoto, Yasser Mohammad, Conversational Informatics A Data-Intensive Approach with Emphasis on Nonverbal Communication --, Springer, in press. [Nitschke 2013] Christian Nitschke, Atsushi Nakazawa and Toyoaki Nishida, " I see what you see: Point of Gaze Estimation from Corneal Images", Asian Conference on Computer Vision (ACPR2013), [Sumi 2010a] Sumi, Y., Yano, M., & Nishida, T. Analysis environment of conversational structure with nonverbal multimodal data. In International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction (pp. 44:1 44:4). New York, NY, USA: ACM, [Mohammad 2009] Yasser Mohammad, Toyoaki Nishida, Shogo Okada. Unsupervised Simultaneous Learning of Gestures, Actions and their Associations for Human-Robot Interaction, in: Proc. IROS 2009: The 2009 IEEE/RSJ International Conference on Intelligent RObots and Systems, pp , [Yano 2012] Yano M. Construction of 3-dimensional recording environments for multi-party conversation with RGB-depth sensors. Master thesis, Department of Intelligence Science and Technology, Kyoto University (in Japanese)