Powering AI Robots with Deep Learning

Transcription

1 Powering AI Robots with Deep Learning NVIDIA Deep Learning Day 2017 COEX Intercontinental Hotel, Seoul, Korea, Tuesday, October 31, 2017 Byoung-Tak Zhang Seoul National University School of Computer Science and Engineering & Surromind Robotics, Inc.

2 Outline AI Robots Come of Age.. 3 Autonomous Mobile Robots Home Robots, Human-Like Robots Deep Learning for AI Robots. 12 Perception, Action, Cognition RoboCup@Home Challenge New AI. 30 2

3 1. AI Robots Come of Age

4 Humans (NI) and Machines (AI) Introspectionism Psyche Behaviorism Cybernetics Mind (= Computer) Cognitivism Symbolic AI Brain Connectionism Neural Nets (ML) Body Action Science Autonomous Robots 2010 Environment Embodied Mind Mind Machine ( = Smart Machine) 4

5 Early AI Robots (Autonomous Mobile Robots) Shakey (SRI) RHINO (U. Bonn) Cart (Stanford Univ.) CoBot (CMU) 5

6 RoboCup (1997~) 6

7 Home Robots PR2 Fetches Beer (Willow Garage) Dash at Hotel (Sevioke) PR2 Making Popcorns (TU Munich) SpotMini (Boston Dynamics) 7

8 Human-Like Robots Humanoid Robot Nao (Aldebaran) Life-Like Robots (Hanson Robotics) Emotion Robot Pepper (SoftBank) Atlas (Google Boston Dynamics) 8

9 Robot Life in a City =gpzc88hkgcu&t=80s Obelix (University of Freiburg, Germany) 9

10 AI Robots for the 4 th Industrial Revolution Cognitive Smart Machines Body (HW, Device) Mind (SW, Data) 10

11 Enabling Technologies for AI Robots Perception Object recognition Person tracking Control Manipulation Action control Navigation Obstacle avoidance Map building & localization Interaction Vision and voice Multimodal interaction Computing Power Cloud computing GPUs, parallel computing Neural processors 11

12 2. Deep Learning for AI Robots

13 Traditional Machine Learning vs. Deep Learning 13

14 Deep Learning Revolution Big Data + Parallel Computing + Deep Learning From programming to learning Automated- or self-programming Paradigm shift in S/W Self-improving systems Intelligence explosion

15 Power of Deep Learning Multiple boundaries are n eeded (e.g. XOR problem) Multiple Units More complex regions are needed (e.g. Polygons) Multiple Layers Big Data + Deep Learning => Automatic Programming

16 AI / Deep Learning Growth AlphaGo

17 Deep Learning for Voice and Dialogue Speech LSTM-RNN (Recurrent Neural Networks) End-to-End Memory Networks (N2N MemNet) CNN + RNN for Question Answering Sukhbaatar, Sainbayar, Jason Weston, and Rob Fergus. "End-to-end memory networks." Advances in Neural Information Processing Systems Gao, Haoyuan, et al. "Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question." Advances in Neural Information Processing Systems

18 Interaction: Conversational Interface Amazon Echo Google Home SKT Nugu 18

19 Deep Learning for Robotic Grasping (Levine et al, 2016) (C) , SNU Biointelligence Lab, 19

20 Deep Reinforcement Learning for Action Control BRETT (Univ. of California, Berkeley) 20

21 Deep Learning for Perception ImageNet Large-Scale Visual Recognition Challenge Image Classification/Localization 1.2M labeled images, 1000 classes Convolutional Neural Networks (CNNs) has been dominating the contest since non-cnn: 26.2% (top-5 error) 2012: (Hinton, AlexNet)15.3% (Using GPUs) 2013: (Clarifai) 11.2% 2014: (Google, GoogLeNet) 6.7% (pre-2015): (Google) 4.9% Beyond human-level performance 21

22 t e m p o r a l Deep Learning for Video Analysis Use 3D CNNs to model the temporal patterns as well as etwor k s for H um an A ct ion R ecognit ion the spatial patterns 3D C onvolut ional N eur al N etwor k s for H um an A ct ion R ecognit ion Figure 2. Extraction of multiple features from contiguous frames. M ultiple 3D convolutions can be applied to contiguous frames toextract multiple features. A s in Figure 1, the sets of connections are color-coded so that the shared weights are in the same color. Note that all the 6 sets of connections do not share weights, resulting in two different feature maps on the right. hardwired 7x7x3 3D convolution 2x2 subsampling 7x6x3 3D convolution A. Karpathy, L. Fei-Fei, et al., CVPR x3 subsampling 7x4 convolution full connnection tions. poral coded In 3D pping res. set of lower-level feature maps. Similar to the case ofinput: 2D convolution, this can be achieved by applying 7@60x40 multiple 3D convolutions with distinct kernels to the same location in the previous layer (Figure 2) A 3D CNN A r chit ect ur e H1: 33@60x40 C2: 23*2@54x34 S3: 23*2@27x17 C4: 13*6@21x12 S5: 13*6@7x4 C6: 128@1x1 Based on the3d convolution described above, a variety Figureof CNN architectures can be devised. In the following, S. Ji, 3. K. AYu, 3D CNN et al., architecture PAMI, 2013 for human action recognition. T his architecture consists of 1 hardwired layer, 3 convowedescribea 3D CNN architecturethat we have devel-

23 Deep Learning for Autonomous Driving (NVIDIA) 23

24 VQA Challenge (2016) Visual Question Answering (VQA) is a new dataset containing open-ended questions about images. These questions require an understanding of vision, language and common sense to answer. 254,721 images (MSCOCO and abstract scenes) 3 questions per image (764,163 total) 10 ground truth answers per question 3 plausible (but likely incorrect) answers per question Open-ended and multiple-choice answering tasks Winner (UC Berkeley & Sony) 66.9% accuracy on real-image openended QA. Naver, Samsung, SNU, Postech ( 현재 SNU 1 등중 )

25 Cambot (SNU) [Kim et al., NIPS-2016] 25

26 Deep Hypernets for Visual Dialogue Deep hypernetworks with hierarchical concept structure are used as knowledge base for Q&A Hierarchical formulation P(x) P(h h ) P(h h ) P(h x) P(x) h h n 1 n n Joint probability of hidden variables h (s) i in the s th layer exp( E( hs)) P( hs ) exp( E( h )) E( h ) h( s( h )) j j j j s( h ) w h w h h... w h h ( j) ( j) ( j) ( j) ( j) ( j) ( j) ( j) j i1 i1 i1i 2 i1 i2 i1i 2 i1 ik i1 i1, i2 i1, i2,... ik Learning is done by adjusting s(h j ) towards maximizing likelihood P(x W) w N ( n) ln P( x W ) N ( k ) xi x 1 i x 2 ik Data xi x 1 i x 2 ik P( x W ) i 1 1, i2,..., i n k [Kim et al., IJCAI-2017] Knowledge Base Construction Fc7 Feature of Convolutional Neural Networks Neural Word Embedding Using Word2vec Preprocessing Hide-and-seek MountainCloud Chair Swimming Play Pororo House DinnerCrong Soccer Hi What 1. K.-M. Kim, C.-J. Nan, J.-W. Ha, Y.-J. Heo, and B.-T. Zhang, Pororobot: A Deep Learning Robot That Plays Video Q&A Games, AAAI 2015 Fall Symposium on AI for Human-Robot Interaction (AI-HRI 2015), J.-W. Ha, K.-M. Kim, B.-T. Zhang, Automated Visual-Lingusitc Knowledge Construction via Concept Learning from Cartoon Videos, In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI 2015), B.-T. Zhang, J.-W. Ha, M. Kang, Sparse Population Code Models of Word Learning in Concept Drift, In Proceedings of Annual Meeting of the Cognitive Science Society (Cogsci), 2012.

27 Learning from Cartoon Videos Image 개수 : Word 개수 : Episode 개수 : [Ha et al., AAAI-2015]

28 Pororobot (SNU) [Ha et al., AAAI-2015] 28

29 AUPAIR: Autonomous Personal AI Robot AUPAIR (SNU & Surromind Robotics) Winning the

30 3. New AI

31 Human Intelligence and Artificial Intelligence

32 Dual Process Theories of Mind

33 New AI (System 1) and Old AI (System 2) New AI Old AI

34 Humans (NI) and Machines (AI) Introspectionism Psyche Behaviorism Cybernetics Mind (= Computer) Cognitivism Symbolic AI Brain Connectionism Neural Nets (ML) Body Action Science Autonomous Robots 2010 Environment Embodied Mind Mind Machine ( = Smart Machine) 34

35 Autonomous Machine Learning 1G: Supervised Learning (1980s~2000) 2G: Unsupervised Learning (2000~Present) 3G: Autonomous Learning (Next Generation) Decision Trees Kernel Methods Multilayer Perceptrons Deep Networks Markov Networks Bayesian Networks Complex Adaptive Systems Perception-Action Cycle Lifelong Continual Learning c SNU Biointelligence Laboratory, 35

36 Technology Parallel Computing Autonomous Sequential Reactive AI with Deep Learning Narrow AI 1980 Future of AI Superhuman AI (Embodied Brain-Like) Cognitive AI Follows given goals and methods Human-Level AI Works out own goals Agency Works out own methods, follows given goals 2030 Free Will Time 2050 Modified from Eliezer Yudkowsky & David Wood 36