What Is And How Will Machine Learning Change Our Lives. Fair Use Agreement

Size: px

Start display at page:

Download "What Is And How Will Machine Learning Change Our Lives. Fair Use Agreement"

Kenneth Gilbert
5 years ago
Views:

1 What Is And How Will Machine Learning Change Our Lives Raymond Ptucha, Rochester Institute of Technology 2018 Engineering Symposium April 24, 2018, 9:45am Ptucha 18 1 Fair Use Agreement This agreement covers the use of all slides in this document, please read carefully. You may freely use these slides, if: You send me an telling me the conference/venue/company name in advance, and which slides you wish to use. You receive a positive confirmation back from me. My name (Ptucha) appears on each slide you use. (c) Raymond Ptucha, rwpeec@rit.edu Ptucha

2 Machine Learning Machine learning is giving computers the ability to analyze, generalize, think/reason/behave like humans. Machine learning is transforming medical research, financial markets, international security, and generally making humans more efficient and improving quality of life. Inspired by the mammalian brain, deep learning is machine learning on steroids- bigger, faster, better! Ptucha 18 4 AI (Artificial Intelligence) technology is now poised to transform every industry, just as electricity did 100 years ago. Between now and 2030, it will create an estimated $13 trillion of GDP growth. Andrew Ng Chairman and CEO, Landing AI Ptucha

Interest in Machine Learning Growing Faster Over Time Interest over time for keywords machine learning, deep learning 100 75 50 25 Oct, 2012 April, 2014 Feb, 2016 https://trends.google.

3 Interest in Machine Learning Growing Faster Over Time Interest over time for keywords machine learning, deep learning Oct, 2012 April, 2014 Feb, Oct, 2017 Machine learning, cs229 is the most popular course at Stanford Their deep learning class, cs231 went from 150 to 350 to 750 in 2015/16/17 respectively Ptucha 18 6 We Live in a Flattened World Interest by by Region for for Machine Deep Learning Ptucha

4 The point of Singularity intelligence The point of singularity is when computers become smarter than humans. time Evolution of biology Advancement of technology Ptucha 18 8 Question 1 Do you think machines will ever be as intelligent as Machines??? Ptucha

Unleashing of Intelligence Machines will slowly match, then quickly surpass human capabilities.

Tomorrow it may be considered irresponsible for a human to relinquish control from a car that

5 Unleashing of Intelligence Machines will slowly match, then quickly surpass human capabilities. Today it is exciting/scary/fun to drive next to an autonomous car. Tomorrow it may be considered irresponsible for a human to relinquish control from a car that has faster reaction times, doesn t drink/text/get distracted/tired, and is communicating with surrounding vehicles and objects. Ptucha : The Year of AI: The Wall Street Journal, Forbes, and Fortune NEC Face Recognition SONY Playstation Virtual Reality Evolutionary Reinforcement Learning Ptucha

2017: The Year of AI: The Wall Street Journal,

Detection & Segmentation YOLO v2 Object Detection

6 2017: The Year of AI: The Wall Street Journal, Forbes, and Fortune DeepBach NVIDIA Autonomous Car Detection & Segmentation YOLO v2 Object Detection Ptucha Some Things to Look for in Ptucha

7 Some Things to Look for in 2018 Faceshift GDC Apple iphone X, Animoji Yourself Ptucha Some Things to Look for in 2018 NVIDIA Drive Ptucha

AI Jobs Already deciding who gets and how much credit for credit card companies. Clerical tasks can all be automated, reducing human errors. Insurance claims being assisted by AI agents.

8 AI Jobs Already deciding who gets and how much credit for credit card companies. Clerical tasks can all be automated, reducing human errors. Insurance claims being assisted by AI agents. AI lawyers can memorize every case ever presented and one day may recommend sentencing. Conversation bots may take over call centers. In law enforcement, money laundering, fraud, and cyber crimes will be detected by AI bots. In healthcare, AI assistants aiding doctors in making better diagnosis. Ptucha Ptucha

9 Question 2 Would you encourage someone to pursue a career in clerical task or a nursing field? Ptucha The Human Brain We ve learned more about the brain in the last 5 years than we have learned in the last 5000 years! It controls every aspect of our lives, but we still don t understand exactly how it works. Ptucha

10 The Brain on Pattern Recognition Airplane, Cat, Car, Dog STL-10 dataset /08/blindsight.html Ptucha The Brain on Pattern Recognition Despite Changes in Deformation: Ptucha

Size, Pose, Angle: Tardar Sauce Grumpy Cat

11 The Brain on Pattern Recognition Despite Changes in Occlusion: Ptucha The Brain on Pattern Recognition Despite Changes in Size, Pose, Angle: Tardar Sauce Grumpy Cat Ptucha

12 The Brain on Pattern Recognition Despite Changes in Background Clutter: Ptucha The Brain on Pattern Recognition Despite Changes in Class Variation Ptucha

13 Teaching Computers to See It took evolution 540M years to develop the marvel of the eye-brain. Lets say a child collects a new image every 200msec. By age 3, this child has processed over 250M images. 5 "#$%&'/'&) 60'&)/#", 60#",/h. 12h./1$2 3651$2'/2. 32.' = 2365 Today s computers can do this in a few days Ptucha Neural Nets on Pattern Recognition Instead of trying to code simple intuitions/rules on what makes an airplane, car, cat, and dog We feed neural networks a large number of training samples, and it will automatically learn the rules! Lets take a glimpse into the magic behind this! Ptucha

14 Artificial Neuron x 0 q 0 x 1 q 1 q x 2 2 q n g ( ) Note, x 0 is the bias unit, x 0 =1 h q (x)! =! #! $! %! ' ( = ( # ( $ ( % ( ' x n dendrites ' h *! = +! # ( # +! $ ( $ + +! ' ( ' = +.! / ( / h *! = + ( 1! Axons Activation function /0# Ptucha Artificial Neural Networks Artificial Neural Network (ANN) A network of interconnected artificial neurons that mimic the properties of a biological network of neurons. Input Hidden Output Ptucha

15 4-Layer ANN Fully Connected Topology Input Layer Hidden Layer 1 Hidden Layer 2 Output Layer D 1 D 2 D 3 D C A image would have 400 input nodes a nodes b nodes C nodes, where C is the number of classes Backpropagation (~1985) uses!" for learning!# Learning happens in the weights- each line is a weight. Ptucha Question 3 Do artificial neurons faithfully mimic real neurons in the brain? Ptucha

16 Two Most Important Deep Learning Fields Convolutional Neural Networks (CNN) Examine high dimensional input, learn features and classifier simultaneously Recurrent Neural Networks (RNN) Learn temporal signals, remember both short and long sequences Ptucha Two Most Important Deep Learning Fields Convolutional Neural Networks (CNN) Examine high dimensional input, learn features and classifier simultaneously Recurrent Neural Networks (RNN) Learn temporal signals, remember both short and long sequences Ptucha

Fully Connected Layers? 200 200 pixel image. 40K input fully connected to 40K hidden (or output) layer. 1.6 billion weights!

17 Fully Connected Layers? pixel image. 40K input fully connected to 40K hidden (or output) layer. 1.6 billion weights! Generally don t have enough training samples to learn that many weights. Ranzato CVPR 14 Ptucha Convolution Filter Ranzato CVPR 14 Convolution filters apply a transform to an image. The above filter detects vertical edges. Ptucha

Locally Connected Layer 200 200 pixel image. 40K input.

Ranzato CVPR 14 Ptucha 18 45 Can we formulate so each filter has similar statistics across all

18 Locally Connected Layer pixel image. 40K input. Four filters, each fully connected 40K =16M weights.getting better! Ranzato CVPR 14 Ptucha Locally Connected Layer pixel image. 40K input. Four filters, each fully connected 40K =16M weights.getting better! Can we formulate so each filter has similar statistics across all locations? Ranzato CVPR 14 Ptucha

Convolution Layer 200 200 pixel image. 40K input. Four 10 10 filters, each fully connected 40K 10 10 4=16M weights.getting better! Require each filter has same statistics across all locations.

19 Convolution Layer pixel image. 40K input. Four filters, each fully connected 40K =16M weights.getting better! Require each filter has same statistics across all locations. Learn filters. Ranzato CVPR 14 Ptucha Convolution Layer Ranzato CVPR pixel image. 40K input. Four filters, each fully connected 40K =16M weights.getting better! Require each filter has same statistics across all locations. Learn filters. To learn four filters we have =400 parameters- great! Ptucha

Many Flavors of CNNs LeNet-5, LeCun 1989 AlexNet, Krizhevsky 2012

He 2015 DenseNet, Huang 2017 Ptucha 18 49 Image Convolution

input image size 3 3 filter sliding over input image Vert pad

20 Many Flavors of CNNs LeNet-5, LeCun 1989 AlexNet, Krizhevsky 2012 VGGNet, Simonyan 2014 GoogLeNet (Inception), Szegedy 2014 ResNet, He 2015 DenseNet, Huang 2017 Ptucha Image Convolution output By padding (filterwidth-1)/2, output image size matches input image size 3 3 filter sliding over input image Vert pad input Horiz pad Ptucha

21 Max Pooling- Reducing the Size of an Image cs321n, Karpathy, Li Ptucha Convolution Neural Network (CNN) Building Block Pooling Convolution Image Deng ICML 14 Ptucha

22 Putting it All Together Convolution Pooling Whole System Ptucha Learning Filters 32 Learned Filters, each Filtered images, each is Input image Use zero padding Ptucha

Learning Filters 32 Learned Filters, each 5 5 3 32 Filtered images, each is 28 28 1 Input image 28 28 3 Use zero

23 Learning Filters 32 Learned Filters, each Filtered images, each is Input image Use zero padding Ptucha Question 4: What are the two key building blocks in a convolutional neural network? Ptucha

24 CNN Visualization Zeiler, Fergus, 2014 Ptucha CNN Visualization Zeiler, Fergus, 2014 Ptucha

CNN as Vector Representation Typical CNN Architecture Input Image 2D Plot of fc8 Feature

out, these fully connected layers are excellent descriptors of the input image!

25 CNN as Vector Representation Typical CNN Architecture Input Image 2D Plot of fc8 Feature Vector Image of fc8 Feature Vector Ptucha CNN as Vector Representation As it turns out, these fully connected layers are excellent descriptors of the input image! For example, you can pass images through a pre-trained CNN, then take the output from a FC layer as input to a SVM classifier. (image2vec) Images in this vector space generally have the property that similar images are close in this latent representation. Ptucha

Vision Tasks Classification Classification + Localization Object

label Evaluation metric: Accuracy CAT Classification + Localization

26 Vision Tasks Classification Classification + Localization Object Detection Instance Segmentation Single Object Multiple Objects Ptucha Classification vs. Classification + Localization Classification Input: Image Output: Class label Evaluation metric: Accuracy CAT Classification + Localization Input: Image Output: Class label, Box coordinates Evaluation metric: Intersection over Union (IoU) (CAT,x,y,w,h) Ptucha

.. pt68x pt68y 137 outputs Of course, need GT for thousands of faces

27 Facial feature points Localization Each face has 68 points, so CNN would output: Face? pt1x pt1y pt2x pt2y... pt68x pt68y 137 outputs Of course, need GT for thousands of faces to train model. ptucha Ptucha FAIR Mask R-CNN, COCO + Places Workshop, ICCV 2017 Ptucha

28 FAIR Mask R-CNN, COCO + Places Workshop, ICCV 2017 Ptucha FAIR Mask R-CNN, COCO + Places Workshop, ICCV 2017 Ptucha

29 FAIR Mask R-CNN, COCO + Places Workshop, ICCV 2017 Ptucha ImageNet Amazon Turk did bulk of labeling 14M labeled images 20K classes Russakovsky et al., M images, 1000 categories Image classification, object localization, video detection Ptucha

ImageNet: Examples of Hammer Ptucha 18 81 Deep Learning- Surpassing The Visual Cortex s Object Detection and Recognition Capability Traditional Computer Vision and Machine Learning Top-5 error on

30 ImageNet: Examples of Hammer Ptucha Deep Learning- Surpassing The Visual Cortex s Object Detection and Recognition Capability Traditional Computer Vision and Machine Learning Top-5 error on ImageNet Deep Convolution Neural Networks (CNNs) Error Introduction of deep learning AlexNet ZFNet 6.7 Trained Human (genius intellect) GoogleLeNet Human (Karpathy) ResNet Similar effect demonstrated on voice and pattern recognition CUImage 2018 moved to Kaggle SENet Year Ptucha

31 AI vs. IA Artificial Intelligence (AI) is the subject of developing machines that can think, act, or reason like humans. Intelligent Augmentation (IA) is the subject of enhancing human abilities- making us faster, smarter, more efficient. Note- according to CB insights 1, AI startup funding in 2016 was $4.2B, up 8 from just four years ago. Central to this is the exponential growth in deep learning the past few years. 1 Ptucha IA vs AI Examples Intelligent Augmentation Enterprise Automate mundane tasks Virtual assistants AR glasses for workers, doctors, repair Autonomous Vehicles Robots/Industrial IoT Highway assist Campus/controlled driving Humans take over for exceptions Collaborative robots work with humans for hard, unsafe, or repetitive Efficient and safer factories Drones Intelligent, but with human overseeing drones for inspections (cell towers, remote locations) Artificial Intelligence Machines perform tasks better and faster than humans All vehicles autonomous Illegal to drive own car Fully automated factories All humans replaced by robots on factory floor Drones don t require human supervision Ptucha

32 An engineer over estimates what she can do in 5 years and under estimates what she can do in 20 years The impact of IA (human-enhancing automation using deep learning and other machine learning techniques) would be bigger in the medium term than most think, while full automation is further away than some recent reporting might indicate Ptucha Question 5 Do we know what technologies will help us make the leap from intelligent augmentation to artificial intelligence? Ptucha

Andrew Ng, 2017 https://www.deeplearning.

33 Andrew Ng, Deep Learning Specialization, Five courses: 1. Neural Networks and Deep Learning 2. Improving Deep Neural Networks 3. Structured Machine Learning Projects 4. Convolutional Neural Networks 5. Sequence Models Ptucha Li, Johnson, Yeung l Ptucha

34 Thank you!! Ray Ptucha Ptucha

Lecture 11-1 CNN introduction. Sung Kim

Lecture 11-1 CNN introduction. Sung Kim Lecture 11-1 CNN introduction Sung Kim 'The only limit is your imagination' http://itchyi.squarespace.com/thelatest/2012/5/17/the-only-limit-is-your-imagination.html Lecture 7: Convolutional