What Is And How Will Machine Learning Change Our Lives Raymond Ptucha, Rochester Institute of Technology 2018 Engineering Symposium April 24, 2018, 9:45am Ptucha 18 1 Fair Use Agreement This agreement covers the use of all slides in this document, please read carefully. You may freely use these slides, if: You send me an email telling me the conference/venue/company name in advance, and which slides you wish to use. You receive a positive confirmation email back from me. My name (Ptucha) appears on each slide you use. (c) Raymond Ptucha, rwpeec@rit.edu Ptucha 18 2 1
Machine Learning Machine learning is giving computers the ability to analyze, generalize, think/reason/behave like humans. Machine learning is transforming medical research, financial markets, international security, and generally making humans more efficient and improving quality of life. Inspired by the mammalian brain, deep learning is machine learning on steroids- bigger, faster, better! Ptucha 18 4 AI (Artificial Intelligence) technology is now poised to transform every industry, just as electricity did 100 years ago. Between now and 2030, it will create an estimated $13 trillion of GDP growth. Andrew Ng Chairman and CEO, Landing AI https://landing.ai/ai-transformation-playbook/?utm_source=mlylist&utm_medium=buttonlink&utm_campaign=playbook Ptucha 18 5 2
Interest in Machine Learning Growing Faster Over Time Interest over time for keywords machine learning, deep learning 100 75 50 25 Oct, 2012 April, 2014 Feb, 2016 https://trends.google.com/trends/explore?q=machine%20learning Oct, 2017 Machine learning, cs229 is the most popular course at Stanford Their deep learning class, cs231 went from 150 to 350 to 750 in 2015/16/17 respectively Ptucha 18 6 We Live in a Flattened World Interest by by Region for for Machine Deep Learning https://trends.google.com/trends/explore?date=today%205-y&q=machine%20learning,deep%20learning Ptucha 18 7 3
The point of Singularity intelligence The point of singularity is when computers become smarter than humans. time Evolution of biology Advancement of technology Ptucha 18 8 Question 1 Do you think machines will ever be as intelligent as Machines??? Ptucha 18 10 4
Unleashing of Intelligence Machines will slowly match, then quickly surpass human capabilities. Today it is exciting/scary/fun to drive next to an autonomous car. Tomorrow it may be considered irresponsible for a human to relinquish control from a car that has faster reaction times, doesn t drink/text/get distracted/tired, and is communicating with surrounding vehicles and objects. Ptucha 18 11 2017: The Year of AI: The Wall Street Journal, Forbes, and Fortune NEC Face Recognition SONY Playstation Virtual Reality Evolutionary Reinforcement Learning Ptucha 18 13 5
2017: The Year of AI: The Wall Street Journal, Forbes, and Fortune DeepBach NVIDIA Autonomous Car Detection & Segmentation YOLO v2 Object Detection Ptucha 18 14 Some Things to Look for in 2018 http://research.nvidia.com/sites/default/files/pubs/2017-10_progressive-growing-of/karras2017gan-paper.pdf Ptucha 18 15 6
Some Things to Look for in 2018 Faceshift GDC Apple iphone X, Animoji Yourself Ptucha 18 16 Some Things to Look for in 2018 NVIDIA Drive Ptucha 18 17 7
AI Jobs Already deciding who gets and how much credit for credit card companies. Clerical tasks can all be automated, reducing human errors. Insurance claims being assisted by AI agents. AI lawyers can memorize every case ever presented and one day may recommend sentencing. Conversation bots may take over call centers. In law enforcement, money laundering, fraud, and cyber crimes will be detected by AI bots. In healthcare, AI assistants aiding doctors in making better diagnosis. Ptucha 18 18 http://www.marketwatch.com/story/this-chart-spells-out-in-black-and-white-just-how-many-jobs-will-be-lost-to-robots-2017-05-31 Ptucha 18 19 8
Question 2 Would you encourage someone to pursue a career in clerical task or a nursing field? Ptucha 18 20 The Human Brain We ve learned more about the brain in the last 5 years than we have learned in the last 5000 years! It controls every aspect of our lives, but we still don t understand exactly how it works. Ptucha 18 21 9
The Brain on Pattern Recognition Airplane, Cat, Car, Dog STL-10 dataset http://thebraingeek.blogspot.com/ 2012/08/blindsight.html Ptucha 18 22 The Brain on Pattern Recognition Despite Changes in Deformation: Ptucha 18 23 10
The Brain on Pattern Recognition Despite Changes in Occlusion: Ptucha 18 24 The Brain on Pattern Recognition Despite Changes in Size, Pose, Angle: Tardar Sauce Grumpy Cat Ptucha 18 25 11
The Brain on Pattern Recognition Despite Changes in Background Clutter: Ptucha 18 26 The Brain on Pattern Recognition Despite Changes in Class Variation Ptucha 18 27 12
Teaching Computers to See It took evolution 540M years to develop the marvel of the eye-brain. Lets say a child collects a new image every 200msec. By age 3, this child has processed over 250M images. 5 "#$%&'/'&) 60'&)/#", 60#",/h. 12h./1$2 3651$2'/2. 32.' = 2365 Today s computers can do this in a few days Ptucha 18 28 Neural Nets on Pattern Recognition Instead of trying to code simple intuitions/rules on what makes an airplane, car, cat, and dog We feed neural networks a large number of training samples, and it will automatically learn the rules! Lets take a glimpse into the magic behind this! Ptucha 18 29 13
Artificial Neuron x 0 q 0 x 1 q 1 q x 2 2 q n g ( ) Note, x 0 is the bias unit, x 0 =1 h q (x)! =! #! $! %! ' ( = ( # ( $ ( % ( ' x n dendrites ' h *! = +! # ( # +! $ ( $ + +! ' ( ' = +.! / ( / h *! = + ( 1! Axons Activation function /0# Ptucha 18 30 Artificial Neural Networks Artificial Neural Network (ANN) A network of interconnected artificial neurons that mimic the properties of a biological network of neurons. Input Hidden Output Ptucha 18 31 14
4-Layer ANN Fully Connected Topology Input Layer Hidden Layer 1 Hidden Layer 2 Output Layer D 1 D 2 D 3 D C A 20 20 image would have 400 input nodes a nodes b nodes C nodes, where C is the number of classes Backpropagation (~1985) uses!" for learning!# Learning happens in the weights- each line is a weight. Ptucha 18 32 Question 3 Do artificial neurons faithfully mimic real neurons in the brain? Ptucha 18 33 15
Two Most Important Deep Learning Fields Convolutional Neural Networks (CNN) Examine high dimensional input, learn features and classifier simultaneously Recurrent Neural Networks (RNN) Learn temporal signals, remember both short and long sequences Ptucha 18 41 Two Most Important Deep Learning Fields Convolutional Neural Networks (CNN) Examine high dimensional input, learn features and classifier simultaneously Recurrent Neural Networks (RNN) Learn temporal signals, remember both short and long sequences Ptucha 18 42 16
Fully Connected Layers? 200 200 pixel image. 40K input fully connected to 40K hidden (or output) layer. 1.6 billion weights! Generally don t have enough training samples to learn that many weights. Ranzato CVPR 14 Ptucha 18 43 Convolution Filter Ranzato CVPR 14 Convolution filters apply a transform to an image. The above filter detects vertical edges. Ptucha 18 44 17
Locally Connected Layer 200 200 pixel image. 40K input. Four 10 10 filters, each fully connected 40K 10 10 4=16M weights.getting better! Ranzato CVPR 14 Ptucha 18 45 Locally Connected Layer 200 200 pixel image. 40K input. Four 10 10 filters, each fully connected 40K 10 10 4=16M weights.getting better! Can we formulate so each filter has similar statistics across all locations? Ranzato CVPR 14 Ptucha 18 46 18
Convolution Layer 200 200 pixel image. 40K input. Four 10 10 filters, each fully connected 40K 10 10 4=16M weights.getting better! Require each filter has same statistics across all locations. Learn filters. Ranzato CVPR 14 Ptucha 18 47 Convolution Layer Ranzato CVPR 14 200 200 pixel image. 40K input. Four 10 10 filters, each fully connected 40K 10 10 4=16M weights.getting better! Require each filter has same statistics across all locations. Learn filters. To learn four filters we have 4 10 10=400 parameters- great! Ptucha 18 48 19
Many Flavors of CNNs LeNet-5, LeCun 1989 AlexNet, Krizhevsky 2012 VGGNet, Simonyan 2014 GoogLeNet (Inception), Szegedy 2014 ResNet, He 2015 DenseNet, Huang 2017 Ptucha 18 49 Image Convolution output By padding (filterwidth-1)/2, output image size matches input image size 3 3 filter sliding over input image Vert pad input Horiz pad https://github.com/vdumoulin/conv_arithmetic Ptucha 18 50 20
Max Pooling- Reducing the Size of an Image cs321n, Karpathy, Li Ptucha 18 51 Convolution Neural Network (CNN) Building Block Pooling Convolution Image Deng ICML 14 Ptucha 18 52 21
Putting it All Together Convolution Pooling Whole System Ptucha 18 53 Learning Filters 32 Learned Filters, each 5 5 32 Filtered images, each is 28 28 Input image 28 28 Use zero padding Ptucha 18 54 22
Learning Filters 32 Learned Filters, each 5 5 3 32 Filtered images, each is 28 28 1 Input image 28 28 3 Use zero padding Ptucha 18 56 Question 4: What are the two key building blocks in a convolutional neural network? Ptucha 18 57 23
CNN Visualization Zeiler, Fergus, 2014 Ptucha 18 61 CNN Visualization Zeiler, Fergus, 2014 Ptucha 18 62 24
CNN as Vector Representation Typical CNN Architecture Input Image 2D Plot of fc8 Feature Vector Image of fc8 Feature Vector Ptucha 18 63 CNN as Vector Representation As it turns out, these fully connected layers are excellent descriptors of the input image! For example, you can pass images through a pre-trained CNN, then take the output from a FC layer as input to a SVM classifier. (image2vec) Images in this vector space generally have the property that similar images are close in this latent representation. Ptucha 18 64 25
Vision Tasks Classification Classification + Localization Object Detection Instance Segmentation Single Object Multiple Objects Ptucha 18 65 Classification vs. Classification + Localization Classification Input: Image Output: Class label Evaluation metric: Accuracy CAT Classification + Localization Input: Image Output: Class label, Box coordinates Evaluation metric: Intersection over Union (IoU) (CAT,x,y,w,h) Ptucha 18 66 26
Facial feature points Localization Each face has 68 points, so CNN would output: Face? pt1x pt1y pt2x pt2y... pt68x pt68y 137 outputs Of course, need GT for thousands of faces to train model. ptucha Ptucha 18 73 FAIR Mask R-CNN, COCO + Places Workshop, ICCV 2017 Ptucha 18 75 27
FAIR Mask R-CNN, COCO + Places Workshop, ICCV 2017 Ptucha 18 76 FAIR Mask R-CNN, COCO + Places Workshop, ICCV 2017 Ptucha 18 77 28
FAIR Mask R-CNN, COCO + Places Workshop, ICCV 2017 Ptucha 18 78 ImageNet Amazon Turk did bulk of labeling 14M labeled images 20K classes Russakovsky et al., 2015 1.2M images, 1000 categories Image classification, object localization, video detection Ptucha 18 80 29
ImageNet: Examples of Hammer Ptucha 18 81 Deep Learning- Surpassing The Visual Cortex s Object Detection and Recognition Capability Traditional Computer Vision and Machine Learning Top-5 error on ImageNet Deep Convolution Neural Networks (CNNs) Error 30 22.5 15 7.5 0 28 2010 26 2011 Introduction of deep learning 15.4 2012 AlexNet 11.2 2013 ZFNet 6.7 Trained Human (genius intellect) 5.11 2014 2015 GoogleLeNet Human (Karpathy) 3.57 2015 ResNet Similar effect demonstrated on voice and pattern recognition 2.99 2016 CUImage 2018 moved to Kaggle 2.25 2017 SENet Year Ptucha 18 82 30
AI vs. IA Artificial Intelligence (AI) is the subject of developing machines that can think, act, or reason like humans. Intelligent Augmentation (IA) is the subject of enhancing human abilities- making us faster, smarter, more efficient. Note- according to CB insights 1, AI startup funding in 2016 was $4.2B, up 8 from just four years ago. Central to this is the exponential growth in deep learning the past few years. 1 https://www.cbinsights.com/blog/artificial-intelligence-startup-funding/ Ptucha 18 83 IA vs AI Examples Intelligent Augmentation Enterprise Automate mundane tasks Virtual assistants AR glasses for workers, doctors, repair Autonomous Vehicles Robots/Industrial IoT Highway assist Campus/controlled driving Humans take over for exceptions Collaborative robots work with humans for hard, unsafe, or repetitive Efficient and safer factories Drones Intelligent, but with human overseeing drones for inspections (cell towers, remote locations) Artificial Intelligence Machines perform tasks better and faster than humans All vehicles autonomous Illegal to drive own car Fully automated factories All humans replaced by robots on factory floor Drones don t require human supervision https://www.cbinsights.com/blog/ai-vs-intelligence-augmentation-applications/ Ptucha 18 84 31
An engineer over estimates what she can do in 5 years and under estimates what she can do in 20 years The impact of IA (human-enhancing automation using deep learning and other machine learning techniques) would be bigger in the medium term than most think, while full automation is further away than some recent reporting might indicate https://www.cbinsights.com/blog/ai-vs-intelligence-augmentation-applications/ Ptucha 18 85 Question 5 Do we know what technologies will help us make the leap from intelligent augmentation to artificial intelligence? Ptucha 18 86 32
Andrew Ng, 2017 https://www.deeplearning.ai/ Deep Learning Specialization, Five courses: 1. Neural Networks and Deep Learning 2. Improving Deep Neural Networks 3. Structured Machine Learning Projects 4. Convolutional Neural Networks 5. Sequence Models Ptucha 18 87 Li, Johnson, Yeung 2017 http://cs231n.stanford.edu/ l Ptucha 18 88 33
Thank you!! Ray Ptucha rwpeec@rit.edu https://www.rit.edu/mil Ptucha 18 89 34