Recent developments in artificial intelligence: Deep learning & neural networks

Transcription

1 1 Recent developments in artificial intelligence: Deep learning & neural networks Lambert Schomaker Artificial Intelligence & Cognitive Engineering Institute Center for Data Science & Systems Complexity DSSC Faculty of Science & Engineering

2 Overview I. History of neural networks and recent successes in AI II. Examples of industrial applications III Lessons learned in interaction with companies

3 Part I. History of neural networks and recent successes in AI

4 Deep learning / Recent advances in ML 4 Google self-driving cars Predicting internet user interests ( cookies ) Twitter-based epidemiology ( flu tweets ) Create a van Gogh or Munch version of a photograph Coloring of B/W movies Learning to play Atari Breakout, Pacman etc. AlphaGO: computer wins at playing GO etc. Most of the successes are based on neural networks

5 5 Connection weights were electric resistors

6 Multilayer Perceptron (1983) Rumelhart & McClelland generalized error backpropagation: Werbos (1974) Universal function approximator by summation of scaled and translated sigmoids Handles more complex mappings Training requires a lot of computing and data Limitations become clear (~1996)

7

8 The GO example 8 Board game, black/white Enclose the opponent GO: possible states (chess: 1047 states) Google/DeepMind: Very limited game knowledge, bootstrap with a limited data set of expert games. Using two neural networks: board value estimator and best move selector Computer won 15 March 2016 from human world champion Lee Sedol

9 GO and convolutional neural nets NN1 NN2 NN1: Learn the egocentric value of any given board pattern, from human expert games Then train NN2 to detect the policy to choose the best move, given a board state by playing 30 million times against itself Later, the improved AlphaGo Zero version even learns completely from scratch without human input

10 More successes Automatic completion of missing regions in images Estimating where on earth and when in time some photo was taken Automatic coloring of B/W movies Generating believable (dream-like) images of unseen rooms in houses based on a training set Morphing a photograph into a painting, in the style of a known painter

11 11

12 Example: finding DATE blocks in handwritten manuscripts Zhenwei Shi (2016) 12

13 Convolutional neural nets Nh hidden, Wij Convolve input image with Nh k*k pixel kernels & paint Nh feature maps of M*M pixels Subsample the feature maps to reduce the amount of information while retaining the essence Repeat ad libitum, in a pipeline

14 What does a CNN learn? 14 Yann Lecun, ICML 2013 early middle late middle late

15 Pixel view+joystick control of old video games 15 Score

16 Pixel view+joystick control of old video games Breakout 16

17 Not only with images! Current projects at AI in my group A trainable image search engine (Monk) for historical handwriting with >100M images Dead-Sea Scrolls: up to 200 TB disk space=big data Project with Philips: metal-defect classification Project with Liebherr: in-the-field excavator error message prediction via SIM loggings Project with KLM: inflight fuel optimization Machine translation (language->language) EU project: Mantis predictive maintenance 17

18 What do you mean 'big data'? Data that are too large and unwieldy to get fast answers with off-the shelf software tools Millions of rows: instances, samples Tens of thousands of column values, attributes per instance Common database and statistics software will fail Commercial providers try to follow (Azure, Watson) but the field is moving fast and your in-house expertise is still very much needed

19 1 Discipline #dimensions #instances #classes Goal Challenge Astronomy star features 2x Star classification, brown dwarf classification While the dimensionality is limited, the absolute values of measurements are important and precise 12 2 Genomics 5x104 4x Disease classification on tissue using RNA expression(affymetrix) array data While dimensionality is huge, measurements are noisy. The information is in their correlations: remove 'normal' cell state information, by, e.g., PCA 3 Catalysis 3x in spectra 102 time steps 10 Ligand effectivity in catalysis using Raman spectroscopy. Determine, in time, which cells in an array are delivering promising results in the ongoing reactions. Classification and regression, time series. Currently, peakdetection heuristics are used to track chemical reactions. The redundancy in the spectral patterns is ignored, missing important information during expensive chemical processes 4 Pattern Recognition 5x103 3x108 5x104 Retrieve and sort instances from dozens of large book collections in a live 24/7 machine learning setup Bootstrap from one single label per class to reliable, large training sets. We want to translate our successes in continuous learning (Monk) to other disciplines

20 Schomaker, L. (2016). Design considerations for a large-scale image-based text search engine in historical manuscript collections, Information Technology. 59, ISSN: more than a billion files, for one problem...

21 But note: Many small-data problems can be very complicated Heterogeneity is a problem Several databases Codes, text, numerical values What do they mean? Relation to actual business process?

22 Part II. Examples of industrial applications

23 Predictive maintenance in the data center In 8 years: 50% of 516 disks is replaced: Failure occurs in clusters

24 Predictive maintenance in the data center In 8 years: 50% of 516 disks is replaced: Failure occurs in clusters But what is the optimal, cost-effective maintenance policy with maximal QOS? When to replace (1) big racks, (2) individual blade servers (3) or individual hard disks?

25 Current industrial practice 1 April 2016 A lot of sensors and raw data logging Data are rarely used, and if: ad hoc Analysis requires human efforts, in teams with different competences: floor operator, database operator, analytics expert This is a slow process, requires a lot of meetings By the time data are analysed the problem is not relevant anymore 14

26 Current industrial practice.. 1 April 2016 This is slow, requires a lot of meetings By the time data are analysed the problem is not relevant anymore In order to benefit from current advances in data science and AI: CLOSE THE LOOP: CONNECT BACK TO THE LOCAL PROCESS Data analytics should be an integrated and highly automated part of the process, with reduced human intervention 14

27 Example: fleet maintenance 1 April 2016 International manufacturer of excavators (20k employees world wide) Each excavator from polar to jungle regions has a GSM/SIM with regular status feedback of these assets to the central office Everything is logged But how to use that data? 14

28 Example: fleet maintenance 1 April 2016 The machine generates messages (codes), over regular time intervals. Manual analysis yields interesting things (e.g.: operators in hot countries switch the machine on for the airco only) but systematic, closed loop analysis is not done yet We did two analyses: Are there customer-related usage patterns? Can we predict the (error) messages with a NN? 14

29 Example: fleet maintenance 1 April Are there customer-related usage patterns? k-means clustering 7 usage patterns over customers

30 Example: fleet maintenance 1 April 2016 Are there customer-related usage patterns? 7 usage patterns over customers (no effect of mechanics workshops) So what? You can now decide for tailored servicing options on the basis of the way the machines are treated (bronze, silver, gold maintenance contracts) 14

31 Example: fleet maintenance 1 April 2016 Can you predict error states? 14

32 Example: fleet maintenance 1 April 2016 Can you predict error states? LSTM neural network 95% accuracy of message code prediction Emmanuel Okafor 14

33 Example: fleet maintenance 1 April 2016 Can you predict error states? LSTM neural network 95% accuracy of message code prediction So what? Prediction allows to detect outliers/faulty machines, and future problems This helps optimizing maintenance 14

34 Next example: Replacing human visual intelligence In maintenance, visual inspection relies on the operators, their experience and intuition If the human operators would only tell the machine what is 'good' and 'bad', it will learn by itself if there are enough examples (hundreds/thousands): <photo-001><dented> <photo-002><scratched> <photo-003><rust> <photo-004><good>..

35 Next example: Replacing human visual intelligence Results: 95% accuracy in a 6-class problem using convolutional neural networks (Sheng He)

36 Part III. Lessons learned in interaction with companies

37 No free lunch All of this only works... if the data are readily available: No procedures and committees that decide whether 'it is necessary' Time needs to be reserved for new operator tasks: training the machine by labeling (tagging) patterns. This will pay off! The improved feedback loop needs to be integrated in normal operations, not as an alien process

38 In practice unrealistic expectancies: to enjoy benefits without the need to change anything in current operations hiring external experts is a good idea, right? Explaining them everything is extremely time consuming. The detailed knowledge is often at the core of your business model. Do you want to outsource your brain? The excavator company (20k) had only 1 data scientist If companies hear that data labeling is needed, at least initially, they bail out. missing an opportunity of innovation

39 In practice Legacy systems are not compatible with modern data-science tools Existing staff is trained in traditional engineering and skeptical People are afraid of losing their job (Am I still needed after labeling the data?) Management does not really want to innovate But after decades of experience, it is indeed unlikely to have a lot of easy low-hanging fruit No quick fixes, AI/Deep Learning is not easy

40 Expectancies Companies will need to look at the current operations and see where feedback loops can be closed and data can be used effectively This will also introduce new risks: the monitoring process needs to be monitored itself (cf. the risks of high-frequency trading on Wall Street) It is not about single diamonds: The beneficial effects will be the result of a well-designed architecture, redefining information flows at multiple levels But it is wise to take an initial single problem as the paradigmatic example for closed-loop data science.

41 Expectancies The developments in AI/machine learning are still going on at an increasingly fast pace, with exciting breakthroughs, almost every month As a company, it is important to keep a balance between stability and innovation Under these conditions, it is risky to listen to outsiders with exciting promises your own skeptical insiders There is no data like more data, esp. if it is labeled.