Is Data the New Oil and AI the New Factory? If So, What Does It Mean for the Research Community?

Transcription

1 Is Data the New Oil and AI the New Factory? If So, What Does It Mean for the Research Community? Robert L. Grossman Center for Translational Data Science University of Chicago October 26, 2018 Version 1.4

2 First Industrial Revolution 1760 s Move from hand crafted items to the mechanical production of items by machines First water power and then steam power first mechanical loom Second Industrial Revolution 1870 s Factories with assembly lines supporting mass production Electrical power Railways and the telegraph first Model T Sources: mechanical loom: Model T assembly line: Industrial robot: Third Industrial Revolution 1970 s Electronic and IT systems Analog and mechanical devices replaced by digital devices Industrial robots controlled by micro processors used in factories 1969 First Programmable logic controller, Modicon 084

3

4 1. A Fifty Year Perspective on Data Science

5 AI - blue Big data - red Data mining - gold Hadoop - green Source of Image: Google Trends, trends.google.com

6 Math & Statistics Computer Science & Engineering Data Science Scientific discipline of interest

7 Algorithms + Data Software & Hardware Scientific or Business Problem

8 John Tukey in a 1962 Paper Viewing Data Analysis as a New Science All in all I have come to feel that my central interest is in data analysis [emphasis added], which I take to include, among other things: procedures for analyzing data, techniques for interpreting the results of such procedures, ways of planning the gathering of data to make its analysis easier, more precise or more accurate, and all the machinery and results of (mathematical) statistics which apply to analyzing data. *John Tukey, The Future of Data Analysis, Annals of Mathematical Statistics, 1962.

9 Tukey Identified Four Drivers in The formal theories of statistics. 2. Accelerating developments in computers and display devices. 3. The challenge, in many fields, of more and ever larger bodies of data. 4. The emphasis on quantification in an ever wider variety of disciplines.

10 Computationally Intensive Statistics Data Mining & KDD Predictive Analytics Big Data & Data Science AI (redux) / Deep Learn. Direct marketing POS Internet Mobile OTO PageRank Hamiltonian Monte Carlo Transfer Learning GNU Project Linux Virtualization DevOps AnalyticOps

11 x-100x x-100x x simulation science data science x experimental science

12 2. The Triumphant Return of AI

13 Google s AlphaGo Board after Black s 37 th move in Game 2 between Lee Sodel and Google s AlphaGo (Black). Michael Redmond s comment on AlphaGo s 19 th move: "This is a creative move. It is a move which I believe I have never seen before in a top professional game. AlphaGo analyzed hundreds of thousands of games and generated labeled data through simulation. Amazon s Alexa Amazon dot will answer your questions, tell you jokes, read to you, and order things for you. AVS uses Amazon s Lex Amazon s DL technology for Automatic Speech Recognition (ASR) for converting speech to text and Natural Language Understanding (NLU) for understanding the intent of text.

14 Many neurons linked together in three internal (hidden) layers Single artificial neuron b parameter Source of Image: Nielsen, Michael A. "Neural networks and deep learning." (2015). x 1 f activation function x 2 x n f(x w + b) = f(x 1 w 1 + x 2 w x n w n + b) inputs output An example of an activation function f

15 How Well Does This Work? Between 2011 and 2015, the error rate for image captioning fell from 25% to 3% using deep neural networks (DNN), better than human error rate of 5%.* Between 2013 and 2015, DNN improved Google s single word recognition rate from from 23% to 8%.** In 2016, DNN improved Google Translate BLEU score (English-French) by 7 points, about twice as good as was obtained in the previous four years.** Google Translate using DNN was rolled out in 9 months and launched in November The [system] demonstrated overnight improvements roughly equal to the total gains the old one had accrued over its entire lifetime. ** Source: *From not working to neural networking, The Economist, June 25, **Gideon Lewis-Kraus, The Great AI Awakening, New York Times Magazine, December 14, 2016.

16 Why Does This Work? Data. We have lots and lots of labeled data. Hardware. Neural networks can be thought of as a sequence of matrix operations and GPUs provide enough computational power to train models with billions of parameters that work quite well. We can build models on large sets of images, speech, or video and then reuse these models as components in other models for related domains using a technique called transfer learning. We don t need to engineer features the neural network finds them from labeled data.

17 Transfer Learning Source project provides this data Target project provides this data inputs standard embeddings over large datasets ( pretraining ) in source domain training on target domain data outputs Start the modeling with standard embeddings of text, images, speech, signals, etc. (the green portion of the architecture above)

18 In some sense, Deep Learning is eating the world. Compare: Marc Andressen, Why Software Is Eating The World, Wall Street Journal, August 20, From a broader perspective, Machine Learning (ML) continues to eat the world, as it has been doing for the last 20 years drive by the exponentially growth in the amount of data and the computational power available to estimate parameters.

19 3. Data Commons as one of the Platforms for Data Science

20 What about Biomedical Research The commoditization of sensors is creating an explosive growth of data. It can take weeks to download large datasets, it is difficult to set up compliant computing infrastructure, and it can take months to integrate & format the data for analysis. There is not enough funding for every researcher to house all the data they need The availability of cloud computing

21 NCI Genomic Data Commons* The GDC was launched in 2016 with over 4 PB of data. Used by users per day and over 100,000 researchers each year. Based upon an open source software stack that can be used to build other data commons. *Source: NCI Genomic Data Commons: Grossman, Robert L., et al. "Toward a shared vision for cancer genomic data." New England Journal of Medicine (2016):

22 4. A Note of Caution

23 Understanding Salmon (A Cautionary Tale) Source: Salmo salar, (Atlantic Salmon), wikipedia.org

24 Methods Subject. One mature Atlantic Salmon (Salmo salar) participated in the fmri study. The salmon was approximately 18 inches long, weighed 3.8 lbs, and was not alive at the time of scanning. Task. The task administered to the salmon involved completing an openended mentalizing task. The salmon was shown a series of photographs depicting human individuals in social situations with a specified emotional valence. The salmon was asked to determine what emotion the individual in the photo must have been experiencing. Design. Stimuli were presented in a block design with each photo presented for 10 seconds followed by 12 seconds of rest. A total of 15 photos were displayed. Total scan time was 5.5 minutes. Source: Craig M. Bennett, Abigail A. Baird, Michael B. Miller, and George L. Wolford, Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: An argument for multiple comparisons correction, retrieved from

25 Several active voxels were discovered in a cluster located within the salmon s brain cavity (Figure 1, see above). The size of this cluster was 81 mm 3 with a cluster-level significance of p = Due to the coarse resolution of the echo-planar image acquisition and the relatively small size of the salmon brain further discrimination between brain regions could not be completed. Out of a search volume of 8064 voxels a total of 16 voxels were significant. Source: Craig M. Bennett, Abigail A. Baird, Michael B. Miller, and George L. Wolford, Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: An argument for multiple comparisons correction, retrieved from

26 Poisoned Stop Signs misclassification An adversary can poison an image so that it is likely to be misclassified. Source: Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song, Robust Physical-World Attacks on Deep Learning Visual Classification, arxiv: v5 [cs.cr] 10 Apr 2018.

27 5. The Broader Implications Data as the New Oil and AI as the New Factory

28 x-100x x-100x data science x simulation science x experimental science

29 Caption: 2D tsne Clustering of 32 TCGA Projects. Patients in different projects are presented by different dot colors or shapes. Source: Figure 6 from Zhenyu Zhang, Kyle Hernandez, Jeremiah Savage, Shenglai Li, Dan Miller, Stuti Agrawal, Francisco Ortuno, Allison Heath, Lou Staudt and Robert L. Grossman, The Uniform Genomic Analysis of Data in the NCI Genomic Data Commons, to appear.

30 Is More Different? Do New Phenomena Emerge at Scale in Data? Source: P. W. Anderson, More is Different, Science, Volume 177, Number 4047, 4 August 1972, pages

31 Mathematics & statistics Computer science & engineering 1. Develop data commons for accelerating research. Translational data science Collaborate with researchers in biology, medicine, healthcare, & the environ. to solve challenging problems that have an impact. 2. Make discoveries over the data commons. 3. Translate the discoveries.

32 Second Industrial Revolution 1870 s Factories with assembly lines supporting mass production Electrical power Railways and the telegraph first Model T Third Industrial Revolution 1970 s Electronic and IT systems Analog and mechanical devices replaced by digital devices Industrial robots controlled by micro processors used in factories 1969 First Programmable logic controller, Modicon 084 Fourth Industrial Revolution 2020 s Systems powered by ML, DL, & AI and abundant data Physical devices replaced by virtualized devices defined by software Autonomous devices that can sense their environment and react to voice commands first Amazon Alexa developers conference

33 The West may have sparked the fire of deep learning, but China will be the biggest beneficiary of the heat the AI fire is generating. The global shift is the product of two transitions: from the age of discovery to the age of implementation, and from the age of expertise to the age of data.

34 6. Summary

35 Databases organize datasets around a project, experiment, or department. Data Warehouse Data warehouses organize the data for an organization (and are enabled by enterprise computing) Data commons organize the data for a scientific discipline, community, or field and provide a platform for machine learning, deep learning, and AIbased discovery.

36 Contact Information Robert L. Grossman ctds.uchicago.edu

37 Abstract Is Data the New Oil, and, If So, What Does it Mean for the Research Community? A question currently being debated is whether data is the new oil and, if so, what does it mean for different communities, from the business community to the research community. Over the past decade, the computing infrastructure and tools for working with data at scale have evolved, with advances including cloud computing, the re-emergence of AI, and data commons. This talk will cover the power and promise of data commons to globally and collaboratively accelerate research, especially biomedical research.