Exploiting the Unused Part of the Brain

Transcription

1 Exploiting the Unused Part of the Brain Deep Learning and Emerging Technology For High Energy Physics Jean-Roch Vlimant

2 A 10 Megapixel Camera

3 CMS 100 Megapixel Camera

4 CMS Detector

5 CMS Readout Highly heterogeneous system Raw data is 100M channels sampled every 25 ns : 1Pb/s 50EB per day in readout and online processing.

6 HLT 1-3 khz 1000 Gb/s L1 105 Hz 40 MHz Event Filtering 1000 Gb/s From Big Data to Smart Data with ultra fast decision

7 Why Deep Learning LHC Data Processing may use deep learning methods in many aspects (attend other relevant talk at the Caltech booth) Large volume of data and simulated data to analyze Several class of LHC Data analysis make use of classifier for signal versus background discrimination. Use of BDT on high level features Increasing use of MLP-like deep neural net Deep learning has delivered super-human performance at certain class of tasks (computer vision, speech recognition,...) Use of convolutional neural net, recurrent topologies, long-short-term-memory cells,... Deep learning has advantage at training on raw data Several levels of data distillation in LHC data processing Neural net computation is highly parallelizable Going beyond fully connected networks with advanced deep neural net topologies Multi-classification of LHC events from particle-level information Charged particle tracking with recurrent and convolutional topologies Particle identification and energy regression in highly granular future calorimeter... 7

8 Advanced Machine Learning and Deep Learning (my selection) 8

9 Machine Learning in a Nutshell Balazs Kegl, CERN

10 Scene Labeling Karpathy, Fei-Fei, CVPR 2015 Create a description of images Generate a decay process description from collision representation, with application to triggers 10

11 Scenery Interpretation Farabet et al. ICML 2012, PAMI 2013 Group and classify what each pixel belongs to Real-time video processing with deep learning Multiple applications to pileup mitigation, object identification, tracking. All from raw data 11

12 Attention Learning Identify people from faces with multiple attention filters Object identification, noise subtraction,... 12

13 Text Processing Question and Answer machine, language translation, semantic arithmetic,... Can the raw data of detector be interpreted as texts and translated into physics descriptions? 13

14 Embedded Symmetries p4 group Introduction of convolutional layers was a ground-breaking advancement Research on embedding more fundamental symmetries into neural nets p4m group T.S. Cohen, M. Welling ICML2016 Symmetries operate on the data or internal representation of data Next is to implement symmetries of physics to build physics-specific NN 14

15 Toolkit and Services Lots of libraries out there, several key components in each major languages. Lots of big-data analytics services offered Common theme of going for spark-hdfs support Question of having in-house software or embracing external libraries is very much alive 15

16 Application to Intensity and Energy Frontiers (a selected few) 16

17 NOVA Event Classification Slides on Paolo 17

18 Particle Jet Identification Top Tagger arxiv: Almeida, Backovic, Cliche, Lee, Perelstein Neural net W tagger arxiv: , Oliveira, Kagan, Mackey, Nachman, Schwartzman W to QCD discrimination Train 18

19 3D Calorimetry Imaging 100GeV Photon 100GeV Pi0 LCD Calorimeter configuration 5x5 mm Pixel calorimeter 28 layer deep for Ecal 70 layer deep for Hcal Photon and pion particle gun Classification, regression and combined models 19

20 Irregular Geometry Challenge Projective Geometry Hexagonal cells The images we are dealing with are not as regular as standard images. Need for specific new treatment and methods to feed neural nets Variable Depth Segmentation 20

21 Collision Event Classification Full event classification using reconstructed particle 4-vectors Recurrent neural nets, Long short term memory cells Dedicated layer with Lorentz boosting Step toward event classification with lower level data : low level feature as opposed to analysis level variables 21

22 Ordering Challenge Text have natural order. RNN/LSTM can correlate the information to internal representation There is underlying order in collision events. Smeared through timing resolution. No natural order in observable 22

23 Charged Particle Tracking Perfect example of pattern recognition Data sparsity is not common in image processing Several angles to tackle the problem. Deep Kalman filter, RNN to learn dynamics, sparse image processing,... Kaggle challenge in preparation 23

24 Challenge of Tracking 24

25 HEP Trk.X LBL, FNAL, Caltech consortium sponsored by the DOE Preparation of simulated data Accurate, fast and light Explore new approaches to charged particle tracking using advanced pattern recognition techniques Recurrent neural nets in learning track kinematics Convolutional neural nets for pattern recognition Hough-like transformation from hit position space to track parameter space Advanced Kalman filter parallelization... exploration has started Tracking competition (a la kaggle) in preparation 25

26 Other Applications Outliers selection Anomaly detection Data quality automation Detector control Experiment control Data popularity prediction Computing grid control Denoising with auto-encoder Fast simulation... 26

27 Accelerating and Emerging Technologies 27

28 Supermicro Server Caltech Servers 2 compute nodes : Intel Xeon CPU E GHz processors per node (28 cores per CPU) with 8 NVIDIA Pascal GTX 1080 Theoretical Peak Performance : 80 Tflops Theano, Tensorflow, Keras MPI training Spearmint hyper-optimization Many thanks to our partners Nvidia and Supermicro 28

29 ALCF Cooley 126 compute nodes : Two 2.4 GHz Intel Haswell E v3 processors per node (6 cores per CPU, 12 cores total) and NVIDIA Tesla K80 Theoretical Peak Performance : 293 Tflops Development Project with 8k core hours Theano, Tensorflow, Keras MPI ready Spearmint experimental 29

30 CSCS Piz Daint 5272 compute nodes : Intel Xeon 2.60GHz (8 cores, 16 virtual cores with hyperthreading enabled, 32GB RAM) and NVIDIA Tesla K20X Theoretical Peak Performance : Pfops Scratch capacity : 2.7 PB Development Project with 36k core hours Theano, Tensorflow, Keras MPI ready Spark experimental stage 30

31 OLCF Titan compute nodes : 2.2GHz AMD Opteron 6274 processors per node (16 cores per CPU) and NVIDIA Tesla K20X Theoretical Peak Performance : 20 Pflops Allocation in preparation

32 Distributed Learning Titan X GTX 1080 Deep learning with elastic averaging SGD Revisiting Distributed Synchronous SGD Implementation with Spark and MPI for the Keras framework

33 Performance Ran on the supermicro 8GPU server here at SC16 Normalized to 2 GPU performance point Please add a factor 2x 7x max speedup Linear speedup with number of workers Saturation in co-scheduling GPU (2 workers on the same GPU) 33

34 Performance Ran on ALCF Cooley 126 GPU cluster Normalized to 2 GPU performance point Please add a factor 2x 14x max speedup Non-linear speedup with number of GPU Linear speedup with number of GPU 34

35 Applicability 35

36 Training vs Inference GPUs are the workhorse for parallel computing Enable training large models, with large dataset Deep learning facility clusters Emergence of smaller GPU Not dedicated to training Strike the balance between Tflops/$ for inference Deployment on the grid 36

37 Neuromorphic Hardware Implementing plasticity in hardware Process signal from detector and adapt to categories of pattern (unsupervised) Post-classified from data analysis or rate throttling NCCR consortium assembling to develop this technology further, with our use case in mind 37

38 Cognitive Computing Spiking neural net as processing units : Cognitive Computing Processing Unit : CCPU Adopt a new programming scheme, translate existing software See Rebecca Carney's talk for more details 38

39 Summary Impressive achievement and promise of modern machine learning and deep learning From realistic to speculative applicability to field of High Energy and Frontier Physics Emerging tools and technology to embrace Thanks to our sponsors 39