Convolutional Neural Networks

Transcription

1 Convolutional Neural Networks Convolution, LeNet, AlexNet, VGGNet, GoogleNet, Resnet, DenseNet, CAM, Deconvolution Sept 17, 2018 Aaditya Prakash

2 Convolution

3 Convolution

4 Demo Convolution

5 Convolution in Neural Networks

6 Convolution in Neural Networks

7 Convolution in Neural Networks

8 Convolution in Neural Networks

9 Stride 1-D

10 Stride 1-D

11

12

13

14

15

16

17

18 Stride 2-D

19 Padding

20 Features / Filters 1-D 2-D

21 Features / Filters 2-D

22 Features / Filters 2-D 3-D

23 Features / Filters 2-D 3-D 2-D Multichannel

24 Features / Filters 3-D

25 2-D Convolution

26 2-D Convolution

27 Pooling 1-D 2-D

28 Convolutional Neural Network

29 Convolutional Neural Network

30 LeNet *Original LeNet-5 has two FCL at the end, and filter sizes are slightly different

31 AlexNet (2012 Winner) AlexNet architecture: [227x227x3] INPUT [55x55x96] CONV1: 96 11x11 filters at stride 4, pad 0 [27x27x96] MAX POOL1: 3x3 filters at stride 2 [27x27x96] NORM1: Normalization layer [27x27x256] CONV2: 256 5x5 filters at stride 1, pad 2 [13x13x256] MAX POOL2: 3x3 filters at stride 2 [13x13x256] NORM2: Normalization layer [13x13x384] CONV3: 384 3x3 filters at stride 1, pad 1 [13x13x384] CONV4: 384 3x3 filters at stride 1, pad 1 [13x13x256] CONV5: 256 3x3 filters at stride 1, pad 1 [6x6x256] MAX POOL3: 3x3 filters at stride 2 [4096] FC6: 4096 neurons [4096] FC7: 4096 neurons [1000] FC8: 1000 neurons (class scores)

32 VGG Net

33 GoogLeNet (2014 winner)

34 Peasant s network vs Google s

35 GoogLeNet (2014 winner)

36 Inception

37 Inception

38 Inception

39 Inception

40 Going Deeper

41 Resnet (2015 winner)

42 Resnet - comparison with other nets

43 Resnet - Number of layer comparison

44 Depth - The Stigma

45 Depth - The Stigma

46 DenseNet

47 ResNeXt Residual Block Aggregated Residual Block (ResNeXt)

48 Squeeze and Excitation Network - winner 2017

49

50 Convolutional Neural Network

51 CNN filter response - See: Interactive visualization with MNIST Higher activations -> Object Location

52 CNN filter response See: Interactive visualization with MNIST - Higher activations -> Object Location - Problem: Does not capture object structure

53 Class activation map

54 Class activation map

55 Class activation map

56 Class activation map - Class activation map is the obtained by taking the output of GAP and learning weights that maximize the discriminative activations for a given class.

57 Class activation map - Problem: Identifies only one object.

58 Our work Multi-Structure Region of Interest (MSROI) Perform weak localization like CAM, but detect multiple salient objects.

59 Multi-Structure Region of Interest Standard Convolution Layer - Add one more dimension to feature maps - classes. MS-ROI Convolution Layer

60 Multi-Structure Region of Interest Standard Convolution Layer - Add one more dimension to feature maps - classes. - Learns class invariant feature maps. MS-ROI Convolution Layer

61 Multi-Structure Region of Interest Standard Convolution Layer MS-ROI Convolution Layer - Add one more dimension to feature maps - classes. - Learns class invariant feature maps. - For training, replace softmax with sigmoid in order to prevent squeezing of the probabilities of classes that are not ground-truth.

62 MSROI - No Free Lunch - For a color image of decent size and with many filters per layer and several layers deep, this number is huge.

63 MSROI - No Free Lunch - For a color image of decent size and with many filters per layer and several layers deep, this number is huge. Solution - Make number of classes very small by using Synsets - hierarchy of classes in Imagenet - Share feature maps across classes to jointly learn lower level features

64 MSROI - Fine-grained is overkill - Most CNN models, including CAM, are trained on Imagenet, which has 1000 classes. Some of the classes are fine-grained like different breeds of dog.

65 MSROI - Fine-grained is overkill - Most CNN models, including CAM, are trained on Imagenet, which has 1000 classes. Some of the classes are fine-grained like different breeds of dog. Intuition, they will have similar semantic map, because of similar object structure.

66 Where do we look? SALICON Dataset

67 Where do we look?

68 Class Activation Map (CAM) CAM where is learned for every class c and for layer d

69 Multi-Structure Region of Interest CAM where is learned for every class c and for layer d MSROI Map - denotes threshold which signifies presence of a class

70 MSROI - Details CAM where is learned for every class c and for layer d MSROI Map - denotes threshold which signifies presence of a class denotes Multi-structure map generated using MSROI. Compare this with CAM map (M) It is sum over all classes with total activations Zlc beyond some threshold.

71 MSROI - Details CAM where is learned for every class c and for layer d MSROI Map - denotes threshold which signifies presence of a class denotes Multi-structure map generated using MSROI. Compare this with CAM map (M) It is sum over all classes with total activations Zlc beyond some threshold. For training use sigmoid instead of softmax to prevent losing information about other objects

72 MSROI - examples on Kodak images

73 Deconvolution / Transpose Convolution / Fractional Convolution

74 Semantic Segmentation

75 Demo Deconv

76 Artistic Style Transfer - Feed the artistic image through the VGG net and compute and save the Gram matrix G. - Feed the photograph through the VGG net and save the feature maps F. - Generate a white noise image. Through backpropagation, iteratively update this image until it has a feature map and a Gram matrix that are close to F and G, respectively.

77 Deep Dream Inceptionism: Going Deeper into Neural Networks Album

78

79

80

81 Fooling CNNs Adversarial Examples and Rubbish Classes Answers Due to high dimensional dot products Occurs in both linear (ReLu) & Non-linear models Direction of perturbation matters not specific point Also occurs in Shallow networks not just DNN Regularisation doesn t prevent fooling examples Adversarial training is good regularization Extremely low probability (not observed in test)

82 Summary - ConvNets stack CONV,POOL,FC layers - Trend towards smaller filters and deeper architectures - Trend towards getting rid of POOL/FC layers (just CONV) - Typical architectures look like [(CONV-RELU)*N-POOL?]*M-(FC-RELU)*K,SOFTMAX where N is usually up to ~5, M is large, 0 <= K <= 2. - but recent advances such as ResNet/GoogLeNet challenge this paradigm Credits CS231n Convnets Chris Colah s awesome blog Chris Burger - Style transfer images Convolutional Neural Networks - Nervana Systems DeepVis - Jason Yosiniki Fooling CNNs - Anh Nguyen More demos - Yann LeCun