To Infinity And Beyond. Computer Vision for Astronomy

Transcription

1 To Infinity And Beyond Computer Vision for Astronomy

2 Ryan foxrow.com

3 1. Image Processing 2. Computer Vision 3. To Infinity and Beyond

4 How computers see

5 How computers see

6 How computers see

7 How computers see

8 Image Processing vs Computer Vision

9 Image processing Computer Vision Low level algorithms Higher level of abstraction Works on individual pixels Works on whole images or image sets Building blocks for larger applications Understand image contents

10 Image processing Computer Vision

11 Use OpenCV $ pip install opencv-python

12 Use OpenCV >>> import cv2 >>> img = cv2.imread( my_image.jpg )... >>> cv2.imwrite( new_image.jpg, img)

13 Image Processing Convolution Feature extraction Feature descriptors Image Segmentation Thresholding Erosion/Dilation Contours Histograms Image registration Panoramas Stacking

14 Convolution

15 Convolution X / 16 = (0*1 + 2*2 + 1*1 + 4*2 + 8*4 + 6*2 + 1*1 + 2*2 + 2*1)/16 = 4

16 Convolution X / 16 = (2*1 + 1*2 + 5*1 + 8*2 + 6*4 + 8*2 + 2*1 + 2*2 + 9*1)/16 = 5

17 Convolution Blur Kernel:

18 Feature Extraction Edge Detection Corner Detection Hough Transform Feature Descriptors

19 Canny Edge Detector 1. Blur the image 2. Calculate gradient 3. Places with strong gradients are likely to be edges

20 Canny Edge Detector cv2.canny()

21 Harris Corner Detector cv2.cornerharris()

22 Hough Transform cv2.houghlines() cv2.houghcircles()

23 Hough Line Transform

24 Feature Descriptors

25 SIFT and SURF Scale Invariant Feature Transform Speeded Up Robust Features

26 Feature Descriptors ORB BRISK BRIEF KAZE AKAZE

27 ORB Oriented FAST and Rotated BRIEF cv2.orb_create()

28 ORB

29 Image Segmentation

30 Thresholding cv2.threshold()

31 Thresholding

32 Erosion and Dilation Grow or shrink regions in an image

33 Erosion cv2.erode()

34 Dilation cv2.dilate()

35 Contours cv2.findcontours()

36 Contours

37 Contours

38 Histograms

39 Histograms numpy.histogram()

40 Histogram Equalization This is not a typical Hubble image

41 Histogram Equalization This is a typical Hubble image

42 Histogram Equalization This is a typical Hubble image

43 Histogram Equalization

44 Histogram Equalization

45 Image Registration

46 Panoramas

47 Panoramas

48 Stacking

49 Stacking Sources of noise: - Cosmic rays - Atmospheric distortion - Lens imperfections - Sensor imperfections - Thermal noise - Readout noise - Clouds Airplanes Pedestrians People blinking

50 Stacking

51 Stacking

52 Computer Vision Object detection Object recognition Face recognition Reverse image search Duplicate detection OCR QR codes Photogrammetry Neural nets

53 Object Detection Feature matching Histogram backprojection Haar cascades* Neural nets*

54 Feature Matching

55 Feature Matching

56 Feature Matching

57 Feature Matching

58 Object Recognition Cascade classifier using known characteristics Neural nets using??? characteristics

59 Face Recognition Haar cascades Most human faces share some characteristics: - Forehead lighter than eyes Eyes are darker than cheekbones Ears extend out to the sides Nose is about halfway between top of head and chin Etc. etc.

60 Face Recognition cv2.cascadeclassifier()

61 Face Recognition

62 Reverse Image Search

63 Astrometry.net

64 Astrometry.net

65 Astrometry.net Locate the brightest stars Find relations between them Search a catalog of known relations

66 Astrometry.net

67 Duplicate Detection

68 Perceptual Hashes Low-avalanche hash functions dhash - Difference hash

69 dhash

70 dhash Convert to grayscale Resize to postage stamp Calculate the difference between adjacent pixels

71 dhash Convert to grayscale Resize to postage stamp Calculate the difference between adjacent pixels

72 dhash Convert to grayscale Resize to postage stamp Calculate the difference between adjacent pixels 0b x9ffffffffffe2fa1

73 dhash 0x9ffffffffffe2fa1 0x9ffffffffffe2fa1 0x0d0f3f1f8f469391

74 dhash Allows for fuzzy matching as well - Hamming distance 0b b $ pip install imagehash imagehash.dhash

75 Optical Character Recognition pytesseract.image_to_string()

76 Optical Character Recognition United States

77 QR Codes

78 Photogrammetry Physical measurements from imagery

79 Photogrammetry Maps Orthomosaics Point clouds Contour lines Length/Area/Volume measurement Terrain classification

80 Photogrammetry

81 Neural Networks Image Classification - VGG16, VGG19, ResNet Object Recognition - YOLO, Faster R-CNN OCR Much more!

82 Image Classification What s in an image? grass, outdoor, people, large, field, park, group, sitting, table, man, standing, grassy, cake, crowd, display, ball, riding, horse, air, umbrella

83 Object Recognition

84 To Infinity and Beyond

85 How can you use it? load DJI_0241.jpg highlight car truck person save load pano/dji_0002.jpg pano/dji_0003.jpg pano/dji_0004.jpg pano/dji_0005.jpg stitch save

86 How can you use it? OpenCV - opencv.org Google, Azure, AWS Astropy - astropy.org Deep Sky Stacker - deepskystacker.free.fr Hugin - hugin.sourceforge.net OpenDroneMap - github.com/opendronemap

87 load DJI_0241.jpg highlight car truck person save load pano/dji_0002.jpg pano/dji_0003.jpg pano/dji_0004.jpg pano/dji_0005.jpg stitch save

88 DIL Drone Imaging Language

89

90 Languages

91 Languages 1. Don t have to be complex 2. 3.

92 Datetime formatting

93 Regular expressions

94 String formatting mini-language

95 Languages 1. Don t have to be complex 2. Trade generality for simplicity 3.

96 Simplicity load highlight stitch show save

97 Languages 1. Don t have to be complex 2. Trade generality for simplicity 3. Are interfaces

98 Ryan foxrow.com github.com/foxrow/dil