Image Processing. COMP 3072 / GV12 Gabriel Brostow. TA: Josias P. Elisee (with help from Dr Wole Oyekoya) Image Processing.

Transcription

1 Image Processing COMP 3072 / GV12 Gabriel Brostow TA: Josias P. Elisee (with help from Dr Wole Oyekoya) 1

2 2

3 Motivation and Goals Grounding in image processing techniques Concentrate on algorithms used in machine vision, graphics, medical imaging Best sensors ever! 3

4 Motivation and Goals Grounding in image processing techniques Concentrate on algorithms used in machine vision, graphics, medical imaging Best sensors ever! With a few problems 4

5 Transmission interference 5

6 Compression artefacts 6

7 Spilling 7

8 Scratches, Sensor noise 8

9 Bad contrast 9

10 Resolution Super resolution? 10

11 Super resolution 11

12 Removing motion blur Cropped subwindow Original image [Images from Amit Agrawal] 12 After motion blur removal

13 Removing motion blur 13

14 14

15 Removing motion blur 15

16 Syllabus (1 st half) 1. The digital image 2. Image segmentation* 3. Image transformations 4. Morphological operations* 5. Image Filtering *= Homework will be assigned 16

17 Features and Object Recognition GV12/3072

18 Syllabus (2 nd half) 6. Filtering Applications + Edges* 7. Corner detection 8. Feature Characterization 9. Color images* 10.Template matching *= Homework will be assigned GV12/3072

19 Why Now? Medicine Automatic or assisted diagnosis Image-guided surgery Biology / Agriculture Film and television Surveillance and police work Military Why are these sectors paying more attention? 20

20 Course content Lots of material! Some mathematics Calculus (light) Geometry and matrix algebra Probability and statistics (light) Some programming Matlab 21

21 Lectures and notes TA: Josias Elisee: J.Elisee at cs.ucl.ac.uk Extra help: Wole Oyekoya: w.oyekoya at cs.ucl.ac.uk Mondays 16:00-17:00 in Medawar Lankester LT Wednesdays 09:00-11:00 in MPEB 1.02 Lab sessions Monday (Malet Place Eng 4.06) Monday (Malet Place Eng 4.06) Moodle! 22

22 Assessment Exam 80% Four Courseworks 20%. Implement and test algorithms in Matlab Honor System 25

23 Unassessed CW Assignment Matlab introduction Start matlab: % matlab or % /opt/matlab7/bin/matlab Download any simple image Load it into matlab: >> I = imread( foo.jpg ); 28

24 Unassessed CW Assignment Display the image in Matlab: >> imshow(i); Print the image data array: >> I (Ha! It s a trap! use Ctrl-C to make it stop) Print the size of the image array and create a subimage: >> size( I ) >> Isubwindow = I(72:92, 62:82); >> imshow(isubwindow); 29

25 Unassessed CW Assignment Start the Matlab help tool (Help menu). In the Contents pane to the left of the window. Click on MATLAB. Go through the Getting Started section. Continue to the Using MATLAB section when you have time. 30

26 IP is Only Part of the Picture See Machine Vision and/or Computational Photography + Capture (GV15 / M085 / 3085) Why? To work on fun projects! MRI of GJB A Computational Investigation into the Human Representation and Processing of Visual Information 31

27 3D Gesture Interfaces (Xbox 360) Build Your Own 3D Scanner: Optical Triangulation for Beginners (Lanman + Taubin) 32

28 Developing Drosophila eye (30 hours) With Franck Pichaud Epithelial Morphogenesis & Cell Polarity LMCB, Cell Biology Unit, MRC, UCL Needed Innovations: - Locate & track branching structures - Propagate confidence to neighbors 33

29 Stop Motion Animation Super Lego Mario (Level 1) Video Annotation, Navigation, and Composition UIST2008, 34

30 Next Time: The Digital Image 35

31 The Digital Image 37

32 Outline What is an image? What is a pixel? How do we store them? 38

33 What is an image? 39

34 Image as 2D signal Signal: function depending on some variable with physical meaning Image: continuous function 2 variables: xy - coordinates 3 variables: xy + time (video) Brightness is usually the value of the function But can be other physical values too: temperature, pressure, depth 40

35 Example 2d images ultrasound temperature camera image 41 CT

36 Image? >> t=rand(256,256); >> imshow(t) 46

37 Where do images come from? Digital cameras MRI scanners Computer graphics packages Body scanners Laser range finders Many more 49

38 Where do images come from? Digital cameras MRI scanners Computer graphics packages Body scanners Laser range finders Many more 50

39 The digital camera A Charge Coupled Device (CCD). Sensor array Image array Lens ADC 51

40 Capturing photons From: Lecture Notes EAAE and/or Science Nuggets

41 53

42 The sensor array Can be < 1cm 2. An array of photosites. Each photosite is a bucket of electrical charge. They contain charge proportional to the incident light intensity during exposure. 55

43 Analog to Digital Conversion The ADC measures the charge and digitizes the result. Conversion happens line by line. The charges in each photosite move down through the sensor array. ADC RAM 56

44 ADC RAM 57

45 Blooming The buckets have finite capacity Photosite saturation causes blooming 58

46 Dark Current Yohkoh satellite, 9 years apart.. CCDs produce thermally-generated charge. They give non-zero output even in darkness. Partly, this is the dark current. Fluctuates randomly. How can we reduce dark current? From: Lecture Notes - EAAE 59

47 Dark Current CCDs produce thermally-generated charge. They give non-zero output even in darkness. Partly, this is the dark current. Fluctuates randomly. How can we reduce dark current? From: Lecture Notes - EAAE 60

48 61

49 What is a pix-el? (0,0) x f(x,y) (x,y) y 63

50 Not a little square! A Pixel Is Not A Little Square, A Pixel Is Not A Little Square, A Pixel Is Not A Little Square! (And a Voxel is Not a Little Cube), Alvy Ray Smith, MS Tech Memo 6, Jul 17,

51 Not a little square! Gaussian reconstruction filter Illustrations: Smith, MS Tech Memo 6, Jul 17,

52 Not a little square! Illustrations: Smith, MS Tech Memo 6, Jul 17, 1995 Cubic reconstruction filter 66

53 Not a little square! Graphics: Dick Lyon,

54 Sampling 1D Sampling in 1D takes a function, and returns a vector whose elements are values of that function at the sample points. 69

55 Sampling 2D Sampling in 2D takes a function and returns an array; we allow the array to be infinite dimensional and to have negative as well as positive indices. 70

56 Greyscale digital image 71

57 Nyquist Frequency Half the sampling frequency of a discrete signal processing system 72

58 Sampling grids 73

59 Retina-like sensors 74

60 Quantization Real valued function will get digital values integer values Quantization is lossy!! After quantization, the original signal cannot be reconstructed anymore This is in contrast to sampling, as a sampled but not quantized signal can be reconstructed. Simple quantization uses equally spaced levels with k intervals b k 2 75

61 Quantization

62 Quantization

63 Usual quantization intervals Grayvalue image 8 bit = 2^8 = 256 grayvalues Color image RGB (3 channels) 8 bit/channel = 2^24 = 16.7Mio colors 12bit or 16bit from some sensors Nonlinear, for example log-scale 78

64 Photo: Paulo Barcellos Jr. 79

65 Winding Down 80

66 Properties Image resolution Geometric resolution: How many pixel per area Radiometric resolution: How many bits per pixel 81

67 Image resolution 512x x x

68 Geometric resolution 144x144 72x72 36x36 18x18 9x9 4x4 83

69 Radiometric resolution

70 Lossless vs. Lossy Name some formats? 85

71 Aliasing and SNR What is the disadvantage of low sampling resolution? What is the disadvantage of high sampling resolution? 86

72 Finish Next week: Image segmentation 87

73 Unassessed Assignment Use matlab to change the geometric and radiometric quantization resolution in one of your images. For each level of sampling and quantization, plot the image function, as in slides 67 & 68, and compare the approximations to the true intensity function that you get at each level. 94