Image Processing COMP 3072 / GV12 Gabriel Brostow TA: Josias P. Elisee (with help from Dr Wole Oyekoya) 1
2
Motivation and Goals Grounding in image processing techniques Concentrate on algorithms used in machine vision, graphics, medical imaging Best sensors ever! 3
Motivation and Goals Grounding in image processing techniques Concentrate on algorithms used in machine vision, graphics, medical imaging Best sensors ever! With a few problems 4
Transmission interference 5
Compression artefacts 6
Spilling 7
Scratches, Sensor noise 8
Bad contrast 9
Resolution Super resolution? 10
Super resolution 11
Removing motion blur Cropped subwindow Original image [Images from Amit Agrawal] 12 After motion blur removal
Removing motion blur 13
14
Removing motion blur 15
Syllabus (1 st half) 1. The digital image 2. Image segmentation* 3. Image transformations 4. Morphological operations* 5. Image Filtering *= Homework will be assigned 16
Features and Object Recognition GV12/3072
Syllabus (2 nd half) 6. Filtering Applications + Edges* 7. Corner detection 8. Feature Characterization 9. Color images* 10.Template matching *= Homework will be assigned GV12/3072
Why Now? Medicine Automatic or assisted diagnosis Image-guided surgery Biology / Agriculture Film and television Surveillance and police work Military http://www.cs.ubc.ca/spider/lowe/vision.html Why are these sectors paying more attention? 20
Course content Lots of material! Some mathematics Calculus (light) Geometry and matrix algebra Probability and statistics (light) Some programming Matlab 21
Lectures and notes http://www.cs.ucl.ac.uk/staff/g.brostow/classes/ip2010/ TA: Josias Elisee: J.Elisee at cs.ucl.ac.uk Extra help: Wole Oyekoya: w.oyekoya at cs.ucl.ac.uk Mondays 16:00-17:00 in Medawar Lankester LT Wednesdays 09:00-11:00 in MPEB 1.02 Lab sessions Monday 11-13 (Malet Place Eng 4.06) Monday 17-18 (Malet Place Eng 4.06) Moodle! http://moodle.ucl.ac.uk/ 22
Assessment Exam 80% Four Courseworks 20%. Implement and test algorithms in Matlab Honor System 25
Unassessed CW Assignment Matlab introduction Start matlab: % matlab or % /opt/matlab7/bin/matlab Download any simple image Load it into matlab: >> I = imread( foo.jpg ); 28
Unassessed CW Assignment Display the image in Matlab: >> imshow(i); Print the image data array: >> I (Ha! It s a trap! use Ctrl-C to make it stop) Print the size of the image array and create a subimage: >> size( I ) >> Isubwindow = I(72:92, 62:82); >> imshow(isubwindow); 29
Unassessed CW Assignment Start the Matlab help tool (Help menu). In the Contents pane to the left of the window. Click on MATLAB. Go through the Getting Started section. Continue to the Using MATLAB section when you have time. 30
IP is Only Part of the Picture See Machine Vision and/or Computational Photography + Capture (GV15 / M085 / 3085) Why? To work on fun projects! MRI of GJB A Computational Investigation into the Human Representation and Processing of Visual Information 31
3D Gesture Interfaces (Xbox 360) Build Your Own 3D Scanner: Optical Triangulation for Beginners (Lanman + Taubin) 32
Developing Drosophila eye (30 hours) With Franck Pichaud Epithelial Morphogenesis & Cell Polarity LMCB, Cell Biology Unit, MRC, UCL Needed Innovations: - Locate & track branching structures - Propagate confidence to neighbors 33
Stop Motion Animation Super Lego Mario (Level 1) http://www.youtube.com/watch?v=hmwwfnkvbyy Video Annotation, Navigation, and Composition UIST2008, http://www.danbgoldman.com/ 34
Next Time: The Digital Image 35
The Digital Image 37
Outline What is an image? What is a pixel? How do we store them? 38
What is an image? 39
Image as 2D signal Signal: function depending on some variable with physical meaning Image: continuous function 2 variables: xy - coordinates 3 variables: xy + time (video) Brightness is usually the value of the function But can be other physical values too: temperature, pressure, depth 40
Example 2d images ultrasound temperature camera image 41 CT
Image? >> t=rand(256,256); >> imshow(t) 46
Where do images come from? Digital cameras MRI scanners Computer graphics packages Body scanners Laser range finders Many more 49
Where do images come from? Digital cameras MRI scanners Computer graphics packages Body scanners Laser range finders Many more 50
The digital camera A Charge Coupled Device (CCD). Sensor array Image array Lens ADC 51
Capturing photons From: Lecture Notes EAAE and/or Science Nuggets 2000 52
http://www.astro.virginia.edu/class/oconnell/astr121/im/ccd-fullframearc-fsu.jpg 53
The sensor array Can be < 1cm 2. An array of photosites. Each photosite is a bucket of electrical charge. They contain charge proportional to the incident light intensity during exposure. 55
Analog to Digital Conversion The ADC measures the charge and digitizes the result. Conversion happens line by line. The charges in each photosite move down through the sensor array. ADC RAM 56
ADC RAM 57
Blooming The buckets have finite capacity Photosite saturation causes blooming 58
Dark Current Yohkoh satellite, 9 years apart.. CCDs produce thermally-generated charge. They give non-zero output even in darkness. Partly, this is the dark current. Fluctuates randomly. How can we reduce dark current? From: Lecture Notes - EAAE 59
Dark Current CCDs produce thermally-generated charge. They give non-zero output even in darkness. Partly, this is the dark current. Fluctuates randomly. How can we reduce dark current? From: Lecture Notes - EAAE 60
61
What is a pix-el? (0,0) x f(x,y) (x,y) y 63
Not a little square! A Pixel Is Not A Little Square, A Pixel Is Not A Little Square, A Pixel Is Not A Little Square! (And a Voxel is Not a Little Cube), Alvy Ray Smith, MS Tech Memo 6, Jul 17, 1995 64
Not a little square! Gaussian reconstruction filter Illustrations: Smith, MS Tech Memo 6, Jul 17, 1995 65
Not a little square! Illustrations: Smith, MS Tech Memo 6, Jul 17, 1995 Cubic reconstruction filter 66
Not a little square! Graphics: Dick Lyon, 2006 67
Sampling 1D Sampling in 1D takes a function, and returns a vector whose elements are values of that function at the sample points. 69
Sampling 2D Sampling in 2D takes a function and returns an array; we allow the array to be infinite dimensional and to have negative as well as positive indices. 70
Greyscale digital image 71
Nyquist Frequency Half the sampling frequency of a discrete signal processing system 72
Sampling grids 73
Retina-like sensors 74
Quantization Real valued function will get digital values integer values Quantization is lossy!! After quantization, the original signal cannot be reconstructed anymore This is in contrast to sampling, as a sampled but not quantized signal can be reconstructed. Simple quantization uses equally spaced levels with k intervals b k 2 75
Quantization 11 10 01 00 76
Quantization 11 10 01 00 77
Usual quantization intervals Grayvalue image 8 bit = 2^8 = 256 grayvalues Color image RGB (3 channels) 8 bit/channel = 2^24 = 16.7Mio colors 12bit or 16bit from some sensors Nonlinear, for example log-scale 78
Photo: Paulo Barcellos Jr. 79
Winding Down 80
Properties Image resolution Geometric resolution: How many pixel per area Radiometric resolution: How many bits per pixel 81
Image resolution 512x512 1024x1024 512x1024 82
Geometric resolution 144x144 72x72 36x36 18x18 9x9 4x4 83
Radiometric resolution 256 128 64 32 16 8 4 2 84
Lossless vs. Lossy Name some formats? 85
Aliasing and SNR What is the disadvantage of low sampling resolution? What is the disadvantage of high sampling resolution? 86
Finish Next week: Image segmentation 87
Unassessed Assignment Use matlab to change the geometric and radiometric quantization resolution in one of your images. For each level of sampling and quantization, plot the image function, as in slides 67 & 68, and compare the approximations to the true intensity function that you get at each level. 94