Part 1: Course Introduction Achim J. Lilienthal AASS Learning Systems Lab, Dep. Teknik Room T1209 (Fr, 11-12 o'clock) achim.lilienthal@oru.se Course Book Chapters 1 & 2 2011-04-05
Contents 1. Introduction digital images human visual perception, optical illusions, e-m spectrum example application person tracking with mobile robots example image understanding tiny images approach 2. Course Contents 3. Digital Image Acquisition image formation model image sampling and quantization, zooming and shrinking
Contents Introduction Digital Images
Introduction Digital Images Digital Images a finite set of digital values (picture elements = pixels) each pixel is associated to a position in a 2D region each pixel has a value digital image of a rat magnification of the rat s nose
Introduction Digital Images Digital Images can be thought of as a matrix (raster image / raster map) of grey levels / intensity values magnification of the rat s nose 94 100 104 119 125 136 143 153 157 158 103 104 106 98 103 119 141 155 159 160 109 136 136 123 95 78 117 149 155 160 110 130 144 149 129 78 97 151 161 158 109 137 178 167 119 78 101 185 188 161 100 143 167 134 87 85 134 216 209 172 104 123 166 161 155 160 205 229 218 181 125 131 172 179 180 208 238 237 228 200 131 148 172 175 188 228 239 238 228 206 161 169 162 163 193 228 230 237 220 199
Introduction Digital Images Digital Images types dimensionality and nature of pixel values binary (bilevel) grey scale color false-color multi-spectral semantic (thematic),... 3D Digital Images picture elements are called voxels (from "volumetric" and "pixel") not addressed here
Introduction Electromagnetic Spectrum The Electromagnetic Spectrum we perceive only a small range of colours of the electromagnetic spectrum (~ 430nm 790nm) gamma rays, X rays, ultraviolet light, visible spectrum, infrared, microwaves, radio waves,...
Introduction Electromagnetic Spectrum The Electromagnetic Spectrum fundamental equations relation between wavelength (λ) and frequency (ν): relation between energy and frequency: E = hν λ = c ν
Introduction Electromagnetic Spectrum The Electromagnetic Spectrum we perceive only a small range of colours of the electromagnetic spectrum (~ 430nm 790nm) objects are perceived by the light they reflect achromatic light: all wavelengths are reflected equally chromatic light: some wavelengths are reflected predominantly
Contents Introduction Biological Vision
Introduction Visual Perception Metaphysics All men by nature desire to know. An indication of this is the delight we take in our senses; for even apart from their usefulness they are loved for themselves; and above all others the sense of sight. Aristotle (384 BC 322 BC)
Introduction Visual Perception The Human Eye
Introduction Visual Perception What happens? photons are reflected at objects pattern of reflected photons is sensed biological vision: with photoreceptors ( pixel) computer vision: with a (digital) camera and further processed as a multidimensional signal biological vision: in the visual cortex computer vision: DIP, computer vision Vision from Per-Erik Forssén "Visual Object Detection"
Introduction Visual Perception Image Formation Pinhole Camera Model from Per-Erik Forssén "Visual Object Recognition"
Introduction Visual Perception Image Formation Pinhole Camera Model focal length between 17 mm (min. refractive power, objects farther than 3m) and 14 mm (max. refractive power) 15 / 100 = h / 17 h = 2.55 mm focal length (min. refractive power)
Introduction Visual Perception The Human Eye sphere (diameter ~ 20 mm)
Introduction Visual Perception The Human Eye cornea constant thickness lens with fixed focal length responsible for ~ 75% of the refraction
Introduction Visual Perception The Human Eye cornea lens constant thickness lens with fixed focal length responsible for ~ 75% of the refraction can be contracted zoom (to a plane) shape of lens is varied to focus on objects at different distances IR and UV light are absorbed by proteins in the lens structure
Introduction Visual Perception The Human Eye cornea lens constant thickness lens with fixed focal length responsible for ~ 75% of the refraction can be contracted zoom (to a plane) 2D image on the retina represents the light pattern reflected from a thin plane in the 3D spatial world, the lens is focused on
Introduction Visual Perception The Human Eye pupil opening varies from 2 to 8 mm regulates the amount of light reaching the retina
Introduction Visual Perception The Human Eye pupil opening varies from 2 to 8 mm regulates the amount of light reaching the retina aperture of a camera source: Wikipedia (http://en.wikipedia.org/wiki/aperture)
Introduction Visual Perception The Human Eye pupil opening varies from 2 to 8 mm regulates the amount of light reaching the retina aperture of a camera light reaches the retinal surface (spherical, inner wall of the eyeball) photoreceptors "translate" light into electrical pulses distributed over the retinal surface non-uniform resolution
Introduction Visual Perception Foveal/Peripheral View
Introduction Visual Perception Foveal/Peripheral View
Introduction Visual Perception Foveal/Peripheral View
Introduction Visual Perception Foveal/Peripheral View
Introduction Visual Perception The Human Eye pupil opening varies from 2 to 8 mm regulates the amount of light reaching the retina aperture of the eye light reaches the retinal surface (spherical, inner wall of the eyeball) photoreceptors are distributed over the retinal surface cones & rods
Introduction Visual Perception The Human Eye two classes of light receptors distributed over the retinal surface cones (bright-light vision phototopic) 6-7 million around fovea colour & bright-light vision fine details cones with peak sensitivity for long, medium and short wavelengths (red, green, blue) only cones in the fovea
Introduction Visual Perception The Human Eye two classes of light receptors distributed over the retinal surface cones (bright-light vision phototopic) 6-7 million around fovea colour & bright-light vision fine details red, green, blue rods (dim-light vision scotopic) 75-150 million coarse details "night vision"
Introduction Visual Perception Receptor Distribution in the Human Eye no receptors where the optic nerve emerges (blind spot) radially symmetric distribution around the fovea except from the blind spot distribution of rods and cones around the fovea
Introduction Visual Perception Why do we sometimes have red eyes in photos?
Introduction Visual Perception
Introduction Visual Perception The Fovea responsible for sharp vision (reading, watching television,...) circular indentation (diameter ~ 15 mm) approx. 330 000 cones in this area (~ a 15 x 15 mm 2 square sensor)
Introduction Visual Perception The Fovea responsible for sharp vision (reading, watching television,...) circular indentation (diameter ~ 15 mm) approx. 330 000 cones in this area (~ a 15 x 15 mm 2 square sensor) resolution that can be achieved with a CCD chip? 10 MP camera 7.2 x 5.3 mm 2 (260 000 pixels / mm 2 ) 590 000 "pixels" on 1.5 x 1.5 mm 2 (260 000 "pixels" / mm 2 )
Introduction Visual Perception Receptor Position in the Human Eye photo-receptors turned away from the lens!
Introduction Brightness Adaptation in the Human Eye human eye can adapt over 10 orders of magnitude! 6 orders in phototopic vision (cones) accomplished by brightness adaptation (changes in the overall sensitivity) much smaller range for each brightness adaptation level B a subjective brightness is a log function of the light intensity brightness discrimination poor at low levels of illumination better with increasing illumination
Introduction Sensation vs Perception Ganglion Cells 125 million rods & cones 1 million ganglion cells implement local neighbourhood operations (local receptive field) respond if there is a difference between "center and surround" (center-surround cells) contrast-sensitive vision absolute intensity / color not available to the brain important for colour constancy
Introduction Visual Perception Image Formation in the Human Eye perceived breightness is not a simple function of intensity! Mach bands stripes appear darker near a more intense stripe (and vice versa) caused by inhibitory neural connections
Introduction Visual Perception Image Formation in the Human Eye perceived breightness is not a simple function of intensity! Mach bands stripes appear darker near a more intense stripe (and vice versa) caused by inhibitory neural connections simultaneous contrast a regions' perceived breightness depends on the intensity in the neighbourhood
Introduction Visual Perception perceived breightness is not a simple function of intensity! simultaneous contrast a regions perceived breightness depends on the intensity in the neighbourhood
Introduction Sensation vs Perception Sensation operation of basic sensory systems result of physical stimuli and low-level processes Perception involve higher-level processes in the percipient memories expectations emotions state of fatigue or alertness "The Great Ideas of Psychology" (TTC)
Introduction Visual Perception Biological Vision development responded to evolutionary necessities
Introduction Visual Perception Biological Vision bear pixels?
Introduction Visual Perception Importance of Context Torralba et al., CVPR 2007, Short Course
Introduction Visual Perception Importance of Context Torralba et al., CVPR 2007, Short Course
Introduction Visual Perception Image Formation in the Human Eye perceived breightness is not a simple function of intensity! Mach bands stripes appear darker near a more intense stripe (and vice versa) caused by inhibitory neural connections simultaneous contrast a regions perceived breightness depends on the intensity in the neighbourhood optical illusions
Introduction Optical Illusions Optical Illusions the eye / brain fills in nonexisting information perceives geometrical properties of an object wrongly characteristic of the human visual system and not yet fully understood... (some examples follow)
Introduction Optical Illusions concentrate on the dot in the middle...... and move your head back and forth
Introduction Optical Illusions movement created only in the brain
Introduction Optical Illusions concentrate on the cross in the middle...... and the moving circle turns green!... after a while the violet circles disappear!!
Introduction Optical Illusions 1. Relax and stare for 30s - 45s to the four dots in the centre 2. Then look slowly to a white wall (large uniformly coloured area) close to you 3. You will see a bright spot forms at the wall 4. Now blink a few times 5. What do you see? Whom do you see?
Introduction Optical Illusions
Contents Introduction Image Processing
1 Introduction Image Processing Image Processing versus Image Analysis world imaging image analysis data image computer graphics knowledge image understanding, computer vision image processing
1 Introduction Image Processing Image Processing versus Image Analysis world visualisation imaging image analysis image processing data image computer graphics knowledge image understanding, computer vision
Introduction Image Processing Fundamental Steps in problem Lara Croft has to get out of a room
Introduction Image Processing Fundamental Steps in problem image acquisition
Introduction Image Processing Fundamental Steps in problem image acquisition preprocessing
Introduction Image Processing Fundamental Steps in problem image acquisition preprocessing segmentation
Introduction Image Processing Fundamental Steps in problem image acquisition preprocessing segmentation representation and description model of objects
Introduction Image Processing Fundamental Steps in problem image acquisition preprocessing segmentation representation and description model of objects recognition and interpretation what are these objects?
Introduction Image Processing Fundamental Steps in problem image acquisition preprocessing segmentation representation and description model of objects recognition and interpretation what are these objects? solution
Contents Course Contents
2 Course Contents Filtering in the Spatial Domain (Image Enhancement) "Lena" with noise Median filtering edge detection
2 Course Contents Fourier Transform original image power spectrum after Fourier transformation inverse transform of filtered power spectrum
2 Course Contents Image Restoration?
2 Course Contents Binary Image Operations original image thresholding closing
2 Course Contents Segmentation? original image segmented (binary) image
2 Course Contents Morphological Image Processing & Shape Description... after morphological closing grey image... after segmentation... after skeletonization
2 Course Contents Colour Representation and Use RGB space CIE s chromaticity diagram
2 Course Contents Classification and Introduction to Pattern Recognition? original image result of classification
Contents Digital Image Acquisition
3 Digital Image Acquisition Digital Image Representation f(x,y) as a matrix of real numbers f (0,0) f (0,1)... f (0, N 1) f (1,0) f (1,1)... f (1, N 1) f ( x, y) = = ( aij ) f ( M 1,0) f ( M 1,1)... f ( M 1, N 1) elements of the matrix are called pixels (2D)
3 Image Formation and Image Sampling Image Formation Model illumination i(x,y) from a source reflectivity r(x,y) = reflection / absorption in the scene f(x,y) = i(x,y) r(x,y) i ~ 0.1 lm/m 2 (full moon) 1000 lm/m 2 (office) 10'000 lm/m 2 (cloudy day) 90'000 lm/m 2 (sunny day)
3 Image Formation and Image Sampling Image Formation Model illumination i(x,y) from a source reflectivity r(x,y) = reflection / absorption in the scene f(x,y) = i(x,y) r(x,y) r = 0.01 (black velvet) 0.65 (stainless steel) 0.80 (flat white wall) 0.90 (silver-plated metal) 0.93 (snow)
3 Image Formation and Image Sampling Image Formation Model illumination i(x,y) from a source reflectivity r(x,y) = reflection / absorption in the scene f(x,y) = i(x,y) r(x,y) Image Sampling digital image can be seen as a 2D function f(x,y) x and y are the spatial coordinates f(x,y) is the grey level / intensity at position (x,y) a digital image must be sampled (digitized) in space (x,y): image sampling in amplitude f(x,y): grey-level quantization
3 Digital Image Acquisition Image Sampling and Quantization conversion of continuous input signal to a digital form continuous signal digitized image
3 Digital Image Acquisition Image Sampling and Quantization conversion of continuous input signal to a digital form sample f(x,y) in both coordinates (sampling) continuous signal
3 Digital Image Acquisition Image Sampling and Quantization conversion of continuous input signal to a digital form sample f(x,y) in both coordinates (sampling) continuous signal
3 Digital Image Acquisition Image Sampling and Quantization conversion of continuous input signal to a digital form sample f(x,y) in both coordinates (sampling) sample f(x,y) in amplitude (quantization)
3 Digital Image Acquisition Image Sampling uniform same sampling frequency everywhere adaptive higher sampling frequency in areas with greater detail (not very common) determines the spatial resolution
3 Digital Image Acquisition Image Sampling spatial resolution: smallest discernible detail in the image (line pairs per mm, for example) 512 256 128 64 32
3 Digital Image Acquisition Image Quantisation greylevel quantization 32 256 82
Part 1: Course Introduction Achim J. Lilienthal Thank you! AASS Learning Systems Lab, Dep. Teknik Room T1209 (Fr, 11-12 o'clock) achim.lilienthal@oru.se Course Book Chapters 1 & 2 2011-04-05