Table of contents Vision industrielle 2002/2003 Session - Image Processing Département Génie Productique INSA de Lyon Christian Wolf wolf@rfv.insa-lyon.fr Introduction Motivation, human vision, history, digital images, methodology Image Acquisition Image formation, sensors, cameras, scanners, illumination, color Fourier Transformation Noise, mage filtering in the frequency domain, correlation Image Processing linear and non linear filtering, histograms Image Analysis thresholding, gradients, laplacian, edge detection, region segmentation Mathematical morphology, 3D binary images, erosion, dilation, neighborhood transformations, stereo vision, shape from X Questions Questions to the Lecture PDF files: http://rfv.insa-lyon.fr/~wolf/enseignement 2 Local and semi-local smoothing Linear noise filtering: example Instead of restoration (by inverting the degradation effects), local treatment on the image enhances its quality. Original image with noise Filtered image Examples: Salt and pepper noise Holes or cuts in the object contours... Replace each pixel by the mean of its neighbors: f'(x,y)=( f(x-,y-) + f(x,y-) + f(x-,y) + f(x,y) )/ Replace each pixel by the mean of its neighbors: u= v= F( x, y) = f( x+ v, y+ u) u= v= 3 Mean filter h = : introduction D convolution The continuous case: Commutative operation! The discrete case: Roberts filter h r h l = 0 0 = 0 0 f h x origin h(x) = filter kernel Filtered pixel 5 6
2D convolution Why flip the kernel? The discrete case: h f f h h 0 y N- x 0 M- f y x f h y x flipped kernel Reason: so that f h = h f 7 8 The impulse response (IR) 2D Dirac image : D case: 2D case: Commutativity: Properties The impulse response of the operator : Associativity (combination of filters): The filter kernel equals the response of the filter to a Dirac impulse: Distributivity (over addition): Transfer function: the Fourier transform of the impluse response: H=FT(h) Contains for each frequency the complex factor by which a periodic structure is multiplicated. Linearity: α,β... scalars 0 Shift invariance: Shift operator: Properties Linear and shift-invariant operators Thought experiment: what kind of linear and shift-invariant operators can there exist? Let s consider an operator with following properties: Linearity Shift invariance h is the impulse response of this operator 2
Iterative application A linear and shift invariant operator is equivalent to a convolution with the impulse response of the operator!!! Iteratively convolving an image N times by a filter of size M corresponds to the application of a single convolution of a filter of size M* = 2 ( (M-) / 2 ) N ) + 2 x a filter of size 3 = x filter of size 5 3 x a filter of size 3 = x filter of size 7 One kernel: may reduce complexity Several kernels: may reducing programming effort 2 2 g3 = 2 2 6 2 g 5 = 6 26 6 6 2 6 2 36 2 6 6 2 6 6 3 Separability The Gaussian and the average filter are separable: f g = (( f gx) gy) = ( f gy) gx) in the frequency domain in the spatial domain corresponds to a multiplication in the frequency domain: 2 G = 2 2 6 2 2 Gy = 2 Gx = [ 2 ] 2 [ 2 ] 2 2 since = 2 B = Bx = By = [ ] 25 5 5 since [ ] = f(x, y) h(x, y) F(u, v).h(u, v) Can lead to a large decrease in computational complexity (depending on the filter operations) Example: filter bank etc. Advantage: Computational complexity for a MxN filter: Non-separable implementation: MxN operations Separable implementation: M+N operations 5 6 Border treatment What do we do with missing pixels on the border? 0 M- Cyclic replacement (only option in the frequency domain!!) Zeros The average filter ( blur ) The obvious filter, corresponds to a convolution with the following kernel: g = g = u= v= F(x,y)=f g F( xy, ) = f( x+ vy, + u) u= v= Interpolation (linear, quadratic,...) 7 8
Increasing the filter size ( blur more ) Gaussian low pass filter ( soften ) g = 25 Bigger kernel = more blur Discrete approximation of a 2D Gaussian filter: Weighted mean, the center pixels are more important than the border pixels Gxy (, )= 2πσxσy e 2 2 x y + 2 σx σy 2 g = 2 2 6 2 Computational complexity! g 5 = 6 26 6 6 2 6 2 36 2 6 2 6 6 6 20 Comparison: Gaussian & average filter Gaussian low pass filter: details Original image Average filter Gaussian filter 2 22 Sharpen (rehaussement) Sharpened image = (A-) original image + high pass filtered image 0 0 0 8 0 Laplacian filter (high pass filter) A=: standard laplacian filter A>: a part of the original image is added to the high pass Strong sharpen Stronger sharpening lets the noise in the image appear! A=2 Noise!!! Example: scanner noise appears after sharpening 23 2
... Adaptive smoothing The coefficients in the filter mask depend on the data: Pixel to filter Position in the mask Problem: edge smoothing of linear filters The min/max filter Goal: Contrast enhancement The original pixel value is compared to the minimum of the pixels in the neighborhood (=m) and the maximum (=M): Discontinuity function: Normalization constant: 2 25 32 8 8 25 36 57 Minimum m=2 Maximum M=57 The center value is closer to m, i.e. the new value is m=2. 25 26 Original image Min/max: example Filtered image The median filter Non-linear filter developed in order to preserve edges. Chooses the median value of the the neighboring pixels, i.e. the value at rank K/2 in the ordered list of gray values. 2 25 32 8 8 25 36 57 K/2 K replace by 25 2 8 25 25 32 36 8 57 Problem: edge smoothing of linear filters The median filter is very efficient against noise, avoids blurring and preserves colors transitions around the objects. 27 28 The median filter Example: salt and pepper noise Original image Filtered with the median filter The mode filter The mean value is not taken from the whole ordered list L of gray values, but from a well chosen compact part of size K/2: 2 25 32 8 8 25 36 57 2 8 25 25 32 36 8 57 2 8 25 2 8 25 25 8 25 32 25 From the chosen interval, the median value is taken 2 30
The symmetric nearest neighbor filter Goal: smoothing regions while preserving edges and thin structures. P 2 P 3 P P M Q Examples Original image SNN iteration SNN 3 iterations Q Q 3 Q 2 Step : For each couple (P i, Q i ): Step 2: mean or median 3 32 Image enhancement The image enhancement problem is not well defined. When is an image better than another? Contrast enhancement: Frequency filtering (as in image restoration) Histogram modification Color processing Contrast techniques Histogram techniques Statistical distribution of the pixel values f(x,y) independent of their position: Appearance frequency of each gray level (color) Pixel count Histo(i)=Card{Pixel(x,y)=i} grayvalues Operations: f (x,y) = T(f(x,y)) T is a function on the gray value, independent of the pixel position! 33 3 Gray level correction functions Stretch (recadrage) f (x,y) = T(f(x,y)) Dilation of the bright zones Gray value correction functions Correction of the nonlinearity of film or monitor Extraction of an intensity window Dilation of the dark zones Suppression of the 2 highest bits in the gray value. Inversion of the image 7 6 5 3 2 0 35 36
Redistribute the colors [a..b] between [0..255] Histogram stretch Histogram equalization Redistribute the colors linearly between [0..255]: The gray levels most frequently used get a higher dynamic range The cumulative distribution function 83 gray levels are redistributed into the whole dynamic range of 0..255 A uniform histogram is created! Alternative histogram shapes ( histogram specification ): Exponential Cubic Logarithmic 37 38 Histogram equalization Histogram equalization: example Contrast is increased at the most populated range of brightness values of the histogram (or "peaks"). Automatically reduces the contrast in very light or dark parts of the image associated with the tails of a normally distributed histogram. Equalization maximizes the entropy in the image. The histogram is not totally flat, due to the discrete nature of the histogram. A really flat histogram can be computed by separating classes. 3 0 Histogram equalization: example Iterative histogram peak sharpening (Amincissement itératif d histogramme) Shift pixels between histograms so that bins that are larger than their neighborhood receive pixels from this neighborhood. For each bin i, consider the neighborhood [i-m, i+m]: if H(i) > A A = mean (H(i-m),...,H(i+m)): r = (H(i)-A)/H(i) move rh(i+j) pixels from i+j to i+j- move rh(i-j) pixels from i-j to i-j+ for j =..m for j =..m i-m i i+m 2
Pseudo Colors A color image is created from a grayscale image by assigning colors to different gray values. Background: humans are able to distinguish only a few gray values but millions of different colors. False colors Modification of the colors of an image which is originally in color: Create unnatural colors (purple sun, green sky,...) Change the spectrum to match human perception Emphasize single objects by changing their colors (contrast against the background).... Example: composite image 3