Digital Image Processing

Digital Image Processing Part : Image Enhancement in the Spatial Domain AASS Learning Systems Lab, Dep. Teknik Room T9 (Fr, - o'clock) achim.lilienthal@oru.se Course Book Chapter 3-4-

Contents. Image Enhancement in the Spatial Domain. Grey Level Transformations 3. Histogram Processing 4. Operations Involving Multiple Images 5.

Contents

Neighbourhood Relations Between Pixels a pixel has 4 or 8 neighbours in D depending on the neighbour definition: 4-neighborhood each neighbor must share an edge with the pixel 8- neighborhood each neighbor must share an edge or a corner with the pixel

Basics of the pixel value in the output image is calculated from a local neighbourhood in the input image the local neighbourhood is described by a mask with a typical size of 3x3, 5x5, 7x7, pixels filtering is performed by moving the mask over the image the centre pixel in the output image is given a value that depends on the input image and the weights of the mask

Basics of filter subimage defines coefficients w(s,t) used to update pixel at (x,y)

Linear filter subimage defines coefficients w(s,t) response of the filter at point (x,y) is given by a sum of products a g ( x, y) = w( s, t) f ( x + s, y + t) s= at= b also called convolution (convolution kernel) b (-,-) (-,) (-,) (,-) (,) (,) (,-) (,) (,)

Linear Implementation generic code: How to Deal With the Border? for P(x,y) in image for F(u,v) in filter Q(x,y) += F(u,v) P(x-u,y-v) end end limit excursion of the centre of the mask smaller image set outside pixel value zero border effects mirroring border pixel values border effects modify filter size along the border slower

Smoothing Spatial Filters (Averaging Filters) for blurring removal of small (irrelevant) details, bridging small gaps for noise reduction but: edges are also blurred

Smoothing Spatial Filters (Averaging Filters) for blurring removal of small (irrelevant) details, bridging small gaps for noise reduction Smoothing Spatial Filters 3x3 Mean Filter / Box Filter need for normalization to conserve the total energy of the image (sum of all greylevels) can cause "ringing" no good model of blurring in a defocused camera turns a single "point" into a "box" x /9

Smoothing Spatial Filters Mean Filter original Mean 5x5 Mean x

Linear in Matlab f = imread('bubbles.tif'); g = imfilter(f, w, filtering_mode, boundary_options, size_options); filter matrix w filtering modes 'corr' or 'conv' only important in the case of asymmetric filters 'corr' (no mirroring) is the default

Linear in Matlab f = imread('bubbles.tif'); g = imfilter(f, w, filtering_mode, boundary_options, size_options); boundary options P padding with (default) 'replicate' replicate values at the outer border 'symmetric' mirror reflecting across the outer border 'circular' repeating the image like a periodic function

Linear in Matlab f = imread('bubbles.tif'); g = imfilter(f, w, filtering_mode, boundary_options, size_options); size options 'same' same size as the input image (cropped padded image) 'full' full size of the padded image default is 'same'

Linear in Matlab f = imread('bubbles.tif'); g = imfilter(f, ones(8)/64, 'replicate'); imshow(g); 7 px

Linear in Matlab f = imread('bubbles.tif'); g = imfilter(f, fspecial('average',3), 'replicate'); imshow(g);

Smoothing Spatial Filters Mean Filter square box filter generates defects axis aligned streaks blocky results output input example from "A Gentle Introduction to Bilateral Filtering and its Applications", Sylvain Paris, Pierre Kornprobst, Jack Tumblin, and Frédo Durand, SIGGRAPH 8

box profile pixel weight pixel position unrelated pixels related pixels unrelated pixels from "A Gentle Introduction to Bilateral Filtering and its Applications", S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, SIGGRAPH 8

strategy to solve problems with box filters use an isotropic (i.e. circular) window use a window with a smooth falloff box kernel Gaussian kernel from "A Gentle Introduction to Bilateral Filtering and its Applications", S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, SIGGRAPH 8

Smoothing Spatial Filters Gaussian Filter weighted average D Gaussian kernel higher weight in the centre to decrease blurring Why a Gaussian? simple model of blurring in optical systems smooth /6 x also a Gaussian in the frequency domain 4

input from "A Gentle Introduction to Bilateral Filtering and its Applications", S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, SIGGRAPH 8

box average input from "A Gentle Introduction to Bilateral Filtering and its Applications", S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, SIGGRAPH 8

Gaussian box average input blur from "A Gentle Introduction to Bilateral Filtering and its Applications", S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, SIGGRAPH 8

Gaussian profile pixel weight pixel position unrelated pixels uncertain pixels related pixels uncertain pixels unrelated pixels from "A Gentle Introduction to Bilateral Filtering and its Applications", S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, SIGGRAPH 8

Gaussian profile spatial parameter σ input small σ large σ limited smoothing strong smoothing from "A Gentle Introduction to Bilateral Filtering and its Applications", S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, SIGGRAPH 8

Gaussian profile spatial parameter σ how to set σ? depends on the application common strategy: proportional to image size e.g. % of the image diagonal property: independent of image resolution depends on image content smooth "object areas" larger σ but don't smooth edges smaller σ from "A Gentle Introduction to Bilateral Filtering and its Applications", S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, SIGGRAPH 8

Smoothing Spatial Filters Median Filter take the values of the input image corresponding to the desired sub-window (3x3, 5x5, ) sort them take the middle value (example: 3x3 the 5th largest) forces pixels with distinct grey levels to be more like their neighbours very good at reducing salt-and-pepper noise less blurring than linear filters of the same size

Smoothing Spatial Filters Median Filter take the median value over the sub-window X ray image of a circuit board Average 3x3 Median 3x3

Median Filter in Matlab f = imread('bubbles.tif'); g = medfilt(f, [3 3]); imshow(g); original image median 8x8 median 3x3 median 3x

Smoothing Spatial Filters Median Filter take the median value over a sub-window original image median 3x3 median 5x5 median x

Smoothing Spatial Filters Median Filter take the values of the input image corresponding to the desired sub-window (3x3, 5x5, ) sort them take the middle value (example: 3x3 the 5th largest) forces pixels with distinct grey levels to be more like their neighbours very good at reducing salt-and-pepper noise less blurring than linear filters of the same size nonlinear filter (order statistics filter) no equivalent in the frequency domain (order statistics filter)

Order Statistics Filters (Fractile Filters) median min, max useful in mathematical morphology percentile generalization of median, min, max 3 7 5% percentile 4 5 3 4 5 7 8 8 min (%) median (5%) max (%)

Order Statistics Filters (Fractile Filters) median min, max useful in mathematical morphology percentile generalization of median, min, max order statistics filters are nonlinear filters order statistics filters do not have an equivalent in the frequency domain

Sharpening Spatial Filters highlight fine detail (also noise) enhance edges use image differentiation ( st order) f f( x+ ε, y) f( x, y) = lim x ε ε

Sharpening Spatial Filters highlight fine detail (also noise) enhance edges use image differentiation ( st order) f f( x+ ε, y) f( x, y) = lim x ε ε f f x f i+, j i, j

Sharpening Spatial Filters highlight fine detail (also noise) enhance edges use image differentiation Sharpening Spatial Filters D approximation to st Order Derivation equivalent to the D convolution mask f x f ( x + ) f ( x) -

Sharpening Spatial Filters highlight fine detail (also noise) enhance edges use image differentiation Sharpening Spatial Filters D approximation to st Order Derivation equivalent to the D convolution mask f f( x + ) f( x ) x -

5 Gradient and Magnitude of the Gradient Sharpening Spatial Filters Based on the Gradient Roberts Prewitt Sobel y f x f y f x f f + + = = y f x f f,

Sharpening Spatial Filters Prewitt gradient edge detector masks approximate G x and G y in - - - f f f + = G x + G y x y - -

Sharpening Spatial Filters Sobel Operators masks approximate G x and G y in detects horizontal and vertical edges f f f + = G x + G y x y - - - - - -

Sharpening Spatial Filters Sobel Operators weight is supposed to smooth by emphasizing the centre - - - - -

Sharpening Spatial Filters Sobel Operators detection of vertical dark-light edges - - -

Sharpening Spatial Filters Sobel Operators combination of all the directional responses

Sharpening Spatial Filters comparison between Sobel and Prewitt operator Sobel (~ G x + G y ) Prewitt (~ G x + G y )

Sharpening Spatial Filters Roberts (cross gradient operators) masks approximate G x and G y in f f f + = G x + G y x y - -

Sharpening Spatial Filters highlight fine detail (also noise) enhance edges uses image differentiation Sharpening Spatial Filters D approximation to st order derivation approximation to nd order derivation f = f ( x + ) + f ( x ) f ( x) x equivalent to the D convolution mask -

5 Sharpening Spatial Filters Laplace Filter Laplacian (second order derivative) y f x f + = ) ( ) ( ) ( x f x f x f x f + + =

5 Sharpening Spatial Filters Laplace Filter Laplacian (second order derivative) filter masks to implement the Laplacian add the "digital implementation" of the two terms in the Laplacian (9 rotation symmetry) y f x f + = -4 ) ( ) ( ) ( x f x f x f x f + + =

5 Sharpening Spatial Filters Laplace Filter Laplacian (second order derivative) filter masks to implement the Laplacian add the "digital implementation" of the two terms in the Laplacian (9 rotation symmetry) add also diagonal terms (45 rotation symmetry) y f x f + = -4-8 ) ( ) ( ) ( x f x f x f x f + + =

Sharpening Spatial Filters Laplace Filter Laplacian (second order derivative) = f x + f y f x = f ( x + ) + f ( x ) f ( x) filter masks to implement the Laplacian add the "digital implementation" of the two terms in the Laplacian (9 rotation symmetry) add also diagonal terms (45 rotation symmetry) -4 negative values re-scale -8

Sharpening Spatial Filters Laplace Filter detection of edges independent of direction isotropic with respect to 9 rotations - - 4 - -

Sharpening Spatial Filters Laplace Filter Laplace filter + original image sharpening - - 5 - -

Sharpening Spatial Filters Laplace filter + original image sharpening

Sharpening Spatial Filters nd order vs. st order Laplacian (second order derivative) f f f = + = f ( x + ) + f ( x ) f ( x) x y x thinner edges not so strong response to a step better response to fine details double response to edges rotation independent one mask for all edges

Sharpening Spatial Filters Unsharp Masking analog equivalent used in publishing industry basic idea: subtract blurred version of an image from original image to generate the edges

Sharpening Spatial Filters in Matlab f = imread('bubbles.tif'); g = imfilter(f, fspecial('laplacian',.5)); g = imfilter(f, fspecial('unsharp',.5)); %...

AASS Learning Systems Lab, Örebro University

Agenda. Image Smoothing Revisited. Bilateral Filtering 3. Applications of Bilateral Filtering 4. Efficient Implementation Heavily based on: "A Gentle Introduction to Bilateral Filtering and its Applications", Sylvain Paris, Pierre Kornprobst, Jack Tumblin, and Frédo Durand, SIGGRAPH 8, see http://people.csail.mit.edu/sparis/siggraph7_course and "Bilateral Filtering for Gray and Color Images", C. Tomasi, R. Manduchi, Proc. Int. Conf. Computer Vision

Image Smoothing Revisited Agenda

Image Smoothing Revisited General Strategy for Smoothing Images adjacent pixels tend to belong to the same object images typically vary slowly over space non-smoothness due to noise noise values less correlated than the signal smoothing = making adjacent pixels look more similar smoothing strategy: pixel average over its neighbors however, basic assumption not true at edges we are interested in edge preserving image smoothing

Image Smoothing Revisited Mean Filter Profile pixel weight pixel position unrelated pixels related pixels unrelated pixels

Image Smoothing Revisited Gaussian Filter Profile pixel weight pixel position unrelated pixels uncertain pixels related pixels uncertain pixels unrelated pixels

Image Smoothing Revisited Gaussian Filter linear convolution weights independent of spatial location well-known operation can be computed efficiently does smooth images but smoothes too much: edges are blurred only spatial distance matters, no edge term

Bilateral Filtering Agenda

Definition of the Bilateral Filter Blur from Averaging Across Edges input * output * * same Gaussian kernel everywhere

Definition of the Bilateral Filter Bilateral Filter Limits Averaging Across Edges input * output * [Aurich 95, Smith 97, Tomasi 98] * kernel shape depends on image content

Definition of the Bilateral Filter Bilateral Filter The Additional Edge Term again: weighted average of pixels. BF new q S not new [ I] = p Gσ r W s σ p new ( p q ) G ( I I ) p q I q normalization factor space weight range weight I please note the different notation: I f

D Illustration Bilateral Filter D Illustration D image = line of pixels better visualized as a plot pixel intensity pixel position

D Illustration Gaussian blur p q GB I = G [ ] p σ q S ( p q ) space I q space Bilateral filter [Aurich 95, Smith 97, Tomasi 98] q p range BF [ I] p = Gσ W s σ r p q S space normalization ( p q ) G ( I I ) p range q I q space

Bilateral Filtering Is This a Linear Filter? BF [ I] = p Gσ r W s σ p q S ( p q ) G ( I I ) p q I q

Bilateral Filtering No! BF[I ] p + BF[I ] p BF[I +I ] p BF [ I] = p Gσ r W s σ p q S ( p q ) G ( I I ) p q I q

D Illustration BF [ I ]p = Wp Gσ ( p q ) q S s Gσ r ( I p I q ) I q p p q output reproduced from [Durand ] input