Linear Filters Tues Sept 1 Kristen Grauman UT Austin. Announcements. Plan for today 8/31/2015. Image noise Linear filters. Convolution / correlation

Transcription

1 8/3/25 Linear Filters Tues Sept Kristen Grauman UT Austin Announcements Piazza for assinment questions A due Friday Sept 4. Submit on Canvas. Plan for today Imae noise Linear filters Examples: smoothin filters Convolution / correlation

2 8/3/25 Imae Formation Slide credit: Derek Hoiem Diital camera A diital camera replaces film with a sensor array Each cell in the array is liht-sensitive diode that converts photons to electrons Slide by Steve Seitz Diital imaes Sample the 2D space on a reular rid Quantize each sample (round to nearest inteer) Imae thus represented as a matrix of inteer v alues. 2D D Adapted from S. Seitz 2

3 8/3/25 Diital color imaes Diital color imaes Color imaes, RGB color space R G B Imaes in Matlab Imaes represented as a matrix Suppose we have a NxM RGB imae called im im(,,) = top-left pixel value in R-channel im(y, x, b) = y pixels down, x pixels to riht in the b th channel im(n, M, 3) = bottom-riht pixel in B-channel imread(filename) returns a uint8 imae (values to 255) Convert to double format (values to ) with im2double row column Slide credit: Derek Hoiem R G B 3

4 8/3/25 Main idea: imae filterin Compute a function of the local neihborhood at each pixel in the imae Function specif ied by a f ilter or mask say in how to combine v alues f rom neihbors. Uses of filterin: Enhance an imae (denoise, resize, etc) Extract inf ormation (texture, edes, etc) Detect patterns (template matchin) Adapted from Derek Hoiem Motivation: noise reduction Ev en multiple imaes of the same static scene will not be identical. Common types of noise Salt and pepper noise: random occurrences of black and white pixels Impulse noise: random occurrences of white pixels Gaussian noise: variations in intensity drawn from a Gaussian normal distribution Source: S. Seitz 4

5 8/3/25 Gaussian noise >> no ise = ra ndn( size (im) ).*s ima ; >> ou tput = i m + nois e; What is impact of the sima? Fi: M. Hebert Motivation: noise reduction Ev en multiple imaes of the same static scene will not be identical. How could we reduce the noise, i.e., iv e an estimate of the true intensities? What if there s only one imae? First attempt at a solution Let s replace each pixel w ith an averae of all the values in its neihborhood Assumptions: Expect pixels to be like their neihbors Expect noise processes to be independent from pixel to pixel 5

6 8/3/25 First attempt at a solution Let s replace each pixel w ith an averae of all the values in its neihborhood Movin averae in D: Source: S. Marschner Weihted Movin Averae Non-uniform w eihts [, 4, 6, 4, ] / 6 Source: S. Marschner Movin Averae In 2D Source: S. Seitz 6

7 8/3/25 Correlation filterin Say the av erain window size is 2k+ x 2k+: Attribute uniform weiht to each pixel Loop over all pixels in neihborhood around imae pixel F[i,j] Now eneralize to allow different weihts dependin on neihborin pixel s relativ e position: Non-uniform w eihts Correlation filterin This is called cross-correlation, denoted Filterin an imae: replace each pixel with a linear combination of its neihbors. The f ilter kernel or mask H[u,v] is the prescription f or the weihts in the linear combination. Averain filter What v alues belon in the kernel H f or the mov in av erae example? ? box filter

8 8/3/25 Smoothin by averain depicts box filter: white = hih value, black = low value oriinal filtered What if the filter size was 5 x 5 instead of 3 x 3? Boundary issues What is the size of the output? MATLAB: output size / shape options shape = full : output size is sum of sizes of f and shape = same : output size is same as f shape = valid : output size is difference of sizes of f and full same valid f f f Source: S. Lazebnik Boundary issues What about near the ede? the filter window falls off the ede of the imae need to extrapolate methods: clip filter (black) wrap around copy ede reflect across ede Source: S. Marschner 8

9 8/3/25 Boundary issues What about near the ede? the filter window falls off the ede of the imae need to extrapolate methods (MATLAB): clip filter (black): imfilter(f,, ) wrap around: imfilter(f,, circular ) copy ede: imfilter(f,, replicate ) reflect across ede: imfilter(f,, symmetric ) Source: S. Marschner Gaussian filter What if we want nearest neihborin pixels to hav e the most inf luence on the output? This kernel is an approximation of a 2d Gaussian f unction: Remov es hih-f requency components f rom the imae ( low-pass f ilter ). Source: S. Seitz Smoothin with a Gaussian 9

10 8/3/25 Gaussian filters What parameters matter here? Size of kernel or mask Note, Gaussian f unction has inf inite support, but discrete f ilters use f inite kernels σ = 5 w ith x kernel σ = 5 w ith 3 x 3 kernel Gaussian filters What parameters matter here? Variance of Gaussian: determines extent of smoothin σ = 2 w ith 3 x 3 kernel σ = 5 w ith 3 x 3 kernel Matlab >> hsize = ; >> sima = 5; >> h = fspecial( aussian hsize, sima); >> mesh(h); >> imaesc(h); >> outim = imfilter(im, h); % correlation >> imshow(outim); outim

11 8/3/25 Smoothin with a Gaussian Parameter σ is the scale / width / spread of the Gaussian kernel, and controls the amount of smoothin. for sim a=:3 : h = fspe cial( 'auss ian, fsize, si ma); out = im filte r(im, h); imshow(o ut); pause; end Keepin the two Gaussians in play straiht More noise Wider smoothin kernel Slide credit: David Forsyth Properties of smoothin filters Smoothin Values positive Sum to constant reions same as input Amount of smoothin proportional to mask size Remove hih-frequency components; low-pass filter

12 8/3/25 Filterin an impulse sinal What is the result of f ilterin the impulse sinal (imae) F with the arbitrary kernel H? a b c d e f h i? Convolution Conv olution: Flip the filter in both dimensions (bottom to top, riht to left) Then apply cross-correlation F Notation for convolution operator H Conv olution Convolution vs. correlation Cross-correlation For a Gaussian or box f ilter, how will the outputs dif f er? If the input is an impulse sinal, how will the outputs dif f er? 2

13 8/3/25 Predict the outputs usin correlation filterin * =? * =? 2 - * =? Practice with linear filters? Oriinal Source: D. Lowe Practice with linear filters Oriinal Filtered (no chane) Source: D. Lowe 3

14 8/3/25 Practice with linear filters? Oriinal Source: D. Lowe Properties of convolution Shift invariant: Operator behav es the same ev ery where, i.e. the v alue of the output depends on the pattern in the imae neihborhood, not the position of the neihborhood. Superposition: h * (f + f 2) = (h * f ) + (h * f 2) Properties of convolution Commutativ e: f * = * f Associativ e (f * ) * h = f * ( * h) Distributes ov er addition f * ( + h) = (f * ) + (f * h) Scalars f actor out kf * = f * k = k(f * ) Identity : unit impulse e = [,,,,,, ]. f * e = f 4

15 8/3/25 Separability In some cases, f ilter is separable, and we can f actor into two steps: Conv olv e all rows Conv olv e all columns Separability In some cases, f ilter is separable, and we can f actor into two steps: e.., h What is the computational complexity advantae for a separable filter of size k x k, in terms of number of operations per output pixel? f f * ( * h) = (f * ) * h Effect of smoothin filters Additive Gaussian noise Salt and pepper noise 5

16 8/3/25 Median filter No new pixel v alues introduced Remov es spikes: ood f or impulse, salt & pepper noise Non-linear f ilter Median filter Salt and pepper noise Median filtered Plots of a row of the imae Matlab: output im = medf ilt2(im, [h w]); Source: M. Hebert Median filter Median f ilter is ede preserv in 6

17 8/3/25 Filterin application: Hybrid Imaes Aude Oliva & Antonio Torralba & Philippe G Schyns, SIGGRAPH 26 Application: Hybrid Imaes Gaussian Filter A. Oliva, A. Torralba, P.G. Schyns, Hybrid Imaes, SIGGRAPH 26 Laplacian Filter unit impulse Gaussian Laplacian of Gaussian Aude Oliva & Antonio Torralba & Philippe G Schyns, SIGGRAPH 26 7

18 8/3/25 Aude Oliva & Antonio Torralba & Philippe G Schyns, SIGGRAPH 26 Summary Imae noise Linear filters and convolution useful for Enhancin imaes (smoothin, remov in noise) Box filter Gaussian filter Impact of scale / width of smoothin filter Detectin f eatures (next time) Separable filters more efficient Median filter: a non-linear filter, ede-preservin Comin up Thursday: Filterin part 2: filterin for features (edes, radients, seam carvin application) See readin assinment on w ebpae Friday: Assinment is due on Canvas :59 PM 8