02/02/10. Image Filtering. Computer Vision CS 543 / ECE 549 University of Illinois. Derek Hoiem

Transcription

1 2/2/ Image Filtering Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem

2 Questions about HW?

3 Questions about class? Room change starting thursday: Everitt 63, same time

4 Key ideas from last class Lighting Ambiguity between light source and albedo Shading is a strong cue for shape Interreflections, multiple sources, ambient light, etc. make lighting and shadows complicated Color constancy Color can be rebalanced by making assumptions (e.g., average pixel is gray)

5 Lightness Perception from Ted Adelson

6 Lightness Perception from Ted Adelson

7 By nickwheeleroz, on Flickr

8 By nickwheeleroz, on Flickr

9 Karsch et al. in review

10 Today s class How can we represent color? What is image filtering and how do we do it? What are some useful filters and what do they do? What is linear separability? Thinking in the frequency domain

11 Color spaces How can we represent color?

12 Color spaces: RGB,, Some drawbacks Strongly correlated channels Non-perceptual,,,, Image from:

13 Color spaces: HSV

14 Color spaces: L*a*b* Perceptually uniform color space

15 If you had to choose, would you rather go without luminance or chrominance?

16 If you had to choose, would you rather go without luminance or chrominance?

17 Most information in intensity Only color shown constant intensity

18 Most information in intensity Only intensity shown constant color

19 Most information in intensity Original image

20 The raster image (pixel matrix)

21 The raster image (pixel matrix)

22 Image filtering Image filtering: compute function of local neighborhood at each position Why bother? Modify images Denoise, resize, enhance contrast, etc. Extract information from images Edge detection, matching, find distinctive points, etc.

23 Example: box filter g[, ] Slide credit: David Lowe (UBC)

24 Image filtering g[, ] f [.,.] h[.,.] h[ m, n] = k, l f [ k, l] g[ m + k, n + l] Credit: S. Seitz

25 Image filtering g[, ] f [.,.] h[.,.] h[ m, n] = k, l f [ k, l] g[ m + k, n + l] Credit: S. Seitz

26 Image filtering g[, ] f [.,.] h[.,.] h[ m, n] = k, l f [ k, l] g[ m + k, n + l] Credit: S. Seitz

27 Image filtering g[, ] f [.,.] h[.,.] h[ m, n] = k, l f [ k, l] g[ m + k, n + l] Credit: S. Seitz

28 Image filtering g[, ] f [.,.] h[.,.] h[ m, n] = k, l f [ k, l] g[ m + k, n + l] Credit: S. Seitz

29 Image filtering g[, ] Q? f [.,.] h[.,.] h[ m, n] = k, l f [ k, l] g[ m + k, n + l] Credit: S. Seitz

30 Box Filter What does it do? g[, ] Replaces each pixel with an average of its neighborhood Achieve smoothing effect (remove sharp features) Slide credit: David Lowe (UBC)

31 Smoothing with box filter

32 Practice with linear filters? Original Source: D. Lowe

33 Practice with linear filters Original Filtered (no change) Source: D. Lowe

34 Practice with linear filters? Original Source: D. Lowe

35 Practice with linear filters Original Shifted left By pixel Source: D. Lowe

36 Practice with linear filters 2 -? Original (Note that filter sums to ) Source: D. Lowe

37 Practice with linear filters 2 - Original Sharpening filter - Accentuates differences with local average Source: D. Lowe

38 Sharpening Source: D. Lowe

39 Other filters 2 Sobel Vertical Edge (absolute value)

40 Other filters Q? Sobel - Horizontal Edge (absolute value)

41 Filtering vs. Convolution 2d filtering h=filter2(f,g); or h=imfilter(f,g); 2d convolution h=conv2(f,g); ], [ ], [ ], [, l n k m g l k f n m h l k + + = ], [ ], [ ], [, l n k m g l k f n m h l k =

42 Key properties of linear filters Linearity: filter(f + f 2 ) = filter(f ) + filter(f 2 ) Shift invariance: same behavior regardless of pixel location: filter(shift(f)) = shift(filter(f)) Any linear shift invariant operator can be represented as a convolution Source: S. Lazebnik

43 More properties Commutative: a * b = b * a Conceptually no difference between filter and signal Associative: a * (b * c) = (a * b) * c Often apply several filters one after another: (((a * b ) * b 2 ) * b 3 ) This is equivalent to applying one filter: a * (b * b 2 * b 3 ) Distributes over addition: a * (b + c) = (a * b) + (a * c) Scalars factor out: ka * b = a * kb = k (a * b) Identity: unit impulse e = [,,,, ], a * e = a Source: S. Lazebnik

44 Important filter: Gaussian Weight contributions of neighboring pixels by nearness x 5, σ = Slide credit: Christopher Rasmussen

45 Gaussian filters Remove high frequency components from the image (low pass filter) Convolution with self is another Gaussian So can smooth with small width kernel, repeat, and get same result as larger width kernel would have Convolving two times with Gaussian kernel of width σ is same as convolving once with kernel of width σ 2 Separable kernel Factors into product of two D Gaussians Source: K. Grauman

46 Separability of the Gaussian filter Source: D. Lowe

47 Separability example 2D convolution (center location only) The filter factors into a product of D filters: Perform convolution along rows: * = Followed by convolution along the remaining column: * = Source: K. Grauman

48 Separability Why is separability useful in practice?

49 Smoothing with Gaussian filter

50 Smoothing with box filter

51 Thinking in terms of frequency

52 Thinking in terms of frequency Intensity Image Fourier Image

53 Signals can be composed + = More:

54 Fourier Transform Fourier transform is in terms of frequency magnitude and phase Phase encodes spatial information (indirectly) Can convert back and forth losslessly To filter, multiply the Fourier images

55 Fourier Matlab demo

56 Some practical matters

57 Practical matters How big should the filter be? Values at edges should be near zero Rule of thumb for Gaussian: set filter half width to about 3 σ

58 g Practical matters What is the size of the output? MATLAB: filter2(g, f, shape) shape = full : output size is sum of sizes of f and g shape = same : output size is same as f shape = valid : output size is difference of sizes of f and g full same valid g g g g g f f f g g g g g g Source: S. Lazebnik

59 Q? Practical matters What about near the edge? the filter window falls off the edge of the image need to extrapolate methods: clip filter (black) wrap around copy edge reflect across edge Source: S. Marschner

60 Practical matters methods (MATLAB): clip filter (black): imfilter(f, g, ) wrap around: imfilter(f, g, circular ) copy edge: imfilter(f, g, replicate ) reflect across edge: imfilter(f, g, symmetric ) Source: S. Marschner

61 Things to remember Several options for color spaces Linear filtering is sum of dot product at each position Sometimes useful to think of images/filters in frequency domain Careful around edges

62 Next class Applications of filters Using filters for denoising and downsampling Using filters for matching Image pyramids, filter banks, and texture

63 Questions