Images and Filters. EE/CSE 576 Linda Shapiro

Transcription

1 Images and Filters EE/CSE 576 Linda Shapiro

2 What is an image? 2

3 3

4 . We sample the image to get a discrete set of pixels with quantized values. 2. For a gray tone image there is one band F(r,c), with values usually between and For a color image there are 3 bands R(r,c), G(r,c), B(r,c) 4

5 Image Operations (functions of functions) F( ) = 5

6 Image Operations (functions of functions) F( ) = 6

7 Image Operations (functions of functions) F( ) =

8 Image Operations (functions of functions) F(, ) = 8

9 Local image functions F( ) = 9

10 Image filtering g[, ] f [.,.] h[.,.] h[ m, n] = k, l g[ k, l] f [ m + k, n + l] Credit: S. Seitz

11 Image filtering g[, ] f [.,.] h[.,.] h[ m, n] = g[ k, l] f [ m + k, n + l] k, l Credit: S. Seitz

12 Image filtering g[, ] f [.,.] h[.,.] h[ m, n] = g[ k, l] f [ m + k, n + l] 2 k, l Credit: S. Seitz

13 Image filtering g[, ] f [.,.] h[.,.] h[ m, n] = g[ k, l] f [ m + k, n + l] 3 k, l Credit: S. Seitz

14 Image filtering g[, ] f [.,.] h[.,.] h[ m, n] = g[ k, l] f [ m + k, n + l] 4 k, l Credit: S. Seitz

15 Image filtering g[, ] f [.,.] h[.,.] ? 9 h[ m, n] = g[ k, l] f [ m + k, n + l] 5 k, l Credit: S. Seitz

16 Image filtering g[, ] f [.,.] h[.,.] ? h[ m, n] = g[ k, l] f [ m + k, n + l] 6 k, l Credit: S. Seitz

17 Image filtering g[, ] f [.,.] h[.,.] h[ m, n] = g[ k, l] f [ m + k, n + l] 7 k, l Credit: S. Seitz

18 Box Filter What does it do? g[, ] Replaces each pixel with an average of its neighborhood Achieve smoothing effect (remove sharp features) Slide credit: David Lowe (UBC) 8

19 Smoothing with box filter 9

20 Practice with linear filters? Original 2 Source: D. Lowe

21 Practice with linear filters Original Filtered (no change) 2 Source: D. Lowe

22 Practice with linear filters? Original 22 Source: D. Lowe

23 Practice with linear filters Original Shifted left By pixel 23 Source: D. Lowe

24 Practice with linear filters 2 -? Original 24 Source: D. Lowe

25 Practice with linear filters 2 - Original Sharpening filter - Accentuates differences with local average 25 Source: D. Lowe

26 Sharpening 26 Source: D. Lowe

27 Other filters 2 Sobel Vertical Edge (absolute value) 27

28 Other filters Sobel - Horizontal Edge (absolute value) 28

29 Basic gradient filters Horizontal Gradient Vertical Gradient - - or - or - 29

30 Gaussian filter * = Input image f Filter h Output image g

31 Gaussian vs. mean filters What does real blur look like?

32 Important filter: Gaussian Spatially-weighted average x 5, σ = 32 Slide credit: Christopher Rasmussen

33 Smoothing with Gaussian filter 33

34 Smoothing with box filter 34

35 Gaussian filters What parameters matter here? Variance of Gaussian: determines extent of smoothing 35 Source: K. Grauman

36 Smoothing with a Gaussian Parameter σ is the scale / width / spread of the Gaussian kernel, and controls the amount of smoothing. 36 Source: K. Grauman

37 2D edge detection filters Laplacian of Gaussian or LoG filter Gaussian x derivative of Gaussian is the Laplacian operator (sum of 2 nd derivatives): Often approximated by -4 37

38 First and second derivatives What are these good for? Original First Derivative x Second Derivative x, y

39 Subtracting filters Original Second Derivative Sharpened

40 Combining filters * * for some = = It s also true:

41 Combining Gaussian filters * =? More blur than either individually (but less than )

42 Separable filters Compute Gaussian in horizontal direction, followed by the vertical direction. Much faster! * = Not all filters are separable. Freeman and Adelson, 99

43 Sums of rectangular regions If an image will be repeatedly convolved with different box filters, we can precompute a summed area table i j s(i,j) = ΣΣf(k,l) k= l= How do we compute the sum of the pixels in the red box? After some pre-computation, this can be done in constant time for any box. This trick is commonly used for computing Haar wavelets (a fundemental building block of many object recognition approaches.)

44 Sums of rectangular regions The trick is to compute an integral image. Every pixel is the sum of itself and its neighbors to the upper left. Sequentially compute using:

45 Sums of rectangular regions The trick is to compute an integral image. Every pixel is the sum of itself and its (modified) N and W neighbors minus its (modified) NW neighbor. Compute sequentially using: ? =2 Result:

46 Sums of rectangular regions Area of red rectangle is found using: A + D B - C A B C D

47 Linear vs. Non-Linear Filters a. original image with Gaussian noise, b. Gaussian filtered, c. median filtered, d. bilateral filtered e. original image with shot noise, f. Gaussian filtered, g. median filtered, h. bilateral filtered

48 Spatially varying filters Some filters vary spatially. The bilateral filter is the product of a domain kernel (Gaussian) and a data dependent range kernel. d(i,j,k,l) = exp[(-(i-k) 2 +(j-l) 2 )/2σ d2 ] is the domain kernel r(i,j,k,l) = exp[- f(i,j)-f(k,l) 2 /2σ r2 ] is the range kernel w(i,j,k,l) = d(i,j,k,l) * r(i,j,k,l) is their product g(i,j) = Σ k,l f(k,l) w(i,j,k,l) / Σ k,l w(i,j,k,l) is the bilateral filter from Szeliski text

49 Constant blur: same kernel everywhere input * output * * Same Gaussian kernel everywhere. Slides courtesy of Sylvian Paris 49

50 Bilateral filter: kernel depends on intensity Maintains edges when blurring! input * output * * The kernel shape depends on the image content. Slides courtesy of Sylvian Paris 5

51 Borders What to do about image borders: black fixed periodic reflected 5

52 Image Sampling F( ) = F( ) =

53 Image Scaling This image is too big to fit on the screen. How can we reduce it? How to generate a halfsized version?

54 Image sub-sampling /8 /4 Throw away every other row and column to create a /2 size image - called image sub-sampling

55 Image sub-sampling /2 /4 (2x zoom) /8 (4x zoom) Why does this look so bad?

56 Down-sampling Aliasing can arise when you sample a continuous signal or image occurs when your sampling rate is not high enough to capture the amount of detail in your image Can give you the wrong signal/image an alias formally, the image contains structure at different scales called frequencies in the Fourier domain the sampling rate must be high enough to capture the highest frequency in the image

57 Subsampling with Gaussian pre-filtering G /4 G /8 Gaussian /2 Solution: filter the image, then subsample Filter size should double for each ½ size reduction.

58 Finale Filtering is just applying a mask to an image. Computer vision people call the linear form of these operations convolutions. They are actually correlations, since the true convolution inverts the mask. There are many nonlinear filters, too, such as median filters and morphological filters. Filtering is the lowest level of image analysis and is taught heavily in image processing courses. 58