06: Thinking in Frequencies. CS 5840: Computer Vision Instructor: Jonathan Ventura

Transcription

1 06: Thinking in Frequencies CS 5840: Computer Vision Instructor: Jonathan Ventura

2 Decomposition of Functions Taylor series: Sum of polynomials f(x) =f(a)+f 0 (a)(x a)+ f 00 (a) 2! (x a) 2 + f 000 (a) (x a) !

3 Decomposition of Functions Fourier series: Sum of sinusoids Asin(ωx +ϕ) Amplitude Frequency Phase

4 Example: Sound Time Domain Frequency Domain Slide: Hoiem

5 Waves What does frequency tell us? Rate of change / speed

6 Waves What does phase tell us? Position

7 Frequency Spectra How do we represent this signal with sines?

8 Frequency Spectra = + =

9 Frequency Spectra = + =

10 Frequency Spectra = + =

11 Frequency Spectra = + =

12 Frequency Spectra = + =

13 Frequency Spectra Need infinite series to recover signal. = A å k= 1 1 sin(2 p kt ) k

14 Frequencies in 2D Fourier transform

15 Fourier Transform MATLAB: fft2 and ifft2 (inverse) Phase is complex part x y (0,0) (zero frequency) Horizontal frequency Spatial Domain Vertical frequency Frequency Domain

16 Fourier Transform Example Spatial Domain Right of center Frequency Domain

17 Fourier analysis in images Spatial domain Frequency domain

18 Signals can be composed + = More:

19 Quiz Question What does the Fourier transform of this image look like?

20 Quiz Question Spatial Domain Frequency Domain (enlarged)

21 Quiz Question What does the Fourier transform of this image look like?

22 Quiz Question Spatial Domain Frequency Domain

23 Quiz Question What does the Fourier transform of this image look like?

24 Quiz Question Spatial Domain Frequency Domain

25 The Convolution Theorem Convolution in spatial domain = multiplication in frequency domain F[ g * h] = F[ g]f[ h] g * h = F -1 [F[ g]f[ h]]

26 Filtering in spatial domain * =

27 Filtering in frequency domain FFT FFT = Inverse FFT Slide: Hoiem

28 Quiz Question F[h] = g * h = F -1 [F[ g]f[ h]] What will this filter do?

29 Quiz Question F[h] = g * h = F -1 [F[ g]f[ h]] Nothing all frequencies are preserved. all-pass filter

30 Fourier Transform of Gaussian Fourier Transform of Gaussian is still Gaussian

31 Quiz Question What happens when we multiply by a Gaussian in the frequency domain?

32 Gaussian Smoothing Convolution with a Gaussian removes high frequencies low pass filter

33 Filtering Why does the Gaussian give a nice smooth image, but the square filter give edgy artifacts? Gaussian Box filter

34 Gaussian

35 Box Filter

36 Quiz Question How do we keep the high frequencies and remove the low frequencies, in the spatial domain?

37 Quiz Question - = How do we keep the high frequencies and remove the low frequencies, in the spatial domain? Convolve with Gaussian and subtract from original image.

38 Quiz Question How do we keep the high frequencies and remove the low frequencies, in the frequency domain?

39 Quiz Question Filter by all pass minus Gaussian high pass filter I h $%& I = I )*+) h,$$ I h $%& I = I )*+) (h,$$ h $%& ) I = I )*+)

40 Quiz Question How do I reconstruct an image from its low frequency and high-frequency images?

41 Quiz Question + = How do I reconstruct an image from low-passed frequency and high-passed images? Add them together!

42 Quiz Question How do I reconstruct an image from low-passed and high-passed images, in frequency domain?

43 Quiz Question ifft2 + = How do I reconstruct an image from low-passed and high-passed images, in frequency domain? Add them together!

44 Sub-sampling by a factor of 2 Throw away every other row and column to create a 1/2 size image

45 Aliasing problem 1D example (sinewave): Source: S. Marschner

46 Aliasing problem 1D example (sinewave): Source: S. Marschner

47 Aliasing in video Slide by Steve Seitz

48 Aliasing in graphics Source: A. Efros

49 Sampling and aliasing

50 Nyquist-Shannon Sampling Theorem Sampling frequency should be ³ 2 f max v v v good bad

51 Anti-aliasing Smooth (blur) before down-sampling Forsyth and Ponce 2002

52 Subsampling without pre-filtering 1/2 1/4 (2x zoom) 1/8 (4x zoom) Slide by Steve Seitz

53 Subsampling with Gaussian prefiltering Gaussian 1/2 G 1/4 G 1/8 Slide by Steve Seitz

54 Salvador Dali invented Hybrid Images? Salvador Dali Gala Contemplating the Mediterranean Sea, which at 30 meters becomes the portrait of Abraham Lincoln, 1976

55

56

57 Why do we get different, distance-dependent interpretations of hybrid images??

58 Clues from Human Perception 1. When we see an image from far away, we are effectively subsampling it 2. Perceptual cues in the mid-high frequencies dominate perception Early Visual Processing: Multi-scale edge and blob filters

59 Campbell-Robson contrast sensitivity curve Contrast (decreasing) Frequency

60 Perception Why do we get different, distance-dependent interpretations of hybrid images??

61 Hybrid Image in FFT Combine low frequencies from one image with high frequencies from another. Hybrid Image Low-passed Image High-passed Image

62 Next time Image pyramids and edges Reading: Szeliski 3.5, 4.2