EE4830 Digital Image Processing Lecture 7. Image Restoration. March 19 th, 2007 Lexing Xie ee.columbia.edu>

Size: px

Start display at page:

Download "EE4830 Digital Image Processing Lecture 7. Image Restoration. March 19 th, 2007 Lexing Xie ee.columbia.edu>"

Drusilla Reed
5 years ago
Views:

1 EE4830 Digital Image Processing Lecture 7 Image Restoration March 19 th, 2007 Lexing Xie ee.columbia.edu> 1

2 We have covered 2 Image sensing Image Restoration Image Transform and Filtering Spatial Domain processing

3 Lecture Outline 3 What is image restoration Scope, history and applications A model for (linear) image degradation Restoration from noise Different types of noise Examples of restoration operations Restoration from linear degradation Inverse and pseudo-inverse filtering Wiener filters Blind de-convolution Geometric distortion and its corrections

4 Degraded Images 4 What caused the image to blur? Can we improve the image, or undo the effects?

5 Image enhancement: improve an image subjectively. Image restoration: remove distortion from image in order to go back to the original objective process. 5

6 Image Restoration 6 Started from the 1950s Application domains Scientific explorations Legal investigations Film making and archival Image and video (de-)coding Consumer photography Related problem: image reconstruction in radio astronomy, radar imaging and tomography See [Banham and Katsaggelos 97]

7 A Model for Image Distortion 7 Image enhancement: improve an image subjectively. Image restoration: remove distortion from image, to go back to the original objective process

8 A Model for Image Distortion 8 Image restoration Use a priori knowledge of the degradation Modeling the degradation and apply the inverse process Formulate and evaluate objective criteria of goodness

9 Usual Assumptions for the Distortion Model 9 Noise Independent of spatial location Exception: periodic noise Uncorrelated with image Degradation function Linear Position-invariant Divide-and-conquer step #1: degraded only by noise.

10 10 Common Noise Models 0, ) ( 0, )! ( ) ( ), (,, ) ( 2 ) ( 2 1 ) ( 1 / ) ( 2 / ) ( = = = = z for ae z p Exponential z for e a b z a z p b a Gamma Erlang a z for e a z b z p Rayleigh e z p Gaussian az az b b b a z z σ µ πσ a R,a I zero mean, independent Gaussian multiplicative noise on signal magnitude additive noise

11 Visual Effects of Noise 11 Original image shown on the right with the annotated dimensions. a b d

12 Recovering from Noise 12 Overview of noise reduction Observe and estimate noise type and parameters apply optimal (spatial) filtering (if known) observe result, adjust filter type/parameters Example noise-reduction filters [G&W 5.3] Mean/median filter family Adaptive filter family Other filter family e.g. Homomorphic filtering for speckle noise [G&W 4.5, Jain 8.13]

13 Recall: Butterworth LPF Recovering from Periodic Noise Butterworth bandreject filter 13 [G&W 5.4]

14 Lecture Outline 14 Scope, history and applications A model for (linear) image degradation Restoration from noise Different types of noise Examples of restoration operations Restoration from linear degradation Inverse and pseudo-inverse filtering Wiener filters Blind de-convolution Geometric distortion and example corrections

15 Recover from Degradation 15 Degradation function Linear (eq 5.5-3, 5.5-4) Homogeneity Additivity Position-invariant (in cartesian coordinates, eq 5.5-5) linear filtering with H(u,v) convolution with h(x,y) point spread function Divide-and-conquer step #2: linear degradation, noise negligible.

16 Point Spread Functions 16

17 Point Spread Functions 17 Spatial domain Frequency domain

18 Inverse Filter 18 Assume h is known: low-pass filter H(u,v) Inverse filter Recovered Image H(u,v) [EE381K, UTexas]

19 Inverse Filtering Example 19 loss of information

20 Inverse Filtering under Noise 20 H(u,v) = 0, for some u, v In the noisy case: [EE381K, UTexas]

21 Pseudo-inverse Filtering 21 [Jain, Fig 8.10]

22 Back to the Original Problem 22 Pseudo-inverse filter: Can the filter take values between 1/H(u,v) and zero? Can we model noise directly?

23 Wiener Filter 23 Find optimal linear filter W(u,v) such that the Mean Square Error between and is minimized (1) orthogonal condition (2) correlation function

24 Observations about Wiener Filter 24 If no noise, S ηη 0 Pseudo inverse filter If no blur, H(u,v)=1 (Wiener smoothing filter) More suppression on noisier frequency bands

25 1-D Wiener Filter Shape Wiener Filter implementation 25 Where K is a constant chosen according to our knowledge of the noise level. [Jain, Fig 8.11]

26 Wiener Filter Example 26 * H (u, v) W(u, v) = 2 H(u, v) + K [EE381K, UTexas]

27 Wiener Filter as a LMS Filter 27 [Young et. al., Fundamentals of Image Processing, TU-Delft]

28 Wiener Filter Example 28 Wiener filter is robust to noise, and preserves high-frequency details.

29 Wiener Filter Example 29 Ringing effect visible, too many high frequency components? (a) Blurry image (b) restored w. regularized pseudo inverse (c) restored with wiener filter [UMD EE631]

30 Wiener Filter 30 How much de-blurring is just enough? [Image Analysis Course, TU-Delft]

31 Improve Wiener Filter 31 Constrained Least Squares Wiener filter emphasizes high-frequency components, while images tend to be smooth

32 Improve Wiener Filter (1) 32 Constrained Least Squares Wiener filter emphasizes high-frequency components, while images tend to be smooth Blind deconvolution Wiener filter assumes both the image and noise spectrum are know (or can be easily estimated), in practice this becomes trial-and-error since noise and signal parameters are often hard to obtain.

33 Maximum-Likelihood (ML) Estimation h(x,y) H(u,v) unknown Assume parametric models for the blur function, original image, and/or noise Parameter set θ is estimated by θ ml = arg{max p(y θ )} Solution is difficult in general Expectation-Maximization algorithm Guess an initial set of parameters θ Restore image via Wiener filtering using θ Use restored image to estimate refined parameters θ... iterate until local optimum θ 33 To explore more: D. Kundur and D. Hatzinakos, "Blind Image Deconvolution," IEEE Signal Processing Magazine, vol. 13, no. 3, May 1996, pp

34 Geometric Distortions 34 Modify the spatial relationships between pixels in an image a. k. a. rubber-sheet transformations Two basic steps Spatial transformation Gray-level interpolation

35 Spatial Distortion Examples 35

36 Recovery from Geometric Distortion 36

37 Recovery from Geometric Distortion 37 Rahul Swaminathan, Shree K. Nayar: Nonmetric Calibration of Wide-Angle Lenses and Polycameras. IEEE Trans. Pattern Anal. Mach. Intell. 22(10): (2000)

38 Epilogue: Estimating Distortion 38 Calibrate Use flat/edge areas ongoing work [Tong et. al. ICME2004]

39 Summary 39 Image degradation model Restoration from noise Restoration from linear degradation Inverse and pseudo-inverse filters, Wiener filter, blind deconvolution Geometric distortions Readings G&W Chapter 5, Jain (at courseworks) M. R. Banham and A. K. Katsaggelos "Digital Image Restoration, " IEEE Signal Processing Magazine, vol. 14, no. 2, Mar. 1997, pp Gratefully attribute many of the image examples to G&W book, Min Wu (UMD), Joanneum Research (Austria), John Conway (Chalmers, Sweden), B. Evans and H. T. Pai (UT Austin), Daniel Garcia- Romero (UMD), NASA Langley Research Center, photo.net, Matlab Toolbox, and whoever annotated in the slides.

Image Restoration. Lecture 7, March 23 rd, Lexing Xie. EE4830 Digital Image Processing

Image Restoration. Lecture 7, March 23 rd, Lexing Xie. EE4830 Digital Image Processing Image Restoration Lecture 7, March 23 rd, 2008 Lexing Xie EE4830 Digital Image Processing http://www.ee.columbia.edu/~xlx/ee4830/ thanks to G&W website, Min Wu and others for slide materials 1 Announcements