Stochastic Image Denoising using Minimum Mean Squared Error (Wiener) Filtering

Stochastic Image Denoising using Minimum Mean Squared Error (Wiener) Filtering L. Sahawneh, B. Carroll, Electrical and Computer Engineering, ECEN 670 Project, BYU Abstract Digital images and video used for various tasks is often subject to distortion and noise. When information is known about the distortions and noise introduced into the image, their effects can be at least partially negated through image restoration techniques. In this paper, we explore the derivation of the inverse filter and the Wiener filter. We look and their efficacy in restoring images corrupted by various types of degradation and noise. We have also investigated the ability to estimate the parameters of unknown noise by analyzing the histograms oniform regions of interest in the degraded image. I. INTRODUCTION The demands for efficient image denoising and restoration methods has grown with increasing usage of digital images and movies in various applications. Regardless of how good imaging sources are, there is always potential for improvement through image restoration techniques. A digital image is generally encoded as a matrix of gray-level or color values. An image may be defined as a two-dimensional function, x[u, v], where u and v are spatial coordinates for a pixel, or picture element. In the case of gray-level images, u and v specify a point on a two-dimensional (2D) grid and x[u, v] is a real value. In the case of color images, x[u, v] is a triplet oalues for the red, green, and blue components [][2]. Image denoising is one of the fundamental challenges in the field of image processing and is important in a wide variety of applications such as object recognition, photo enhancement, and image restoration. Image denoising may be defined as the process of recovering the original image from a noisy or degraded image by using a priori knowledge of the degradation phenomenon. Hence, restoration algorithms are based on modeling the degradation and applying the inverse process to recover the original image. As Fig. shows, the general practice is to model the image degradation process as a degradation function that, together with a noise term, operates on an input image x[u, v] to produce a degraded image, y[u, v] = H(x[u, v]) + n[u, v], () where H is the degradation operator and n[u, v] is the noise [3][2]. Therefore, the objective of restoration is to obtain an estimate, ˆx[u, v] of the original image given some knowledge about the degradation function, H and the additive noise contaminated with the image, n[u, v]. Typically, the more knowledge about the degradation process and noise, the closer the estimate ˆx[u, v] will be to original image, x[u, v]. The degradation process in the spatial domain can be thought of as a convolution of the degradation function and the input image, with the assumption that the degradation function H is a spatial linear invariant process. The degraded image is then given by y[u, v] = h[u, v] x[u, v] + n[u, v]. (2) The degradation function h[u, v] is referred to as the point spread function (PSF), a term that arises from letting h[u, v] operate on a point of light to obtain the characteristics of the degradation for any type of input. The point spread function is analogous to the impulse response function in any signal processing setup. From our knowledge of signal and systems theory the convolution operation in the spatial domain is a multiplication in the frequency domain. Using the Fourier transform, Eq. (2) may be represented in the frequency domain as Y [, ] = H[, ]X[, ] + N[, ], (3) where H[, ], X[, ] and N[, ] are the Fourier transforms of the corresponding terms in the spatial domain, h[u, v], x[u, v] and n[u, v] respectively. Although a large number of approaches have been proposed for image denoising, the problem of noise suppression remains an open challenge in situations characterized by additional image degradation and low signal-to-noise ratios [4]. The earliest attempts at image restoration were based on the concept of inverse filtering, in which the transfer function of the degrading system is inverted to yield a restored image. The inverse filter transfer function is chosen so that the estimate image in the frequency domain is simply the degraded image divided by the degradation function, Substituting Eq. (3) into Eq. (4) yields ˆX[, ] = Y [, ] H[, ]. (4) ˆX[, ] = X[, ] + N[, ] H[, ]. (5) This tells us that even with exact knowledge of the degradation function, H[, ] we are still unable to recover the original image exactly because N[, ] is unknown. Moreover, the second term of Eq. (5), N[, ]/H[, ], could easily dominate the estimate of the image if the denominator has zeros or very small values. Therefore, the poor performance of direct inverse filtering in general motivates more powerful improvements by using the minimum mean square error (Wiener) filtering approach. Wiener filters are a class of optimum linear filters which involve linear estimation of a desired signal sequence from

2 Noise n(u, v) x(u, v) y(u, v) Degradation Restoration Filter x(u, v) Function h (Wiener Filter) Fig.. Degradation Process with additive noise A model of image degradation/restoration process Denoising and Restoration Process another related sequence. In the statistical approach to the solution of the linear filtering problem, we assume the availability of certain statistical parameters (e.g. mean and correlation functions) of the useful signal and unwanted additive noise. The problem is to design a linear filter for the noisy data that minimizes the effect of the noise at the filter output according to some statistical criterion. A useful approach to this filter optimization problem is to minimize the mean-square value of the error signal that is defined as the difference between some desired response and the actual filter output. For stationary inputs, the resulting solution is commonly known as the Wiener filter. However, in general, the Wiener filter is inadequate for dealing with situations in which nonstationarity of the signal and/or noise is intrinsic to the problem. In such situations, the optimum filter is assumed to have a time-varying form. A highly successful solution to this more difficult problem is found in the Kalman filter [][4]. II. NOISE MODELS Having introduced the degradation function, the second term in the degradation process is the additive noise. Modeling its effect and behavior is crucial to image restoration. Generally, noise models that appear in literature are constructed either in the spatial domain or in the frequency domain. In this work, we consider three of the most common spatial domain noise models. Gaussian noise normally arises in an image due to factors such as electronic circuit or sensor noise, low illumination, and thermal noise. Its probability density function (PDF) is given by p Z (z) = ( exp 2πσ 2 ) (z m)2 2σ 2, (6) where m is the mean and σ 2 is the variance. Uniform noise is often used to represent quantization noise in images. It has a flat PDF between a lower bound a and upper bound b, p Z (z) = { b a if a z b 0 otherwise. The mean is m = a+b 2 and the variance is σ 2 = (b a)2 2. Salt and pepper noise occurs when pixels either spike or are dropped. Its causes include analog to digital conversion errors and transmission errors. The PDF for salt and pepper noise is P a for z = a p Z (z) = P b for z = b (8) 0 otherwise, (7) where m = ap a + bp b and σ 2 = (a m) 2 P a + (b m) 2 P b, given that b > a. The parameters of spatial domain noise may be known from the sensor specification or a priori knowledge of the noise distribution. However, in most cases it is necessary to estimate these parameters from the sample images being denoised. The common approach is to select a region of interest (ROI) in an image with as uniform a background as possible. Since the variability of intensity values in the region will be due primarily to noise, the normalized histogram of the ROI can be viewed as a mean-shifted approximation of the intensity PDF for the noise. Let z i be a discrete random variable that denotes intensity levels in an image, and p(z i ), i =, 2,..., L, be the corresponding normalized histogram, where L is the number of possible intensity values. The assumption is that a histogram component p(z i ) is an estimate of the probability of occurrence of intensity value z i. One of the principal approaches for describing the shape of a histogram is via its central moments, which are defined as [2] L µ n = (z i m) n p(z i ), (9) i=0 where n is the number of moment order, and m is the mean L m = zp(z i ). (0) i=0 The second central moment is the variance, L µ 2 = (z i m) 2 p(z i ). () i=0 From Eq. (9), µ 0 = L i=0 p(z i) = because the histogram is assumed to be normalized, making the sum of all its components. Furthermore, µ = 0 by definition. Generally, the shape of the histogram identifies the closest PDF. In other words, if the shape looks Gaussian, then mean and variance is all that is needed to specify the PDF, whereas in the Uniform case mean and variance can be used to solve for the lower bound a and upper bound b of the PDF. However, the PDF for salt and pepper noise is estimated differently because only the probabilities of white and black pixels are needed. So, the heights of the peaks within the histogram that correspond to white and black pixels are the estimates of P a and P b in Eq. (8). Figure 2 highlights a ROI that we selected for a dandelion image and Figs. 3 to 5 show the histograms of pixel intensities in the ROI when the image has been corrupted with Gaussian, uniform, and salt and pepper noise. III. THEORY Let the degraded, noisy image y[u, v] be given by y[u, v] = h[u, v] x[u, v] + n[u, v], where h[u, v] is the degradation or point spread function, x[u, v] is the noiseless image, n[u, v] is the noise, and u and v are a pixel coordinates. We wish to find an LTI filter with

3 250 Histogram of ROI 200 number of pixels 50 00 50 Fig. 2. The ROI used to construct histograms and estimate the moments of the noise. The selected region has relatively uniform intensity in the original image and is highlighted in the lower left corner of the image. 00 Histogram of ROI 0 0 50 00 50 200 250 300 intensity levels Fig. 5. Histogram of pixel intensities in the ROI with salt and pepper noise. Note the small spikes at 0 and 255, representing the black and white pixels. Fig. 3. number of pixels 80 60 40 20 0 0 50 00 50 200 250 300 intensity levels Histogram of pixel intensities in the ROI with Gaussian noise. where E denotes the expected value. Parseval s theorem states that x[n] 2 = N X(k) 2. N n= k=0 Since they are directly proportional, we can minimize the spatial domain mean squared error by minimizing the frequency domain mean squared error, ɛ E X[, ] ˆX[, ] 2, impulse response g[u, v] that gives us the MMSE estimate of x[u, v], The MSE ɛ is given by Fig. 4. number of pixels 50 40 30 20 0 ˆx[u, v] = g[u, v] y[u, v]. { } ɛ = E (x[u, v] ˆx[u, v]) 2, (2) u v Histogram of ROI 0 0 50 00 50 200 250 300 intensity levels Histogram of pixel intensities in the ROI with uniform noise. where and are horizontal and vertical frequencies. By the multiplication property of the Fourier transform, ˆx[u, v] becomes ˆX[fu, ] = G[, ]Y [, ] in the frequency domain, and y[u, v] becomes H[, ]X[, ] + N[, ]. Omitting the function parameters for notational convenience, we plug in the expression for ˆX to get ɛ E X GY 2. Substituting in Y then gives ɛ E X G (HX + N) 2. Rearranging the terms, we have ɛ E ( GH)X GN 2. Using the definition of an absolute square, we get ( )( ) ɛ E ( GH)X GN ( GH)X GN,

4 where denotes the complex conjugate. Multiplying out the terms and distributing the expected value across the sums gives ɛ ( { (( )( ) } E GH)X ( GH)X { (( E{( GH)X(GN) ) } GN } E GH)X ) + E{(GN)(GN) }. Both G and H are constant with respect to the expected values, so they can be pulled out of the expected values to get ɛ ( (( GH)( GH) )E { X 2} (( GH)G )E{XN } (( GH) G)E{X N} Fig. 6. The original dandelion image. + (GG )E { N 2} ). We assume that the noise is independent of the original image and that either the noise or the image is zero-mean, so E{XN } = E{X N} = 0. Also, let the power spectral densities of the original image and the noise be { X[fu S x [, ] = E, ] 2} S n [, ] = E{ N[fu, ] 2}. We can now write ɛ (( GH)( GH) S x + (GG )S n ). (3) To find the G that minimizes the MSE, we take the derivative of Eq. (3) with respect to G(, ) and set it equal to zero. According to complex analysis, G acts like a constant when differentiating with respect to G. Thus, the derivative gives us (G S n H( GH) S x ) = 0. Since the sum over all the frequencies is zero and all the frequencies are independent of each other, the expression inside the sums must be zero for each individual frequency as well. Thus, we have Solving for G gives G S n H( GH) S x = 0,. G HS x = S n + HH. S x Taking the complex conjugate of both sides and noticing that HH = H 2, we have G = H S x S n + H 2 S x. Finally, we write the equation in terms of the signal-to-noise ratio SNR[, ] = Sx[fu,fv] S n[,], H[, ] G[, ] = H[fu, ] 2. (4) + SNR[,] Fig. 7. The dandelion image with a 20 pixel motion blur (left) and the restored dandelion image using the inverse filter (right). To gain a better intuitive understanding of the Wiener filter, we can rewrite Eq. (4) as H[fu, f G[, ] = v ] 2 H[, ] H[fu, ] 2. (5) + SNR[,] It can now be seen that the Wiener filter is simply the inverse of the original degradation function weighted according to the relative strengths of the signal and noise at each frequency. As the noise at a particular frequency becomes large relative to the signal, SNR becomes small and the rightmost fraction in Eq. (5) also becomes small. Thus, G[, ] will be small at that frequency and will attenuate the noise. As the noise at a particular frequency becomes small relative to the signal, SNR becomes large and Eq. (5) approaches the inverse of the original degradation function, H[,]. IV. RESULTS AND ANALYSIS We found that the inverse filter works extremely well when the point spread function is known and there is no noise. Figure 6 shows our original dandelion test image. We degraded the dandelion image with motion blur across 20 pixels and then restored it to the original image using the inverse filter as shown in Fig. 7. In this case, the restored image is nearly identical to the original. When a little noise is introduced into the blurred image, the inverse filter no longer works as effectively. The inverse filter seeks to undo the blurring, which smooths over sudden changes in intensity in the original image. Adding noise to the degraded image creates new sudden changes in blurred intensities, which get sharpened and amplified when the blurring

5 Fig. 8. Image corrupted with a small blur and imperceptible Gaussian noise (left) and the image recovered using the inverse filter (right). Note how the imperceptible noise is magnified to give the recovered image a grainy appearance. operation is inverted. Figure 8 shows the image corrupted by a small blur and an imperceptible amount of noise. It also shows the result of inverse filtering the degraded image. The inverse filter amplifies the noise enough that it gives the restored image a grainy look. When non-negligible amounts of noise and blur are present in an image, the inverse filter fails to produce recognizable results. This is because the value of H[, ] in Eq. (5) is small for some of the frequencies where non-trivial amounts of noise are present. It thus amplifies the noise so much that the original signal is completely masked out. The Wiener filter, on the other hand, does a decent job of approximating the original image. It is able to do this by attenuating the frequencies with poor signal to noise ratios that dominated the image produced by the inverse filter. Figure 9 shows a blurred, noisy image and its reconstructions using the inverse filter and the Wiener filter. The Wiener filter also performs fairly well with uniformly distributed noise and a Gaussian blur. Figure 0 shows an image with a 2 pixel standard deviation Gaussian blur and uniform noise in the range -. to., where 0 is black and is white. The image recovered through Wiener filtering is sharper and has less noise. The Wiener filter performs moderately well on salt and pepper noise. An image corrupted with a small motion blur and salt and pepper noise is given in Fig., along with the Wiener filter reconstruction. Note that the reconstructed image contains grainy dark and light spots where black or white pixels were blurred into their surroundings. Wiener filtering is not the ideal denoising operation for salt and pepper noise simply because it is a linear spatially invariant operator. It can only remove the white and black pixels by blurring them into the pixels around them. Non-linear operations like the median filter perform better in this case. Finally, we used the ROI and histograms shown in Figs. 2 to 5 to estimate the power spectrum of the noise, S n [, ]. We still assume knowledge of the power spectrum of the original signal, as estimating it goes beyond the scope of this paper. We obtained nearly identical results when using our estimated noise power spectrum versus the actual noise power spectrum, as shown in Fig. 2. This indicates that the estimate is accurate enough to be effective for filtering. Fig. 9. Image corrupted with significant amounts of blur and Gaussian noise (top left), the image recovered using the inverse filter (top right), and the image recovered using the Wiener filter (bottom). Note that with significant amounts of noise and blur, the inverse filter fails to produce an image that is even recognizable as a dandelion. Fig. 0. Image corrupted by a Gaussian blur with std = 2 and uniform noise in [.,.] (left) and its Wiener filter reconstruction (right). Fig.. Image corrupted by a small motion blur as well as with salt and pepper noise (left) and its Wiener filter reconstruction (right). The salt and pepper grains both occurred with probability.05, leaving a.7 chance that a pixel was unmodified. Note how small clusters of salt or pepper grains can leave light or dark spots in the resulting Wiener filtered image.

6 Fig. 2. The wiener filtered dandelion image from Fig. 9 usin the actual noise power spectrum (left) and the estimated noise power spectrum (right). The two images are extremely similar, indicating that our estimate of the noise power spectrum is accurate. V. CONCLUSION Although the inverse filter works well when no noise is present, the Wiener filter performs much better and is more versatile. However, the Wiener filter assumes knowledge of the degradation function and the power spectra of both the noise and the original image. Most image restoration methods require some knowledge of the degradation function, but the Wiener filter in particular presents the additional difficulty of knowing the power spectra [2]. We showed that the noise power spectrum can be effectively estimated by analyzing a relatively uniform region of interest in the degraded image. However, obtaining the spectrum of the original signal is more difficult. This requirement makes the Wiener filter less useful in many practical applications. Moreover, the Wiener filter provides a sound theoretical foundation upon which other restoration techniques are based. REFERENCES [] A. Buades, B. Coll, and J. M. Morel, A review image denoising algorithm, with new one, Society for Industrial and Applied Mathematics, vol. Vol. 4, No. 2, p. pp. 490530, 2005. [2] R. C. Gonzalez and R. E. Woods, Digital Image Processing. Person Eduaction, 2007. [3] T. F. Chan and J. Shen, Image Processing and Analysis: Varitional, PDE, wavelet, and Stochaistic methods. Society for Industrial and Applied Mathematics, 2005. [4] A. Mishra, A. Wong, D. Clausi, and P. Fieguth, Adaptive nonlinear image denoising and restoration using a cooperative bayesian estimation approach, in Computer Vision, Graphics Image Processing, 2008. ICVGIP 08. Sixth Indian Conference on, dec. 2008, pp. 62 627.