Image De-Noising Using a Fast Non-Local Averaging Algorithm RADU CIPRIAN BILCU 1, MARKKU VEHVILAINEN 2 1,2 Multimedia Technologies Laboratory, Nokia Research Center Visiokatu 1, FIN-33720, Tampere FINLAND Abstract: - Recently the non local (NL) means algorithm was introduced in order to suppress additive Gaussian noise in images. Although its very good performances for Gaussian noise, the NL-means method has two disadvantages. Its computational complexity is very large and the impulse rejection is poor. In this paper, we propose a simplified version of the NL-means algorithm with a much smaller computational complexity and better behavior in mixed (Gaussian and impulsive) noise. Experimental results obtained with our proposed method and several other algorithms are also presented. Key-Words: - Image de-noising, Image Restoration, Sigma filter, Noise estimation, Mean Squared Error. 1 Introduction In all imaging systems, the digital pictures are distorted by various factors. Among many interferences, one of the most important is the noise. There are different sources of noise such as: the recording medium (film, digital sensor), the transmission medium, measurement and quantization errors [1], [2]. Although the manufacturing technology of the CCD and CMOS sensors are very advanced, the snapped picture must always be digitally processed prior storage or displaying. Even for the latest manufacturing technologies, of the camera sensors, the signal-to-noise ratio into the captured image can be very low 1. As a consequence, de-noising is a very important research topic in image processing and various algorithms have been implemented to reduce different types of noise. Some algorithms are designed for better suppression of Gaussian noise while others are tuned to different noise distributions [3]. There are many proposed methods that aims of reducing the additive noise (see [4], [5] and the references therein) while others are designed to deal with signal dependent noise [6]. Denoising algorithms can be classified into two main classes: methods that operate in pixel domain and approaches that perform filtering in transform domain. A well known example from the first class is the sigma filter [7] that uses the knowledge of the additive noise variance to select the pixels from the neighborhood of 1 Especially in low light conditions. the current pixel and the output is the average of these. The anisotropic filters [6] uses some more elaborated techniques to define the neighborhood of the current pixel and they also operate at pixel level. The wavelet thresholding techniques [8], [9] operate in transform domain modifying the wavelet coefficients of the input image (some methods use DCT, FFT and other transforms). Nowadays, more and more mobile devices, such as mobile phones and PDA s, are equipped with digital cameras and related applications. As a consequence, image processing algorithms must be implemented in such devices. Due to the relatively reduced computational power of a mobile device, one important requirement for such algorithms is to possess a low computational complexity 2. In this paper, we propose a new de-noising algorithm for Gaussian and mixed (Gaussian and impulsive) noise removal. Our proposed method, which is a modified version of the approach in [5], have a lower computational complexity and better behavior in removing mixed noise. The paper is organized as follows: in Section 2 the sigma filter and the NL averaging method are briefly described in a common framework. Based on this, in Section 3, we introduce our new approach and we include it in the same framework showing the differences and similarities. Experiments showing the performances of several algorithms, in terms of MSE and 2 Of course, processing quality is of main interest as well.
visual quality, are described in Section 4 and Section 5 concludes the paper. 2 Existing approaches In this section, we briefly describe two existing algorithms that are related with our proposed method: the sigma filtering [7] and the Non Local (NL) averaging method from [5]. We first show the link between these two methods and then, based on these considerations, we introduce our proposed filtering scheme in the next section. We emphasize here that there are many other de-noising techniques that belong to this type of filters, (see for example [6] and [10]), but they use different techniques to compute the filter weights. The sigma filter and the NL-means filter share in common the fact that filter weights are computed based on some discrepancy measure between the current pixel and the neighboring pixels. The model of the distorted input image is assumed to be: y(i, j) = x(i, j) + n(i, j), i, j I (1) with i, j being the horizontal and vertical pixel coordinates, I is the image domain, x(i, j) the original noise free image, n(i, j) the additive noise and y(i, j) the observed noisy image. Both de-noising algorithms from [5] and [7] operate in pixel domain and are based on averaging the pixels that are in some neighborhood of the current pixel. Basically the two algorithms are implemented in two steps: the first step is the neighborhood selection and secondly the averaging of the neighboring pixels. Formally, both algorithms can be described by the following simple formula: ŷ(i, j) = k,l Ω W k,l y(k, l) k,l Ω W k,l, (2) where Ω I is a rectangular M M search window centered at the current pixel, ŷ(i, j) is the output of the filter at the current pixel and W k,l are some weights. The difference between the sigma filter and the NL averaging come from the definition of the weights W k,l. In the case of sigma filtering, the weights corresponding to the pixels from the search window Ω are selected as follows: { 1, if y(i, j) y(k, l) < 2σn W k,l = 0, otherwise k, l Ω (3) where y(i, j) is the current pixel, y(k, l) is a pixel from Ω and σ n is the noise standard deviation. The weights of the NL averaging filter are selected in a more complicated manner [5]: k,l W k,l = e i,j h 2, k, l Ω (4) In (4), h 2 is a constant parameter of the algorithm. Since NL-means was designed for Gaussian distributed noise, a natural way is to select this parameter proportional to the noise variance as suggested in [5]. The similarity function k,l i,j is the distance between the pixels y(i, j) and y(k, l) and it is defined as follows: k,l i,j = m,p G m,p (y(i + m, j + p) y(k + m, l + p)) 2 k, l Ω. (5) with G m,p being a Gaussian kernel of size L L and variance a and L 1 2 m, p L 1 2, (see [5] for more details). As we can see from (3), in the case of sigma filtering, the weights associated with the neighboring pixels that have the value close to the current pixel 3 are equals to unity while the other weights are set to zero. In the case of the NL averaging, in order to measure the closeness between the current pixel y(i, j) and the pixel y(k, l), the weighted mean squared error between two rectangular L L windows centered at y(i, j) and respectively y(k, l) is computed by (5). The pixels y(k, l) that give smaller distance to y(i, j) have the largest weights (see (4)). Finally, the output of the filter at the current position is computed by (2) in both algorithms. We mention here, that the name non-local averaging come from the fact that this algorithm was initially introduced with Ω = I (search window was the entire image). This makes the computational complexity prohibitive therefore, in [5] the solution Ω I was proposed. 3 They are inside the interval [y(i, j) 2σ n, y(i, j) + 2σ n].
3 Proposed approach The two filtering methods described in the above section have some advantages and disadvantages. The main advantage of the sigma filtering is its simplicity and good de-noising properties in Gaussian noise environments. However, for non-gaussian noise its performance degrades. The NL averaging method has better filtering ability, in Gaussian noise environments, compared with the sigma filter and other well known approaches (see [5] for more details). The two main disadvantages of this method are its increased computational complexity and the poor performance for non-gaussian noise. Both of the above algorithms have poor performance in non-gaussian noise due to the fact that computation of the weights W k,l is based on Gaussian assumption. For instance (3) comes from the observation that 95.5% of the samples of a Gaussian distributed variable lies within [µ 2σ, µ + 2σ] with µ and σ being the mean and the standard deviation of the additive noise. The weights of the NL-means filter are computed in (4) based on some distance measure which is computed as a weighted mean squared error. This is also very sensitive to impulses since any impulse that appear inside the local window of size L L centered around y(i, j) or centered around y(k, l) will increase the value of k,l i,j and decrease the corresponding weight. As a consequence, in the vicinity of an impulse the number of selected neighbors is too small, therefore the noise reduction of the NL-averaging is poor. We propose here an alternative way to obtain W k,l in order to improve the impulse rejection (another way to improve the impulse rejection capabilities is to implement some nonlinear function in (2) when computing the output pixel). Basically, the weights are computed as in (4) but the difference measure i,j k,l is given by the following equation: k,l i,j = (y F(i, j) y F (k, l)) 2, k, l Ω. (6) where y F (i, j) and y F (k, l) are some estimates of the noise-free pixels. The first step of our approach is to compute a rough estimate of the filtered image denoted as y F (i, j). We propose to use for this step the multistage median filter that have good filtering performances for mixed noise [3] 4. The weights are computed by (6) and (4) using the filtered pixels y F (i, j) and the output pixel 4 Other nonlinear filters can be implemented as well. is given by (2). Computing the weights W k,l based on the rough estimate y F (i, j) which does not contain impulses provide the robustness or our proposed method. In our approach, the impulses are rejected by the nonlinear filter and the weights are not influenced by the presence of an impulse into the vicinity of the current pixel. For instance, if the current pixel y(i, j) is an impulse, its corresponding pixel y F (i, j) is impulse free. As a consequence, the current pixel is averaged together with other impulse free pixels, which leads to the reduction of the impulses 5. We should emphasize here that, also in our approach, the value of the parameter h 2 in (4) is chosen proportional to the variance of the Gaussian component of the additive noise. If the additive noise is purely impulsive (zero variance of the Gaussian component), the value of this parameter is limited to a small value. Another difference between our approach and the approach from [5] is that we are using the variance of the noise from y F (i, j) to setup the parameter h 2 and not the input noise from y(i, j). This is motivated by the fact that the similarity measure k,l i,j is computed on y F (i, j). Comparing (5) and (6) we can see that our proposed approach has a much less computational complexity than the NL averaging method due to the fact that the difference measure k,l i,j is simply the squared difference of two pixels and not a weighted mean squared error 6. Computation of y F (i, j) does not increase too much the complexity. 4 Experimental results In this section, we compare the performances of our proposed algorithm with the sigma filter from [7], multistage median (MSM) filter of [3] and NL averaging method from [5]. The experiments are conducted for two types of additive noise. The first noise model is Gaussian with zero mean and variance σ 2 n = 50 and the second one is a mixture of Gaussian and impulsive noise. The following parameters of the algorithms were used in our experiments: the size of the search window M = 21, the size of the Gaussian kernel in (5) was L = 7, a = 1.5 and h 2 = σ 2 n for the NL averaging algorithm as was suggested in [5]. For our proposed de-noising method we have used M = 21, 5 Also a method that uses a nonlinear function in (2), for impulse rejection, is under consideration. 6 In fact its complexity is closer to the complexity of sigma filter.
Table 1: The MSE of the compared algorithms for Gaussian additive noise. Image Proposed MSM Sigma NL Lena 31.28 35.22 29.27 23.40 Barbara 38.59 46.46 30.56 20.71 Airplane 26.92 33.62 27.08 22.32 Boat 35.12 38.61 32.19 26.69 Table 2: The MSE of the compared algorithms for mixed additive noise. Image Proposed MSM Sigma NL Lena 37.41 39.25 224.10 221.67 Barbara 49.31 54.96 214.92 207.34 Airplane 38.92 41.97 213.18 211.93 Boat 45.73 47.56 212.84 210.53 Original Clean Image Input Image Proposed a) Multistage Median b) Sigma filter c) NL averaging d) e) f) Figure 1: Gaussian noise case: a) original image b) input noisy image, c) proposed method, d) multistage median, e) sigma filter, f) NL averaging h 2 = 3σFn 2 and the size of the local window used to compute y F (i, j) was 5 5 (σfn 2 is the variance of the residual Gaussian noise component from y F (i, j)). The sigma filter and MSM filter were implemented with a 5 5 window size. The mean squared error between the filtered image and the original clean image, for all algorithms, are shown in Tab. 1 for Gaussian noise and in Tab. 2 for mixed noise. From these tables we can clearly see that for Gaussian additive noise the better performance is obtained with NL averaging method while the worst case result is obtained with the multistage median filter. The performances of our proposed algorithm and the sigma filter are situated between these two. For the mixed noise, the results presented in Tab. 2 show that our proposed method have the best denoising capability compared with the other three filters. As expected the sigma filter and the NL averaging method fail to discard the impulses from the input image thus giving larger levels of MSE. To visually compare the performances of the four algorithms, in Fig. 1 and Fig. 2 we show the original, the input noisy image and the filtered images for Gaussian and mixed noise respectively. For mixed noise environments our proposed method clearly outperforms the other algorithms, providing smoother output compared with the MSM filtering and better im-
Original Clean Image Input Image Proposed a) Multistage Median b) Sigma filter c) NL averaging d) e) f) Figure 2: Mixed noise case: a) original image b) input noisy image, b) proposed method, c) multistage median d) sigma filter, e) NL averaging pulse rejection compared to the sigma and NL-averaging methods. Moreover, comparing our proposed method with the sigma filter, for Gaussian noise case, we can see that although sigma filter gives better MSE values, our proposed method provides better overall visual quality (for such large noise level, after sigma filtering some noisy pixels still remain in the output image. The visual effect of those pixels is similar to the impulsive noise as can be easily seen in Fig. 1 e)). Noise variance estimate is used in the sigma filtering (see (3)), in the NL averaging method and in our proposed algorithm (to setup the value of the parameter h 2 in (4)). In our experiments, we have used the algorithm introduced in [11] to estimate the variance of the Gaussian noise component from the images and use it to setup the parameters of the compared algorithms. The final comparison concerns the computational complexity and the processing time. Besides the good impulse rejection performances another target of our new implementation is to reduce the computational complexity of the NL-averaging method. We have measured the processing time of all compared algorithms and they are summarized in Tab. 3. These re- Table 3: The processing time of the compared algorithms in seconds. Proposed 16.22 Multistage Median 8.83 Sigma filter 2.38 NL-averaging 639.39 sults were obtained on a PC with 3Ghz processor and for an image of size 256 256. From Tab. 3 we can see that multistage median filer and sigma filter have shorter processing time compared with our approach. However, they provide weaker performances in terms of visual quality and MSE than our proposed algorithm. Clearly the running time of our implementation is much smaller (about 30 times) compared to the running time of the NL-averaging. 5 Conclusions In this paper, we have proposed a new algorithm for noise reduction in images. Comparing with the NL-
averaging method, our filtering approach has the advantage of lower computational complexity and robustness in mixed noise environments. Further improvements of this de-noising technique are under consideration and they will be the subject of future publications. References: [1] R. C. Gonzalez and R. E. Woods, Digital Image Processing -second edition, Prentice Hall, USA, 2002. [10] V. Katkovnik, A Multiresolution Nonparametric Regression and Image De-noising, in Proceedings of IEEE International Conference on Image Processing ICIP 2003, Barcelona, Spain, 2003, pp. 883 887. [11] R. C. Bilcu and M. Vehvilainen, A New Method for Noise Estimation in Images, in Proceedings of IEEE International Workshop on Nonlinear Signal and Image Processing NSIP 2005, Saporo, Japan, May 2005. [2] A. Rosenfeld and A. C. Kak, Digital Picture Processing -second edition, Academic Press, USA, 1982. [3] J. Astola and P. Kuosmanen, Fundamentals of Nonlinear Digital Filtering, New York: CRC Press, USA, 1997. [4] R. C. Bilcu and M. Vehvilainen, A New Image De-Noising Technique Based on Image Decomposition and Sigma Filtering, in WSEAS Transactions on Communications, vol. 4, no. 8, pp. 579 586, Aug. 2005. [5] A. Buades, B. Coll, and J. M. Morel, Image Denoising by Non-local Averaging, in Proceedings of IEEE International Conference on Audio Speech and Signal Processing ICASSP 2005, Philadelphia, USA, 2005. [6] A. Foi, R. C. Bilcu, V. Katkovnik, and K. Egiazarian, Anisotropic Local Approximations for Pointwise Adaptive Signal-Dependent Noise Removal, in Proceedings of European Signal Processing Conference EUSIPCO 2005, Antalya, Turkey, 2005. [7] J. S. Lee, Digital Image Smoothing and the Sigma Filter, in Computer Graphics Image Processing, vol. 24, pp. 255 269, 1983. [8] D. L. Donoho and I. M. Johnstone, Threshold Selection for Wavelet Shrinkage of Noisy Data, in Proceedings of the 16th IEEE International Conference on Engineering Advances: New Opportunities for Biomedical Engineers, Baltimore, USA, Nov. 1994, pp. A24 A25. [9] D. L. Donoho, De-Noising by Soft- Thresholding, in IEEE Transactions on Information Theory, vol. 41, no. 3, pp. 613 627, May 1995.