On the evaluation of edge preserving smoothing filter Shawn Chen and Tian-Yuan Shih Department of Civil Engineering National Chiao-Tung University Hsin-Chu, Taiwan ABSTRACT For mapping or object identification, the edges possess important information. It would be desirable to preserve the edges in the original image, while applying smoothing filter to reduce the influence of noise. A number of filters are available for this purpose, including the Mean Filter, Median Filter, Kuwahara Filter, and Symmetric Nearest Neighbor Filter (SNNF). This study investigates the evaluation scheme for the performance of these filters. Traditional numerical indices, and the recently developed Smoothing/Sharpening measures are applied for comparison. From the experiments with simulated noises, it is found that the Smoothing/Sharpening measures do provide intrinsic information. However, what the meaning of the quality is still remains for discussion. 1. INTRODUCTION Over the past few years, various filters have been proposed based on numerous purposes. Consequently, criteria quantifying the performances of the filters are desired. Two general classes of criteria are used as the basis for such evaluations. [Gonzalez and Woods (1992)]: a) Objective fidelity criteria: a simple and convenient mechanism for quantitating the differences between two images by letting functions represents images. b) Subjective fidelity criteria: human observers evaluate different images and averaging their evaluations. In this study, objective fidelity criteria are preferred. Following are indices commonly used. a) Mean Absolute Error (MAE): M 1 N 1 1 ˆ, x= 0 y= 0 ( ) (, ) MAE = f xy f xy (1) MN b) Root Mean Square Error (RMSE): 1 e = f x y f x y rms [ ˆ (, ) (, )] M 1 N 1 2 σ e MN x = 0 y = 0 c) Signal to Noise Ratio (SNR): = (2)
2 S σ SNR = = 10 log 10 MSE σ 2 e (3) Where σ 2 is the variance of the original image and 2 σ is the variance of the error. e d) Peak Signal to Noise Ratio (PSNR): PSNR = 10 log 10 ( peck-to-peak value of the referenced image) 2 2 σ e (4) e) Entropy: M 1 N 1 = x= 0 y= 0 ( ) ( Entropy P x, y log P x, y ) (5) Where P(x,y) denotes the probability of the pixel(x,y) occurs. These methods evaluate errors among the whole image providing general information. As for the edges which are poorly represented in terms of the number of pixels, a small number of significant errors in edge preservation will not be represented well in the overall error measure. In order to appropriately evaluate filters designed for preserving edges, two measures are proposed to accomplish this goal [Judith et al., 1999]. 2. SHARPENING VS. SMOOTHING Sharpening and smoothing are two effects induced by a filter. Most performance measures capture both effects in a single number, such as the RMSE, can not conform to visual judgment. Therefore, it is reasonable to consider these two factors separately especially while evaluating the performances of edge-preserving filters. The following steps estimate sharpening and smoothing values. a) Plot a scattergram: A scattergram is plotted of the pixel of the gradient magnitude of the original image versus those of the gradient magnitude of the filtered version. Figure 1 shows an example. f x f = f y (6) 2 f f f = mag( f ) = + x y 2 (7) Where f denotes the gradient vector of function I(x,y) f denotes the gradient magnitude of function I(x,y)
Figure 1 Scattergram of gradient magnitude images of original image (x-axis) and a filtered version (y-axis) b) Fit lines: Use a robust estimation method to fit a line y = ax + b through the two sets. Achieving a density-independent estimate of the factors in which edges are sharpened and flat regions are smoothed. a, b = argmin y ax b A A + ( ) ( ) ( ab, ) ( x, y) A ( ab, ) ( x, y) B ( ) a, b = argmin y ax b B B (10) ( + ) (11) Where a A indicates the smoothing induced by the filter. a B gives an indication of the sharpening effect of the filter. a 1 and a 1 are required, so the values are clipped at 1 if necessary. A B c) Weight slopes: The slopes found are weighted with the relative number of points used for the estimate, in order to specify the number of pixels actually used to estimate these values. (, ) = ( 1) Smoothing f I a (, ) = ( 1) Sharpening f I a A B A A + B B A + B (12) (13) Where 1 a = is substituted to obtain numbers in the same range [0, >. A a A The two values can be considered to be an amplification factor of edges, and an attenuation factor of flat regions, respectively. If an image is only filtered without any further processing, it can be used to compare the filter operation.
3. SMOOTHING FILTERS There are two categories of image enhancement techniques: spatial domain methods and frequency domain methods. The spatial domain refers to the image plane itself, the approaches in this category are based on direct manipulation of pixels in an image. Frequency domain methods are based on modifying the Fourier transform of an image. In this paper, to understand the performances of filters with different intensions, eight filters are examined in the spatial domain. a) Mean filter: Mean filtering is a simple, intuitive and easy to implement method of smoothing images, i.e. reducing the amount of intensity variation between one pixel and the next. It is often used to reduce noise in images. b) Median filter: The median filter is normally used to reduce noise in an image, somewhat like the mean filter. However, it often does a better job than the mean filter of preserving useful detail in the image. c) Adaptive filter: The Wiener filter is the optimal filter for restoration in the presence of noise by minimizes the root mean-square error (rms). d) Symmetric nearest neighbor filter: The SNN filter compares each pixel to its 8-connected neighbors. The neighbors are inspected in symmetric pairs around the center, i.e. N S, W E, NW SE, and NE SW. Select half the number of pixels in a square window by selecting one pixel nearest in gray value to the center pixel from each pair of pixels. For a (2n+1) (2n+1) window centered at the pixel(x,y) in the image, from each pair of pixels {( x + i, y+ j)(, x i, y j) } where n i, j + n, Select g ( x+ i, y+ j) if g ( x, y) g( x+ i, y+ j) g( x, y) g( x i, y j) Select g ( x i, y j) if g ( x, y) g( x+ i, y+ j) g( x, y) g( x i, y j) < ; > ; Otherwise, select (x,y). e) Gaussian smoothing: The Gaussian smoothing operator is a 2-D convolution operator that is used to blur images and remove detail and noise. In this sense it is similar to the mean filter, but it uses a different kernel that represents the shape of a Gaussian (`bell-shaped') hump. The degree of smoothing is determined by the standard deviation of the Gaussian. f) Kuwahara filter: A Kuwahara filter is a non-linear edge-preserving smoothing filter. It takes a kernel around the objective pixel in the input image of size J=K=4L+1, where L is an integer. This kernel is split into four overlapping regions as shown in Figure 2.
Figure 2 Four, square regions defined for the Kuwahara filter. In this example L=1 and thus J=K=5. Each region is [(J+1)/2] x [(K+1)/2]. In each of the four regions (i=1, 2, 3, 4), the mean brightness µ i, and the variance σ 2 i of the k 2 pixel grey-values are measured. The output value of the center pixel in the kernel is the mean value of the region having the smallest variance. g) Unsharp masking: The unsharp filter is a simple sharpening operator which derives its name from the fact that it enhances edges (and other high frequency components in an image) via a procedure which subtracts an unsharp, or smoothed, version of an image from the original image. 4. EXPERIMENT In the experiment, one portrait image was used as base image, shown in Figure 3. The goal of this experiment was to see how the mentioned measures perform, and if measures really conform to the operation of the filters. Deliberately corrupting an image with noise allows us to test the resistance of an image-processing operator to noise and assess the performance of various noise filters. In the instruction to the subject, three types of noises, Gaussian Noise, Salt and Pepper Noise, and Speckle Noise were added to the image, shown in Figure 3. Table 1 Quality measures for images corrupted by simulated noises. RMSE MAE SNR PSNR Entropy Sharpening Smoothing Gaussian 18.01 14.3765 8.45 23.02 7.67-0.09 0.02 Salt and Pepper 13.33 11.06 25.64 7.46-0.03-0.96 1 4648 Speckle 13.23 11.13 25.70 7.57-0.15 1.61 10 7114 Three corrupted types of images processed with eight different filters, and also quality measures calculated by previously mentioned methods are shown in Appendix A, B, and C.
(a) original (b) with Gaussian noise (variance=0.005) (c) with salt and Pepper noise (p=1%) (d) with speckle noise (variance=0.01) Figure 3 Original image with simulated noises. 5. CONCLUSION Our experiment shows that: a) General measures including MAE, RMSE, SNR, PSNR, and Entropy have no identical trend indicating which filter outperforms others. b) Unsharp filter cannot remove noises by comparing almost any measures we used, and this inference conform visual judgment as well. c) Sharpening and smoothing measures correlate quite well with human perception. However they cannot offer further information other than these two effects at present. d) SNNF as well as kuwahara performs better than other filters since they not only remove noises, but retain the edges and details.
6. REFERENCE D. de Ridder, R.P.W. Duin, P.W. Verbeek, and L.J. van Vliet. On the application of neural networks to non-linear image processing tasks. In Proceeding International Conference on Neural Information Processing 1998 vol. I, pages 161-165, 1998. Fisher, R., S. Perkins, A. Walker and E. Wolfart, 2000. Hypermedia Image Processing Reference, Artificial Intelligence, the University of Edinburgh, http://www.dai.ed.ac.uk/homes/rbf/hipr/hiprsrc/html/usrguide.htm. Gonzalez, R. C., and Woods, R. E., 1992, Digital Image Processing, Addison Wesley. Harwood, D., M. Subbarao, H. Hakalahti and L.S. Davis, 1987. A New Class of Edge-Preserving Smoothing Filters, Pattern Recognition Letters, 6:155-162 J. Dijk, D. de Ridder, P.W. Verbeek, J Walraven, I.T. Young, and L.J. van Vilet. A quantitative measure for the perception of sharpening and smoothing in images. 5 th Annual Conference of the Advanced School for Computing and Imaging, ASCI, Delft, 1999, pp. 291-298. Mark A Schulze and Qing X Wu. Nonlinear Edge-Preserving Smoothing of Synthetic Aperture Radar Images. In Proceeding of the New Zealand Image and Vision Computing 95 Workshop, pp. 65-70. (Christchurch, New Zealand, August 28-29, 1995.)
APPENDIX A: CORRUPTED IMAGES WITH GAUSSIAN NOISE (a)mean filter 3 3 (b)median filter 3 3 (c)snn-mean filter 3 3 (d)snn-median filter 3 3 (e)adaptive filter 3 3 (f)gaussian filter 3 3
(g)kuwahara filter 3 3 (h)unsharp filter 3 3 Figure 4 Corrupted images with Gaussian noise smoothed by different filters. Table 2 Quality measures for smoothed images RMSE MAE SNR PSNR Entropy Sharpening smoothing Mean 11.07 7.5412 12.68 27.25 11.52 0.0083 2.4465 Median 10.4 7.7352 13.22 27.79 7.49 0.0004 1.9397 Adaptive 8.89 6.6959 14.58 29.15 12.28-0.0013 2.0769 SNN-mean 12.2 9.3764 11.84 26.41 9.54 0.0077 0.7777 SNN-median 12.69 9.6921 11.49 26.06 8.55 0.0096 0.7093 Kuwahara 13.51 9.9203 11.5 25.52 9.54 0.0102 0.5792 Unsharp 62.39 51.2055-2.34 12.23 7.44 1.0197 0.0059 Gaussian 12.03 9.6063 11.95 26.52 16 0.0008 0.2802 APPENDIX B: CORRUPTED IMAGES WITH SALT AND PEPPER NOISE (a)mean filter 3 3 (b)median filter 3 3
(c)snn-mean filter 3 3 (d)snn-median filter 3 3 (e)adaptive filter 3 3 (f)gaussian filter 3 3 Figure 5 (g)kuwahara filter 3 3 (h)unsharp filter 3 3 Corrupted images with salt and pepper noise smoothed by different filters.
Table 3 Quality measures for smoothed images RMSE MAE SNR PSNR Entropy Sharpening smoothing Mean 10.33 5.7503 13.27 27.85 11.48 0.018 0.2646 Median 6.2 2.8217 17.71 32.28 7.4 0.0018 0.0462 Adaptive 12.4 4.3158 11.69 26.26 11.88 0.0052 0.1173 SNN-mean 7.19 3.7229 16.43 31 9.42 0.0412 0.2287 SNN-median 8.04 4.0701 15.46 30.03 8.43 0.0503 0.2595 Kuwahara 19.53 3.9174 8.3 22.32 9.41 0.0517 0.1945 Unsharp 28.62 16.1291 4.43 18.99 7.64 0.7243 0.0015 Gaussian 9.21 3.0876 14.27 28.85 15.7 0.0076 0.0729 APPENDIX C: CORRUPTED IMAGES WITH SPECKLE NOISE (a)mean filter 3 3 (b)median filter 3 3 (c)snn-mean filter 3 3 (d)snn-median filter 3 3
(e)adaptive filter 3 3 (f)gaussian filter 3 3 (g)kuwahara filter 3 3 (h)unsharp filter 3 3 Figure 6 Corrupted images with speckle noise smoothed by different filters. Table 4 Quality measures for smoothed images RMSE MAE SNR PSNR Entropy Sharpening smoothing Mean 10.26 6.5716 27.91 0.5979 11.49 0.0129 1.6084 Median 9.52 6.9684 28.56 0.7386 7.44 0.0004 0.9568 Adaptive 7.31 5.4535 30.85 0.0158 12.04 0.0005 1.1628 SNN-mean 10.83 8.1972 27.44 0.3381 9.48 0.0139 0.4683 SNN-median 11.75 8.8663 26.73 0.4278 8.50 0.0179 0.3533 Kuwahara 11.14 8.4996 27.19 1.0373 7.49 0.0181 0.3623 Unsharp 50.71 41.6313 14.03 0.6737 7.68 1.1220 0.0068 Gaussian 9.13 7.3169 28.92 0.1842 15.99 0.0018 0.2189