EE64 Final Project Luke Johnson 6/5/007 Analysis of the SUSAN Structure-Preserving Noise-Reduction Algorithm Motivation Denoising is one of the main areas of study in the image processing field due to many factors. Due to the nature of imaging systems and detectors, any image will always have some sort of noise obscuring the actual scene being imaged. In astronomy and other fields that tend to work in low signal-to-noise regimes, this noise can be highly problematic in that it obscures the underlying image to the point where it can be unrecognizable. Intelligent noise-reduction algorithms make it possible to extract data from these noisy images that is both qualitatively and quantitatively meaningful. Without such algorithms, these fields would be severely limited. Another field where denoising is critical is computer vision. Even a small amount of noise can fool an otherwise intelligent object recognition algorithm so denoising must be an integral part of any computer vision application. Given such wide application, it is no surprise that noise reduction in image processing has inspired a large volume of scientific research and has generated more and more advanced algorithms over the years. The SUSAN algorithm was published in 1997 by Stephen Smith and J. Michael Brady and while it has since been surpassed by more advanced algorithms, it offers a good conceptual introduction to the concepts on which many modern denoising algorithms are based. The SUSAN principle The SUSAN principle, when introduced in the late 1990 s, was a new approach to image processing that inspired much scientific research. In the past decade the original paper has been cited almost 00 times and has led to advanced image processing techniques that achieve very impressive results. In Smith and Brady (1997) the SUSAN principle is applied to three separate but related image processing problems: edge detection, corner detection, and denoising. This project will focus on the application of the SUSAN principle to denoising so the edge and corner detection algorithms will not be discussed. The essence of the SUSAN noise reduction filter is to take a noisy image and replace each pixel in the image with a weighed average of its neighbors. This is similar to linear denoising methods in which the image is convolved with a weighting kernel that replaces each pixel in the image with a weighted sum of its nearest neighbors. What sets the SUSAN method apart from these linear methods, and results in a very non-linear filter, is that the weighting kernel changes with each pixel and is highly dependent on the image structure. As a result of this non-linearity, the performance of the filter will vary dramatically from one image to another depending on both the spatial and brightness structure of the image and the types of noise present. This makes traditional statistical analysis of the filter s performance very difficult so in lieu of mathematical analysis I shall present a qualitative and quantitative analysis of the filter s performance on some images that are widely used in the image processing community.
Algorithm The key to the SUSAN denoising algorithm is the determination of the Univalue Segment Assimilating Nucleus (USAN). In theory, the USAN consists of the pixels within the mask that correspond to the same image section as the pixel at the center of the mask. This is expressed in the brightness domain as the pixels inside the mask whose brightness is closest to that of the center pixel. The S in SUSAN stands for smallest and is derived from the feature detector where the pixel associated with the smallest USAN in its local area is identified as an edge or a corner pixel. In the noise reduction algorithm, SUSAN is a somewhat meaningless term because the size of the USAN is not that useful, however this is called the SUSAN filter because the USAN is used as the local smoothing kernel. The SUSAN filter uses a circular mask with radius 3.4 pixels for a mask that covers 37 pixels total. It is circular in order to make the filter response as isotropic as possible. I m not sure why the authors chose a fixed radius of 3.4 pixels instead of letting the radius of the mask vary, however the 37 pixel mask seems to provide a large enough area to get good smoothing results in most cases. The underlying assumption used to preserve structure is that the neighboring pixels which are part of the same image feature as the nucleus will be of similar brightness. Thus, for each pixel in the image, the mask is placed with the pixel in question at the center of the mask, and the USAN is calculated by evaluating the brightness between each mask pixel and the nucleus and then comparing it to a threshold. For example, a binary comparison could be used: 1if I ( r) I ( r0 ) t c( r, r0 ) = 0 if I ( r) I ( r0 ) > t where r is the coordinate of a mask pixel represented in vector form, r 0 is the nucleus position, I is the brightness, and t is the brightness difference threshold. In this case, the value of c would indicate whether or not the pixel r was part of the USAN or not. In the actual implementation of the filter, this comparison function is modified to be a Gaussian weighting function so that it is smoother and therefore provides more desirable performance. In this case: c( r, r 0 ) = e I ( r) I ( r ) 0 t This algorithm also makes the assumption that pixels that are spatially nearer to the nucleus pixel are also more likely to be part of the same image feature so it also applies a Gaussian weighting to each pixel that takes into account its spatial distance from the nucleus. Therefore, the overall weighting for a given pixel in the mask is: r ( I ( x, y) I ( x i, y j)) + σ t w( x, y) = e where x,y are the image coordinates of the mask pixel, i and j are the respective distances from the nucleus in the x and y directions, r = i + j and σ is a spatial smoothing factor. Once these weights have been calculated, the final step is to sum each pixel in the mask multiplied by its weight and then normalize by the USAN area (the sum of all the weights). This value is then used as the new value for the center pixel. One important thing to note is that the SUSAN algorithm does not include the center pixel of the mask in the kernel so when calculating the weighted average either omit the center pixel or else give it a weight of zero. In some cases
this may result in the USAN having a zero or near-zero area. In this case it is assumed that since the center pixel is the only pixel in its USAN, it must be impulse noise. If this happens, replace the center pixel with the median of its eight neighbors instead of using the SUSAN weighting kernel. The final algorithm for calculating the new value of pixel x,y is: J ( x, y) = i, i 0 j, j 0 I( x+ i. y+ i, i 0 j, j 0 e j) * e r ( I ( x+ i, y+ j) I ( x, y )) + σ t r ( I ( x+ i, y+ j) I ( x, y)) + σ t I have written a Matlab function, SUSAN_filter.m, that runs this filtering algorithm on a given image. The m-file is included in the ZIP archive attached to this report. To run the function, make sure the m-file is in your path and type help SUSAN_filter for information on how to call the function. Analysis In this analysis section, I will present a basic demonstration and then a quantitative and qualitative investigation of the properties of the SUSAN structure-preserving noise-reduction filter. The first section is a demonstration of the filter and its ability to remove different types of noise from images as well as some discussion of the filter s properties. The second section will focus on a quantitative analysis and comparison to other denoising methods and the third section will be a more qualitative comparison on the SUSAN filter to other modern non-linear noise reduction techniques. Demonstration of SUSAN filter Figure 1a shows image used for this part of the analysis. It is named test,pgm and can be found in the ZIP archive attached to this report. It was created by Smith and Brady to showcase the efficacy of the SUSAN denoising algorithm and serves as a good example because it contains simple shapes that make it very obvious which features in the image are being preserved and which aren t. Figure 1b and 1c show the image after 1 pass with the SUSAN filter and the residual between images 1b and 1a. From this figure it can be seen that in the presence of no noise, there is very little degradation of the image structure due to the filtering algorithm. This means that the filter is almost idempotent and can therefore be used multiple times on the same image in order to further reduce noise without causing too much degradation of the actual image structure. In figure a, Gaussian noise of standard deviation 15 has been added to the image. Figure b shows the result of applying the SUSAN filter with spatial smoothing factor 3.5 and brightness threshold 75 applied once to the image. Figure c shows the result of a second pass with the SUSAN filter, this time with smoothing factor 8 and brightness threshold 10. Figure d shows the absolute difference between figure c and a. It can be seen that in areas where the image features have a higher brightness variation than the noise, this algorithm does quite well, however in areas where the variations in brightness of the underlying image are on the order of or less than the noise in the image, then the filter has trouble separating the noise from the image features.
Figure 1a): Test image b): filtered image c): residual (contrast enhanced) Figure a): Gaussian noise added (rms = 15.1) b): one filter iteration (rms = 3.51) c): two iterations (rms =.80) d): residual (contrast enhanced)
Figure 3a): image with impulse noise (rms = 4.8) b): after one iteration (rms = 6.88) c): after two iterations (rms = 5.7) d): median filtered image (rms = 6.81) The SUSAN denoising algorithm is also designed to filter out impulse noise. A demonstration of this can be seen in figure 3. Figure 3a shows the test image altered so that % of the area of the image is randomly set to brightness 0 or 55. The results of the first and second pass of SUSAN denoising can be seen in figures 3b and 3c. The spatial smoothing factor is 0.3 for both and the brightness threshold is set at 150 for the first pass and 5 for the second. As can be seen, there is a significant reduction of the noise level however some obvious impulses still remain in the image. This is a fundamental weakness of the SUSAN principle, if there are two noisy pixels close enough together, they will be seen as part of an actual image feature instead of as a corruption of the image. One way to combat this problem could be to enforce some sort of continuity on the USAN but I have not had time to explore that possibility. One final thing to note is that although the SUSAN filter does not remove all of the impulse noise
from the image, due to the structure-preserving nature of the filter it actually achieves a better RMS error than the traditional 3 x 3 median filter. The final noise type to be analyzed is Poisson noise, encountered when imaging very faint objects. The MATLAB command imnoise was used to generate the noisy image in figure 4a. The final filtered image can be seen in figure 4c, the noise level has been significantly reduced. To see how this demo was created, see the file SUSAN_demo.m that is included in the ZIP archive. It will walk through this process step by step. Figure 4a): Poisson noise (rms = 7.65) b ) : one iteration (rms = 3.04) c) : two iterations (rms =.8) Quantitative analysis and comparison One of the key elements in using the SUSAN noise reduction algorithm is the determination of the proper values to use as the spatial smoothing, σ, and brightness threshold, t. In their original paper, Smith and Brady state that the performance of the filter is not strongly dependent on the value of t and that it only needs to be changed from its default value of 10 in extreme circumstances. I suppose that depends on the definition of extreme circumstances because in my explorations of the algorithm, I found it just as necessary to set the proper brightness threshold as it is to set the proper spatial smoothing parameter. Generally the noisier the image is, the higher the value of t needs to be and for images with low noise, a t value of less than 10 is desired in order to better preserve the structure of the image. In the script SUSAN_demo.m there is a section that plots the value of the RMS error as a function of the spatial smoothing factor and another that plots it as a function of t given a noisy image. An example of this for the Lena image can be seen in figure 5. Figure 5): Effect of varying t and σ
I was not able to determine an efficient method for finding good values of these parameters, so I settled for using a brute force method that simply plugs in an array of values for t, then having calculated the one that gives the best RMS error, it uses that value of t and plugs in an array of values for σ to minimize the RMS error again. The process can then be repeated until it converges at the best values for σ and t. I found that one iteration of this process was generally sufficient for my purposes. One other consideration is how many iterations of the filter to use. Due to the SUSAN filter s preservation of image structure, it can be run multiple times in order to further suppress noise. For each image there is an optimal number of filter iterations that minimize the RMS error, any further iterations past that will start to degrade the structure of the image. Again, I was not able to determine an efficient method for this so I used a similar guess-and-check algorithm as was used for determining σ and t. With most images, one or two iterations was the perfect amount but for images with lots of noise, more than were needed to achieve the lowest RMS error. The main problem with my formulation of a way to find the optimal parameters for filter iterations, σ, and t is that it requires a measurement of the RMS error to be made. In most real situations, this is impossible to do so another metric of image quality would have to be used in order to get the SUSAN filter to realize its full potential. The MATLAB script SUSAN_MSE_SNR.m uses this brute force algorithm to calculate a curve of peak signal to noise (PSNR) in a noisy image versus the final RMS error after filtering with the SUSAN filter. Using the Lena image, Gaussian noise was added to the image and a determination of the best filter parameters was run. Using those parameters, five iterations of noise addition and filtering were performed in order to get a reasonable statistical sample of the filter s performance. This process was repeated for 13 different PSNR values in order to produce the curve seen in figure 6. It can be seen that the SUSAN denoising algorithm works for a wide range of noise levels and even manages to obtain a factor of ten reduction in RMS for an image with initial noise variance equal to 6,500. The performances of four other noise reduction algorithms are also shown on this graph. The BLS-GSM filter (Portilla, 003b) results are taken from the results reported at http://decsai.ugr.es/~javier/denoise/description/results.htm. The other three results were generated from a Monte Carlo simulation in the same way as the SUSAN filter results. The curve for the Total Variation filter was generated by the script TV_MSE_SNR.m in conjunction with the function tv.m. The tv.m function is written by Guy Gilboa and can be found at http://visl.technion.ac.il/~gilboa/pde-filt/tv_denoising.html. Total Variation (Rudin 199) is an iterative non-linear denoising method that is especially effective at reducing Gaussian noise in an image. As can be seen, it performs similarly to the SUSAN filter at most noise levels. The reason the SUSAN filter performs so much better than this algorithm at very low PSNR is that I limited the Total Variation algorithm to a maximum of 400 iterations due to limited time. If it were allowed an unlimited number of iterations then I imagine that it would achieve similar performance to the SUSAN filter at the lowest PSNR levels. I also tested two different implementations of bilateral filtering against the SUSAN filter. Much like the SUSAN filter, the bilateral filter uses a weighted average where the weighting function is a Gaussian in both the spatial and brightness domains. The bilateral filter also allows the filter window to vary in size so it has four parameters that need to be adjusted to find the best implementation of the filter. The curve for the Tomasi (1998) bilateral filter was generated by using the script bilat_mse_snr.m in conjunction with the function bfilter.m. The
bfilter.m function was written by Douglas Lanman and can be found at: http://www.mathworks.es/matlabcentral/fileexchange/loadfile.do?objectid=1191&objecttype=file. Also, much like the SUSAN filter, the performance of the bilateral filter is heavily dependent upon the parameters used when implementing it. I modified the brute force code used with the SUSAN filter to work with the bilateral filter with a fixed window size of 5 pixels. It is possible that I could have achieved better results with the bilateral filter had I allowed the window size to also be adjusted to minimize RMS error but I did not have enough time to do so. Another factor that may have affected the results from this bilateral filter is that the function bfilter.m only takes values on the closed interval [0, 1] so in order to get it to work properly, I clipped all grayscale values outside the interval [0, 55], scaled them all down by 55, filtered the image, then re-scaled it up by a factor of 55. This is different from the other filters I ran where I allowed the grayscale to take on any value, positive or negative. If I had enough time to re-do these simulations I would alter the other scripts so they also clipped the values outside the range of [0, 55] because that would be a more realistic scenario, but unfortunately I didn t realize this problem until it was too late. As such, the RMS error values at low PSNR may not be truly representative of the noise reduction algorithms. Regardless of this, the bilateral filter is similar in performance to the SUSAN filter. One of the reasons that the SUSAN filter outperforms the bilateral filter at some of the noise levels is probably because I did not optimize the window size for the bilateral filter. Had I done that, it probably would have achieved lower average RMS errors than the SUSAN filter. 80 SUSAN filter vs. other methods 70 60 SUSAN filter (Smith 1997) total variation (Rudin 199) bilateral filter (Tomasi 1998) PSNR (db) 50 40 bilateral approximation (Paris 006) BLS-GSM (Portilla 003b) 30 0 10 0 0 5 10 15 0 5 RMSE Figure 6: Performance of SUSAN against other denoising methods
The other bilateral filter I tested uses the function bilateralfilter.m which can be found at http://people.csail.mit.edu/jiawen/#code. The script that I used to generate this curve is Bilat_MSE_SNR.m. The function bilateralfilter.m is written by Jiawen Chen and is based on Paris (006). From my limited understanding of this filter, it appears to estimate the proper filter parameters by looking at the edges of the input image. This is very nice because it is very time consuming to have to tweak four different parameters (spatial smoothing, brightness smoothing, window size, and number of iterations to run) for every image that needs to be denoised. As the results from figure 6 show, however, it doesn t work out nearly as well in practice. Without any other prior knowledge about the image, this algorithm falls well short of the others that were tested. It is true that if the function is fed an edge image that is less noisy than the image to be denoised then it can achieve much better performance; however I was mainly interested in how well it could do with no additional information other than the noisy image. It is also worth noting that this approximation to the bilateral filter runs much faster than any of the other filtering algorithms used so it appears to trade image quality for speed and ease of use. Qualitative analysis and comparison My previous analyses of the effectiveness of the noise-reduction algorithm focused on the RMS error between the denoised image and the actual image. Such comparisons are nice because they allow for a quantitative evaluation of how well the algorithm is working. However, lower RMS error does not necessarily mean a superior image. There are many other factors that are less quantifiable that determine the quality of an image. This section shows how well the SUSAN filter works on natural images and compares it to a few of the more modern filters. Figure 7a shows the Lena image with Gaussian noise of standard deviation 5 added. Figure 7b shows the result of applying 3 iterations of the SUSAN filter with spatial smoothing parameters 1.4, 3, and 0 and brightness thresholds 100, 10, and 5. Figure 7c and 7d show the results of filtering with the BLS-GSM method (Portilla 003b) and the iterative steering kernel regression method (Takeda 006). It can be seen that both of the more recent methods outperform the SUSAN filter in both image quality and RMS error. More information about the iterative steering kernel regression can be found at http://www.soe.ucsc.edu/%7ehtakeda/kerneltoolbox.htm. Images for figures 7a, 7c, 7d, 8a, 8c, 8d, 9a, 9c, 9d, 10b, and 10c were taken from http://www.soe.ucsc.edu/~htakeda/kernelreg/kernelreg.htm. The second example demonstrates the SUSAN filter s ability to reduce compression artifacts. A poor quality JPEG-compressed image can be seen in figure 9a. Obvious blocking and other compression artifacts are easily visible. Figure 9b shows the result after one iteration of the SUSAN filter with spatial smoothing factor 3 and brightness threshold 5. Figure 9c and 9d show the results after filtering with the bilateral filter and iterative kernel regression algorithms. The bilateral filter is very similar to the SUSAN filter in that it uses a Gaussian kernel in both the spatial and brightness domain in order to non-linearly smooth the image. As can be seen, although the SUSAN filter is slightly outperformed by the newer filters, it is able to achieve decent results. The final example of the SUSAN filter is on a color photograph of JFK that suffers from grain noise due to the film it was originally captured on. Film grain noise is a highly non-linear effect so it is a good test for any filter. The original image is shown in figure 10a, the SUSAN filtered image in figure 10b, the bilateral filtered image in 10c, and an image filtered with the
iterative steering kernel is shown in 10d. The parameters used in the SUSAN filter were σ = 5 and t = 0. Since there is no zero-noise image, instead of computing RMS error the difference image between the original image and the filtered image can be compared. These residuals can be seen in figure 11. The bilateral filter and the SUSAN filter both appear to do too much smoothing to the image, making it look more like a cartoon. This is apparent both from observing the filtered images and also from looking at the difference image. The fine structure of the facial lines and hair appears in the difference image, showing that it was subtracted out of the original. The iterative steering kernel seems to do a much better job preserving the fine structure of the image while at the same time smoothing the noise. To see how the figures in this section were created refer to the MATLAB script SUSAN_qualitative.m. Figure 7a): Image with Gaussian noise (rms = 5) b): SUSAN filter result (rms = 7.99) c): Portilla 003b result (rms = 6.64) d): Iterative steering kernel result (rms = 6.66)
Figure 8a): image with compression artifacts (rms = 9.76) b): Result of SUSAN filter (rms = 8.60) c): result of bilateral filter (rms = 8.5) d): Result of iterative steering kernel (rms = 8.49)
Figure 9a): image corrupted by film grain noise c): Results of bilateral filter b): Results of SUSAN filter d): Results of iterative kernel steering
Figure 10: difference images from SUSAN filter, bilateral filter, and iterative kernel steering methods. Conclusions The SUSAN structure-preserving noise-reduction algorithm has been shown to be effective in reducing noise while preserving underlying image structure. It has been shown to be especially effective in high signal regimes and in images with very sharp edges however it has trouble with large amounts of noise. In particular, the algorithm can have trouble determining the proper weighting kernel if there is too much noise present. The most obvious example of this is when there are two pixels corrupted by impulse noise that are very close together. In this case, the algorithm will by fooled into thinking that the two pixels are an image feature and average them together instead of ignoring them and averaging the pixels around them that are not as noisy. High amounts of Gaussian noise can also cause the algorithm to have trouble determining the proper weighting kernel to use and result in either a blotchy image or an overly smoothed image depending on the smoothing parameters used. Despite the shortcomings of the SUSAN filter, its underlying method: using a Gaussian kernel to reconstruct each pixel as a non-linearly weighted average of its neighbors dependent on both the spatial and luminous structure of the image, is a fundamentally strong principle. It has inspired others to build upon the work of Smith and Brady in order to develop more advanced and more effective denoising algorithms that have yielded very impressive results.
References Paris, S. and F. Durand, A Fast Approximation of the Bilateral Filter using a Signal Processing Approach, ECCV, 006). Portilla, J., V Strela, M Wainwright, and E P Simoncelli, Image Denoising using Scale Mixtures of Gaussians in the Wavelet Domain, IEEE Transactions on Image Processing. vol 1, no. 11, pp. 1338-1351, November 003. Rudin, L., S. Osher, and E. Fatemi, ``Nonlinear Total Variation based noise removal algorithms", Physica D, 60 59-68, 199. Smith, Stephen M. and J. Michael Brady, SUSAN -- A New Approach to Low Level Image Processing, International Journal of Computer Vision, 1997. Takeda, H., "Kernel Regression for Image Processing and Reconstruction", M.S. Thesis, Electrical Engineering, UC Santa Cruz, March 006. Takeda, H., S. Farsiu, and P. Milanfar, "Kernel Regression for Image Processing and Reconstruction", IEEE Transactions on Image Processing, Vol. 16, No., pp. 349-366, February 007. Takeda, H., S. Farsiu, and P. Milanfar, "Robust Kernel Regression for Restoration and Reconstuction of Images from Sparse Noisy Data", Proceedings of the International Conference on Image Processing (ICIP), Atlanta, GA, October 006. Tomasi, C. and R. Manduchi, "Bilateral Filtering for Gray and Color Images", Proceedings of the 1998 IEEE International Conference on Computer Vision, Bombay, India. Web links http://people.csail.mit.edu/jiawen/#code http://visl.technion.ac.il/~gilboa/pde-filt/tv_denoising.html http://decsai.ugr.es/~javier/denoise/software/index.htm http://www.soe.ucsc.edu/~htakeda/kernelreg/kernelreg.htm http://www.mathworks.es/matlabcentral/fileexchange/loadfile.do?objectid=1191&objecttype=file.