I-GIL KIM A THESIS PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF ENGINEER

Transcription

1 IMAGE DENOISING USING HISTOGRAM-BASED NOISE ESTIMATION By I-GIL KIM A THESIS PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF ENGINEER UNIVERSITY OF FLORIDA

2 2008 I-Gil Kim 2

3 To my family, especially my wife Jiyoung and little Julia 3

4 ACKNOWLEDGMENTS I am deeply grateful to my mentor Dr. Paul W. Chun from the bottom of my heart. He always encouraged me and gave me valuable advice. I also offer my gratitude to Dr. Fred J. Taylor, my supervisory committee chair, for his direction and guidance. I would like to thank my committee members. Finally, I express my gratitude to my father, Dr. Koojin Kim. He instilled in me in the great love I have for the natural world and taught me to be curious about science. Special thanks go to my family and friends for their love, support, and sacrifices that allowed me to be me throughout my years of study. 4

5 TABLE OF CONTENTS ACKNOWLEDGMENTS...4 LIST OF TABLES...6 LIST OF ABBREVIATIONS...9 ABSTRACT...11 CHAPTER 1 INTRODUCTION...13 page Conventional Noise Level Estimation Methods...13 Conventional Noise Reduction Methods BACKGROUNG AND RERATED REASEARCH...15 Bosco s Noise Estimation (2005, IEEE)...15 Amer s Noise Level Estimation (2005, IEEE)...16 Shin s Noise Level Estimation (2005, IEEE) NOISE LEVEL ESTIMATION USING HISTOGRAM COMPRESSION...20 Drawbacks of Conventional Noise Level Estimation Methods...20 Proposed Noise Level Estimation Method based on Histogram Compression NOISE REMOVAL FILTERING...34 Bilateral Noise Reduction Filtering...34 Improvement of Bilateral Filtering...36 Impulse Noise Removal...37 Performance Test of the Proposed Noise Reduction Method by PSNR CONCLUSION...40 APPENDIX: TEST IMAGES FOR PERFORMANCE COMPARISON...41 LIST OF REFERENCES...43 BIOGRAPHICAL SKETCH

6 LIST OF TABLES Table page 6-1 PSNR for test images

7 LIST OF FIGURES Figure page 2-1 Differences computation in homogeneous areas Absolute noise histogram Fine structure and texture images Special masks Shin s noise estimation algorithm Four types of image Homogeneous regions Color digital image acquisition Histogram compression on luminance of noisy image Relationship between σ X and σ W Graphical analysis of the histogram compression for four different types of noisy images Graphical analysis of the effect of histogram compression Homogeneous image performance test Less-homogeneous image performance test Complex Structure image performance test Non-homogeneous image performance test Homogeneous and less homogeneous image performance test Complex structure, and non-homogeneous image performance test Pixel selection for filtering based on σ Central pixel outlier detection Impulse noise reduction by using detecting central pixel outlier

8 A-1 The Kodak homogeneous and non-homogeneous photo images...41 A-2 Complex structure non-homogeneous photo images

9 LIST OF ABBREVIATIONS b ij B Non-overlapping image block Blue component of RGB color space B* Homogeneous block E k f[k] f [ k ] g[k] G Absolute difference error between true and estimated noise level Original signal Estimated original signal by filtering Contaminated signal Green component of RGB color space h[ k, ς ] Local filter MSE n[k] P s PSNR R Mean squared error Noise signal Weight of the proposed method Peak signal-to-noise ratio (db) Red component of RGB color space RGB RGB red, green, and blue color space W White Gaussian noise W [ k, ς ] Weight of bilateral filter W s [ k, ς ] Space weight of bilateral filter W R [ k, ς ] Range weight of bilateral filter X Y Y i YUV Original noise-free image Noisy image Luminance of YUV color space Luminance and chrominance color space 9

10 Z σ ij Histogram modified noisy image Standard deviation of intensity for each block b ij σ min Minimum standard deviation σ, σ Standard deviation of signal X, Y, and W X σ Y,, W ς Neighboring point 10

11 Abstract of Thesis Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Engineer IMAGE DENOISING USING HISTOGRAM-BASED NOISE ESTIMATION Chair: Fred J. Taylor Major: Electrical and Computer Engineering By I-Gil Kim August 2008 Exact noise level estimation is very useful in digital image processing. For example, some noise removal algorithms use a noise level estimation to adjust the aggressiveness of noise removal. If an estimated noise level is too low, too much noise will remain in the denoised image. If an estimated noise level is too high, the features of the original images will be removed from the denoised image. Accurate noise level estimation will produce better results in the restored image. Most conventional noise estimation methods are only focused on estimating the noise level from a noisy image including a lot of homogeneous (flat) regions. So, the typical methods induce excessive overestimation and underestimation if the given noisy image doesn t have any homogeneous regions. I propose a new noise level estimation method that uses histogram modification to find more exact noise level from the given noisy image. This new noise estimation method uses a proposed noise reduction filter based on a bilateral filter. It makes use of block-based noise estimation, in which an input image is assumed to be contaminated by the additive white Gaussian noise and impulse noise. A noise reduction filtering process is performed by the filter bilateral-based. Coefficients of the noise filter are selected as functions of the standard deviation 11

12 of the Gaussian noise that is estimated from a given noisy image by the proposed method. To accurately estimate the amount of noise in various types of noisy images, images are categorized into four different types based on their homogeneity: homogeneous, less homogeneous, complex structure, and non-homogeneous images. In the proposed method, by using histogram modification on a noisy image, fluctuation in a non-homogeneous region can be effectively suppressed without influencing its noise level. This process increases homogeneous regions in a given noisy image and improves the performance of noise estimation for most noisy images regardless of their homogeneity. To show the effectiveness of the proposed method, it is tested on various kinds of noisy images ranging from homogeneous to non-homogeneous and its performance is compared with that of three conventional noise estimation methods. The proposed noise estimation and reduction method can be efficiently used in various image and video-based applications such as digital cameras and digital television and is superior because of its performance and simplicity. 12

13 CHAPTER 1 INTRODUCTION Nowadays many people use camera phones and low-end digital cameras in their everyday life. These devices are particularly subject to noise reduction, when images are acquired in lowlighting conditions. However these devices are often used in poor lighting-conditions due to their portability. In order to obtain an acceptable picture, sometimes it is necessary to amplify the image signal taken under low light environments such as indoor scenes. However, when boosting an already degraded image signal, noise in the image also can be amplified. An effective noise reduction filter should change its strength according to the noise level in the noisy image. Here, we need some measure to estimate the noise level in the given noisy image. In general, we can get noise level information from the noise standard deviation σ. Hence many noise filters depend on σ to adaptively change their smoothing effects [1]-[4]. Conventional Noise Level Estimation Methods Conventional noise level estimation techniques can be categorized into three main classes [8]: Block-based, filter-based, and adaptive-based methods. Before 1990, filter-based methods were commonly used. Such methods perform a pre-filtering operation in which the noisy image is blurred to suppress image structure. A difference image is computed by subtracting the filtered image from the original one and then noise level is estimated using the difference image, which is assumed to contain only the noise signal. After 1990, block-based methods started to play a role in noise level estimation. Briefly, the block-based method partitions the image into a sequence of blocks. The estimation of standard deviation of a noisy image is carried out by the properly calculating the weighted noise level obtained by averaging the noise levels of the most homogeneous blocks. An adaptive method is a hybrid method which uses elements of the filterbased and the block-based methods. Its performance depends on the first noise level estimation. 13

14 Efficient techniques based on the block based approach are described in [5] and [6]. A comparison of various methods for determining an approximation of the noise level in an image is given in [7]. Conventional Noise Reduction Methods A large number of different noise reduction methods have been proposed so far. Traditional denoising methods can be generalized into two main groups: spatial domain filtering and transform domain filtering. Spatial domain filtering methods have long been the mainstay of signal denoising and manipulate the noisy signal in a direct fashion. Conventional linear spatial filters like Gaussian filters try to suppress noise by smoothing the signal. While this works well in the situations where signal variation is low, such spatial filters result in undesirable blurring of the signal in situations where signal variation is high. To overcome these drawbacks, a number of new spatial filtering methods such as total variation techniques [9], anisotropic filtering techniques [10], and bilateral filtering techniques [11,12] have been introduced to suppress noise while preserving signal characteristics in the regions of high signal variation. Bilateral filtering is a non-linear and non-iterative filtering technique which utilizes both spatial and amplitudinal distances to preserve signal detail. On the other hand, transform domain filtering methods transform the noisy signal into the frequency domain and manipulate the frequency coefficients to suppress signal noise before the signal is transformed back into the spatial domain. These techniques had been introduced by Wiener filtering [13] and wavelet-based methods [14,15] I propose novel approaches to noise level estimation and noise reduction that improve bilateral filtering. The proposed methods are robust in situations where the given noisy image is characterized by non-homogeneity and provide improved noise suppression. 14

15 CHAPTER 2 BACKGROUNG AND RERATED REASEARCH In this section, I introduce three recent noise estimation algorithms, showing good performance in IEEE journal. Bosco s and Amer s method are block-based approaches, and Shin s method is adaptive approach. These three conventional methods will be compared with my proposed method in chapters 3 and 4. Bosco s Noise Estimation (2005, IEEE) In most of block-based noise estimation methods, noise level is obtained by averaging the noise levels of the most homogeneous blocks. In order to overcome inaccuracy of averaging, Bosco proposed new method using difference histogram approximation. The detail algorithm is as follows: 1. Tessellate an input noisy image into a number of 3 3(5 5) nonoverlapping image blocks 2. Compute absolute differences between the central pixel and its neighborhood for each blocks (Figure 2-1) 3. Select a block as a homogeneous block if the 8 differences are very small 4. Compute noise histogram for selected homogeneous blocks from accumulated differences (Figure 2-2) 5. Obtain noise level(σ) by considering 68% of the noise samples. The index on the x-axis of the noise histogram allowing reaching the 68% of the noise samples Figure 2-1. Differences computation in homogeneous areas 15

16 In Bosco s method, histogram approximation method is exploited instead of averaging local standard deviations usually used in conventional method for noise level estimation. In some of case, it provides better result than other conventional methods. The main weakness of this method is that it takes more time and storage to accumulate differences. Figure 2-2. Absolute noise histogram. The index on the x-axis allowed to reach the 68% of the total samples represents the current noise level estimation. Amer s Noise Level Estimation (2005, IEEE) Selection of homogeneous areas in the given noisy image is very important in most of conventional noise estimation methods. Accuracy of block-based noise level estimation highly relies on selecting homogeneous blocks. However, many conventional methods were found to have difficulties finding homogeneous region in noisy image containing fine structures or textures (Figure 2-3). Amer introduced a new approach to overcome this drawback using special masks to find homogeneous regions. The following steps describe the proposed method: 1. Tessellate an input noisy image into a number of 3 3(5 5) nonoverlapping image blocks 2. Use eight high-pass operators and special masks for corners to stabilize the homogeneity estimation for each block. (Figure 2-4) 16

17 3. Compute summation of the absolute values of all eight quantities from step 2) 4. Select the block as a homogeneous block if the summation is less than threshold 5. Average local standard deviation of the selected homogeneous blocks Amer s method gives better noise estimation performance for the images which have texture and fine structure by using special masking if the noise level of noisy image is not too high. The number of detected homogeneous regions is rapidly decreased when given noise level is high. So it is hard to find homogeneous areas in this case. Figure 2-3. Fine structure and texture images. Figure 2-4. Special masks Shin s Noise Level Estimation (2005, IEEE) Shin s noise estimation algorithm is based on both block-based and filtering-based approaches. It selects homogeneous blocks by a block-based approach and then filters the 17

18 selected homogeneous blocks using a filtering-based approach. Block-based methods require a low computational load whereas filtering-based approaches yield a stable estimate. Fig. 2 shows the flowchart of Shin s estimation noise estimation method using adaptive Gaussian filtering. Each step of Shin s method is described as follows. 1. Tessellate an input noisy image into a number of 3 16 nonoverlapping image blocks (b ij ). 2. Compute the standard deviation (σ ij ) of intensity for each block b ij and find the minimum standard deviation (σ min ). 3. Select homogeneous blocks (B*) whose standard deviations of intensity are close to σ min 4. Compute Gaussian filter coefficients (σ 1st is set to σ min ) 5. Gaussian filtering on the selected blocks (B*). 6. Compute the standard deviation(σ 2nd ) of the difference image between noisy and filtered images within selected blocks (B*), which gives the estimated noise standard deviation ( ˆσ n =σ 2nd ) In general, block-based approaches tend to overestimate the noise level in good quality images and underestimate it in highly noisy images. This underestimation can be compensated by adaptive Gaussian filtering of Shin s method. However, its performance is highly depend on the first estimated noise level (σ 1st ) and execution time is usually longer than other block-based approaches because of the second filtering-based estimation. 18

19 Figure 2-5. Block diagram of Shin s noise estimation algorithm 19

20 CHAPTER 3 NOISE LEVEL ESTIMATION USING HISTOGRAM COMPRESSION In this chapter I will address the drawbacks of conventional methods for noise level estimation in digital images degraded by noise and compare performances of each method with the proposed method for four types of noisy image. Drawbacks of Conventional Noise Level Estimation Methods Most conventional methods for estimating the noise standard deviation are generally based on the following steps: 1. Detect homogeneous regions in the noisy image (because fluctuations in flat areas, pixels are supposed to be due exclusively to noise). 2. Compute the local standard deviation in the detected homogeneous regions. 3. Repeat steps 1) and 2) until the whole image has been processed. 4. Finally estimate the noise level (or standard deviation of noise) by averaging the computed local standard deviations. This method has two major drawbacks. First, if there are no homogeneous regions or too small areas of flat region in a given noisy image, it is hard to detect the homogeneous regions. An insufficient number of detected homogeneous regions may result in overestimation or underestimation. This will induce wrong estimation for noise level. Second, the process of detecting homogeneous regions in a noisy image requires a high number of computations. The detection process for a non-homogeneous image may be just time wasted work and the noise estimation based on such detection has a high probability of being wrong. Theses drawbacks originate from the fact that most conventional noise estimation methods are only focused on noisy images including a lot of homogeneous regions. In order to make this problem clear, I have categorized noisy images into four types as following: 1. Homogeneous image: image contains a lot of homogeneous regions. 20

21 2. Less homogeneous image: image contains very few homogeneous regions in a limited area. 3. Complex structure image: original noise-free image has very complex structure, so it is not easy to find homogeneous regions due to local fluctuations. 4. Non-homogeneous image: there are no homogeneous regions in the given noisy image. A B C D Figure 3-1. Four types of image. A) homogeneous image, B) less homogeneous image, C) complex structure image, D) non-homogeneous image Figures 3-1 shows examples of four different types of image based on homogeneity. There are homogeneous regions in Figure 3-1 A) and B), but it is not easy to find homogeneous regions in Figure 3-1 C) and D) by using conventional noise estimation methods. Figure 3-2 shows homogeneous regions detected by Bosco[1] method and violet blocks in each image are the detected areas. 21

22 A B C D Figure 3-2. Homogeneous regions. A) homogeneous image, B) less homogeneous image, C) complex structure image, D) non-homogeneous image Proposed Noise Level Estimation Method based on Histogram Compression In this section the proposed noise estimation method is shown to overcome the drawbacks of conventional noise estimation methods by means of histogram compression on intensity of image that tries to exploit correlation among R, G, and B components of color image regardless of the number of homogeneous regions in the given noisy image. The best way to estimate an exact noise level from a given noisy image is to make the image homogeneous without influencing its noise deviation. In other words, if we can suppress deviation in the original noise-free image without changing deviation of noise in a noisy image, it is easy to find more exact noise levels from the suppressed noisy image. 22

23 Nowadays most digital images are in color and each pixel of the image has red (R), green (G), and blue (B) components. Each RGB component in a noisy color image has a noise and the noise tends to have Gaussian distribution in a digital image acquisition system. Noise X Y X = 0.3R X + 0.5G X + 0.3B X R channel : R X + W R G channel : G X + W G B channel : B X + W B where σ WR = σ WG = σ WB Y = 0.3(R X + W R ) + 0.5(G X + W G ) + 0.3(B X + W B ) Figure 3-3. Color Digital image Acquisition Figure 3-3 shows the structure of a current digital image acquisition system. Generally most common noise in digital image acquisition can be modeled by a white Gaussian noise with the same standard deviation σ W. In order to reduce the correlation between the RGB components, a conversion from RGB to YUV color space as follows: Y i (Luminance) = 0.3R i + 0.6G i + 0.1B i U i (Chrominance1) = B i - Y i V i (Chrominance2) = R i Y i However, if component noise is uncorrelated in RGB space, it is correlated in YUV space. Also the component noise and RGB component are independent to each other, so they are 23

24 uncorrelated in RGB space. A linear operation such as histogram modification in YUV space may affect uncorrected RGB components and correlated component noise. Histogram compression, one of several histogram modification methods, can be useful to suppress deviation of RGB component without affecting deviation of component noise. X = 0.2R X + 0.5G X +0.3B X Y = ax + b (a=0.01, b = 128) Y = 0.2R Y + 0.5G Y +0.3B Y Figure 3-4. Histogram Compression on Luminance of Noisy Image. The histogram plots the number of pixels in the image (vertical axis) with a particular brightness value (horizontal axis) The noisy image and original noise-free image have luminance correlation among RGB components in YUV color space. X = 0.2R X + 0.5G X + 0.3B X Y = X + W (1) = 0.2R Y +0.5G Y +0.3B Y (2) = 0.2(R X +W R ) + 0.5(G X +W G ) + 0.3(B X +W B ) (3) where X is original noise-free image and Y is a noisy image. W is an additive white Gaussian noise. Because X and W are independent of each other, variation of Y has the following relation to variation of X. σ Y 2 = σ X 2 + σ W 2 (4) If we do histogram compression on the luminance of noisy image Y Z = ay-b, (5) 24

25 α= Y - Z = (1-a)Y + b Z = Y - α = 0.2*R Z + 0.5*G Z + 0.3*B Z (6) = 0.2(R Y - α) + 0.5(G Y - α) + 0.3(B Y - α) (7) where Z is compressed noisy image in YUV. From equation (6) and (7), the R component of Z, R z becomes R Z = R Y - α = R X +W R -(1-a)Y - b = R X +W R -(1-a)(X+W) - b = [R X (1-a)X+b] + [W R -(1-a)(0.2W R +0.5W G +0.3W B )] If the histogram compression equation (5) has parameters a=0.01 and b=128, the term (1-a) can be ignored as in the following equation: R Z [R X -X+128] + [0.8W R -0.5W G -0.3W B )] = P + W p (8) where P = R X -X+128 and W p = 0.8W R -0.5W G -0.3W B. R z can be divided into the RGB component term P and noise term W p. From equation (5) and a=0.01, the variance relation between Z and Y becomes σ z 2 = 0.01σ zy 2 σ z 2 << σ Y 2 (9) In other words, equation (9) means that the variation of Y is much higher than that of Z and also the deviations of R components of Y and Z have a similar relation. σ RZ 2 << σ RY 2 (10) σ P 2 + σ WP 2 << σ RX 2 + σ W 2 from equation (2),(3), and (8) σ P σ WR σ WG σ WB 2 << σ RX 2 + σ W 2 25

26 White Gaussian noise in Figure 3-3 has the same standard deviation (σ W = σ WR = σ WG = σ WB ), so equation (10) becomes σ 2 P σ 2 W << σ 2 2 RX + σ W 2 The noise term σ W can be eliminated by approximation σ P 2 << σ RX 2 (11) The deviation relation of equation (8) is σ RZ 2 = σ P 2 + σ W 2 (12) Equations (11) and (12) show that the standard deviation of the R component is suppressed by histogram compression on luminance of a noisy image but the standard deviation is not suppressed. The numbers of homogeneous regions in a histogram-compressed noisy image are increased with remaining component noise. In order to see the effect of histogram compression more clearly, graphical analysis by using relations among σ X, σ Y, and σ W in equation (4) is helpful to understand the proposed method. Figure 3-4 shows the relation and the values of σ Y on each pixel of noisy image are distance from the origin by using the Pythagoras relationship between σ X and σ Y. Each blue point shows the relative locations of local standard deviation σ X, σ Y, and σ W of 3x3 blocks for R components. Even if σ X and σ W are unknown in a given noisy image, we can get the values of local standard deviation σ X before the noise level σ W is added to original noise free image. If blue points are close to the vertical axis, the local blocks of the points are homogeneous including homogeneous regions in the noisy image. There are a lot of homogeneous blocks in Figure 3-4 because the given noisy image is homogeneous. Hence, many blue points can be found near the vertical axis. 26

27 σ x 3x3 block masking for getting local σ Y σ w σ Y Figure 3-5. Relationship between σ X and σ W. Each blue pixel shows relative locations of local standard deviation of 3 x 3 block. However, the result of this graphical analysis may be different for the four types of image based on homogeneity as categorized in the previous section. In the case of a non-homogeneous image, most of the blue points may be apart from the vertical axis. Figure 3-5 shows the graphical analysis for the four different types of image and the effect of histogram compression on R components. The blue points in a red box indicate that the local blocks of the points are homogeneous. In the case of complex structures and non-homogeneous images, there are very few homogeneous blocks. So it is not easy to find homogeneous regions using conventional noise estimation methods. However, after histogram compression, σ X can be suppressed without influencing distribution of σ W in all four types of images. The total number of points which are included in the red box in a complex structure and non-homogeneous images is increased. In other words, homogeneous regions are increased in the given noisy images by histogram compression. 27

28 Homogeneous Less homogeneous Complex Structure Non-homogeneous Histogram Compression Histogram Compression Histogram Compression Histogram Compression Suppressed Suppressed Suppressed Suppressed Figure 3-6. Graphical analysis of the histogram compression for four different types of noisy images. Another way to show the effect of histogram compression is to compare local standard deviations before and after histogram compression for original noise-free image X, noisy image Y, and additive white Gaussian noise W separately. Figure 3-6 shows the results. It is apparent that the local standard deviation of the original noise-free image σ X can be suppressed by histogram compression. Histogram compression doesn t affect the local standard deviation of white Gaussian noise. From two graphical analyses in Figure 3-5 and 3-6, we can see that homogeneous regions can be increased by histogram compression even if there are very few such homogeneous regions in the given noisy image. With the increased number of homogenous regions, it becomes easier to more exactly estimate the overall noise level. 28

29 σ X σ X X X σ W Histogram Compression σw W W σ Y σ Y Y Y Figure 3-7. Effect of histogram compression. The proposed noise level estimation method is based on the following steps: 1. Perform histogram compression on luminance of noisy image. 2. Detect homogeneous regions in the image gotten from the step (1): homogeneous blocks can be selected by the local standard deviation which is less than a threshold 3. Estimate noise level from the detected homogeneous regions: estimated noise level is calculated by averaging the local standard deviation of detected homogeneous blocks or finding the most frequent local standard deviation by using histogram approximation. Performance of the proposed noise level estimation method was tested by comparing the results with those obtained using other three noise estimation methods: Bosco[1], Amer[2], and Shin[3], introduced in chapter 2. 29

30 A B C D Figure 3-8. Performance Test for Homogeneous Image. A) Bosco s method, B) Amer s method, C) Shin s method, D) Proposed method Figure 3-7 illustrates the performance comparison for the homogeneous image shown in Figure 3-1. Values on the horizontal axis represent actual given noise levels (standard deviations) and those on the vertical axis represent estimated noise levels (standard deviations). Red line is an ideal case of noise estimation in which the estimated amount of noise is equal to the amount of noise actually added. Blue points represent actual estimated noise levels obtained by using each NLE method. Conventional noise level estimation methods show good performance because there are a lot of homogeneous regions. However, underestimation appears at high noise levels. The proposed method shows better performance at both low and high noise levels. For the less-homogeneous image test shown in Figure 3-8, overestimation at low noise levels and underestimation at high noise levels using the conventional methods are induced. However, the proposed method still shows good performance. 30

31 A B C D Figure 3-9. Performance Test for Less-homogeneous Image. A) Bosco s method, B) Amer s method, C) Shin s method, D) Proposed method A B C D Figure Performance Test for Complex Structure Image. A) Bosco s method, B) Amer s method, C) Shin s method, D) Proposed method 31

32 A B C D Figure Performance Test for Non-homogeneous Image. A) Bosco s method, B) Amer s method, C) Shin s method, D) Proposed method The proposed method proves particularly strong when applied to a complex structure or non-homogeneous image as seen in Figure 3-9 and In these cases, some of the conventional methods show less acceptable performance due to excessive overestimation and underestimation. Neither is a problem in using the proposed method. To evaluate the performance of the algorithm, the estimation E k = σ n -σ e error is first calculated. E k is the difference between the true and the estimated noise level. The average and the standard deviation of the estimation error are then computed from all the measures. In Figure 3-11, the performance comparison is the actual test result for 24 Kodak homogeneous and less homogeneous photo images in Appendix A ( Figure A-1) which are used as test images in image processing. The left plot A) in Figure 3-11 reveals that the estimation error using the proposed 32

33 method is lower than that of other NLE methods for all noise levels. Interestingly, in the right plot B), the standard deviation of the error using the proposed method is significantly less. A B Figure Performance Test for Homogeneous and Less homogeneous Image. A) Mean of absolute error of difference between actual noise and estimated noise level, B) Standard deviation of error A B Figure Performance Test for Complex structure, and Non-homogeneous Images. A) Mean of absolute error of difference between actual noise and estimated noise level, B) Standard deviation of error Figure 3-12 shows the performance comparison for complex structure and nonhomogeneous images in Appendix A ( Figure A-2) which can not be successfully tested in conventional noise estimation methods. The proposed method shows significantly better noise level estimation for less homogeneous, complex structure, and non-homogeneous images, all of which remain problematic in the area of current image processing. 33

34 CHAPTER 4 NOISE REMOVAL FILTERING The proposed noise level estimation method described in chapter 4 has been inserted in a noise reduction system for white Gaussian noise. Basically its performance depends on the estimation of standard deviation σ for a noisy image. In order to remove noise based on the estimated noise level using the proposed method, a bilateral noise reduction filter is used because it is a non-linear filtering technique which utilizes both spatial and amplitudinal distances to better preserve signal detail. Bilateral Noise Reduction Filtering Consider a 2-D signal f that has been degraded by a white Gaussian noise n. The contaminated signal g can be expressed as follows: g[k] = f[k] + n[k] where k=(x,y). The goal of noise reduction from a noisy image is to suppress noise n and extract original noise free image f from noisy image g. In spatial filtering techniques, an estimate of f is obtained by applying a local filter h to g. f[k] = h[k,ς] ⅹ g[k] In a conventional linear spatial filtering method, the local filter is defined based on spatial distances between the particular point in the signal at center pixel location (x,y) and its neighboring points. In the case of Gaussian filtering, the local filter is defined as in the following equation: 2 ς h[ k, ς ] = exp 2 2σ s where ς represents a neighboring point. Such filters operate under the assumption that an amplitudinal variation within the neighborhood is small and that the noise signal has large amplitudinal variations. The noise signal can be suppressed by smoothing the signal over the 34

35 local neighborhood. The problem of this assumption is that important signal detail is also characterized by large amplitudinal variation. Therefore, such filters may induce an undesirable blurring of signal detail. A simple and effective solution for this problem is to use bilateral filtering, which is firstly introduced by Tomasi et al. [11] and developed from the Bayesian approach by Elad [12]. In bilateral filtering, a local filter is defined based on a combination of the spatial distances and the amplitudinal distances between a center point at (x, y) and its neighboring points. This can be formulated as a product of two local filters. One is an enforcing spatial locality and the other is an enforcing amplitudinal locality. In the Gaussian case, the bilateral filter can be defined by the following equations: f [ k ] = N ς = -N W [ k, ς ] W [ k, ς ] = W [ k, ς ] W S 1 N ς = -N W [ k, ς ] R [ k, ς ] g[ k - ς ] W [ k, ς ] W s R [ k, ς ] 2 ς exp 2σ s [Y [ k ] Y [ k ς ] exp 2σ R = 2 = 2 2 ] The main advantage of defining the filter in this manner is that it allows for non-linear filtering to enforce both spatial and amplitudinal locality at the same time. The estimated amplitude at a particular point is influenced by neighboring points if the neighboring points have similar amplitudes which are more than those with distant different amplitudes. This can reduce smoothing across signal regions which have large but consistent amplitudinal variations, thus it is better to preserve such signal detail. Furthermore, the 35

36 normalization term of the above formulation helps the bilateral filter smooth away small amplitudinal differences associated with the noise in smooth regions. Improvement of Bilateral Filtering It was advantageous to improve the bilateral noise removal filtering to obtain a more robust detection of the outliers. The proposed method uses the estimated local standard deviation σ. The bilateral filter averages only pixels that are similar to the central pixel in the filter mask. For example, assume that the mask is a 5x5 window including 9 valid pixels as seen in Figure 4-1. In order to improve the performance of bilateral filtering, two slightly different central pixels can be considered instead of P. Those are the σ-biased center P-σ and P+σ. An interval whose width is directly proportional to σ is used to select the pixels that can be averaged safely. Only similar pixels are selected for filtering by the interval centered on P because it maximizes the number of selected pixels. Figure 4-1. Pixel Selection for Filtering Based on σ Pixels are successively averaged using different weights depending on their spatial locality and locality. The final filtered pixel is obtained by the following equation: h[ k, ς ] ς = N ς = N = ς = N W [ k, ς ] P ς = N W [ k, ς ] ς P-σ P P+σ where W[k,ς] represents the weight associated to the ς-th pixel. 36

37 Impulse Noise Removal If the central pixel P is an outlier (defected by impulse noise), unfortunately there are no pixels similar to it. In this case the σ-biased centers of P in the previous section are not useful to improve bilteral filtering because there will be no pixels which are included by the corresponding intervals. outlier Min Max P-σ P P+σ Figure 4-2. Central Pixel Outlier Detection Unfortunately, an outlier can be located in the center pixel of the filter mask in Figure 4-2. To overcome this problem, we need to consider the minimum (Min) and the maximum (Max) value which is contained in the neighbor pixels of the central pixel. These values are useful to determine whether the central element in the mask is correct or affected by impulse noise. We can determine that a center pixel may be an outlier if P-σ is greater than the maximum value in the neighbor pixels. After the center pixel is classified as defective, it should be replaced by a weighted average of the remaining neighbor pixels. Figure 4-3 shows the result of proposed impulse noise reduction and bilateral noise reduction in a noisy image including impulse noise and white Gaussian noise. Impulse noise is eliminated by using the proposed method with reduction of white Gaussian noise in the cropped and magnified part of a noisy image. 37

38 Impulse noise reduction Figure 4-3. Impulse Noise Reduction by using Detecting Central Pixel Outlier Performance Test of the Proposed Noise Reduction Method by PSNR PSNR is a measurement of the similarity between two images. In PSNR higher numbers are better. If PSNR is higher than 30dB, it is hard to distinguish between the two images with the human eye. General PSNR formula is the following: MSE = 1 mn m-1 n-1 i= 0 j= 0 I(i, j) - K(i, j) 2 MAX I 2 MAX I PSNR = 10* log10 ( ) = 20* log10( ) MSE MSE where I and K are images compared to each other to show their similarity. The proposed method was applied to four different types of noisy image with different characteristics. Each test image is contaminated by white Gaussian noise with standard deviation of 5 and impulse noise. The PSNR of the restored image was measured for the proposed method as well as conventional noise estimation methods and general bilateral filtering method. A summary of the results is shown in Table 6-1. The proposed method achieves good PSNR gains over other methods for all of the test images. Furthermore, the proposed method achieves better PSNR gains over less homogeneous, complex structure, and non-homogeneous noisy images. 38

39 Table 6-1. PSNR for test images Method Image type PSNR (DB) BOSCO AMER SHIN PROPOSED Homogeneous Less homogeneous Complex-structure Non-homogeneous

40 CHAPTER 5 CONCLUSION The proposed noise level estimation and noise reduction method is valid for estimating the noise level in images affected by additive white Gaussian noise and impulse noise. Conventional noise estimation methods perform well only when applied to a homogeneous image. However, the proposed method can estimate more exact noise level from homogeneous to nonhomogeneous images and also shows good performance in noise reduction. It is particularly strong in estimating the noise level in complex structure and non-homogeneous image. The proposed noise estimation method requires fewer computational resources than conventional methods because there is no special process required to detect homogeneous regions in a non-homogeneous noisy image. Its parallelism structure would speed up performance on H/W implementation. 40

41 APPENDIX A TEST IMAGES FOR PERFORMANCE COMPARISON Figure A Kodak Homogeneous and Non-homogeneous Photo Images 41

42 Figure A-2. Complex Structure Non-homogeneous Photo Images 42

43 LIST OF REFERENCES [1] J.Brailean, R.Kleihorst, S.Efstratiadis, A. Katsaggelos, R. Lagendijk, Noise Reduction Filter for Dynamic Image Sequences: A Review, Proceedings of the IEEE, Vol.83, No.9, Sept [2] S. Battiato, A.Bosco, M. Mancuso, G. Spampinato, Adaptive Temporal Filtering for CFA Video Sequences, In Proceedings of IEEE ACIVS 02 Advanced Concepts for Intelligent Vision Systems 2002, pp , Ghent University, Belgium, September 2002 [3] A. Bosco, K. Findlater, S. Battiato, A. Castorina " A Noise Reduction Filter for Full-Frame Imaging Devices" IEEE Transactions on Consumer Electronics Vol. 49, Issue 3, August 2003 [4] A. Bosco, K. Findlater, S. Battiato, A. Castorina, - "A Temporal Noise Reduction Filter Based on Full-Frame Data Image Sensors" - in Proceedings of IEEE - ICCE 2003 [5] A. Amer, A. Mitiche, and E. Dubois, Reliable and Fast Structure-Oriented Video Noise Estimation, in Proc. IEEE Int. Conf. Image Processing, Montreal, Quebec, Canada, Sep [6] G. Messina, A. Bosco, A. Bruna, G. Spampinato, Fast Method for Noise Level Estimation and Integrated Noise Reduction, IEEE Transactions on Consumer Electronics, vol. 51, No. 3, pp , August, [7] D.-H. Shin, R.-H Park, S. Yang, J.-H. Jung, Block-Based Noise Estimation Using Adaptive Gaussian Filtering, in IEEE Transactions on Consumer Electronics, Vol. 51, No. 1, February, 2005 [8] S. I. Olsen, Estimation of Noise in Images: An Evaluation, Graphical Models and Image Process., vol. 55, pp , July 1993 [9] L. Rudin, S. Osher, Total variation based image restoration with free local constraints, in: Proceedings of the IEEE ICIP, vol. 1, 1994, pp [10] S. Greenberg, D. Kogan, Improved structure-adaptive anisotropic filter, Pattern Recognition Lett. 27 (1) (2006) [11] C. Tomasi, R.Manduchi, Bilateral filtering for gray and color images, in: Proceedings of the ICCV, 1998, pp [12] M. Elad, On the origin of the bilateral filter and ways to improve it, IEEE Trans. Image Process. 11 (10) (2002) [13] N. Wiener, Extrapolation, Interpolation, and Smoothing of Stationary Time Series, Wiley, New York, [14] J. Portilla, V. Strela, M. Wainwright, E. Simoncelli, Image denoising using scale mixtures 43

44 of Gaussians in the wavelet domain, IEEE Trans. Image Process. 12 (11) (2003) [15] Q. Li, C. He, Application of wavelet threshold to image denoising, in: Proceedings of the ICICIC, vol. 2, 2006, pp

45 BIOGRAPHICAL SKETCH I-Gil Kim obtained his B.S. degree in the Department of Electronics Engineering from Hong Ik University, Korea, in After graduation from the university, he moved to the United States to pursue his graduate studies. He received his M.S degree in Electrical Engineering from University of Southern California in He served as a software engineer at the digital media R&D center in Samsung, Korea, from 2003 to He was admitted to the University of Florida in the fall of 2005 and has worked on numerous projects in the fields of image processing and pattern recognition in the department of electrical and computer engineering while completing his engineering degree. 45