Interpolation of CFA Color Images with Hybrid Image Denoising

2014 Sixth International Conference on Computational Intelligence and Communication Networks Interpolation of CFA Color Images with Hybrid Image Denoising Sasikala S Computer Science and Engineering, Vasireddy Venkatadri Institute of Technology, Guntur, India kala.sasiv88@gmail.com Sudhakar Putheti Computer Science and Engineering, Vasireddy Venkatadri Institute of Technology, Guntur, India sudhakarp0101@gmail.com Abstract Interpolation of the CFA images is used to produce full color images by using Demosaicking Process. Denoising is a technique which is used to remove the noise from a noisy image. The noise will be introduced during acquisition, Transmission and reception & storage phases. Some sensors are using Color Filter Array (CFA) to capture images and reproduce those images into full color images with Color Demosaicking. In this paper denoising before demosaicking strategy is used. Fusion based Hybrid denoising technique is effectively removing the noise. In this paper the proposed method, Denoising is done with Wiener Filter and Median Filter. Then the resultant data vectors are fused with PCA Fusion. Fused data is used for demosaicking using Directional Linear Minimum Mean Square Estimation. The proposed method outperforms the existing techniques, which yields better visual quality at boundary locations too. The performance of the proposed method is evaluated on benchmark images and offered significant PSNR values than the existing methods. Keywords Wiener filter, Median Filter, PCA Fusion, PSNR, Demosaicking, Color Filter Array(CFA), RMSE, Wavelet transform. I. INTRODUCTION Now a day s digital color camera s using single sensor digital color camera s to capture full color images. In these cameras the Color Filter Array (CFA s) can be arranged on the top of the signal. CFA is a Mosaic pattern of color filters generated using CCD/CMOS sensors [2] which capture all three primary colors as red, green and blue at the same time. The sensor cell can consists of only one color sample at each pixel. CFA sensor reads the remaining two missing color components with Interpolation. The interpolation of CFA color images is known as the Color Demosaicking (CDM) [2]. The Color Demosaicking is also called as Debayering, CFA Interpolation or Color reconstruction [2]. Digital Color images are corrupted with random variations in the pixel intensity values called noise. Noises [6] can be classified as Gaussian, Salt and Pepper, Speckle, Uniform, Shot and Poisson Noises. Theses noises in CFA images can decrease the quality of the images and causes a very serious Color artifact to the images which are difficult to remove with single denoising technique only. Hence in this paper, Interpolation of CFA color images is done with Hybrid Image Denoising. In the recent researches most of the denoising techniques are applied in wavelet domain. These will be depending on the wavelet transform coefficients of wavelet threshold [1] selection and shrinking for image denoising. Wavelet will decomposes the image into sub images and separate the noise signal from the original signal from the basis in [3]. Adaptive wiener filter is one of the best filtering techniques used to suppress the noise of an image. Data Acquisition Addition of Noise Denoising Method Demosaicking Fig. 1: Architecture of Denoising After Demosaicking In the Fig. 1, generic architecture of denoising after demosaicking of an image is shown. Image is acquired in the first step, and then noise is added in second step. Filters are used to suppress the additive noise in the image. Finally the denoised image can be applied by the demosaicking, to reconstruct the full color images by using the Bayer Patterns. Rajni et.al., described an overview on distinct Denoising methods like Linear and Non Linear filters. Narinder Kaur et.al., [9] defined a Decision Based Median Filter and Weighted Median Filter to suppress Salt & Pepper noise in an image. These filters are used to remove a noise which other filters are failed to remove. Li Shixin, Zhang Xinghui et.al., [8] proposed a local adaptive Wiener filter applied for a 2-D Image. These 2-D images are reconstructed with 1-D windows weighted combination of filters. Sudipta Roy et.al., proposed hybridization of wavelet and bilateral filters for denoising of images of type X-ray, Ultrasound and astronomical images. Lei Zhang et.al., [2] proposed a denoising technique based on the Principal Component Analysis to remove color artifacts generated in single sensor digital cameras with cost effective implementation using Color Filter Array (CFA). Xingbo Wang et.al., put forward an extended Discrete Wavelet Transform for Multispectral filter array demosaicking for spectral correlation and spatial resolution of images. 978-1-4799-6929-6/14 $31.00 2014 IEEE DOI 10.1109/.53 10.1109/CICN.2014.53 193

II. HYBRID IMAGE DENOISING Wiener Filter Data Acquisition Additive Noise Wavelet Decomposition Denoised Image Median Filter Image Reconstruction with IDWT Bayer Pattern DLMMSE method for Demosaicking Fig 2: Architecture of the Proposed Method PCA Fusion Images can be acquired by the digital cameras with a color filter array. The CFA can be attached on the top of the sensor cameras. Such sensors can capture visual scenes into images. While transferring the visual scenes into images, the images might effected by some noise. Noise [6] is a random variation of pixels in images. In the color images the color information or brightness of the image get disturbed with the noise. In the view of removal of noise from the images, some additive noise is also added to the original image. Different types of noises are added to original images and those are shown in Fig 3. The noisy image can be described as: (1) (a) (b) (c) (d) (e) Fig 3: (a) Original Image (b) Gaussian Noise (c) Salt & Pepper (d) Poisson (e) Speckle Noise Gaussian noise in [6] is a statistical noise, which is a random fluctuation of continuous process described by probability distribution. One of the Impulse noise is Salt & Pepper noise. With this noise dark pixels in the bright regions and the bright pixels in the dark regions also generated. The Poisson noise called photon noise or shot noise. Poisson noise is an Electronic noise that occurs from the finite no. of particles Energy. Speckle noise can also be represented by the multiplicative noise. In a Local area the Speckle noise increases the mean of gray level. For further Processing the noisy image can be transferred in to Wavelet domain for image decomposition or analysis [7] & [8]. A. Wavelet Decomposition 2-D Wavelet Decomposition [3], [7] & [8] is used to divide the image into the four sub-bands called approximate, vertical, horizontal and diagonal coefficients. Again the approximate coefficients can be used for further decomposition. This process is called N-level decomposition. (a) (b) (c) Fig. 4 (a) Original Image, (b) 2-D Wavelet Decomposition at 1 level (c) 2-D Wavelet Decomposition at 2 level. Each sub-band can be applied with the Wiener and Median filter separately. By applying these filters the noise in the image can be suppressed. The denoised image can be given the better visual quality than the noisy image and it gives the better Peak signal Noise Ratio (PSNR) than the noisy image. The Wiener filter is said to be an optimal in terms of the mean square error. Wiener minimizes the mean square error of the image in the process of inverse filtering and noise smoothing. (2) Where are the variance and mean of noisy image respectively. Local mean and variance around each pixel is calculated using formulae (3) and (4) as below: LL LH HL HH (3) (4) The wiener filter can creates filtering as pixel wise filtering using the local mean and local variances. The estimation of an image is given in (2). Where is noise variance, if variance of any noise is not given, the wiener filter uses average of all local estimated variances. Wiener is a simple approach, which controls output errors and exploits signal. But Wiener is spatially invariant and it will not work on too blurred image. Median filter in [9] is a non-linear filter, which used to remove salt & pepper noise. Median filter is used to replace 194

the central value with its neighbouring pixels. The pattern of neighbours around the central pixel called the window. The median value will be calculated with sorting of all the neighbour pixel values from the window into numerical order and the middle pixel value will be replaced with all the pixels considered in the window. Median filter is applicable for fixed or different window sizes to detect the impulse noise. Most of the median filters are not at good for real time applications and they failed to yield consistent output in both low & high noise conditions. (a) (b) Fig.5: (a) Salt & Pepper Noise (b) Denoised image with Median Filter To overcome the challenges of both Wiener [7] [8] and Median filters [9], in this paper a Hybrid Denoising Method is defined with PCA Fusion [4]. B. PCA Fusion A process which is used to add two or more images into a single image is known as Image fusion [4] is used to form a high quality of an image. These fused images are very informative than the input image, gathering the information from multiple sources. Good fused image can be performed by Peak Signal Noise Ratio (PSNR), Root Mean Square Error (RMSE). PCA involves mathematical procedure. It transfers the no. of correlated variables into no. of uncorrelated variables. The uncorrelated variables are known as principal components. The PCA does not contain basis vectors. This basis vector depends on the dataset. The compact and optimal description of the data set can be computed by using PCA. In this PCA fusion [4] the spatial domain fusion produces spectral degradation and algorithm is as follows. 1) PCA Fusion algorithm: Arranging the whole data into column vector. The dimension of the resulting matrix A is n x n. Along with each column the empirical mean value will be calculated. The empirical mean can be represented by M e is of dimension 1 * n. Subtracting the mean vector M e from each column of the data matrix A. The dimension of resultant data matrix X is n* n. Find the covariance matrix C of X using the formulae C=X*X T (5) Calculate the Mean Expectation using Cov(X). Compute the eigenvectors V and eigenvalues D of Covariance matrix C in (5). Sort the eigenvalues in decreasing order. Both V and D are of dimension 2x2. Consider the first column of V which corresponds to larger eigenvalues to compute N 1 & N 2. Where N 1 & N 2 are Normalized components. N 1 & N 2 represented in (6) C. Wavelet Reconstruction The Fused image applied with Inverse Discrete Wavelet Transform to reconstruct the Decomposed original image. Reconstruction or synthesis is the process in which assembling all components back. Wavelet analysis involves Down-sample or Decimate. Wavelet reconstruction in [3] [7] [8] involves Up-sampling or Interpolation. The up-sampling is done by zero inserting between every two coefficients. Reconstruction is done with the synthesis of a signal from the wavelet coefficients. Often want to get multi-level or multi-stage reconstruction for a small wave. In Wavelet analysis, when a signal or image decomposed, restore the image that can get the original signal or original image, to this the wavelet multistage reconstructions may be introduced. D. Color Filter Array (CFA) CFA [2] is a mosaic pattern of tiny color filter placed on the top of the pixel sensors of digital cameras to capture full color information. Bayer filter is used to arrange RGB color filters in photo sensors with pattern as 50% green, 25% red and 25% blue. The CDM is used to interpolate the missing color components at each pixel location and to obtain full color CFA images. The CDM process in [2] will difficult to characterize the noise by the combination of noise across channels. These CDM algorithms are used for neighbouring pixels and to calculate approximately the value of a particular pixel. E. Demosaicking with DLMMSE: The directional linear minimum mean square estimation (DLMMSE) demosaicking algorithm in [5] can be applied to produce full color images collected from single sensor digital cameras. In this Strategy, green-red & green-blue different signals are used for optimal directional filtering. The assumptions of Primary difference signal (PDS) between the green and blue or red channels are low-pass, in both horizontal and vertical directions the missing green channels are estimated by using the linear minimum mean square-error estimation (LMMSE) technique in [5]. The different images between different channels like green and blue channel and the green and red channels are also low-pass signal, which are referred as Primary Difference Signal (PDS). (7) (8) Where n is the position index of pixels and and are smooth signals in (7) & (8). The high visual quality of the final restored image is one important advantage by using this demosaicking algorithm. The quality of an image can depends on the estimation accuracy of the missing green samples in the Bayer Pattern. In this paper, the estimation of missing color 195

samples in both horizontal and vertical directions. These two estimates are combined optimally. Instead of using the LMMSE techniques to estimate the x and y values, A good approximation to MMSE. The LMMSE is calculated as (20) (21) The corresponding missing green samples is interpolated as Fig. 6: Row and a column of mosaic data that intersect at a red sampling position The red sample in the center of the Bayer Pattern is denoted by R 0. The green neighbors of the red sample R 0 in horizontal direction is give in (9) & (10) (9) (10) Similarly the vertical direction of red, green neighbors of R 0 given in (11) & (12) (11) (12) Interpolation of missing green samples can be estimated through horizontal and vertical directions in (13) & (14) Similarly the missing red sample is interpolated with the original green channels. With PDS the error is occurred by using the interpolation of missing red and green values, in (15) and (16) the vertical and horizontal directions of the two estimated values obtained in random process. (15) (16) The estimated error can be calculated as follows (17) Associating these values in demosaicking noise namely (18) The optimal minimum mean square error estimation of x is (19) After completing the pixel positions, the directional weighted estimates of the green-red and green-blue as in (7) and in (8). After completion they recovers the green channel of the bayer CFA through estimating the missing green samples, and more important for human visual system. The estimation of missing green sample given in (24) By using this demosaicking technique, the resultant image has the high visual quality and has high smoothing boundary areas. III. EXPERIMENTAL RESULTS To test the act of the proposed method by using a large number of color images. The proposed method PCA fusion with color demosaicking is used to produce full color images. In this section the results are made on two different images. Two Different types of experiments are made. First experimental results are made to evaluating the denoising with the different types of techniques, and comparing these techniques by calculating the PSNR value and MSE value. A. Peak Signal Noise Ratio The ratio between the original signal and the noise signal can be calculated, the ratio between the signals maximum power and the power of corrupting noise. That affects the exact representation of the original image in equation (25). The Mean Square Error can be represented as MSE, the MSE can calculated in equation (26) 1) Assessment of denoising techniques on CFA Images: Let s perform the denoising technique on sensor Images. The images can disturbed with some noises like Gaussian Noise, Thermal noise, Salt & Pepper noise, Poisson noise and Speckle noise. The original Sensor images are available; the performance of the denoising techniques can effectively valuated by using the Peak Signal Noise Ratio (PSNR) in equation (25). 196

Method Original Image with Gaussian Noise Original Image with Salt & Pepper Noise Original Image with Poisson Noise MSE PSNR MSE PSNR MSE PSNR Wavelets 628.391 21.18 614.259 22.56 615.354 23.1 Wiener Filter 539.245 31.12 574.412 26.12 578.24 26.35 Median Filter 569.362 25.26 524.1 32.14 558.301 24.96 Hybrid Denoising Method 512.214 34.59 504.384 35.51 524.872 32.11 Proposed Method 506.195 35.21 501.326 36.91 516.326 33.659 Table 1: Comparative Study of Proposed Method with Other Existing Techniques The proposed method and Hybrid image denoising method can effectively suppresses the noise with the highest PSNR value calculated with equation (25) than the other denoising methods. With the wavelet transform the image can be decomposed to some sub-bands. The wiener filter and the median filters are suppressing the noise effectively. Fig illustrates the denoising result of CFA Kodak fence image. The proposed method is a combination of denoising with demosaicking. Since the proposed method can performs the strategy as denoising before demosaicking. It has better results than the remaining two strategies as denoising after demosaicking and joint denoising and demosaicking, which constitutes state-of-the-art schema. This schema can suppress noise in single sensor digital color camera images. (a) (c) (d) Fig 7: (a) Original Image, (b) One level decomposition of Original Image, (c) Hybrid Denoised reconstructed Image, (d) Image Interpolated with Proposed Method (b) n Fense Image Flower Image House Image MSE PSNR MSE PSNR MSE PSNR 1 501.326 36.91 498.52 38.15 458.25 41.62 2 485.689 39.54 478.21 39.01 439.32 44.28 3 506.23 36.18 495.24 36.18 462.32 42.36 Table 2: Comparative Study of Proposed Method on different images with different levels of decomposition n IV. CONCLUSION Proposed method used to suppress the different types of noises which are generated during acquisition and transmission phases. This method deals with denoising before demosaicking strategy. In this method, PCA Fusion is applied to the information extracted with Wiener & Median filters. The fused data is used for Demosaicking with Directional Linear Minimum Mean Square Estimation (DLMMSE). Proposed method is evaluated on the standard images like Flower, Fence and House Images. Peak Signal Noise to Ratio & Mean Square Error values proven that proposed method outperforms the existing method. V. REFERENCES [1]. Donoho D L, Denoising by soft-thresholding, IEEE Transactions on information Theory, Vol. 41, No. 3, pp. 613-627, 1995. [2]. L. Zhang, X. Wu, and D. Zhang, Color reproduction from noisy CFA data of single sensor digital cameras, IEEE Trans. Image Process., vol.16, no. 9, pp. 2184 2197, Sep. 2007. [3]. Zhao Hong-tu, Yan Jing, The Wavelet Decomposition And Reconstruction Based on The Matlab, Proceedings of the Third International Symposium on Electronic Commerce and Security Workshops(ISECS 10) Guangzhou, P. R. China, 29-31, pp. 143-14. July 2010. [4]. Deepak Kumar Sahu, M.P.Parsai, Different Image Fusion Techniques A Critical Review, International Journal of Modern Engineering Research (IJMER), Vol. 2, Issue. 5, pp-4298-4301. Sep.-Oct. 2012. [5]. Laxmi Murthy Davala. Directional Linear Minimum Mean Square- Error Estimation in Color Demosaicking, International Journal of Advanced Technology & Engineering Research (IJATER), Volume 2, Issue 4, pp.171-183. July 2012. [6]. Mohd Awais Farooque1, Jayant S.Rohankar2, Survey On Various Noises And Techniques For Denoising The Color Image, International Journal of Application or Innovation in Engineering & Management (IJAIEM), Volume 2, Issue 11, pp.217-221, Nov 2013. [7]. Ick Hoon Jang and Nam Chul Kim, Locally Adaptive Wiener Filtering In Wavelet Domain for Image Restoration, IEEE TENCON Speech and Image Technologies for Computing and Telecommunications, pp.25-28, 1997. [8]. Li Shixin, Zhang Xinghui and Wang Jianming A new Local Adaptive Wavelet Image De-noising Method, IEEE Computer Society, ISCCS, pp.154-156, 2011. [9]. Narinder Kaur A New Hybrid Approach to Remove Salt and Pepper Noise from Colorscale Images, IJERT, Vol.3 Issue 1, pp.3356-3340, January-2014. This paper presents a Hybrid Image Denoising of CFA Color Images collected from Single Sensor Cameras. 197