An Effective Directional Demosaicing Algorithm Based On Multiscale Gradients

Transcription

1 79 An Effectie Directional Demosaicing Algorithm Based On Multiscale Gradients Prof S Arumugam, Prof K Senthamarai Kannan, 3 John Peter K ead of the Department, Department of Statistics, M. S Uniersity, Tiruneleli Chief Executie Officer, Nandha Engineering College, Erode. 3 Research Scholar, Department of Computer and Information Technology, M S Uniersity, Tiruneleli ABSTRACT In a typical digital camera, the colors of the scene are captured by a single CCD or CMOS sensor array, where for each pixel the sensor detects a particular color channel. This kind of sensor is called Color Filter Array. The remaining two color channel alues need to be estimated to obtain a complete color image. This technique is called demosaicing or Color Filter Array interpolation. This proect proposes a directional approach to the CFA interpolation problem that makes use of multi scale color gradients. The relationship between color gradients on different scales is used to generate signals in ertical and horizontal directions. It proposes a demosaicing method that uses multi scale color gradients to adaptiely combine color difference estimates from arious directions. Then determine how much each direction should contribute to the green channel interpolation based on these signals. Finally Structural approximation algorithm is used to refine red and blue channel.so the quality of the image is to be improed. This type of approach requires a limited computational cost and gies good performance een when compared to more demanding techniques. The proposed method is easy to implement since it is noniteratie and threshold free. Experiments on test images show that it offers superior obectie and subectie interpolation quality. Index Terms- Demosaicing, Color filter array interpolation, Multiscalecolor gradient, directional interpolation. I. INTRODUCTION Most digital cameras employ single sensor designs because using multiple sensors coupled with beam splitters for each pixel location is costly in hardware. This design choice necessitates the use of color filter arrays. The color channel layout on a color filter array determines which channel will be captured at each pixel location. Many different CFA layouts hae been proposed but the Bayer CFA pattern is the most commonly used design [].The CFA pattern layout plays an important role in the design of a CFA interpolation algorithm. Demosaicing is an important part of the image processing pipeline in digital cameras. The failure of the employed demosaicing algorithm can degrade the oerall image quality considerably. The simplest way to address the demosaicing problem would be to treat each color channel separately and interpolate. The color channel layout for the Bayer CFA pattern is shown in Figure.The quality can be improed by applying the interpolation oer color differences to take adantage of the correlation between the color channels. The gradients are useful for extracting directional data from digital images. Seeral demosaicing methods include the integrated gradients. (a (b (c (d Figure.Bayer color filterarray patternand its (b green, (b red and (d blue samples amilton et al. proposed adaptiely interpolating the green channel in horizontal or ertical directions or a combination of both, based on directional classifiers and thresholds [].The idea of using aailable red and blue channel pixels in initial green channel interpolation is borrowed by many subsequent methods. A possible area for improement is to come up with better classifiers that

2 79 can lead to a more accurate direction decision. Variance of color differences is used to make a hard interpolation direction decision in [3], while linear minimum meansquare error framework is employed to combine directional estimates in []. Another interesting approach is to interpolate the green channel in both directions and then to make a posteriori decision based on sum of gradients in each direction[5].the demosaicing problem has been studied from many other angles. Glotzbach et al.proposed a frequency domain approach where they extracted high frequency components from the green channel and used them to improe red and blue channel interpolation[6]. he full range of demosaicing Gunturk et al. used the strong spectral correlation between high frequency sub bands to deelop an alternating proections method [7]. A comprehensie list of demosaicing approaches, experimental results, and obserations are presented in a recent surey paper[8]. An early demosaicing method proposed Adaptie color plane interpolation in single sensor color electronic camera used deriaties of chrominance samples in initial green channel interpolation, and this technique is gathered by many subsequent algorithms. Many of the demosaicing algorithms proposed directional interpolation with different decision rules. For instance Color demosaicing using ariance of color differences used ariance of color differences to make a hard direction decision. On the other hand, Color demosaicking ia directional linear minimum mean square-error estimation proposed a soft direction decision based on the Linear Minimum Mean Square Error Estimation (LMMSE framework. ere, the directional color differences are considered as noisy obserations of the actual color difference and they are combined optimally. Spatially adaptie color filter array interpolation for noiseless and noisy data improed this directional approach with scale adaptie filtering based on local polynomial approximation (LPA. Another interesting directional approach is to perform interpolation in both directions and then make a posteriori decision. Adaptie homogeneity-directed demosaicing algorithm used local homogeneity of the directional interpolation results and Demosaicing with directional filtering and a posteriori decision used color gradients oer a local window as the decision criteria. Another method proposed using high frequency components extracted from green channel to improe red and blue channel components that are more susceptible to aliasing. Color plane interpolation using alternating proections proposed an alternating proections scheme using the strong interchannel correlate on in high frequency sub bands. Since the method is iteratie, it required a high number of calculations. Another method Adaptie filtering for color filter array demosaicking proposed filtering the input mosaicedcolor components together to presere the high frequency components better. The rest of the paper is organized as follows. Section gies some basic information and describes the proposed method in detail. Section 3 presents experimental results, and section gies a brief discussion. II. PROPOSED ALGORITM A. Algorithm background We hae proposed a directional CFA interpolation method that uses the color difference gradients hae more features to combine color difference estimates from arious directions based on the ratio of total absolute alues of horizontal and ertical color difference. The steps are illustrated below. Input Image Multiscale Gradient Calculation Red & Blue channel Interpolation Red& Blue channel Refinement Initial Directional Color Channel Estimation Fig: System design Color Difference Estimation Combine color difference Estimation from different directions Initial Green channel Interpolation Green channel update Output Image

3 79 Most digital cameras use color filter arrays and this design choice leads to the capture of only a subset of the image data. The simplest way to address the demosaicing problem would be to treat each color channel separately and interpolate the missing color channels. = (, (, (,. (, (, (, ( Where and V denote horizontal and ertical directions ( is the pixel location. Now we hae a true color channel alue. Next take the color difference estimate: C, ( = C, ( = G ( R(, if G is interpolated G( R (, ifrisinterpolated G ( R(, if G is interpolated G( R (, ifrisinterpolated (3 Fig: 3 Bayer mosaic pattern The first step is to get initial directional color channel estimates for red & green rows and columns in the input mosaic image, the directional estimates for the missing red and green pixel alues are calculated. For blue & green rows and columns in the input mosaic image, the directional estimates for the missing blue and green pixel alues are calculated. ere we are calculating horizontal and ertical color channel estimates. The directional color channel estimates for the missing Green pixel alues are, (, = (, (,. (, (, (, (, (, (, =. (, (, (, ( The directional color channel estimates for the missing Red pixel alues are, (, (, (, =. (, (, (, This equation is similar for blue & green rows and columns. We propose a more effectie approach to directional interpolation, where the decision of the most suitable directionof interpolation is made on the basis of the reconstructed green component only. We proposed a directional CFA interpolation method that uses color difference gradients in [9]. The color difference gradient corresponds to taking the difference between the aailable color channel alues two pixels away from the target pixel, doing the same operation in terms of the other color channel by using simple aeraging, and then finding the difference between these two operations. It could be argued that the performance of such an algorithm relies on its ability to successfully combine directional estimates. Fig: Multi scale gradients equation ere we take the difference between the aailable color channel alues one pixel away from the target pixel. (, (, (, = (, (, (

4 793 Next these equations combine the color difference estimates from arious directions. The easiest way of doing that is to optimize the normalizing terms (N in the denominators. The final multiscale gradients equation for red green rows and columns can be gien as follows: (, (, (, = (, (, (, 3 (, 3 (, (, 3 (, = (, (, (, (, ( 3, ( 3, (, (, 3 (5 B.Initial Green Channel Interpolation The first step of the proposed algorithm is to interpolate the missing green channel pixels. We perform this interpolation adaptiely using the multiscale color gradients equation deried aboe.in addition to the horizontal and ertical pixel alue and color difference estimations described in equations(and(3.next we combine the directional color difference adaptiely:, (, =.., ( :,., (, :. / = f = [/ / / ] (6 The weights for horizontal and ertical directions( w, w V are calculated by adding multiscalecolor gradients oer a local window.for a local window size of 5 by 5,the weight for each direction is calculated as follows, = / (, = / (, This section concentrates on estimating missing green pixels from known green and red pixel alues The same technique is used to estimating missing green pixels from known green and blue pixels. We hae directional color difference estimates around eery green pixel to be interpolated. Fig. 5 image Kodak test set C. Green Channel Update After the initial green channel interpolation,we update the results using directional multiscale gradients again, except we ealuate all the directions separately.it is used to improe the green channel results. The four neighbors of the target pixel has its own weight as follows: (7 = / (, = / (, = / (,

5 79 = / (, = (8 The weight (w for each direction w, w, w, w is ( N S E w north south east and west directions calculated by summing multiscale color gradients oer a local window.assuming a 3 by 5 window for horizontal and a 5 by 3 window for ertical components. ere the directional color difference estimates are updated., (, =, (,. (., (,., (,., (,., (, ]. (9 Finally the updated color difference estimate is added to the aailable target pixel to obtain the green channel estimate: (, =, (, (, (, =, (, (, ( For red and blue pixels at green locations,we make use of the multiscale color gradients again. The horizontal and ertical estimations are combined adaptiely using the directional weights ( w, w V defined in equation (. The immediate ertical neighbours of a green pixel are either red or blue pixels. For the red pixel case the interpolation is carried out as follows: wv i, Ri, G' i, Ri, R' ( G( * ( w w w w B' ( G( w R' * ( w V B * ( w G' w i, G' B' * ( w w i, R' G' w B i, B' i, ( By the end of thiseqn(3, all the missing alues are estimated and the full color image is reconstructed. E. Red and Blue Channel Refinement The final step of the proposed method is to refine the interpolated red and blue alues. The equations for doing such refinements by using SA s method[]..let Q(k,l be either red or blue sample and let D(k,l=G(k,l Q(k,l. D. Red and Blue Channel Interpolation For red and blue channel interpolation, we keep the same approach that we employed in[9]. Red pixel alues at blue locations and blue pixel alues at red locations are interpolated using the filter that was proposed. P rb. Where denotes element-wise matrix multiplication and then summation of elements. (, = (, (, ( 3: 3, 3: 3 3, (, = (, (, ( 3: 3, 3: 3, ( Fig:6Reference Bayer pattern. erein, G is a green sample, and P andq represent either red or blue sample respectiely. If P is red, then Q isblue, and ice ersa. (, = (, ( (, (, / (, = (, ( (, (, / (, = (, ( (, (, /

6 795 (, = (, ( (, (, / (3 After the aboe refinements finally interpolate, (, = (, ( (, (, (, (, / ( The end of this equation can be seen that the proposed method produce superior image quality than other demosaicing algorithms. III. EXPERIMENTAL RESULTS We tested the proposed algorithm on the image Kodak test set featured in [8]. The results in terms of CPSNR are compared to the three highest performing methods in a recent surey paper [8], and to the method the sered as the starting point of the proposed algorithm [9]. These methods are Gradient Based Threshold Free(GBTF[9], Local Polynomial Approximation (LPA [], Directional Linear Minimum Mean Square-Error Estimation(DLMMSE [], and ariances of color differences (VCD [3]. The proposed algorithm has the best CPSNR for eery image in the test set. It outperformsthe closest method(gbtfby.6 db on aerage. The comparison results are summarized in Table and a sample image region is shown in Figure 8. No VCD DL LPA GBTF Prop Prop After refinement Ag Table: Comparison of CPSNR alues for different demosaicing methods Ag VCD Fig: 7 A sample chat for proposed demosaicing after refinement Fig:8 Fence region from images no.7 (aoriginal(bvcd(cdlmmse (dlpaici (egbtf (fproposed IV. CONCLUSION In this paper, we hae demonstrated that the relationship between color gradients at different scales can be used to deelop a high quality CFA interpolation method that is easy to implement. Experimental results shows that the effectieness of proposed method out performs other aailable algorithms by a clear margin in terms of CPSNR. Further research efforts can focus on improing the results and applying the multiscale gradients idea to other image processing problems REFERENCES [] Pekkucuksenand Yucelltunbasak, Multiscale Gradients-Based ColorFilter Array Interpolation, IEEE Trans.Image process, ol., no., Jan 3 [] J. E. Adams and J. F. amilton, Adaptie color plane DL LPA GBTF Prop Prop After refinement

7 796 interpolation in single color electronic camera, U.S , Apr. 996 [3] K.-. Chung and Y.-. Chan, Color demosaicing using ariance of color differences, IEEE Trans. ol. 5, no., pp , Oct. 6 [] L. Zhang and X. Wu, Color demosaicking ia directional linear minimum mean square-error estimation, IEEE Trans. Image Process., ol., no., pp , Dec. 5 [5] D. Menon, S. Andrian and G. Calagno, Demosaicing with directional filtering and a posteriori decision, IEEE Trans. Image Process., ol. 6, no., pp. 3, Jan. 7 [6] J. W. Glotzbach, R. W. Schafer, A method of color filter array interpolation with alias cancellation properties, in Proc. IEEE, ol.., pp. [7] B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, Color plane interpolation using alternating proections, IEEE Trans. ol., no. 9, pp , Sep. [8] D. Menon and G. Calagno, Regularization approaches to demosaicking, IEEE Trans. Image Process., ol. 8, no., pp. 9, Oct. 9 [9] Pekkucuksen and Y. Altunbasak, Gradient based threshold free color filter array interpolation, in Proc. IEEE Int. Conf. Image Process., Sep., pp. 37 [] D. Paliy, V. Katkonik, R. Bilcu, S. Alenius, and K. Egiazarian, Spatially adaptie color filter array interpolation for noiseless and noisy data, Int. J. Imag. Syst. Technol., ol. 7, no. 3, pp. 5, 7 [] X. L "Demosaicing by successie approximation," IEEE Trans. Image Process., ol., no. 3, pp , March 5