An Efficient Adaptive Filtering for CFA Demosaicking

Transcription

1 Dev.. Newin et. a. / (IJCSE) Internationa Journa on Computer Science and Engineering An Efficient Adaptive Fitering for CFA Demosaicking Dev.. Newin*, Ewin Chandra Monie** * Vice Principa & Head Dept. of ECE, SATYAM Coege of Engineering & Technoogy, Aravoimozhy ** Add. Director, Department of Technica Education, Chennai. *Mai ID: dev_r_newin@yahoo.com Abstract Most digita sti cameras acquire imagery with a coor fiter array (CFA), samping ony one coor vaue for each pixe and interpoating the other two coor vaues afterwards. The interpoation process is commony known as demosaicking. In genera, a good demosaicking method shoud preserve the highfrequency information of imagery as much as possibe, since such information is essentia for image visua quaity. We discuss in this paper two key observations for preserving high-frequency information in CFA demosaicking: ) the high frequencies are simiar across three coor components, and 2) the high frequencies aong the horizonta and vertica axes are essentia for image quaity. Our frequency anaysis of CFA sampes indicates that fitering a CFA image can better preserve high frequencies than fitering each coor component separatey. This motivates us to design an efficient fiter for estimating the uminance at green pixes of the CFA image and devise an adaptive fitering approach to estimate the uminance at red and bue pixes. Experimenta resuts on simuated images, as we as raw data, verify that the proposed method outperforms the existing methods both visuay and in terms of peak signa-tonoise ratio, at a notaby ower computationa cost. Keywords-demosaicking, biinear, uminance, spectrum I. INTODUCTION To reduce cost and size, most digita sti cameras (DSC) use a singe eectronic sensor covered with a coor fiter array (CFA) to acquire imagery, samping ony one coor vaue for each pixe. To restore a fu-coor image from these CFA sampes, the two missing coor vaues at each pixe need to be estimated a process that is commony referred to as demosaicking. Today, the most popuar CFA pattern is ayer pattern [], a schematic of which is shown in Fig., where green () vaues are samped in a quincuncia attice, whie red () and bue () vaues are in two separate rectanguar attices. Estimating the missing coor vaues is possibe because of substantia correation between neighboring pixe intensities (interpixe correatio. iinear and bicubic demosaicking use isotropic neighborhoods, which may ead to over smoothing of edges. In rea images, the interpixe correations are anisotropic, i.e., the correation is arger aong the edges and smaer across the edges. Severa researchers suggested using this property to improve the performance of demosaicking. The biinear and bicubic interpoation techniques remove the aiasing by eiminating the overapped high-frequency content. This eimination is simiar to ow-pass fitering [2], [3], which coud degrade the quaity of the resutant image, especiay in the more sparsey samped red and bue components. A good demosaicking method shoud remove the aiasing artifacts whie at the same time preserving as much high-frequency information as possibe. To better preserve high-frequency information in demosaicking, the key is to expoit the strong correations across coor components (or panes). Our experimenta resuts further confirm that the detai waveet coefficients of three coor panes are not ony strong correated, but aso simiar to each other [9]. This can expain why coor-difference images are smooth and suitabe for interpoation. Fig.. ayer coor fiter array pattern. The simiarity aso impies that in demosaicking we coud reconstruct a fu-resoution image pane containing the high-frequency information, and then use it to estimate the missing coor vaues. An appropriatey designed ow-pass fiter can be used to reconstruct from a CFA image a furesoution uminance pane, which is then used as a reference to reconstruct the missing coor vaues. However, the fiter proposed in [8] unnecessariy eiminates high-frequency information aong the horizonta and vertica axes, which is important for image quaity, as we wi demonstrate in Section II-. Our frequency anaysis of ayer CFA sampes indicates that the important high-frequency information aong the two axes is preserved, free of aiasing, in the sub samped green pane and, hence, can be extracted by a propery designed fiter. ased on our anaysis, we propose in Section III an efficient ow-pass fiter for uminance of green sampes to better preserve the high-frequency information. The highfrequency information at red/bue pixes is then estimated by adaptive fitering of coor-difference components (Section IV). Incorporating these techniques for preserving highfrequency information, our method outperforms existing stateof-the-art methods both visuay and in terms of peak signato-noise ratio (PSN). II. KEY OSEVATIONS AOUT CFA DEMOSAICKIN In this section, we discuss two key observations that can hep better preserve the high-frequency information in demosaicked images. ISSN :

2 Dev.. Newin et. a. / (IJCSE) Internationa Journa on Computer Science and Engineering A. Simiarity of Inter-coor High-Frequency Content unturk et a. have demonstrated that the highfrequency components of three coor panes are highy correated, with correation vaues ranging from 0.98 to [7]. eneray, a high correation between two variabes does not necessariy mean their equaity. However, the smoothness of coor-difference panes impies that the high-frequency components from different coor components are not ony correated but aso simiar to each other [9]. This approximate equaity of high-frequency information between coor panes has been we expoited in demosaicking. The most popuar way of expoiting this property is through interpoation of the coor-difference images. For exampe, this can be done by estimating a fu-resoution green component first and then using it to predict the red and bue components from coordifference images. The foowing emma expains why this approach works. Lemma : Let F be a fu-resoution reference coor component. Then any other fu-resoution coor component C ε {,,}can be predicted from its sub samped version using C I (C s F s ) + F () where F s is sub samped version of F and I denotes a proper interpoation process.. Importance of Different High-Frequency Components In natura images, the energy spectrum is primariy concentrated in a ow-frequency region, aong the horizonta and vertica axes. A typica exampe is shown in Fig. 2 Fig. 2. (a) A test image and (b) its spectrum To understand the importance of different highfrequency components, we removed from a set of images their high-frequency coefficients outside the centra circe [see Fig. 3(a)], and outside a region more eongated aong the two axes [see Fig. 3(b)], respectivey, whie preserving the same amount of image energy. We found that the atter resuts are generay more visuay peasing than the former, even though the preserved energy is the same. This experiment shows that high-frequency information aong the horizonta and vertica axes has arger impact on the image quaity than that at the corners. This is because the human visua system is more sensitive to the information of these high frequencies. Fig. 3. emoving (a) frequency coefficients outside the centra circe and (b) frequency coefficients outside a region eongated aong both axes, respectivey III. ETTE LUMINANCE FILTEIN AT EEN SAMPLES According to the CFA spectrum A (u, v), the repicas of coor-difference component in the horizonta and vertica directions have the same ampitudes but inverse signs. If we sub-sampe CFA at ony green-pixe ocations, the side components in these two directions wi overap and cance out each other A (u, v) * D (u, v) = s (u, v) (2) Fig. 4. Fourier transforms of (a) fu-resoution coor component, (b) quincunciay samped green component, and (c) rectanguary samped red/bue component. The resutant spectrum has no repicas at the sides, as shown in Fig. 4(b).If we remove the corner components C2 before subsamping the CFA image, the resutant spectrum wi coincide with the uminance spectrum. Specificay, et A (u, v) be the spectrum with the corner components C2 removed, then A (u, v) * D (u, v) = [2 ]( u, v)* D ( u, v) L( u, v)* D ( u, v) (3) 4 The advantage of this approach is that the ow-pass fiter removing corner components C2 has a quincuncia shape and, hence, preserves the important high-frequency information aong the horizonta and vertica axes. One exampe of such quincuncia fiter is shown in Fig. 4(b), which is often used in biinear interpoation. However, we can see that such a quincuncia fiter yieds poor performance because it aso removes a part of component C. This eads to much arger mean square error (MSE) of the estimated uminance. An aternative soution is to simpy modify the Aeysson s fiter with r = 0, as shown in Fig. 5(a). This substantiay reduces the MSE, where we iterativey computed the optima parameters to achieve the best MSE, as Aeysson et a. did [8]. Our Experimenta resuts show that the optima vaues of r2 are typica in a sma range around [0.08, 0.6]. Moreover, the MSE degrades itte around the optima vaue. A arger fiter size can reduce the MSE vaues more. The reduction becomes minimum when the fiter size is arger than 5 X 5. Hence, we fixed our fiter size as 5 X 5 in order to ISSN :

3 Dev.. Newin et. a. / (IJCSE) Internationa Journa on Computer Science and Engineering reduce the computationa cost. The spectrum and the impuse response of the proposed 5 X 5 fiter are shown in Fig. 5(b) and (c). Fig. 5. Proposed ow-pass fiter for green pixe uminance: (a) passband, (b) spectru and (c) impuse response Since the passband of the proposed fiter is fixed, it does not depend on the image content. Despite this simpification, the performance of the redesigned fiter ony degrades sighty in terms of MSE. Computationay, it requires ony five mutipications and 20 additions for each pixe, compared to 2 additions and 68 mutipications for the Aeysson s X fiter. However, the proposed ow-pass fiter is not suitabe for estimating the uminance at red/bue pixe ocations, because the side repicas of the component do not cance out at these ocations. To address this probem we propose using an adaptive fitering scheme, as described in Section IV. IV. ADAPTIVE LUMINANCE FILTEIN AT ED/LUE SAMPLES According to Lemma, we can interpoate the estimated, sub-samped version of uminance (at green-sampe ocations, see Section III) to fu resoution by using the difference images Cr = L - [see Fig. 6(a)] or Cb = L -. However, this is one type of a chicken-and-egg probe as the references must have fu resoution, which requires extension of sub-samped red/bue components to fu resoution. ut, the red/bue components aso require a furesoution reference, such as the fu-resoution uminance. Fig. 6. (a) eference neighboring sampes and (b) the adaptive fiter We propose resoving this probem iterativey. We start with biinear interpoation of red and bue components, separatey; to fu resoution, and then use these estimated components to interpoate a fu-resoution version of the uminance (using Lemma ). Hence, we improve the first estimates of fu-resoution red and bue components by using the estimated uminance as a reference. This process can be repeated severa times, unti no further changes occur or the changes are sufficienty sma. Our experiments show that two iterations are sufficient in most cases. Since the first step, biinear interpoation of red/bue components may fai to preserve high-frequency information F I F, for F {, }, the high-frequency information wi aso be ost in the second step, biinear interpoation of the uminance component. According to Lemma, ^ L I ( L F ) F bi s s L where I bi designates biinear interpoation. To better preserve the high-frequency information we propose using an anisotropic adaptive interpoation for the coor-difference components. We define the adaptive fiter coefficients, w k, k = to 4, shown in Fig. 6(b), to be proportiona to the spatia correation aong the corresponding directions, which can adapt to different neighborhood spatia patterns. In our adaptive fitering, the correations (and, hence, the coefficients) are estimated from the neighborhood pixe vaues. For red-sampe ocations shown in Fig. 6(a), the coefficients are defined as ( ( n 2) n ) n ) w ( ( m 2, m, m, w2 ( ( n 2) n ) n ) w3 (5) ( ( m 2, m, m, w4 Note that these expressions define a type of edge detector; the presence of edge makes the corresponding coefficients sma. Not ony does the edge detector operate directy on the subsamped data, it can aso work on diagona edges. Simiar weights can be used for interpoation at buesampe ocations. Iterating the adaptive fitering can improve the performance of uminance estimation, especiay in the first two iterations. More iteration, however, may negate the improvement. The negation happens because the equivaent fiter size after severa iterations becomes very arge and may invove many pixes that are ony sighty correated. To avoid performance degradation and minimize the computation, we, therefore, perform two iterations in the proposed red/buesampe uminance estimation. Overa, the proposed approach has a notaby better performance (in terms of MSE) compared with the Aeysson s fiter. V. EXPEIMENTAL ESULTS To evauate the proposed method we compare it with five demosaicking techniques Aternating projections (AP) [7] Successive approximation (SA) [6] Primary-consistent soft-decision (PCSD) [4], Frequency seection (FS) [8] Adaptive homogeneity-directed demosaicking [5] (4) iinear interpoation Among these methods, biinear interpoation is the simpest and aso most common reference for performance comparison in the demosaicking iterature. The AP method projects the high frequencies of green component to red and bue components, the SA method iterates its isotropic interpoation, and the FS method seectivey fiters CFA image to estimate the uminance. These three methods are not anisotropic. On the other hand, the PCSD s anisotropic interpoation criterion is obtained from training, and AHD s is ISSN :

4 Dev.. Newin et. a. / (IJCSE) Internationa Journa on Computer Science and Engineering based on some homogeneity simiarity measure. The PSN resuts obtained by the methods under comparison are summarized in Tabe I. TALE: PSN PEFOMANCE (IN DECIELS) COMPAISON FO EACH TEST IMAE FO ED, EEN, LUE PLANES Image iinear AP PCSD SA FS AHD Proposed Our method outperforms the other methods for the overwheming majority of test images. On average, the improvement to biinear is quite arge, up to 8.8 d. The improvement to the AP, PCSD and SA methods is around 0.5 d.0 d, and that to the FS and AHD methods is around.5 d 2.3 d. Consistent performance improvement can be obtained when using S-CIELA [2], [3]. The proposed method produces visuay favorabe resuts with sharper edges compared to most other methods. Our method obtains the best perceptua resuts with sharper edges and fewer zipper artifacts. iinear interpoation yieds burred edges and many coor artifacts. The AP, SA, and FS methods produce severe zipper artifacts around the edges due to the use of isotropic interpoation. The PCSD method can obtain better resuts since it uses an anisotropic interpoation technique, but the training criterion it uses for edge direction estimation may be inaccurate for some image regions, eading to some artifacts. As reported in unturk s review paper [0], the AHD method can produce perceptuay favorabe images with fewer artifacts. The approach uses a type of anisotropic fitering, based on some homogeneity criteria for neighborhood simiarity. Fig. 7.Test Images (referred as Image to 8) However, the interpoation is restricted to one of two directions (horizonta or vertica), which may not work we for textured images, where high-frequency components might need to be preserved in both directions. Our method can better preserve the textured structures due to the adopted adaptive fitering. Our method yieds better PSN performance (see Tabe I), and has much ower computationa cost. VI. CONCLUSION In this paper, we have discussed two important observations for preserving high-frequency information in CFA demosaicking. First, we have shown that due to the simiarity of high-frequency information across three coor components, one fu-resoution component can be used to estimate the high-frequency information of a other components. We have aso shown that high-frequency information aong horizonta and vertica axes is more important for image quaity than that in the spectrum s corners. Our frequency anaysis expains why the ayer CFA pattern can preserve the vita high-frequency information. ased on these observations, we propose an adaptive fitering demosaicking method for better preservation of highfrequency information. We designed an efficient ow-pass fiter on CFA image to estimate the uminance at greensampe ocations. The proposed fiter can preserve more high frequencies than the existing fiters. Next, an adaptive fitering is used to estimate the uminance at red and bue sampes. Our frequency anaysis indicates that the proposed adaptive fiter is we suited for varying image content and can better preserve high frequencies. Experimenta resuts confirm that the proposed method outperforms the existing state-of-the-art methods both visuay and in terms of peak signa-to-noise ratio (PSN), at a notaby ower computationa cost. In CFA demosaicking, anisotropic techniques are usuay required to preserve high frequencies around edges and remove zipper artifacts, but they incur high computationa cost. Our anaysis and proposed method show that a propery designed 5 X 5 fiter can preserve we the high frequencies at green sampe ocations, without incurring much computationa cost. Furthermore, our proposed adaptive fitering interpoation at red/bue sampe ocations can hande the diagona edges more effectivey than many existing anisotropic techniques do, eading to a better perceptua performance. EFEENCES []. ayer, Coor imaging array, U.S. Patent 3 97,065, 976. [2] D. Cok, Signa Processing method and apparatus for producing interpoated chrominance vaues in a samped coor image signa, U.S.Patent , 987. [3] J. Adams, Interactions between coor pane interpoation and other image processing functions in eectronic photography, in Proc. SPIE Cameras and Systems for Eectronic Photography and Scientific Imaging, 995, vo. 246, pp [4] X. Wu and N. Zhang, Primary-consistent soft-decision coor demosaicking for digita cameras, IEEE Trans. Image Process., vo. 3, no.9, pp , Sep [5] K. Hirakawa and T. W. Parks, Adaptive homogeneity-directed demosaicing agorith IEEE Trans. Image Process., vo. 4, no. 3, pp , Mar [6] X. Li, Demosaicing by successive approximation, IEEE Trans. Image Process., vo. 4, no. 3, pp , Mar ISSN :

5 Dev.. Newin et. a. / (IJCSE) Internationa Journa on Computer Science and Engineering [7]. K. unturk, Y. Atunbasak, and. M. Mersereau, Coor pane interpoation using aternating projections, IEEE Trans. Image Process.,vo., no. 9, pp , Sep [8] D. Aeysson, S. Susstrunk, and J. Heraut, Linear demosaicking inspired by human visua syste IEEE Trans. Image Process., vo. 4, no. 4, pp , Apr [9] N.-X. Lian, V. Zagorodnov, and Y.-P. Tan, Edge-preserving image denoising via optima coor space projection, IEEE Trans. Image Process., vo. 5, no. 9, pp , Sep [0]. K. unturk, J. W. otzbach, Y. Atunbasak,. W. Schafer, and.m. Mersereau, Demosaicking: Coor fiter array interpoation, IEEE Signa Process. Mag., vo. 22, no., pp , Jan [] L. Chang and Y. P. Tan, Effective use of spatia and spectra correations for coor fiter array demosaicking, IEEE Trans. Consum. Eectron., vo. 50, pp , Feb [2] W. Lu and Y.-P. Tan, Coor fiter array demosaicking: New method and performance measures, IEEE Trans. Image Process., vo. 2, no.0, pp , Oct [3] X. Zhang, S- Cieab: A Spatia Extension to the CIE LA Deta E Coor Difference Metric 998 [Onine].Avaiabe: stanford. edu/htm/xmei/ scieab/scieab.htm. ISSN :