MANY satellite sensors provide both high-resolution

Transcription

1 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 8, NO. 2, MARCH Improved Additive-Wavelet Image Fusion Yonghyun Kim, Changno Lee, Dongyeob Han, Yongil Kim, Member, IEEE, and Younsoo Kim Abstract Effective image-fusion methods inject the necessary geometric information and preserve the radiometric information. To preserve the radiometric information, the injected high frequency of a panchromatic (pan) image must follow the frequency of the multispectral (MS) image. In this letter, an improved additive-wavelet (AW) fusion method is presented using the àtrous algorithm. The proposed method does not decompose the MS image; thus, it preserves the radiometric information of the MS image and can inject high frequency following the frequency of the MS image using a low-resolution pan image. Experimental results obtained using IKONOS data indicate that the proposed method produces superior-quality images compared with the AWluminance proportional method in a quantitative analysis. Index Terms À trous algorithm, IKONOS, image fusion, panchromatic (pan)-sharpening. I. INTRODUCTION MANY satellite sensors provide both high-resolution panchromatic (pan) (HRP) and low-resolution multispectral (LRM) images because of technological and physical constraints [1], [2]. Image fusion or pan-sharpening has been developed to produce high-resolution multispectral (HRM) image by combining the HRP and LRM images. After the image fusion, users can use the HRM image in various applications [3]. So far, many image-fusion methods have been developed. Some methods, such as the intensity hue saturation (IHS or LHS) transform and principal component analysis, provide superior visual HRM image but produce radiometric distortion [4], [5]. In contrast, the wavelet-based image-fusion methods provide superior radiometric quality. Moreover, these methods make it easy to control the tradeoff between the radiometric and geometric information [6]. More recently, the fusion method, which considers the spectral-response function during the fusion process, has been presented [7]. Currently used wavelet-based image-fusion methods are usually associated with two computation algorithms: the Mallat Manuscript received March 11, 2010; revised May 29, 2010 and July 9, 2010; accepted August 6, Date of publication September 13, 2010; date of current version February 25, Y. Kim and Younsoo Kim are with the Satellite Data Application Department, Satellite Information Research Institute, Korea Aerospace Research Institute, Daejeon , Korea ( yhkari@gmail.com; younsoo@kari.re.kr). C. Lee is with the School of Civil Engineering, Seoul National University of Technology, Seoul , Korea ( changno@snut.ac.kr). D. Han is with the Department of Civil and Environmental Engineering, Chonnam National University, Yeosu , Korea ( hozilla@ chonnam.ac.kr). Yongil Kim is with the Department of Civil and Environmental Engineering, Seoul National University, Seoul , Korea ( yik@snu.ac.kr). Color versions of one or more of the figures in this paper are available online at Digital Object Identifier /LGRS algorithm and the à trous algorithm. The Mallat-algorithmbased dyadic wavelet transform, which uses decimation, is not shift invariant and exhibits artifacts due to aliasing in the fused image. In contrast, the à trous-algorithm-based dyadic wavelettransform method, which does not use decimation, is shift invariant, a characteristic that makes it particularly suitable for image fusion [1], [8]. Therefore, the à trous algorithm works out better than the Mallat algorithm [9]. Generally, the à trous-algorithm-based image fusion can be performed in two ways: 1) by substituting the low frequency of the LRM image by the corresponding high frequency of the HRP image and 2) by adding the high frequency of the HRP image into the LRM image [10]. The first method is the substitute-wavelet (SW) method, and second method is the additive-wavelet (AW) method. The AW method maintains the high frequency of the HRP image and low frequency of the LRM image, in contrast to the SW method which eliminates the low frequency of the LRM image. However, if the same high frequency is injected into every LRM image, the AW method can produce redundant high frequency. As the HRP and LRM images generally have different local radiometry, radiometric distortion can arise in fused images. Redundant high frequency means that the high frequency does not follow the frequency of the LRM image. Also, the SW method can lose some of the information of the LRM image during substitution process [11]. These problems can generate artifacts in fused image. If these artifacts are not taken into account, the fused image will suffer from radiometric and geometric distortions. To overcome the problems of the AW and SW methods, a new AW fusion approach is presented in this letter. The proposed method does not decompose the LRM image. Therefore, this method maintains all the information of the LRM image, which is not done by the SW method. In addition, this method can reduce the redundant high-frequency injection problem using a low-resolution pan (LRP) image. To verify the efficiency of the proposed method, experimental evaluations are carried out on IKONOS data. II. ÀTROUS-ALGORITHM-BASED IMAGE FUSION This section reviews the à trous-algorithm-based fusion methods as presented in [7], [10], and [12] and concisely discusses the characteristics of these methods. To decompose the images, a scaling function with a B 3 cubic spline profile was used. It is assumed that the HRP and LRM images are apriorigeometrically registered and superimposed. We enlarged the LRM image size to the HRP image size using bicubic interpolation X/$ IEEE

2 264 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 8, NO. 2, MARCH 2011 A. AW-Based Methods The AW or AWRGB method injects the high frequency of the HRP image directly into the LRM image. Thus, the radiometric signature of the LRM image is not preserved, as the same high frequency is injected into the LRM image. This implies that the AW method produces redundant geometric information. The AW method can be formulated by the following procedure. 3) Decompose the matched HRP image to n wavelet planes HRP = HRP r + w HRP. (1) 4) Add the n wavelet planes of the HRP image to the LRM image to produce the HRM image w HRP (2) where HRP r is the residual image of the HRP image, HRM i is the ith HRM image, LRM i is the ith LRM image, and n w HRP is the n wavelet planes of the HRP image. The wavelet planes are the sum of the high frequency. Unlike the AW method, the AW-luminance (AWL) method is equivalent to injecting the high frequency into the LRM image proportionally to their original values. Thus, this method maintains the relative radiometric signature between the LRM bands. The AWL method can be formulated by the following procedure. 3) Decompose the matched HRP image to n wavelet planes HRP = HRP r + w HRP. (3) 4) Add the n wavelet planes of the HRP image to the intensity component L New = L + w HRP. (4) 5) Obtain the final fused HRM image from L New HS bands. However, the AWL method is limited to only three bands at a time. Recently, a method extended to L bands fusion method has been proposed. This method is termed the AWL proportional (AWLP) method [7]. It can be formulated as follows: LRM i (1/L) L i=1 LRM i w HRP (5) where L is the number of LRM bands. The AWLP method injects high frequency into the LRM image considering the relative radiometric signature of the LRM bands. To obtain this weighting factor, the ratio between the LRM image and the mean value of all m LRM images was calculated. This facilitated the injection of the high frequency into the LRM bands in a manner proportional to their original values. The AWLP method is a modified version of the AWL method [3]. B. SW-Based Methods The SW method substitutes the wavelet planes of the LRM image for the wavelet planes of the HRP image as follows. 3) Decompose the matched HRP and LRM images to n wavelet planes HRP = HRP r + LRM i = LRM r i + w HRP w LRM. (6) 4) Replace the wavelet planes of the LRM image by the equivalent wavelet planes of the HRP image HRM i = LRM r i + w HRP. (7) The SW method can be rewritten as follows: w LRM. (8) The SW method substitutes the high frequency of the HRP image with the low frequency of the LRM image. Thus, the wavelet planes of the LRM image are discarded. Recently, González-Audícana et al. [12] have proposed the hybrid IHS method. In their method, multiresolution wavelet decomposition is used to implement the high frequency, and they used the IHS transform to inject. This method is expressed as follows: w I (9) w I is the wavelet planes of the intensity component [13]. This method is termed as the SW intensity (SWI) method. Practically, the IHS transform cannot completely separate the radiometric and geometric information of the LRM image [12]. Thus, the SWI method can also lose some of the information of the LRM image during the substitution process owing to the intensity component obtained by the weighting of each of the LRM bands with a set of coefficients. Moreover, it is difficult to determine a clear relationship among the LRM bands of recently launched satellite sensors. Thomas et al. [2] clearly demonstrated the different local radiometry and geometry between the HRP and LRM images.

3 KIM et al.: IMPROVED ADDITIVE-WAVELET IMAGE FUSION 265 In other words, the local high frequency that is visible in the LRM image can be missing in the HRP image, and vice versa. Therefore, the SW-based methods can eliminate both the geometric and radiometric information of the LRM image. In other words, it is necessary to develop an effective fusion method that injects the necessary geometric information and preserves the radiometric information. This implies that the LRM image must be maintained in the fusion process, and the high frequency, which is not present in the LRM image, is injected. This is the main idea of the proposed method. III. PROPOSED METHOD The important point in image fusion is to estimate the missing high frequency of the HRM image. We assumed that the low frequency of the HRM image can be obtained by the LRM image [14] and that the missing high frequency of the HRM image can be extracted through the frequency difference of the HRP and LRP images. This assumption is similar to that the general image-fusion (GIF) method of Wang et al. [1]. The GIF method was developed based on the physical principles of image formation under several physical assumptions. According to the aforementioned assumptions, the final fused image can be obtained using the à trous algorithm as follows: w LRP (10) w LRP denotes the wavelet planes of the LRP image. The LRP image is spatially degraded by filtering with a Gaussian low-pass filter whose frequency response matches the shape of the modulation transfer function (MTF). The MTF of the IKONOS HRP image, as measured using the Nyquist frequency along a track system, is In addition, the à trous wavelet algorithm matches the shape of the MTF of a typical visible near-infrared with a cutoff frequency that is equal to The complementary high-pass filter, yielding a scale level to be injected for 1 : 4 fusion, retains more of the highfrequency component than an ideal filter [15]. The proposed method considers a bandpass injection of high frequency extracted from an HRP image instead of conventional high-pass injection. Thus, the proposed method allows the HRP image to be decomposed into a series of disjointed bandpass channels in the spatial frequency. Subsequently, the difference between the high frequency of the HRP image scale and the low frequency of the LRP image scale is injected. Thus, the low frequency of the LRP image scale is discarded. This implies that the high frequency between the LRM and HRP images is injected while maintaining the LRM image. This will reduce the redundant high frequency that can produce artifacts, and the bandpass injection takes advantage of the noise filtering from the HRP image with respect to classical high-pass injection. As a result, the proposed method preserves the radiometric information of the LRM image and can inject the high frequency following the frequency of the LRM image. This is the main difference between the proposed method and the AWLP method. The proposed method is similar to the high-pass filtering method, as the proposed method injects the high frequency from the signal difference of the LRP and HRP images. The proposed method can be simplified, as in the Choi et al. work [16], as follows: w (HRP LRP) (11) w (HRP LRP) is the high frequency of the difference image from the HRP and LRP images. This method enables the final fused image to be obtained by simple addition of the high frequency of the difference image. The proposed method is similar to the SW and SWI methods. However, in the proposed method, the fusion process is implemented without any decomposition of the LRM image, in contrast to the SW-based methods. Thus, the radiometric and geometric information of the LRM image is maintained in the fused image. We call this method as the improved AW (IAW) method. The proposed method can be reformulated as follows: LRM i HRM i =LRM i + (1/L) L i=1 LRM w (HRP LRP). (12) i This method proportionally injects the high frequency into every LRM image in a manner identical to that of the AWLP method. Thus, this method maintains the relative radiometric signature of the LRM bands. As a result, the fused image can preserve the spectral angle between the LRM and HRM images. We call this method as the IAW proportional (IAWP) method. IV. DATA SETS AND EXPERIMENTAL RESULTS Experiments were conducted to evaluate the performance of the proposed method using the IKONOS data of Daejeon, Korea. For all data sets, the LRM image size was , and the HRP image size was pixels. The experiment areas constitute forests, buildings, and roads. The IAWP method was directly compared with the AWLP method. The other fusion methods were not included in the analysis because the AWLP method outperforms the AW, SW, and SWI methods. EXP denotes interpolated image using bicubic interpolation. A. Visual Analysis The visual performances of the IKONOS data are shown in Fig. 1 for an RGB composition of pixels. The IAWP method produced high-quality image, implying that the IAWP method contains the necessary geometric information as well as rich radiometric information. The AWLP method appears to be slightly sharper than the IAWP method. Fig. 2 shows the profiles of the white line in the fused images. According to the profiles, it is clear that the frequency of the IAWP fused image follows the frequency of the LRM image more closely than the AWLP fused image. The frequency of the AWLP fused image is slightly higher or lower than that of the IAWP fused image.

4 266 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 8, NO. 2, MARCH 2011 Fig. 1. Comparison of fused images as an RGB combination. The presented images are pixels in size at full scale. (a) A 4-m LRM image interpolated at 1 m. (b) HRP image at 1 m. (c) AWLP fused image. (d) IAWP fused image. B. Quantitative Analysis Existing statistical evaluation indicators, such as Q4, spectral angle mapper (SAM), and erreur relative globale adimensionnelle de synthèse (ERGAS), cannot provide reliable measurements for evaluating the image-fusion quality. This implies that quantitative evaluation through downgrading the original data Fig. 2. Profile comparison of transected pixels. (a) Blue band. (b) Green band. (c) Red band. (d) NIR band. is not suitable for very high resolution satellite data [17] [19]. SAM is supposed to measure mainly radiometric distortion, while for ERGAS and Q4, it is impossible to quantify the sensitivity to radiometric and geometric distortion separately.

5 KIM et al.: IMPROVED ADDITIVE-WAVELET IMAGE FUSION 267 TABLE I QUALITY INDEXES OF THE FUSED IMAGES the high frequency which follows the frequency of the LRM image using the LRP image. In the experimental results using IKONOS data, the proposed method demonstrated greater efficiency than the AWLP method. Therefore, the quality not requiring a reference (QNR) index, as recently proposed by Alparone et al. [19], was used in this letter. The QNR index evaluates the quality of the fused image without requiring the HRM image. The QNR index combines the two distortion indexes of the radiometric and geometric distortion indexes. The radiometric distortion index, referred to as D λ, is calculated as D λ = 1 L L Q( ˆM l, ˆM r ) Q( M l, M r ) (13) L(L 1) l=1 r=1 r 1 where ˆM denotes the HRM bands, M represents the LRM bands, and Q represents the universal image-quality-index calculation [20]. This index has zero value if the HRM and LRM images are identical. The geometric distortion index, referred to as D S, is calculated as D S = 1 L Q( L ˆM l,p) Q( M l, P ) (14) l=1 where P denotes the HRP image and P is the LRP image obtained by filtering with an MTF-shaped low-pass filter. The QNR index is defined as QNR =(1 D λ )(1 D S ). (15) The highest value of QNR is one. This is obtained when the radiometric and geometric distortions are both zero. It is important to note that the resulting QNR index is in agreement with the objective evaluations expressed by Q4 and ERGAS [19]. A higher QNR value indicates that most of the radiometric and geometric information of the LRM and HRP images are incorporated during the fusion process. Table I shows the performance comparisons of the fused images. Table I demonstrates that the IAWP method provides a less-distorted fused image compared with that provided by the AWLP method. V. C ONCLUSION In satellite image fusion, the critical issue is how much radiometric information is preserved while simultaneously increasing the geometric information. To cope with this problem, this letter has proposed the IAWP fusion method using the à trous algorithm. The proposed method can be considered as an improvement of the SW and AW methods in the sense that the LRM image is not decomposed and because it injects REFERENCES [1] Z. Wang, D. Ziou, C. Armenakis, D. Li, and Q. Li, A comparative analysis of image fusion methods, IEEE Trans. Geosci. Remote Sens., vol. 43, no. 6, pp , Jun [2] C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, Synthesis of multispectral images to high spatial resolution: A critical review of fusion methods based on remote sensing physics, IEEE Trans. Geosci. Remote Sens., vol. 46, no. 5, pp , May [3] L. Alparone, L. Wald, J. Chanussot, C. Thomas, P. Gamba, and L. Mann Bruce, Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest, IEEE Trans. Geosci. Remote Sens., vol. 45, no. 10, pp , Oct [4] J. Zhou, D. L. Civco, and J. A. Silander, A wavelet transform method to merge Landsat TM and SPOT panchromatic data, Int. J. Remote Sens., vol. 19, no. 4, pp , Mar [5] S. Rahmani, M. Strait, D. Merkurjev, M. Moeller, and T. Wittman, An adaptive IHS pan-sharpening method, IEEE Geosci. Remote Sens. Lett., vol. 7, no. 4, pp , Oct [6] M. Lillo-Saavedra and C. Gonzalo, Spectral or spatial quality for fused satellite imagery? A trade-off solution using the wavelet à trous algorithms, Int. J. Remote Sens., vol. 27, no. 7, pp , Apr [7] X. Otazu, M. González-Audícana, O. Fors, and J. Núnez, Introduction of sensor spectral response into image fusion method. Application to wavelet-based methods, IEEE Trans. Geosci. Remote Sens., vol. 43, no. 10, pp , Oct [8] T. Ranchin, B. Aiazzi, L. Alparone, S. Baronti, and L. Wald, Image fusion The ARSIS concept and some successful implementation schemes, ISPRS J. Photogramm. Remote Sens.,vol.58,no.1/2,pp.4 18, Jun [9] M. González-Audícana, X. Otazu, O. Fors, and A. Seco, Comparison between the Mallat s and the à trous discrete wavelet transform based algorithms for the fusion of multispectral and panchromatic images, Int. J. Remote Sens., vol. 26, no. 3, pp , Feb [10] J. Núnez, X. Otazu, O. Fors, A. Prades, V. Pala, and R. Arbiol, Multiresolution-based image fusion with additive wavelet decomposition, IEEE Trans. Geosci. Remote Sens., vol. 37, no. 3, pp , May [11] T. Stathaki, Image Fusion Algorithms and Applications, 1st ed. Amsterdam, The Netherlands, Elsevier, 2008, ch. 6 [12] M. González-Audícana, J. L. Saleta, R. G. Catalán, and R. García, Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition, IEEE Trans. Geosci. Remote Sens., vol. 42, no. 6, pp , Jun [13] M. Choi, A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter, IEEE Trans. Geosci. Remote Sens., vol. 44, no. 6, pp , Jun [14] J. Lee and C. Lee, Fast and efficient panchromatic sharpening, IEEE Trans. Geosci. Remote Sens., vol. 48, no. 1, pp , Jan [15] B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, MTFtailored multiscale fusion of high-resolution MS and Pan imagery, Photogramm. Eng. Remote Sens., vol. 72, no. 5, pp , May [16] M. Choi, J. Song, D. Seo, D. Lee, and H. Lim, Introduction of a fast substitute wavelet intensity method to pan-sharpening technique, Korean J. Remote Sens., vol. 23, no. 5, pp , Oct [17] L. Alparone, S. Baronti, A. Garzelli, and F. Nencini, A global quality measurement of pan-sharpened multispectral imagery, IEEE Geosci. Remote Sens. Lett., vol. 1, no. 4, pp , Oct [18] Y. Zhang, Methods for image fusion quality assessment A review, comparison and analysis, in Proc. Int. Arch. Photogramm, Remote Sens. Spatial Inf. Sci., Beijing, China, 2008, vol. XXXVII, pp [19] L. Alparone, B. Aiazzi, S. Baronti, A. Garzelli, F. Nencini, and M. Selva, Mutltispectral and panchromatic data fusion assessment without reference, Photogramm. Eng. Remote Sens., vol. 74, no. 2, pp , Feb [20] Z. Wang and A. C. Bovik, A universal image quality index, IEEE Signal Process. Lett., vol. 9, no. 3, pp , Mar