A Comparative Analysis of Image Fusion Techniques for Remote Sensed Images

Transcription

1 roceedings of the Word Congress on Engineering 27 Vo I WCE 27, Juy 2-4, 27, London, U.K. Comparative naysis of Image Fusion Techniques for emote Sensed Images sha Das 1 and K.evathy 2 Department of Computer Science, University of Keraa. 1 asha_das_2@rediffmai.com, 2 revathy_srp@yahoo.com bstract This paper deas with different techniques for registration and fusion of remote sensed images. In this work the ower spatia resoution mutispectra and higher resoution panchromatic images of SOT sateite are used. These images are registered using a registration agorithm that combines a simpe yet powerfu search strategy based on stochastic gradient with the simiarity measure as mutua information, together with a waveet-based muti-resoution pyramid. The agorithm is found to give sub pixe registration accuracy. The study is imited to pairs of images, which are misaigned by rotation and/or transation. The registered images are subjected to a pixe eve mutispectra image fusion process using waveet transform approach. Spectra quaity assessments shows that compared to other conventiona image fusion techniques, this fusion process using waveet transform keeps much of the spectra information in the merged image with respect to the origina mutispectra one. Finay, segmentation is performed on the fused images to vaidate the agorithms used for registration and fusion and the resuts show better accuracy for waveet based methods than the conventiona methods. Index Terms Fusion, image registration, mutua information, segmentation, stochastic optimization. I. ITODUCTIO To address chaenges in the fied of remote sensing, integration of mutipe sensor data is a key component, and as a first step towards this goa, very accurate registration of muti-sensor data is the first requirement for such integration. The aim of the work is to deveop and assess an image registration methodoogy that wi enabe this accurate mutisource integration. This study focuses on precision correction or automatic image registration, with the goa of providing methods that can hande mono- and muti-sensor data with muti-resoution from a spatia or spectra point of view. For combining the spatia and spectra features, data fusion techniques are used [1]. The high-resoution SOT panchromatic (SOT- images and mutispectra SOT (SOT-XS images are fused using the conventiona techniques and the waveet based methods, and the resuts are compared for their spectra quaity. Spectra quaity assessments shows that compared to other conventiona image fusion techniques, the pixe eve mutispectra fusion process using waveet transform appied on these images keeps much of the spectra information in the merged image with respect to the origina mutispectra one. Very accurate registration of panchromatic and mutispectra images is the first requirement for such integration. registration agorithm that combines a simpe yet powerfu search strategy based on a stochastic gradient with the simiarity measure as mutua information, together with a waveet-based muti-resoution pyramid is found to give subpixe registration accuracy. The study is imited to pairs of images, which are misaigned by rotation and/or transation. The resut of registration and fusion of the higher resoution panchromatic image and that of the Mutispectra image are subjected to watershed segmentation. The segmentation accuracy is better for the fusion using waveet-based methods than the other conventiona methods. Section II gives an overview of the registration method adopted and section III summarizes the waveet fusion techniques used in this work. Data sets used and resuts are discussed in section VI and discussions and concusion are given in section V. II. MULTIESOLUTIO IME EISTTIO Conventiona feature based registration technique via the use of round Contro oints (C is aborious, tedious, and time intensive, whereas an automatic intensity based registration approach is more fexibe in the sense that it is image content independent. The main principe behind any intensity based registration approach is to find a set of transformation parameters that gobay optimizes a simiarity measure. Two commony used simiarity measures are mean squared difference (MSD and normaized cross-correation (CC. However, these two simiarity measures are adequate ony for intra-moda registration (i.e., for registration of images taken from the same sensor. For muti-moda image registration probems, mutua information ( is found to be a suitabe simiarity measure. Simiar to CC, the intention is to maximize the mutua information between the two images. IS: WCE 27

2 roceedings of the Word Congress on Engineering 27 Vo I WCE 27, Juy 2-4, 27, London, U.K. Fig. 1. Summary of the waveet-based mutua information registration method Since its introduction, has been used widey in many medica image registration probems, whereas its appication in remote sensing image registration has been imited. Ony recenty, some work has been initiated on registration of remote sensed images using mutua information. For our appication we seected as the simiarity measure as they are found to give good resuts for the remote sensed images taken by different sensors. The first step in registering two images is to decide on the feature space to use for matching. The waveets or waveetike features are used as the feature space for the registration process. Fig.1 summarizes the adopted registration scheme when waveet or waveet-ike information is utiized. oth the reference and input images are first decomposed foowing a muti-resoution waveet or frame decomposition. In order to achieve computationa efficiency, search strategy foows the muti-resoution decomposition, working iterativey from the deepest eve of decomposition (where the image size is the smaest to the top eve of decomposition, i.e., going from coarse to fine spatia resoution. For a eves of decomposition, the simiarity measure between sub-band images of the reference image and input image is successivey computed and maximized. The accuracy of this search increases when going from coarse resoution to fine resoution [2]. Steerabe Simoncei fiters [3] are more robust to transation, rotation and noise than the standard Daubechies waveet fiters and so they are adopted to extract the feature space for matching. The method described by Simoncei enabes one to buid transation and rotation-invariant fiters by reaxing the critica samping condition of the waveet transforms. The next step is to appy the simiarity measure based on Mutua Information ( on the extracted features of the mutispectra and panchromatic images. The mutua information [4] of two images is expressed in terms of the entropy of the images, where entropy is a measure of uncertainty of how we one is abe to predict the gray vaue of an arbitrary point in an image. iven two images and, the mutua information M(, of the images is defined by M(, = H( H( - H(,, (1 with H( and H( the margina entropies of the separate images and H(, the entropy of the joint image. The probem of registering images and is summarized as maximizing M (,, which means that the joint entropy H (, is minimized (i.e. the joint histogram has sharp peaks. t the same time the margina entropies H ( and H ( are maximized to avoid situations where the images are shifted so far apart that ony two sma regions of background coincide. ow the optima transformation to aign the images have to be found out for which the is maximum. The optimization technique, which is impemented in this work, is the Simutaneous erturbation Stochastic pproximation (SS agorithm [5] introduced by Spa. SS is based on an easiy impemented and highy efficient gradient approximation that reies ony on measurements of the objective function to be optimized. It does not rey on expicit knowedge of the gradient of the objective function, or on measurements of the gradient. III. IME FUSIO Image fusion is a too for integrating a high-resoution panchromatic image with a mutispectra image, in which the resuting fused image contains both the high-resoution spatia information of the panchromatic image and the coor information of the mutispectra image. In this study, a pixe eve mutispectra image fusion process using waveet transform approach is performed [6]. Many papers about image fusion based on waveet transform have been pubished in recent years [7]-[12]. The bock diagram of a generic waveet-based image fusion scheme is shown in Fig. 2. Fig. 2. ock diagram of a generic waveet-based image fusion approach Waveet transform based image fusion invoves three steps; forward transform, coefficient combination and backward transform. In the forward transform, two or more registered input images are waveet transformed to get their waveet coefficients. These coefficients represent the approximation, IS: WCE 27

3 roceedings of the Word Congress on Engineering 27 Vo I WCE 27, Juy 2-4, 27, London, U.K. horizonta, vertica and diagona components of the input images. The same process needs to be appied to a source images one by one. Then, these waveet coefficients from the different input images are combined according to certain fusion rues to get fused waveet coefficients. fter seecting the fused waveet coefficients according to a fusion rue, an inverse waveet transform is done to construct the fused image. Impementation of image fusion by means of waveet transform based on substitutive and additive approaches are adopted. In these approaches, first image and the,, and bands of muti-spectra image are decomposed using waveet transform method. fter decomposing the,, and bands of the mutispectra image using waveet transform in eve of transformation, usuay = 2 or 3, we wi have one approximation coefficients, (,, and 3 waveets anes for each band WT WT WT D D D - : is pproximation coefficient at eve or approximation pane - H : is Horizonta coefficient at eve or horizonta waveet pane - V : is Vertica Coefficient at eve or vertica waveet pane - D : is Diagona coefficient at eve or diagona waveet pane Simiary by decomposing the panchromatic high-resoution image we wi have one approximation coefficients, ( and 3 waveets anes for anchromatic image, where means, panchromatic image. Foowing this step, the waveet coefficients sets from two images are combined via substitutive or additive rues. In the case of substitutive method, the waveet coefficient panes (or detais of the,, and decompositions are repaced by the simiar detai panes of the panchromatic decomposition. Then, for obtaining the fused images, the inverse waveet transform is impemented on resutant sets. D D D (2 (3 nother possibiity for fusion decision map construction is using additive fusion method in which the waveet coefficient panes of the high-resoution image is added directy to the waveet coefficient panes of the ow spatia-resoution image in each band. ([ H ([ H ([ H H H H ] [ V ] [ V ] [ V ] [ D ] [ D ] [ D D ] D ] D ] For comparison purpose, the same images are aso fused using Intensity, Hue, Saturation (IHS method and rovey and Mutipicative Transformation methods to evauate the proposed waveet transformation approach. IV. DT SETS USED D ESULTS In this study, SOT sateite images are used as the test images. mutispectra SOT scene consists of three bands- XS1, XS2 and XS3. The first two bands characterize the ground cover in the visibe waveength range and the third band the near infrared waveength range. These bands are acquired with a spatia resoution of 2*2m per pixe. The waveength range of the panchromatic image covers the bands XS1 and XS2 with a ground resoution of 1*1m. fter registration of the ower resoution SOT-XS image into the high resoution SOT- image, a 512*512 pixes subset with various features that is sufficient for fusion purpose is seected. The reason for the choice of SOT imagery is the reative simpe geometric reation between panchromatic and mutispectra scenes but the method described is suitabe for imagery from other scenes as we. Fig. 3(a and Fig. 3(b show the SOT-XS and SOT- images used in this study. (a (b Fig. 3. Test Images, (a SOT-XS (b SOT- The proposed image registration scheme based on the and the stochastic optimization appied on a muti-resoution basis is used as the preprocessing step for the fusion of the SOT panchromatic and SOT-XS images. Each band of the mutispectra SOT-XS image is separatey registered with the image with which the mutispectra image has to be fused. So the red, green and the near infrared (I band images are (4 IS: WCE 27

4 roceedings of the Word Congress on Engineering 27 Vo I WCE 27, Juy 2-4, 27, London, U.K. registered with the higher resoution SOT images. n up-samping of the ower resoution images are performed to make the pixe size equa to that of the higher resoution SOT- images. Tabes I, II and III give the resuts of the registration of each of the bands. TLE I: Mutua Information Using Simoncei Decompositions for and XS1: starting point (,,, Max: o: Iterations = 1 Fusion of SOT- and SOT-XS is performed using substitutive method and additive waveet methods and compared with other conventiona methods ike IHS and rovey methods with respect to statistica and visua methods. Fig. 4(a shows the fused image by Substitutive waveet method and Fig. 4(b shows the fused image by dditive waveet method. Fusion resuts for IHS and rovey methods are shown in Fig. 4(c and Fig. 4(d respectivey. ctua arameters 3 2 Leve Starting Estimated arameters Max: MS error (a (b TLE II: Mutua Information Using Simoncei Decompositions for and XS2: starting point (,,, Max: o: Iterations = 1 ctua arameters 3 2 Leve Starting Estimated arameters Max: MS error TLE III: Mutua Information Using Simoncei Decompositions for and XS3: starting point (,,, Max: o: Iterations = 1 ctua arameters Leve Starting Estimated arameters Max: MS error The registration of the first band XS1 with the SOT- is performed with an MS error of The registration of the second band XS2 with the SOT- is performed with an MS error of.6466 and.4529 for the case of XS3 and SOT-. (c Fig. 4.Fused Images (a WT (Substitutive Method (b WT (dditive Method. (c IHS Fusion (d rovey Fusion The performance of various fusion methods are anayzed by the foowing methods. One of the quantitative assessment criteria that is considered as a spectra quaity index is the deviation index, measuring the normaized goba absoute difference of the fused image with ow-resoution mutispectra image. Spectra quaity of fused images is iustrated in a better way by this quantity. The Fig. 5 shows this parameter for the fused images using various methods. It can be seen that the minimum vaues for a of the three bands beong to the waveet based image fusion methods HIS rvy Sbst ddi Fig 5. Deviation index of fused images (d XS1 XS2 XS1 IS: WCE 27

5 roceedings of the Word Congress on Engineering 27 Vo I WCE 27, Juy 2-4, 27, London, U.K. Contro of the radiometric correspondence of the two images at a oca scae, can be performed by measuring the correation coefficient between the two images. It is obvious from the diagram of Fig. 6 that the fused image with the waveet transform fusion method has the maximum correation in a the three bands. Since the correation coefficient represents the degree of simiarity, the vaue of the correation coefficient is desirabe to be as higher as possibe, but it cannot reach to 1 which means that the fused image is the same with the origina muti-spectra image and no spatia information from panchromatic image is added to the fused image. In this sense, it shows that the waveet based fusion agorithm resuts in a very higher simiarity between origina muti-spectra image and the fused images HIS ry Subs ddi Fig 6: Correation coefficients between SOT-XS image and fused images The histograms of the fused image using waveet transform in the I band have a itte difference with the histogram of the SOT-XS image in the simiar band (Fig.7. In spite of waveet transform fusion method; the two others have changed the histogram of the fused image to a more extent. This means, in fusion by waveet transform, spectra characteristics of fused image is very simiar to spectra characteristics of SOT-XS image a. I and Histogram from XS image c. I and Histogram from IHS b. I and Histogram from WT XS1 XS2 XS d. I and Histogram from rovey Fig 7: Histograms of I band in fused images and SOT-XS image after (a without fusion (b Substitution Waveet fusion (c IHS Fusion (d rovey Fusion (e dditive Waveet Fusion Finay the resuts of fusion are compared by subjecting the fused images to segmentation. We have used a typica segmentation method namey watershed agorithm. The resuts of the segmentation of fused images are shown in Fig. 8. It can be observed that the segmentation accuracy is better for the case of fusion using waveet methods than using other conventiona methods. (a (c Fig 8 esuts of watershed segmentation. Segmentation resut in (a origina MS image. (b Waveet Fused MS image. (c IHS Fused MS image and (d rovey Fused MS (b (d V. DISCUSSIOS D COCLUSIO Image Fusion aims at the integration of disparate and compementary data to enhance the information apparent in the images as we as to increase the reiabiity of the interpretation. This eads to more accurate data and increased utiity in appication fieds ike segmentation and cassification. In remote sensing this finds immense appication as the reduced amount of data in the mutispectra images can be fused through efficient fusion techniques with the high informative panchromatic images to yied better high spectra and spatia resoution images. These high quaity fused images can be used for andmass cassification or for miitary purposes ike target ocaization etc. The quaity of the fused image greaty affects the cassification accuracy. For this a good registration technique is required which greaty affects the quaity of fused image. The entropy based based registration technique which is appied ony quite recenty to the remote sensing fied is found to be giving better resuts for the cassification purposes in this fied. EFEECES [1] C. OHL and J. L. V EDEE, Mutisensor image fusion in remote sensing: concepts, methods and appications, int. j. remote sensing, 1998, vo. 19, no. 5, IS: WCE 27

6 roceedings of the Word Congress on Engineering 27 Vo I WCE 27, Juy 2-4, 27, London, U.K. [2] rene. Coe-hodes, Kisha L. Johnson, Jacqueine LeMoigne, and Iya Zavorin, Mutiresoution egistration of emote Sensing Imagery by Optimization of Mutua Information Using a Stochastic radient, IEEE Transactions On Image rocessing, Vo. 12, o. 12, December 23 [3] E.. Simoncei, and W. T. Freeman, The Steerabe yramid: Fexibe rchitecture for Muti-scae Derivative Computation, roc. ICI-95, [4] F. Maes,. Coignon, D. Vandermeuen,. Marcha,. Suetens, Mutimodaity image registration by maximization of mutua information, IEEE Transactions on Medica Imaging 16 ( [5] J. C. Spa, Mutivariate stochastic approximation using a simutaneous perturbation gradient approximation, IEEE Trans. utomat. Contr., vo. 37, no. 3, pp , 1992 [6] Fusion pproach of Muti-sensor emote Sensing Data ased on Waveet Transform, He uojin Li Keu Hu Deyong 1998 [7] anchin, T. and Wad, L., 2. Fusion of High Spatia and Spectra esoution images: The SIS Concept and Its Impementation. hotogrammetric Engineering & emote sensing Vo. 66, pp [8] Yocky, D.., Image merging and data fusion using the discrete two-dimensiona waveet transform.j. Opt. Soc. m.., Vo. 12, o 9, pp [9] Yocky, D.., Mutiresoution WaveetDecomposition Image Merger of Landsat ThematicMapper and SOT anchromatic Data.hotogrammetric Engineering & emote sensing Vo. 62, o. 3, pp [1] Zhou, J., Civco, D. L., and Siander, J.., waveet transform method to merge Landsat TM and SOT panchromatic data. Internationa Journa of emote Sensing, Vo. 19, o. 4, pp [11] úñez, J., X. Otazu, O. Fors,. rades, V. aà, and. rbio, Mutiresoution-based image fusion with additive waveet decomposition, IEEE Trans. eosci. emote Sensing, vo. 37, pp , May [12] Li, H.,.S. Manjunath, and S.K. Mitra, Mutisensor image fusion using the waveet transform, raph. Modes Image rocess, vo. 57, no. 3,pp , IS: WCE 27