SPATIAL QUALITY EVALUATION OF FUSION OF DIFFERENT RESOLUTION IMAGES

Transcription

1 Jun L SPATIAL QUALITY EVALUATION OF FUSION OF DIFFERENT RESOLUTION IMAGES Jun L Laval Unversty, Canada COPL, Physcs department Jl@phy.ulaval.ca KEY WORDS: Image Fuson, Evaluaton, Spatal Qualty, Blur Parameter, Mult-sensor ABSTRACT Spatal qualty and spectral qualty are the two mportant ndexes that are used to evaluate the qualty of any fused mage. Most of the exstng methods for ths purpose, however, consdered only the spectral qualty except for that proposed by Zhou et al., whle the latter cannot be used to compare drectly the spatal resoluton between the fused mage and orgnal hgh-resoluton. A new spatal qualty assessment approach was presented n the paper based on the blur parameter estmaton. It s based fact that the blur parameter, whch s a measure of the spread of the sensor s pont spread functon (PSF), characterzes the spatal resoluton of the sensor mage. The merged mages derved from IHS, PCA, HPF, and AWL methods are compared n order to evaluate the performance of the proposed spatal qualty measure. Expermental results demonstrate that the spatal qualty measure based on the blur parameter s more effcent n judgng spatal qualty of the fused mage. The blur parameter s an objectve qualty measure based on a basc optcal mechansm and thus can be used to evaluate the effectveness of varous processng or fuson schemes n terms of spatal qualty of the fused mage. It s also shown that the spatal resoluton of the fused mages can not completely match wth the same spatal resoluton of the hgh-resoluton panchromatc mage. The proposed approach s based upon smple concepts, easy to understand and easy to mplement and use. INTRODUCTION The number of commercally avalable mult-sensors and the data they provde are contnually ncreasng. Each sensor has ts msson and applcatons. It s often desrable to smultaneously requre hgh spatal and spectral resoluton n a sngle mage. However, n most cases nstruments are not capable of provdng such data ether by desgn or by observatonal constrants. For example, n remote sensng SPOT PAN satellte data provdes hgh-resoluton (0-m pxels) panchromatc data whle Landsat TM satellte data provdes lower resoluton (30-m pxels) multspectral data. The better soluton to ths problem could come from mage fuson. The ntegraton of spectrally and spatally complementary remote mult-sensor data can facltate vsual and automatc mage nterpretaton. Many fuson methods have been proposed for fusng hgh spectral and spatal resoluton data n order to produce multspectral mages havng the hghest spatal resoluton avalable wthn the data set. However, a few quanttatve methods have been proposed n current lterature to evaluate the qualty of the fused mage (Munechka et al., 993; Blanc et al., 996; Ranchn et al., 996; Wald et al., 997; Zhou et al., 998; Zhang, 999). Most of the evaluatons have been proposed only focus on the spectral qualty of fused mage. Some methods for evaluatng the spectral qualty are based on the calculaton of the mage dfference between the merged mage and standard mage, whch s the desred merged result (Yocky 995, L et al. 995). However, such a standard remote sensng mage s usually not avalable. For remote sensng applcatons, Chavez et al. (99) consdered that the methods used to merge data wth hgh-spatal and hgh-spectral resoluton propertes should not dstort the spectral characterstcs of the orgnal hgh-spectral resoluton data. Not dstortng the spectral characterstcs s very mportant for ensurng that targets that are spectrally separable n the orgnal data are stll separable n the fused data set ( Chavez et al. 99). Wald et al. (997) proposed both a formal approach and some crtera to provde a quanttatve assessment of the spectral qualty of the fused mage. Fve sets of crtera are defned. The above methods only concentrated on the evaluaton of the spectral qualty of a merged wthout the consderaton of spatal qualty. However, no quanttatve methods have been found n current lterature to evaluate the spatal qualty of the fused mages except for Zhou et al. (998). Zhou used the correlaton coeffcents between the hgh-pass fltered SPOT PAN mage as ndex of the spatal qualty. It s based on the fact that the spatal nformaton unque n SPOT PAN s mostly concentrated n hgh frequency doman. But ths method can not drectly compare the spatal qualty of the merged mage wth that of the hgh-resoluton panchromatc mage. Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B. Amsterdam

2 Jun L In ths paper, a new approach was presented to assess the spatal qualty of a fused mage based on blur parameter estmaton. Ths measurement s based on the fact the blur parameter, whch s a measure of the spread of the sensor s pont spread functon (PSF), characterzes the spatal resoluton of the sensor mage. The paper s organzed as follows. A bref revew of qualty evaluaton of the fused mages s gven n secton. Secton 3 presents a new scheme for spatal qualty evaluaton of the fused mages. Expermental results for the spatal qualty of the fused mages based on Intensty-Hue-Saturaton (IHS) (Carper et al., 990), Hgh-Pass Flter (HPF) (Chavez and Bowell, 988), Prncpal Component Analyss (PCA) (Fung and LeDrew, 987) and Intensty Addtve Wavelet (AWL) (Nunez et al. 999; L et al., 999) methods are presented n secton 4. QUALITY EVALUATION OF THE FUSED IMAGES Fused mages provde ncreased nterpretaton capabltes and more relable results snce data from dfferent characterstcs are combned. The fused mages must have the followng propertes: (a). If the fused mage s degraded agan to the orgnal low resoluton, t should concde wth the orgnal low-resoluton mage ; (b). The spatal resoluton of the fused mage should be as dentcal as possble to that of the orgnal hgh-resoluton mage; (c). The mult-spectral set of the fused mages should concde wth the mult-spectral set of the mages that the correspondng sensor would observe wth the orgnal hgh-resoluton. Therefore, the fused mages should be evaluated both spectrally and spatally.. Spectral qualty The mathematcal approach along wth a set of crtera descrbed by Wald et al. s used to evaluate the spectral qualty of the fused mages. The set of crtera s capable of quanttatvely summarzng the performance of the fused mage. It provdes a global vew of the dscrepances between the fused mage and the orgnal one. In the spectral qualty assessment method, the bas s used to express the dfference between the means of the orgnal and the merged mages. The dfference n varances represents the quantty of nformaton added or lost durng the enhancement of the spatal resoluton. The correlaton coeffcent shows the smlarty n tny detal between the orgnal and the merged mages. The standard devaton globally represents the level of error n any pxel.. Spatal qualty The spatal qualty of the fused mage s usually judged by vsual nspecton. However, In remote sensng applcatons the objectve comparson of the vsual qualty of the fused mages s a dffcult and lengthy task to handle. The human vsual system s not equally senstve to varous types of dstorton n an mage. The perceved mage spatal qualty strongly depends on the observed scene, the vewng condtons and observer. Thus, judgment should be determned by subjectve tests on the scene beng vewed, and an exact decson can be rarely gven. Recently, Zhou et al. (998) appled the correlaton coeffcents between the hgh-pass fltered fused mages and the hgh-pass fltered panchromatc mage as an ndex of the spatal qualty. It s based on the fact that the spatal nformaton unque n SPOT PAN s mostly concentrated n hgh frequency doman. The fused mages and SPOT PAN mage are fltered wth a Laplacaan hgh-pass flter resultng n the hgh frequency parts of the data. Then the correlaton coeffcents between the hgh frequency components of fused mages and the hgh frequency components of SPOT PAN were calculated and used to compare the smlarty. However, ths method can not drectly compare the spatal qualty of the merged mage wth that of the hgh-resoluton panchromatc mage. 3 A NEW SCHEME FOR SPATIAL QUALITY EVALUATION OF THE FUSED IMAGES Actually, the spatal qualty of an mage concdes wth the mage resoluton. In other words, the hgher the mage resoluton, the rcher the detal nformaton and the better the spatal qualty of the mage. The mage resoluton s an essental characterstc of an magng sensor nstrument. It generally depends on the physcal characterstcs of the sensor: the optcs and the densty and spatal response of the detector elements. The ultmate resoluton of an nstrument wth a very narrow pont spread functon may be lmted by the pxel sze of the detector nstead of by the performance of the optcal elements. The lne spread functon (LSF) provdes a natural procedure for establshng resoluton crtera for the mage, and the LSF can be derved from the pont spread functon (PSF). Thus, the spatal qualty of the fused mage can be characterzed by pont spread functon the PSF. Here we used blur parameter as an ndex of the spatal qualty of the fused mage. 340 Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B. Amsterdam 000.

3 Jun L Theoretcally, the sensor PSF can be obtaned from the mage of a pont lght source. However, n practce, t s dffcult to create an deal pont lght source that s ncoherent and polychromatc ( Surya, 994). Therefore, the sensor PSF s usually estmated from the observed mage. Here we assume that the PSF s a two-dmensonal Gaussan functon. Gaussan functons have the followng propertes: (). The Gaussan functon s symmetrc about the mean, and the weghts assgned to sgnal values decrease gradually wth dstance from the mean; (). The wdth of the Gaussan functon s determned by ts spread parameter,.e., the standard devaton. As the spread parameter decreases, Gaussan convoluton does less smoothng. Conversely as the spread parameter ncrease, the amount of smoothng s ncreased. The spread parameter of the Gaussan functon s taken to be equal to the blur parameter s, and therefore the PSF s: x + y PSF( x, = exp( ) () ps s To obtan the measured PSF n above equaton (), a fundamental problem s how to determne the blur parameter for a gven mage. The blur parameter s s a measure of the spread of the sensor PSF. For a crcularly symmetrc PSF denoted by h ( x,, t s defned as s = ( x + y ) h( x, dxdy () For a PSF model based on paraxal geometrc optcs, t can be shown that the blur parameter s s proportonal to the blur crcle radus. If R s the blur radus, then s = R. For a PSF model based on a -D Guassan functon, s s the standard devaton of the dstrbuton of -D Guassan functon. Assume that there exst N step edges EDG ( x, ( =,, L, N) along the y-axs on the fused mage f ( x,. Each edge s composed of Np pxels. Let a be the mage denstes of EDG ( x, to the left of the y-axs and b be the heght of the correspondng steps. Each edge-ntensty can be expressed as EDG ( x, = a + b u( x) (3) where u (x) s the standard unt step functon. If G ( x, s the observed mage of each edge and h ( x, s the sensor s PSF, then we have where denotes the convoluton operaton. Now consder the dervatve of have G ( x, = h( x, EDG ( x, (4) G along the gradent drecton. Snce dfferentaton and convoluton commute, we G EDG = h( x, = h( x, b d( x) (5) where d (x) s the drac delta functon along the x axs. The above expresson can be smplfed to obtan where (x) G = b q(x) q s the lne spread functon of the sensor defned by (6) Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B. Amsterdam

4 Jun L + h For any PSF h( x, of a lossless sensor, by defnton, we have Therefore, we obtan q ( x ) = ( x, dy (7) + + h ( x, dxdy = (8) + G ( x, dx = b, ( =,, L, N ) (9) Therefore, gven the observed mage G ( x, of each edge, we can obtan the lne spread functon q (x) of each edge mage from the expresson G q = ( x) + (0) G dx The lne spread functon of the sensor q (x) can be taken to be the average of q (x) of N edge mages. Then N q ( x) = q ( x) () N After obtanng the lne spread functon q (x), the next step s to determne the blur parameter. The second central moment Where x s the frst moment of the LSF. Therefore, we can get the blur parameter by = s l of the LSF can be computed as follows: + s = ( x x) q( x dx () l ) s = s l. The detaled procedure of the proposed algorthm for the blur estmaton s as follows: Step : Calculate the dfference of gray values of adjacent pxels n each row and search pxel set of each edge n G ( x j ) ( G ( x j ) s the mage dervatve at column x j, x j x; j =,, LNp ), whose sgn changes from plus to mnus. Step : Compute the frst moment of each edge by Np j= x j M =, =,, L, N Np G ( x ) j= G ( x ) j j (3) 34 Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B. Amsterdam 000.

5 Jun L where M s the column number where the each edge s located, and Np s total numbers of pxels for each edge. Step 3: Determne the lne spread functon of each edge: G ( x) q ( x) = x Np (4) G ( x) x= x Step 4: Take the average of q (x) of N edges as the lne spread functon of the sensor q (x) accordng to Equaton (). Step 5: Calculate the second central moment Then take the average of Step 6: Compute the blur parameter by s l of the lne spread functon usng followng equaton: N Np s = ( x M ) q( x) (5) l = j= s l over all the rows n the mage. s = s l. j 4 EXPERIMENTAL RESULTS The above assessment technque s tested on fuson of SPOT panchromatc mage of Nngxa area, the western of Chna, acqured on 7 February 993 and the Landsat TM5,4,3 mage, acqured over the same area on 8 May 995. The Landsat-TM mult-spectral mage s geometrcally regstered to the SPOT panchromatc mage accordng to a moment-based approach proposed by Flusser and Suk (994). The TM mages were also up-sampled to the same pxel sze as SPOT PAN mage. The IHS, HPF, PCA, and the addtve wavelet (AWL) methods are employed to fuse SPOT PAN and TM mult-spectral mages. The color composte mage of Landsat TM5,4,3 s shown n Fgure a and fgure b for the SPOT panchromatc mage. Fgure c, d, e and f show the merged results of the Landsat and the panchromatc mages by the IHS, HPF, PCA, and the ntensty addtve wavelet (AWL) methods, respectvely. It can clearly be seen that the above four fuson methods are capable of mprovng the spatal resoluton wth respect to the orgnal Landsat TM mage. To quantfy the behavor of the IHS, HPF, PCA, and AWL methods we compute the blur parameters among the dfferent solutons and orgnal SPOT-PAN mage as well as TM mages. Table presents the blur parameters of orgnal TM, SPOT mages, the fused mages by the IHS, HPF, PCA and AWL approaches. The frst and second lne of table show the blur parameters of TM5,4,3 and SPOT PAN mage. The blur parameter of SPOT PAN mage s less than those of TM5,4,3 mages. Ths means that SPOT PAN mage has hgher spatal resoluton than TM mages, whch s n accordance wth the realty. It s shown from Table that all above four fuson methods have larger the blur parameters of n band5,4,3 than SPOT PAN mage. Ths means that the spatal resoluton of the fused mages based on the above four fusng methods can not completely reach the spatal resoluton of the SPOT PAN mage. On the other hands, the results derved from the above four fuson methods all have less blur parameters than TM mage do. It s obvous that the four fuson methods are capable of mprovng the spatal resoluton of the TM mult-spectral mages. Therefore, mult-sensor mage fuson s usually a trade-off between the spectral nformaton from hgh spectral resoluton sensor and the spatal structure from hgh spatal resoluton sensor. The last lne of table shows that the blur parameters of and n band 4 and band 3 are less than other blur parameters n the same columns. Ths means that band 4 and band 3 derved from the AWL have the best spatal qualty compared wth those from other schemes. The reason s that the resoluton of the SPOT panchromatc mage s added to the soluton wth dscardng the resoluton of TM mult-spectral mages. Thus the detal nformaton from both mages s used. Wth the AWL method, the blur parameter n band 5 does not match wth the ones n band 4 and band3. The reason s that the IHS s nonlnear transformaton. Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B. Amsterdam

6 Jun L (a). Orgnal Landsat TM5,4,3 mages (b). SPOT panchromatc mage (c). The merged result by the IHS method (d). The merged result by the HPF method (e). The merged result by the PCA method (f). The merged result by the AWL method Fgure. The merged results of SPOT PAN and TM mages by the IHS, HPF, PCA, and AWL methods Band 5 Band 4 Band 3 TM SPOT.0408 IHS HPF PCA AWL Table. The blur parameters of the regstered orgnal TM, SPOT mages, the fused mages by the IHS, HPF, PCA and AWL approaches (unt: pxel) For comparson wth the proposed scheme, Zhou s (998) approach n terms of the spatal qualty was also used n the experment to test dfferent fusng technque. In the Zhou s approach, the correlaton coeffcents between the hgh-pass fltered fused SPOT PAN and TM mages and the hgh-pass fltered SPOT PAN mage are taken as an ndex of the spatal qualty. The hgh-pass flter s known as a Laplacan flter as llustrated n equaton (6): 344 Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B. Amsterdam 000.

7 Jun L (mask) = 8 (6) Table shows the ranks of dfferent fusng algorthms n terms of spatal qualty by the proposed and Zhou s approached. The qualty measures wth both the proposed and Zhou s methods are almost consstent. Only t exsts slghtly dfference n band 5 n terms of spatal qualty between the proposed and Zhou s methods. The blur parameter of the PCA method n band 5 s less than that of the AWL method n terms of the proposed qualty measure whle the evaluated result based on Zhou s qualty measure s opposte. But the proposed method concdes wth real results. The reason s that the frst prncpal component mage s replaced by SPOT PAN mage so that the fused mage n band 5 s closer to SPOT PAN mage than other two bands. Band 5 Band 4 Band 3 Rank The proposed Zhou s The proposed Zhou s The proposed Zhou s approach approach approach approach approach Approach PCA AWL AWL AWL AWL AWL AWL PCA HPF HPF HPF HPF 3 IHS IHS IHS IHS IHS IHS 4 HPF HPF PCA PCA PCA PCA Table. Ranks of dfferent fusng algorthms n terms of spatal qualty usng the proposed and Zhou s approaches Among the above four fuson methods, the HPF method s the most smlar to the AWL method. Both the proposed and Zhou s measures show that the HPF method s nferor to the AWL method. The reason s that the flter of the HPF has a fxed kernel sze and resoluton. It s dffcult to fnd one optmal flter for varous ground cover types of dfferent szes. 5 CONCLUSIONS In ths paper, we proposed a new assessment approach of the spatal qualty of a fused mage based on blur parameter estmaton. Ths measurement s based on the fact that the blur parameter, whch s a measure of the spread of the sensor s pont spread functon (PSF), characterzes the spatal resoluton of the sensor mage. The assessment results usng Zhou s evaluaton method may depend on the specfc hgh-pass flter. Ths s because the spatal qualty measure proposed by Zhou s based on the correlaton coeffcents between the hgh-pass fltered merged mages and the hgh-pass fltered panchromatc mage. Compared wth Zhou s spatal qualty measure, the blur parameter s an objectve qualty measure based on a basc optcal mechansm and thus can be used to evaluate the effectveness of varous processng or fuson schemes n terms of spatal qualty of the fused mage. It s also shown that the orgnal mult-spectral (e.g. TM mult-spectral) and hgh-resoluton panchromatc (e.g. SPOT PAN) mages can be also ncluded for a comparson. The proposed approach s based upon smple concepts, easy to understand and easy to mplement and use. ACKNOWLEDGMENTS Ths work was supported by the research grant NNSFC 49700/ from the Natonal Nature Scence Foundaton of Chna. The author would lke to thanks Prof. Yunlong Sheng, Physcs Department, Laval Unversty, Canada, for valuable comments and suggestons. REFERENCES Blanc, P., Blu, T., Ranchn, T., Wald, L., and Alos, R., 998. Usng terated flter banks wth the ARSIS concept for producng 0m Landsat multspectral mages, Internatonal journal of Remote Sensng, Vol. 9, No., pp Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B. Amsterdam

8 Jun L Carper, W.J., Lllesand, T.M., and Kefer, R.W., 990. The use of Intensty-Hue-Saturaton transformaton for mergng SPOT panchromatc and mult-spectral mages data, Photogrammetrc Engneerng and Remote Sensng, Vol.56, No. 4, pp Chavez P. S., Sdes S. C. and Anderson J. A., 99. Comparson of three dfferent methods to merge mult-resoluton and mult-spectral data: TM & SPOT Panchromatc, Photogrammetrc Engneerng and Remote Sensng, Vol. 57, No. 3, pp Chavez, P.S., and Bowell, J., 988. Comparson of the spectral nformaton content of Landsat Mapper and SPOT for three dfferent stes n the Phoenx, Arzona regon, Photogrammetrc Engneerng and Remote Sensng, Vol.54, No., pp Flusser, J., and Suk, T., 994. A Moment-based approach to regstraton of mages wth affne geometrc dstorton, IEEE Transactons on Geoscence and Remote Sensng, Vol. 3, No., pp Fung T. And LeDrew E., 987. Applcaton of prncpal component analyss to change detecton, Photogrammetrc Engneerng and Remote Sensng, Vol.53, No., pp Yocky, D. A., 995. Image mergng and data fuson usng the dscrete two-dmensonal wavelet transform, Journal of the Optcal Socety of Amerca, Part A., Vol., No. 9, pp L, J., Zhou Y.Q., and L, D. R., 999. PCA and wavelet transform for fusng panchromatc and mult-spectral mages, SPIE s Internatonal Symposum on AeroSense: Image Explotaton and Target Recognton; Orlando, Florda, U.S.A, Vol.379, pp , 5-9 Aprl 999. L, H., B.S. Manjunath, and S.K. Mtra, 995. Mult-sensor mage fuson usng the wavelet transform, Graphcal Models and Image Processng, Vol.57, No.3, pp Munechka, C.K., Warnck, J.S., Salvaggo, C., and Schott, J.R., 993. Resoluton enhancement of multspectral mage data to mprove classfcaton accuracy, Photogrammetrc Engneerng and Remote Sensng, Vol. 59, No., pp Nunez, J., Otazu, X., and Fors, O., Prades A., Pala, V., and Arbol, R., 999. Multresoluton-based Image fuson wth addtve wavelet decomposton, IEEE transactons on Geoscence and Remote Sensng, Vol. 37, No. 3, pp Ranchn, T., Wald, L., Mangoln, M., 996. The ARSIS method: a general soluton for mprovng spatal resoluton of mages by means o sensor fuson, n Proceedngs of the nternatonal Conference on Fuson of Earth Data, Ranchn, T. and Wald, L. eds., pp , Cannes, France, France, 996. Surya, G., 994. Three dmensonal scene recovery from mage defocus, Ph.D thess, Department of Electrcal Engneerng, State Unversty of New York, Stony Brook, NY, Dec Wald, L., Ranchn, T. and Mangoln, M., 997. Fuson of satellte mages of dfferent spatal and spectral resolutons: assessng the qualty of resultng mages. Photogrammetrc Engneerng & Remote Sensng, Vol.63, No.6, pp Zhou, J., Cvco, D.L. and Slander, J.A., 998. A wavelet transform method to merge Landsat TM and SPOT panchromatc data, Vol.9, No.4, pp Zhang, Y., 999. A new mergng method and ts spectral and spatal effects, Internatonal Journal of Remote Sensng, Vol. 0, No. 0, pp Internatonal Archves of Photogrammetry and Remote Sensng. Vol. XXXIII, Part B. Amsterdam 000.