Multi-sensor Remote Sensing Image Fusion Based On Retina-Inspired Model

Transcription

1 009 IEEE Sympoium on Indutrial Eletroni and Appliation (ISIEA 009), Otober 4-6, 009, Kuala Lumpur, Malayia Multi-enor Remote Sening Image Fuion Baed On Retina-Inpired Model Haan Ghaemian Shool of Eletrial and Computer Engineering arbiat Modare Univerity ehran, Iran Abtrat Remote ening ytem oberve pixel in different portion of eletromagneti petrum. hee ytem are deigned within many ompeting ontraint, among the mot important being the trade off between the patial reolution and the petral reolution. o ollet more photon and maintain image SNR, the multipetral enor have a larger pixel ompared to panhromati enor. With appropriate algorithm it i poible to ombine thee data and produe imagery with the bet harateriti of both, namely high patial and high petral reolution. hi proe i nown a a ind of data fuion. Some widely performed in the remote ening ommunity are HSI (hue-aturation and intenit tehnique, PCA (prinipal omponent analye) tehnique, and the Brovey tranform tehnique. Reently, the Wavelet tranform ha been ued for merging multi-reolution image. Normally, the objetive of thee proedure i to reate a ompoite image of enhaned interpretability, but, thoe method an ditort the petral harateriti of the multipetral image. hi paper preent a multi-reolution data fuion heme, baed on viual hannel image deompoition. hi paper introdue a general iue of Retina-Inpired image analyi model, and appliation of the model in multipetral image fuion. A qualitative and quantitative omparion ued to evaluate the petral and patial feature performane of the propoed method with the other. Viual and tatitial analye how that the propoed algorithm ignifiantly improve the fuion quality; ompared to fuion method inluding, IHS, PCA, Brovey, and direte Wavelet tranform (DW). In thi method, there i no need to reample image, whih i an advantage over the other method, it an perform in any apet ratio between the panhromati and MSS pixel. Keyword Data Fuion; Multi-enor; Remote Sening; Retina-Inpired Model I. INRODUCION Earth obervation atellite provide data in different portion of the eletromagneti petrum at different patial and petral reolution. For the full exploitation of inreaingly ophitiated multi-oure data, advaned data fuion tehnique are being developed [1-3]. he fued image may provide inreaed interpretation apabilitie and more reliable reult. Many fuion method have been propoed for fuing high petral and patial reolution data in order to produe multi-petral image having the highet patial reolution available within the data et. he higher reolution image i ued a a referene whih the lower reolution image i geometrially regitered. herefore the lower reolution image i up ampled to math the ground ample interval of the higher reolution image. he propoed method in thi paper i a feature level image fuion tehnique. he objetive of the multi-reolution image fuion i to generate hybrid high patial reolution multi-petral image that attempt to preerve the radiometri harateriti of the original low patial reolution multipetral data. In thi tudy, low-reolution multi-petral MSS LandSat image fue with a high-reolution panhromati SPO image to ahieve optimal reolution in the patial and petral domain. Several method have been ued for thi job, uh a the IHS, PCA and DW et. [4-10]. he above method and everal other tehnique have been developed to merge high-reolution panhromati data with low-reolution Multipetral data. Normally, the objetive of thee proedure i to reate a ompoite image of enhaned interpretability, but, thoe method an ditort the petral harateriti of the multipetral image, and the analyi beome diffiult. he biologial retina i more than a imple video enor. It not only onvert optial information into eletrial ignal but perform oniderable proeing on the viual ignal before tranmitting it to a higher viual ytem level. Image fuion an inorporate the proeing priniple of human viion ytem [11]. hi paper preent a multi-reolution data fuion heme, baed on retinal viual hannel deompoition, motivated by analytial reult obtained from "retina baed image analyi, or multiale image deompoition inorporate the viual hannel phenomena" [1]. he energy paing the petral feature are ditributed in the lower frequeny ubband, and the patial feature, edge, are ditributed in the higher frequeny ub-band. By adding the high-ale patial feature (extrated from a panhromati image) to the low-ale patial feature (from M image), the viual-hannel proedure enhane the multipetral image. he retina model i baed on Differene-Of-Gauian (DOG) operator, whih deribe ome of the reeptive field propertie of the ganglion ell [1] /09/$ IEEE 500

2 009 IEEE Sympoium on Indutrial Eletroni and Appliation (ISIEA 009), Otober 4-6, 009, Kuala Lumpur, Malayia Figure 1. A thin piee of retina i enlarged in a photomirograph revealing it layer [11]. II. REINA MODEL he retina i a thin layer of neural tiue in the ba of the eye. It an be deompoed in five layer: three layer of ell bodie and two layer of ynapti interonnetion between the neuron. hi trutural form i depited in Figure 1. Light enter from the ganglion ell layer ide firt, and mut penetrate all ell type before reahing the rod and one. hi i beaue the pigment-bearing membrane of the photoreeptor have to be in ontat with the eye' pigment epithelial layer [11]. he photoreeptor' ell bodie are loated in the outer nulear layer of the retina. he ynapti terminal of the photoreeptor mae ontat with the dendriti field of the bipolar ell and horizontal ell in the outer plexiform layer (OPL). he ell bodie of the bipolar and horizontal ell are loated in the inner nulear layer. he horizontal ell mae onnetion with the ell in the outer nulear layer. he bipolar ell, however, mae onnetion onto the dendrite of the ganglion ell within the inner plexiform layer (IPL). Sine only the bipolar ell lin the ignal in the outer and inner plexiform layer, all the viual ignal mut go through the bipolar ell. Another la of ell loated in the inner nulear layer i the amarine ell. hee ell have no identifiable axon, only dendrite. he dendriti field of the amarine and ganglion ell onnet in the inner plexiform layer. he retinal ganglion ell bodie are loated in the ganglion ell layer, and their dendriti field onnet with the axon terminal of the bipolar a well a with the dendriti field of the amarine ell. otally, OPL propertie are generated by the ynapti triad, whih i ompoed of three ind of interonneted ell [11]: he one ell ontitute a layer of the trandution and regularization proeing. he trandution onvert luminane into eletrohemial potential aimed at the downtream layer. he regularization onit in filtering input ignal with a light low-pa frequenie filtering. he one ell are defined by their hape (midget, diffue ), their type of repone (on, off) and their funtion (petral enibilitie: red, green, and blue olor). Moreover their behavior depend loally on the luminane intenity and ontrat; Figure. A multi-enor image fuion heme in the Retina-Inpired Model he horizontal ell ontitute a layer of trong regularization proeing. he output repone perform from the one ell output to elaborate a patial average of the image intenity; he bipolar ell mae the differene between the horizontal (luminane average) and the one output.so bipolar ell etimate the loal ontrat of the image intenity to inreae viual indie. A one ell, the bipolar ell are variou. We oberve bipolar ell are laified alo by their hape (midget, diffue ), their type of repone (bipolar on, off ), and their funtion (ontrat etimation: red/green, blue/green+red ); he bipolar axon tranmit the OPL output to the IPL area. he IPL Proeing i aumed by ganglion ell: he ganglion ell have a reeptive field organized a onentri irle. he ganglion ell are laified a bipolar ell by their hape, their type of repone (on, off, on+off), and their funtion (patial ontrat etimation and luminane etimation) [11]. Our image fuion arhiteture are motivated by the biologial omputational proee of the human retina. he three different one ell in the retina are enitive to the hort, medium, and long wavelength of the viible petrum. If the retina were imply to tranmit oppoiteontrat image diretly from the photoreeptor to the brain, the reulting viion would probably be oaregrained and blurry. Further proeing in the retina define preie edge to image and allow u to fou on fine detail. he honing of the image tart at the firt ynapti level in the retina, where horizontal ell reeive input from one. he biologial retina not only onvert optial information into eletrial ignal but perform 501

3 009 IEEE Sympoium on Indutrial Eletroni and Appliation (ISIEA 009), Otober 4-6, 009, Kuala Lumpur, Malayia oniderable proeing on the viual ignal before tranmitting it to higher viual ytem level. Image fuion an inorporate the proeing priniple of human viion ytem. hi paper preent a multireolution data fuion heme, baed on retinal viual hannel deompoition, motivated by analytial reult obtained from "retina baed image analyi, or multiale image deompoition inorporate the viual hannel phenomena": the energy paing the petral feature are ditributed in the lower frequeny ubband, and the patial feature, edge, are ditributed in the higher frequeny ubband [5]. By adding the high-ale patial feature (extrated from a PAN image) to the low-ale patial feature (from MSS image), the viual-hannel proedure enhane the multipetral image (ee Fig. ). he retina model i baed on Differene-Of-Gauian operator [1], whih deribe ome of the reeptive field propertie of the ganglion ell. It onit of two Gauian with different variane at poition (x, and an generally be written a: CS x, = α G( r; ) α G( r; ). (1) ( 1 r G( r; ) = exp. () π where r = x + y, α and α are weighting of entre and urround input and G and G are normalized filter (meaning a patial integral of one) that repreent the filtering of the viual equene taing plae repetively in light reeptor (, or enter ignal) and G G in horizontal ell (, or urround ignal). Both filter are patially low-pa. Filter G i more low-pa than G, meaning S >. It orrepond to biologial fat that horizontal ell develop their ignal with more ynape and more ellular integration than reeptor, and are lined to their neighboring horizontal ell through trongly oupling gap juntion. In ummary the output from thee photoreeptor are oppoitely enhaned within band by enter-urround patial opponent proee at the bipolar ell. In later tage (ganglion ell in retina) thee ignal are oppoitely enhaned by enter-urround proee between the different band. Ultimately, we an repreent the funtion of thi model by: f x, = h ( x, f ( x, + h ( x, f ( x, ). (3) ( 1 1 y where f 1 (x, i the high reolution image, h 1 (x, i the high pa filter (photoreeptor ell), f (x, i the low reolution image and h (x, i the low pa filter (horizontal ell). hi allow to generate a patially enhaning multi-petral image f(x,, by adding the high reolution patial feature to f (x,. Figure 3. SPO image and LANDSA image of North-Wet of ehran III. EXPERIMENS AND RESULS In thi experiment, the Landat M image are patially enhaned to 10-m reolution by fuing them with the SPO panhromati data. he tet area i 10-m by 10- m (Figure 3) loated in the north-wet of ehran, Iran, whih inluded variou, international airport, natural par, mall lae, vegetation, agriultural, mountain, bare oil, highway, et. he M image were regitered geometrially onto SPO PAN a a referene image, by eleting 0 ontrol point. For all merging method, exept our propoed method, the M image reample to 10-m reolution, by uing firt order polynomial, and nearet neighbor interpolation algorithm. he purpoe of multiple image fuion i to ahieve omplementary patial and petral benefit from different enor. he merged reult hould be evaluated both petrally and patially. he ombination of M band,3,4 wa eleted beaue thee band mot 50

4 009 IEEE Sympoium on Indutrial Eletroni and Appliation (ISIEA 009), Otober 4-6, 009, Kuala Lumpur, Malayia loely overed the ame portion of the eletromagneti petrum a the PAN image. he PCA, wavelet and the propoed method an merge all multi-petral band with the PAN image at one. Viual evaluation of the 43-band olor ompoite image indiate that the IHS, PCA and Brovey Method hange olor of the ompoite image, whih mean the petral feature, are ditorted by thee method. Due to limitation of pae, Fig.4 how ome part of the original image of M band in RGB olor ompoite. hi part ha hown in PAN-SPO image in figure 3. Fig.5a-e how a 43-band olor ompoite image of the enhaned M data by the propoed method. Some method for evaluating the petral quality are baed on the alulation of the image differene between the merged image and tandard pattern image, whih i the deired merge reult. However uh a tandard remote ening image i uually not available. For remote ening appliation the method ued to merge data with high patial and high petral reolution propertie hould not ditort the petral harateriti of the original high petral reolution data. Not ditorting the petral harateriti i important for alibration purpoe and for inuring that target that are petrally eparable in the original data till eparable in merged data et. hu the petral quality of fued image an be meaured band by band by the average differene between the pixel value of the merged image and orreponding original (the regitered and down-ampled one) M, a hown in equation (4). d ( ) = 1 N N i = 1 f ( i ) b ( i ). (4) where f and b are the pixel value of merged and M image. It i deirable that the fued image ha a mall differene with the orreponding original M image. he data fuion hould not ditort the petral harateriti of the original multi-petral data. he petral quality of the patially enhaned image i meaured band by band in term of orrelation between the pixel value of the original image and patially enhaned one. ρ () = C(f,b ). f b. (5) ABLE1. AVERAGE DIFFERENCE BEWEEN ENHANCED FUSED IMAGE AND ORIGINAL M BANDS FUSION MEHOD IHS BROVEY PCA W1 W REINA M M M M M M Mav ABLE. CORRELAION BEWEEN ENHANCED FUSED IMAGE AND ORIGINAL M BANDS Fuion Method M1 M M3 M4 M5 M7 Mav IHS BROVEY PCA W1 W REINA In thi paper we ued the orrelation oeffiient between the high pa filtered fued SPO PAN and M image and the high pa filtered SPO PAN Image a an index of the patial quality. hi meaurement i baed on the fat that the patial information unique in pot pan i motly onentrated in the high frequeny domain. he higher orrelation between the high frequeny omponent of fued M and SPO PAN Image and the high frequeny omponent of SPO PAN indiate that more patial information from SPO PAN Image i inorporated during fuing. he filter we ued wa a Laplaian a illutrated in equation (6). he reult are preented in able HPF(ma) (6) = ρ () = C(PAN HPF,f ),HPF. (7). PAN f When fuing image derived from low patial high petral reolution and high patial low reolution, it i deirable that thee two type of data hare ome petral imilarity. Otherwie it i diffiult to ahieve optimally petral and patial reult imultaneouly. Equation (15) deribe hybrid evolution of thee riteria. he reult have hown in able 4.he bet reult for both riteria i obtained in Retina model. M =ρ (petral). ρ (patial). (8) S =ρ (petral) +ρ (patial) ABLE 3. CORRELAION BEWEEN HE HIGH FREQUENCY COMPONEN OF FUSED IMAGE AND HIGH FREQUENCY COMPONEN OF SPO PAN M1 M M3 M4 M5 M7 Mav IHS BROVEY PCA W1 W REINA

5 009 IEEE Sympoium on Indutrial Eletroni and Appliation (ISIEA 009), Otober 4-6, 009, Kuala Lumpur, Malayia M S Brovey IHS ABLE 4. HYBRID EVALUAION PCA W1 W Retina may aue variation in petral and patial quality in ome band. he propoed method ahieve the bet petral quality in all band. Comparing with IHS, PCA, Brovey tranform method; the bet petral and patial quality i only ahieved imultaneouly with the propoed data fuion. IV. CONCLUSIONS In thi paper, a retina baed fuion ha been preented to merge SPO PAN with high patial reolution and Landat M with high petral harateriti. A quantitative approah wa ued to ompare petrally and patially, the fued reult derived from PCA, IHS, Brovey, Wavelet tranform and Retina model. he following onluion may be explain from thi reearh. Multi ale image fuion i uually a trade-off between the petral information extrated from multi-petral image and the patial information extrated from high patial reolution image. he retina an ontrol thi trade-off. he propoed method ahieved the bet petral quality in all band omparing with IHS, PCA and Brovey and wavelet method. he bet petral and patial quality i only ahieved imultaneouly with the propoed data fuion. he wavelet tranform method ha advantage over IHS, PCA and Brovey tranform. Method in that the SPO PAN an be adjuted petrally to math eah M band before merging. IHS an ombine three M band with Spot PAN at one time and bring equal petral error to red, green, blue olor image he Brovey tranform i an eaily performed approah being able to ombine M, 3, 4 with SPO PAN. It onerve the leat petral fidelity of M image, ine emphai i plaed on the patial information from SPO PAN. It hould be mentioned that Brovey tranform on ome area lie water area ould not preerve the petral harateriti. IHS method ahieve the bet petral reult in urban and vegetation area. PCA method an deal with many band at one time, however in ome ae mixture of M band in the firt prinipal omponent ha negative impat on the merged reult and REFERENCES [1] D. L. Hall, J. Llina "An Introdution to Multienor Data Fuion," Pro. IEEE, Vol. 85, No. 1, pp. 6-3, [] C. Pohl and J. L. Van Genderen. "Multi-enor image fuion in remote ening: onept, method and appliation", International Journal of Remote Sening, Vol. 19, no. 5, pp , [3] Z. Wang, D. Ziou, C. Armenai, D. Li, and Qingquan Li, "A Comparative Analyi of Image Fuion Method, "IEEE ranation on Geoiene and remote ening, Vol. 3, No.6, pp , June 005. [4] M. Gonzalez, el. al, "Fuion of Multipetral and Panhromati Image uing Improved IHS and PCA merge Baed on Wavelet deompoition", IEEE tranation on Geoiene and Remote Sening, vol. 4, no. 6, pp , 004. [5] H. Ghaemian, "Multienor Image Fuion by Invere Subband Coding" Proeeding of ISPRS-000, CD#3, Commiion II, July 000. [6] V. S. Petrovi, C. S. Xydea, "Gradient-baed multireolution image fuion, "IEEE ranation on Image Proeing, Vol., pp. 8 37, 004. [7] A. Garzelli, F. Nenini, "Interband truture modeling for Panharpening of very high-reolution multipetral image," Information Fuion, Vol. 6, pp. 13-4, 005. [8] M. Choi, "A New Intenity-Hue-Saturation Fuion Approah to Image Fuion With a radeoff Parameter," IEEE ranation on Geoiene and Remote Sening, Vol. 44, No. 6, 006. [9] H. Ghaemian, "Multienor Image Fuion by Multiale Filter Ban", Proeeding of IEEE International Conferene on Image Proeing ICIP001, Otober 001. [10] J. J. Lewi, R. J. O Callaghan, S.G. Niolov, D.R. Bull, C.N. Canagarajah, "Region-baed image fuion uing omplex wavelet," Proeeding of the 7th International Conferene on Information Fuion, Stoholm, Sweden, pp , [11] H. Kolb, "he neural organization of the human retina," in: Heenlively, J.R, Arden, G.B (Ed.), Priniple and Pratie of Clinial Eletrophyiology of Viion, pp. 5 5, [1] S. Shah, and M. D. Levine, "Viual Information Proeing in Primate Cone Pathway, Part I: A Model", IEEE ranation on Sytem, Man, and Cyberneti, Part B, vol. 6, No Figure 4. original M image with 30-m patial reolution 504

6 009 IEEE Sympoium on Indutrial Eletroni and Appliation (ISIEA 009), Otober 4-6, 009, Kuala Lumpur, Malayia Figure 5a. he output reult of IHS tehnique Figure 5b. he output reult of Brovey tehnique Figure 5. he output reult of PCA tehnique Figure 5d. he output reult of Wavelet tehnique Figure 5e. he output reult of Retina tehnique 505