MULTI-SENSOR DATA FUSION OF VNIR AND TIR SATELLITE IMAGERY

Transcription

1 MULTI-SENSOR DATA FUSION OF VNIR AND TIR SATELLITE IMAGERY Nam-Ki Jeong 1, Hyung-Sup Jung 1, Sung-Hwan Park 1 and Kwan-Young Oh 1,2 1 University of Seoul, 163 Seoulsiripdaero, Dongdaemun-gu, Seoul, Republic of Korea, mywhatwhat1@uos.ac.kr, hsjung@uos.ac.kr, psh5759@uos.ac.kr 2 Korea Environment Institute, 370 Sicheong-daero, Sejong, Republic of Korea, ohky@kei.re.kr KEY WORDS: Data Fusion, Visible and Near Infrared(VNIR), Thermal Infrared(TIR), Entropy ABSTRACT: In this paper, we study the data fusion of visible and near infrared (VNIR) and thermal infrared (TIR) imagery. VNIR image gives high spatial resolution and clear edge expression information while TIR image gives Earth s surface temperature information. Frequently, it is not easy to utilize the TIR images in many remote sensing applications, because the TIR images does not have a good spatial resolution enough to describe spatial details. Thus, in order to improve the TIR image s availability and utilization, a multi-sensor data fusion of TIR and VNIR is quite necessary. In this paper, we have proposed an efficient way to fuse the VNIR and TIR images of and ASTER. The panchromatic images is generated by using principal component analysis (PCA) from multi-spectral images, and the fused image is created by merging the panchromatic and thermal images. And finally, the performance test of the proposed fusion method is performed by using entropy concept. The method can be applied to detect hot-spot area such as forest fire and investigate a nuclear test and a surface temperature change in urban area. 1. INTRODUCTION Multi-sensor data fusion is proposed as the process to solve the limitation of acquiring information from singlesensor alone. Via these fusion techniques, many applications and algorithms have been proposed and developed. For example, the fusion of hyperspectral image and digital color image to obtain better information (Liao et al., 2015), land classification of hyperdimensional data (Jimenez et al., 1999), forest type mapping (Kempeneers et al., 2011), radar tracking system (Sun and Deng, 2004), vehicle localization (Lazarus et al., 2007) and so on. One of these application parts, the thermal information of Earth s surface is also an essential topic. From the remote sensed satellite imagery, especially the thermal infrared (TIR) image, we can extract the Earth s surface temperature information with low spatial resolution. For this reason, there are some limitation on using TIR images as source data. Otherwise, the panchromatic and visible and near infrared (VNIR) image gives high spatial resolution and clear edge expression information. However, the panchromatic image does not include the color information or land cover data. VNIR images cannot give the thermal information. To solve these limitation and improve the utilization of each single-sensor images, the multi-sensor data fusion is quite necessary. In this study, we have proposed an efficient way to fuse the VNIR and TIR images. The panchromatic image is used to enhance the spatial details in general. But if the satellite only provides VNIR and TIR images, we have to use substitute image of panchromatic image. Therefore, we have applied principal component analysis to generate the substitute image. Then, we have fused the substitute image and TIR image using optimal fusion scaling factor. Finally, we have represented the performance of the fusion result quantitatively using entropy. 2. METHOD Generally, the image fusion with panchromatic (PAN) and TIR images is performed to improve the utilization of the TIR images. Because the PAN image has high spatial resolution enough to describe spatial details while TIR image shows Earth s surface temperature information. The fusion image is created by merging the high-passed PAN image and modified TIR image. Fusion equation is defined by: II FFFFFF TTTTTT (ii, jj) = ααㆍii PPPPPP HHHH (ii, jj) + II TTTTTT(ii, jj), (1) where II TTTTTT FFFFFF (ii, jj) is the fused image brightness at line ii, pixel jj, αα is a fusion scaling factor, II PPPPPP HHHH (ii, jj) is modified high-passed PAN image brightness, and II TTTTTT(ii, jj) is modified TIR images brightness, respectively. Block-averaging is performed to generate the low-passed PAN image. The high-passed PAN image can be formed by subtraction the

2 low-passed PAN image from original PAN image. The modified high-passed PAN image is defined by threshold value which made by confidence level using Z-score. The modified TIR image can be generated by histogram matching with low-passed PAN image (Jung and Park, 2014). The fusion scaling factor can be defined any real number. 2.1 Principal Component Analysis In this paper, we used and ASTER satellite imagery which don t contain PAN images. For this reason, we essentially generate the substitute image of PAN image. The substitute image of PAN image should have similar wavelength with PAN image. Generally, the PAN image s wavelength can almost cover VNIR images. Thus, we carried out the principal component analysis (PCA) of VNIR images. The PCA process uses an orthogonal transformation to convert the image into a set of values of linearly uncorrelated vectors (Chavez Jr. and Kwarteng, 1989). That is, we can generate the high variance image called principal component 1 (PC1) image from PCA, and this image can successfully replace the PAN image. In ASTER case, the representative image of TIR image also formed by PCA. 2.2 Fusion Scaling Factor α As previously stated, the fusion scaling factor can be defined any real number, i.e., it is important to define the optimal fusion scaling factor. Because it can control the spatial information and thermal information of fused image. We calculated the optimal fusion scaling factor as ratio of root-mean-squares (RMS) of modified high-passed PAN image and modified TIR image (Jung and Park, 2014). The equation is followed: α = NN ww MM ww ii=1+ww jj=1+ww{ss TTTTTT (ii,jj)}2 NN ww MM ww ss PPPPPP HHHH (ii,jj) 2 ii=1+ww jj=1+ww, (2) where NN and MM are number of lines and pixels of modified high-passed PAN image and modified TIR image, w is the half of moving-window size, and ss is the local standard deviations, respectively. 3. EXPERIMENT RESULTS AND DISCUSSION We had tested proposed method using and ASTER satellite imagery. Table 1 shows the tested image list, tested area, and image information. Satellite ASTER Table 1. Tested image list, area and information Tested Area VNIR resolution/size/ number of bands Seosan, Republic of Korea 30m / 1024ⅹ1024 (pixel) Band 1, 2, 3, 4, 5 Busan, Republic of Korea 30m / 1024ⅹ1024 (pixel) Band 1, 2, 3, 4, 5 Mt. Baekdu, North Korea 15m / 1800ⅹ1800 (pixel) Band 1, 2, 3 TIR resolution/size/ Number of bands 120m / 256ⅹ256 (pixel) Band 6 120m / 256ⅹ256 (pixel) Band 6 90m / 300ⅹ300 (pixel) Band 10, 11, 12, 13, 14 For the fusion of VNIR and TIR imagery, we had carried out following steps: (i) generate the PC1 image of VNIR image by PCA, (ii) block-averaging the PC1 image to generate the low-passed PC1 image, (iii) generate the highpassed PC1 image by subtract low-passed PC1 image from original PC1 image, (iv) generate the modified highpassed PC1 image using Z-score, (v) generate the modified TIR image by histogram matching with low-passed PC1 image, (vi) calculate the optimal fusion scaling factor α, (vii) fuse the modified high-passed PC1 image and modified TIR image using calculated optimal fusion scaling factor. Figure 1, 2 and 3 represents partial image, original TIR image and original PC1 of each fusion results, respectively.

3 Figure 1. (a) Fused image, (b) Original TIR image and (c) Original VNIR PC1 image of (Seosan). Figure 2. (a) Fused image, (b) Original TIR image and (c) Original VNIR PC1 image of (Busan). Figure 3. (a) Fused image, (b) Original TIR PC1 image and (c) Original VNIR PC1 image of ASTER. In Figure 1(a) and 1(b) shows the similarity of fused image and original TIR image. Furthermore, it is shown that improvement of spatial details. That is, we can confirm that fused image has both spatial enhancement and thermal information. In this case, the optimal fusion scaling factor was calculated as Figure 2 shows improvement of spatial and thermal information more clearly. As shown in the images, we can surely distinguish between the shoreline and the ocean. And also, the red circle area s brightness value of fused image is high because a nuclear power plant has been located. The case of ASTER, Figure 3, represents successful fusion result of VNIR and TIR imagery as well. Figure 2 and 3 has and optimal fusion scaling factor, respectively. Finally, the performance test of the proposed fusion method is performed by using entropy. Entropy is usually used to measure the quantity of contained information of an image (Bai et al., 2011). A large value of the entropy means the fusion result has abundant information which indicates fine performance (Roberts et al., 2008). Table 2 represents the entropy of each images.

4 Satellite (Area) (Seosan) (Busan) ASTER (Mt. Baekdu) fused image Table 2. each images TIR image VNIR image Ratio (Fused/VNIR) α As shown in the Table 2, the ratio of entropy of fused image and VNIR image are directly proportional to optimal fusion scaling factor α. That is, fused image has both spatial details of VNIR image and temperature information of TIR image. This result suggests a good performance of proposed method. 4. CONCLUSION The multi-sensor data fusion is an useful technique to improve the utilization of remote sensed imagery. In this study, we have proposed the efficient multi-sensor data fusion method of VNIR and TIR imagery for the lack of PAN image satellite such as and ASTER. The method designed to not only solve the substitution of PAN image but control the trade-off between the spatial information and thermal information by optimal fusion scaling factor. Three images have been tested and demonstrated fine performance by calculating the entropy. The proposed method can be applied to detect hot-spot area such as forest fire and investigate a nuclear test and a surface temperature change in urban area. Acknowledgements This study was supported in part by the Space Core Technology Development Program through the National Research Foundation of Korea funded by the Ministry of Science, ICT and Future Planning under Grant NRF- 2014M1A3A3A References Bai, X., Zhou, F. and Xue, B., Fusion of infrared and visual images through region extraction by using multiscale center-surround top-hat transform. Optics Express, 19(9), pp Chavez Jr., P. and Kwarteng, A., Extracting Spectral Contrast in Landsat Thematic Mapper Image Data Using Selective Principal Component Analysis. Photogrammetric Engineering and Remote Sensing, 55(3), pp Jimenez, L. O., Morales-Morell, A. and Creus, A., Classification of Hyperdimensional Data Based on Feature and Decision Fusion Approaches Using Projection Pursuit, Majority Voting, and Neural Networks. IEEE Transactions on Geoscience and Remote Sensing, 37(3), pp Jung, H.-S. and Park, S.-W., Multi-Sensor Fusion of Landsat 8 Thermal Infrared (TIR) and Panchromatic (PAN) Images. Sensors, 14, pp Kempeneers, P., Sedano, F., Seebach, L., Strobl, P. and San-Miguel-Ayanz, J., Data Fusion of Different Spatial Resolution Remote Sensing Images Applied to Forest-Type Mapping. IEEE Transactions on Geoscience and Remote Sensing, 49(12), pp Lazarus, S. B., Ashokaraj, I., Tsourdos, A., Zbikowski, R., Silson, P. M. G., Aouf, N. and White, B. A., Vehicle Localization Using Sensors Data Fusion Via Integration of Covariance Intersection and Interval Analysis. IEEE Sensors Journal, 7(9), pp Liao, W., Huang, X., Coillie, F. V., Gautama, S., Pizurica, A., Philips, W., Liu, H., Zhu, T., Shimoni, M., Moser G.

5 and Tuia, D., Processing of Multiresolution Thermal Hyperspectral and Digital Color Data: Outcom of the 2014 IEEE GRSS Data Fusion Contest. IEEE Journal of Selected Topic in Applied Earth Observations and Remote Sensing, 8(6), pp Roberts, W., Aardt, J. and Ahmed, F., Assessment of image fusion procedures using entropy, image quality, and multispectral classification. Journal of Applied Remote Sensing, 2(1), Sun, S.-L. and Deng, Z.-L., Multi-Sensor Optimal Information Fusion Kalman Filter. Automatica, 40(1), pp