Contents Image Fusion in Remote Sensing Optical imagery in remote sensing Image fusion in remote sensing New development on image fusion Linhai Jing Applications Feb. 17, 2011 2 1. Optical imagery in remote sensing Landsat 1 (1972) What is Remote Sensing? Collection of information about an object using a device (sensor) that is not in contact with the object use electromagnetic energy Wave Picture Frequency Wave # Wavelength (μm) Electromagnetic Spectrum Radiation Energy Levels
PAN 波段 B G R NIR bands 1(B) 2(G) 4(R)and 7(NIR) IKONOS multispectral image (MS) False color composite Bands 1 (B) 2(G) 7(R) 7 8 Forest map of Mexico Resource survey 9 10 Oil spill in Mexico Bay, 2010 11 Events 12
High Spatial resolution Platforms Low PAN B G R NIR Multispectral (MS) and panchromatic (PAN)bands in IKONOS imagery 15 16 Landsat ETM+ -30 m MS -15 m PAN SPOT -20 m MS -10 m PAN IKONOS 1 & 2-4 m MS - 1 m PAN 17 MS image (4m) PAN image (1m)
2. Image fusion in remote sensing What is image fusion? Image fusion is to fuse images of different spatial, spectral and temporal resolutions. PBS fused image B1(B), 2(G) and 3(R) Purpose: Better and accurate visualization. 21 0 0.5 1 km 22 Stage I. in 80 Purpose: Better visualization. Methods Arithmetic operations: +, -, *, /. Image transformation and substitution PCS, IHS, Brovey, Main problem: IHS Wavelet 23 Serious spectral distortion in fused images. 24
PAN band + Fused MS band i PAN image MS band i MS band i A low-resolution PAN image Fused MS band i From: PAN image 25 MS image 26 PCS fusion method PAN Band 1 PCA PC 1 Inverse PCA Band 1 Band 2 Band n PC 2 PC n Band 2 Band n Assumption: PC1 is similar to PAN band. 27 28 Covariance matrix for input channels: 1(R) 2(G) +- - - - - - - - - - - - - - -- - - Band 2 1(R) 2093.97 1(G) 1401.58 1179.11 Band 1 PC2 PC1 Eigenvectors of covariance matrix (arranged by columns): 0.80940 0.58725-0.58725 0.80940 Band 2 29 Band 1 30
IHS fusion method PAN PCS fused image 3(R), 2(G), 1(B) Band 2 IHS I Inverse IHS Band 2 Band 3 H Band 3 Band 4 S Band 4 0 0.5 km Assumption: Intensity (I) is similar to PAN. 31 32 IHS fused image 3(R), 2(G), 1(B) 33 Google Maps 34 Stage II. from 90 Purpose: Reduce spectral distortion in fused images Methods: Multi-resolution analysis techniques. Filtering (1980) Wavelet transform (1992) A trous algorithm (1999) Pyramids (1997) Reconstruction Decomposition Image decomposition and reconstruction
Challenges in image fusion PAN band MS band i Spectral distortion in fused images. Wavelet transform Inverse transform Wavelet-based image fusion 38 Causes for spectral distortion in fused images Haze in images 3. New development on image fusion in remote sensing Un-mixing of mixed pixels 40 The red/nir scatter-plots of a series of IKONOS MS images with different spatial resolutions. MS. PAN. Spatial resolution: (a) 2048 m. (b) 1024 m. (c) 512 m. (d) 256 m. (e) 128 m. (f) 64 m. (g) 32 m. (h) 16 m. (i) 8 m. (j) 4 m. 42
2.3. Un-mixing of sub-pixels in reality (a) a mixed pixel (b) sub-pixels (c) high-resolution pixels Mixed pixel Vegetation pixel Soil pixel 43 A prerequisite for the un-mixing of a mixed MS sub-pixel is the class 44 of the corresponding high-resolution PAN pixel. 3. Development of fusion methods Two improved ratio-based methods; One improved multi-resolution-based method; One method for misaligned MS and PAN data Two methods for mixed MS pixels; two methods for thermal IR images. 45 46 HR fusion method Fused MS band i PAN image 1. Haze-and-Ratio-based fusion method (HR) MS band i Degraded PAN image 47 48
Figure 5.1. The 128 128 details of original and synthetic images. Table 5.2. Standard deviations of error of the fused images. (a) 4-m MS. (b) 16-m MS. (c) 4-m PAN. Fused Band Blue Green Red NIR HR 14.4 21.2 26.9 36.7 HR_no_H 32.8 29.2 27.1 38.1 PANSHARP 15.7 24.5 30.9 42.1 GS 18.6 28.4 35.7 48.2 Exp 21.3 36.2 45.6 62.0 (d) HR. (e) PANSHARP. (f) Gram-Schmidt (GS). 50 CHR fusion method Fused MS band i PAN image 2. Component-specific haze-and-ratio-based fusion method (CHR) 51 MS band i A degraded PAN image derived from a MS 52 component Figure 5.1. The 128 128 details of original and synthetic images. 3. A fusion method based on Phenology of Pixels and Haze (PPH) (a) 4-m MS. (b) 16-m MS. (c) 4-m PAN. (d) Modified SVR. (e) SVR. (f) Modified Brovey. (g) Brovey. 53 54
Figure 7.3. The 128 128 original and synthetic images. (a) 4-m MS. ( b) 1-m PAN. (c) PPH. (d) PPH ignoring haze 4. A method to Reduce Mis-registration Impact (RMI) (e) CBD. ( f) RWM. ( g) AWP. ( h) AWL. (j) AWLP. ( k) PANSHARP. (l) SDM. 55 56 Figure 6.3. The 128 128 original and synthetic images. (a) 4-m MS. (b) 1-m PAN. (c) RMI. (d) HR. 5. A method based on Shadow-oriented Classification of Panchromatic pixels (SCP) (e) PANSHARP. (f) SDM. (g) GS. 57 58 SCP fusion method SCP fusion method PAN (a) 16-m MS. (b) Initial classification. (c) 4-m PAN. (d) Refined classification. PAN Vegetatio n Shado w Building Shadow Tree Soil (e) True 4-m MS 59 60
Figure 8.9. The 23 23 original and synthetic images. Figure 8.10. The 128 128 original and synthetic images. (a) True 4-m MS. (b) 16-m MS. (c) 4-m PAN. (e) RMI. (d) SCP. (a) 4-m MS. (b) 4-m PAN. (c) SCP. (d) RMI. ( e) CBD. (f) CBD. (g) RWM. (h) udwpc. (i) PANSHARP. (j) GS. (f) RWM. (g) udwpc. (h) PANSHARP. (i) GS. 61 62 Segementation 6. A method based on Object-oriented Classification of Panchromatic image (OCP) (a). 4-m PAN image. (b). 32-m MS image. 63 (c). Segmentation result. 64 Figure 9.4. The 33 33 original and synthetic images. 7. A method based on MultiVariate analysis (MV) (a) True 4-m MS. (b) 4-m PAN. (c) 32-m MS. (d) OCP. 8. A method based on Non-linear transform and MultiVariate analysis (NMV) (e) HR. (f) PANSHARP. (g) GS. 65 66
A flowchart of fusion of a thermal-ir band and a MS band MV fusion method A MS band Approximation Spatial details A MS image A variable of n MS bands Approximation Spatial details + + Thermal-IR band Correlation (single MS band, thermal ) is low. 67 Multiple correlation coefficient of n MS bands and a thermal band is higher. Thermal-IR band 68 NMV fusion method Figure 10.1. The 100 100 original and synthetic images. A MS image Approximation Spatial details + (a) 120-m MS (b) 120-m TM6 (c) 480-m TM6 (d) MV A variable of n MS bands and n non-linearly transformed MS bands Thermal-IR band 69 (e) NMV (f) Price s (g) ARSIS (h) PBIM. 70 4. Applications 71 Classification
Haughton Crater - A/B SAR Crater morphology has survived since impact (23 my) Impact crater 74 Sahara Desert SIR-A Radar vs. LANDSAT Lithological map 76 Spectral Geology - Hyperspectral Remote Sensing!
Processed (MNF) Probe -1 Hyperspectral data - South Baffin Island Rock Type MNF Colour Weathering Colour Quartzite Blue White-grey Metasediment Greenish Buff grey Metagabbro Dark red (due to vegetation) Black Carbonate Red (due to vegetation) White Granitoid/gneiss Generally green Buff grey Glaciers 80 GPS Global Positioning system RS Remote Sensing GIS Geographic Information System Mining activities 81 82 Commercial Google Maps Military 84
Mars - Mariner 9 Venus - Radar (Magellan) Lava flows Fractured terrain 86