Remote Sensing Laboratory Dept. of Information Engineering and Computer Science University of Trento Via Sommarive, 14, I-38123 Povo, Trento, Italy CURRENT SCENARIO AND CHALLENGES IN THE ANALYSIS OF MULTITEMPORAL REMOTE SENSING IMAGES Lorenzo Bruzzone E-mail: lorenzo.bruzzone@ing.unitn.it Web page: http://rslab.disi.unitn.it
Outline 1 Background and current trends in multitemporal images 2 Change detection in VHR multispectral images 3 Change detection in VHR SAR images 4 Change detection in multisensor/multisource VHR images 5 Discussion and Conclusion Lorenzo Bruzzone 2
Remote Sensing Laboratory Dept. of Information Engineering and Computer Science University of Trento Via Sommarive, 14, I-38123 Povo, Trento, Italy 1. Background and Current Trends in Multitemporal Images
Trend in Multitemporal Images In the last ten years we had a significant increase in the interest on topics related to the time series and the analysis of multitemporal data: Sharp increase in the number of papers published on the major remote sensing journals (e.g., IEEE Transactions on Geoscience and Remote Sensing, IEEE Geoscience and Remote Sensing Letters, IEEE Journal on Selected Topics in Applied Earth Observation and Remote Sensing, Remote Sensing of Environment, International Journal of Remote Sensing). Increased number of related sessions in international conferences. Increased number of projects related to multitemporal images and data. Lorenzo Bruzzone 4
Trend in Multitemporal Images The increased interest in multitemporal data analysis is due to many issues: Increased number of satellites with increased revisitation time that allow the acquisition of either long time series or frequent bitemporal images. New policy for data distribution of archive data that makes it possible a retrospective analysis on large scale (e.g. the Landsat Thematic Mapper archive). New policies for the distribution of new satellites data (e.g. ESA Sentinel). Lorenzo Bruzzone 5
Remote Sensing Laboratory Dept. of Information Engineering and Computer Science University of Trento Via Sommarive, 14, I-38123 Povo, Trento, Italy 2. Change Detection in Very High Resolution Multispectral Images
After Change Before Change Where we were 10 years ago.. Landsat TM (burned area detection) Lorenzo Bruzzone 7
CD in Multitemporal MS Images: Typical Architecture Corrected t 1 image Comparison Operators: Difference Vector difference Ratio Log-ratio Optical Images Synthetic Aperture Radar (SAR) Images X 1 Comparison X D Analysis Ω ={ω c, ω u } X 2 Corrected t 2 image Difference/Ratio Image Change-detection map Analysis: Pixel-based thresholding Context-based approaches L. Bruzzone, D. Fernandez Prieto, Automatic analysis of the difference image for unsupervised change detection, IEEE Transactions on Geoscience and Remote Sensing, Vol. 38, No.3, pp. 1171-1182, 2000 Y. Bazi, L. Bruzzone, F. Melgani, «An unsupervised approach based on the generalized Gaussian model to automatic change detection in multitemporal SAR images,» IEEE Transactions on Geoscience and Remote Sensing, Vol. 43, No. 4, 874-887, 2005.. Lorenzo Bruzzone 8
CD in Multitemporal MS Images: Example Landsat TM, Pre-event Landsat TM, Post-event Magnitude Difference Image Pixel-Based Change Detection Map Burned area Lorenzo Bruzzone 9
spatial resolution (m) New Satellites with VHR MS Sensors 100 Landsat 1-3 Landsat 4-5 10 SPOT 1-4 Landsat 7 RapidEye SPOT 5 1 Eros A Ikonos QuickBird WorldView 1-2 GeoEye 1 Cartosat-2 Eros B Pleiades-HR 1-2 GeoEye 2 0,1 1970 1980 1990 2000 2010 2020 Lorenzo Bruzzone 10
CD in Multitemporal VHR MS images July 2006 October 2005 Quickbird images of the city of Trento (Italy) Lorenzo Bruzzone 11
CD in Multitemporal VHR MS images October 2005 July 2006 Quickbird images of the city of Trento (Italy) Lorenzo Bruzzone 12
CD in Multitemporal VHR Images: Example Quickbird, October 2004 (true color composition) Quickbird, July 2006 true color composition Magnitude Difference Image Pixel-Based Change Detection Map Lorenzo Bruzzone 13
CD in VHR MS Images Change detection in VHR Images should exploit a top-down approach to the definition of the processing architecture. This approach should: explicitly model the presence of different radiometric changes on the basis of the properties of the considered images extract the semantic meaning of changes; identify changes of interest with strategies designed on the basis of the specific application; exploit the intrinsic multiscale properties of the objects and the high spatial correlation between pixels in a neighborhood. L. Bruzzone, F. Bovolo, A Novel Framework for the Design of Change-Detection Systems for Very-High-Resolution Remote Sensing Images, Proceedings of the IEEE, Vol. 101, pp. 609-630, 2013.. Lorenzo Bruzzone 14
CD in VHR MS Images: Architecture Design Multitemporal data set Auxiliary information Identification of the tree of radiometric changes Detection of all radiometric changes Change Vector Analysis, Context-sensitive techniques, etc. Selection of the strategy for detecting changes of interest Direct extraction of changes of interest Detection of the changes of interest Differential extraction of changes of interest by cancellation Refined detection of the radiometric change of interest Change detection map Lorenzo Bruzzone 15
Identification of the Tree of Radiometric Changes Radiometric Changes(W rad ) Changes due to acquisition conditions (W Acq ) Changes occurred on the ground (W Grd ) Differences in atmospheric conditions (W Atm ) Type of sensor Sensor view angle Differences in acquisition system (W Sys ) Seasonal effects Sensor acquisition mode Natural disasters (W Dis ) Vegetation Phenology (W veg ) Anthropic activity (W Ant ) Environmental conditions (W Env ) Lorenzo Bruzzone 16
Detection of Changes of Interest Direct detection Differential detection by cancellation X 1 X 2 X 1 X 2 Detection of radiometric changes Detection of change of interest 1 Detection of change of interest K Non-relevant change 1 + - Non-relevant change 2 - - + + Non-relevant change N + + Refined detection of the radiometric change of interest Map of changes Map of changes Lorenzo Bruzzone 17
Multilevel Approach: Semantic of Changes Object Meta-level (o) j=1,,jo Classification map, object map, O 1 O 2 O Primitive Meta-level (p) j=1,,jp Geometric or statistic primitives P 1 P 2 P Meta-levels fusion Map of a specific Radiometric change Pixel Meta-level (px) j=1,,jpx Pixel radiometry X 1 X 2 D L. Bruzzone, F. Bovolo, A Novel Framework for the Design of Change-Detection Systems for Very-High-Resolution Remote Sensing Images, Proceedings of the IEEE, Vol. 101, pp. 609-630, 2013.. Lorenzo Bruzzone 18
Example: CD in VHR Optical Images Study area: South part of Trento (Italy). Multitemporal data set: portion (380 430 pixels) of two images acquired by the Quickbird satellite in October 2004 and July 2006. Causes of Change: changes on the ground, seasonal changes, registration noise. October 2004 July 2006 Reference Map Lorenzo Bruzzone 19
Example: CD Architecture Design Multitemporal data set Auxiliary information Identification of the tree of radiometric changes Detection of all radiometric changes Change Vector Analysis, Context-sensitive techniques, etc. Selection of the strategy for detecting changes of interest Direct extraction of changes of interest Detection of the changes of interest Differential extraction of changes of interest by cancellation Refined detection of the radiometric change of interest Change detection map Lorenzo Bruzzone 20
Identification of the Tree of Radiometric Changes W Rad W Sys W Grd w sh w rn W Veg W Ant Shadow changes w at w gl w b Registration noise Apple trees Grassland New buildings Lorenzo Bruzzone 21
Changes Tree and Detection Strategy Identification of the tree of radiometric changes Differential detection by cancellation X 1 X 2 W Rad Detection of radiometric Changes (CVA) w sh W Sys w rn W Grd Detection of w sh + - Detection of w rn + - Shadow changes Registration noise Refined detection of W Grd Map of changes Lorenzo Bruzzone 22
Example: CD Architecture Multiscale analysis for w rn detection Shadow detection Shadow index Comparison Shadow change index w sh detection + - - CVA W rad detection Change-detection map X 1 Magnitude of multispectral change vectors W={W nc, W Grd } X 2 Parcel detection Lorenzo Bruzzone 23
Example: Qualitative Results October 2005 July 2006 Reference Map Change Detection Map CVA Parcel Based Change detection Map Top-down Architecture Lorenzo Bruzzone 24
Example: Quantitative Results 95 90 Overall change detection accuracy (%) 93.91 91.56 90.86 85 80 CVA Pixel-based CVA parcel-based Top-down architecture Technique False Alarms Missed Alarms Total Errors Overall accuracy (%) CVA pixel-based 5005 9924 14929 90.86 CVA parcel-based 3537 10261 13798 91.56 Top-down architecture 1470 8480 9950 93.91 Lorenzo Bruzzone 25
Remote Sensing Laboratory Dept. of Information Engineering and Computer Science University of Trento Via Sommarive, 14, I-38123 Povo, Trento, Italy 3. Change Detection in Very High Resolution SAR Images
May 1999 Multitemporal SAR Images: 10 years ago.. September 1999 ERS SAR images of a flood in the Cat-Tien National Park, Vietnam Lorenzo Bruzzone 27
spatial resolution (m) New Satellites with VHR SAR Sensors 100 JERS-1 ASAR 10 ERS-2 PALSAR RADARSAT-2 RISAT TerraSAR-X TanDEM-X Cosmo-SkyMed 1-4 1 1990 1995 2000 2005 2010 2015 Lorenzo Bruzzone 28
April 2009 Multitemporal SAR Images: New challenges September 2009 Comso-Skymed SAR Images of the Earthquake of L Aquila, Italy COSMO-SkyMed Product ASI Agenzia Spaziale Italiana (2010). All Rights Reserved. Lorenzo Bruzzone 29
CD in VHR SAR images In multitemporal SAR VHR images we have many sources of backscattering changes. Often backscattering changes associated with different sources exhibit characteristics similar to each other. They can be separated only by explicitly modeling the EM behavior of complex objects. To this end it is necessary to bridge the semantic gap between low level features and semantic information: Modelling the interaction between the EM waves and the imaged objects; Extracting the different object components with proper detectors; Combining object components for identifying the objects and the possible changes in their state. Lorenzo Bruzzone 30
Example: Building Detection in VHR SAR Images Building EM model VHR satellite SAR image A. Ferro, D. Brunner, L. Bruzzone, G. Lemoine, On the Relationship Between Double Bounce and Aspect Angle of Buildings in VHR SAR Images, IEEE Geoscience and Remote Sensing Letters, Vol. 8, No.4, pp. 612-616, 2011 Lorenzo Bruzzone 31
Primitives and Semantic: Building Detection Despeckled image Detected lines Detected bright areas Detected shadow areas Detected building fooprints Lorenzo Bruzzone 32
Example: Building Detection in VHR SAR Images A. Ferro, D. Brunner, L. Bruzzone, Automatic Detection and Reconstruction of Building Radar Footprints from Single VHR SAR Images, IEEE Transactions on Geoscience and Remote Sensing, Vol. 51, pp. 935-952, 2013 Lorenzo Bruzzone 33
Change Detection in VHR SAR Images Moving from object detection in single images to object change detection in multitemporal images increases the complexity of the information extraction. In order to define an effective general approach to change detection for VHR SAR images we have to: Decompose the general complex problem in simpler hierarchical problems. Exploit the intrinsic multiscale nature of objects present in VHR images. Model the specific properties of expected changes for extracting the semantic meaning of backscattering changes. Exploit the available prior information on the considered scenario. Lorenzo Bruzzone 34
Example: Building Change Detection Objects level Geometric primitives Bright Areas Building Map Shadow Areas Double bounce area Building Map Building Change Meta-levels Detection Map fusion Building Change Detection Map Pixel Level Problem: Error propagation VHR SAR image t 1 VHR SAR image t 2 Lorenzo Bruzzone 35
Architecture for Change Detection in VHR SAR VHR SAR t 1 image N X LR Prior knowledge on the scene X 1 Comparison X LR Multiscale decomposition n X LR Scale-driven hierarchical change detection CD Map X 2 VHR SAR t 2 image Ratio Log-ratio 0 X LR Modelling of the expected changes F. Bovolo, C. Marin, L. Bruzzone, A Hierarchical approach to Change Detection in Very High Resolution SAR Images for Surveillance Applications, IEEE Transactions on Geoscience and Remote Sensing, Vol. 51,, pp. 2042-2054. 2013. F. Bovolo, C. Marin, L. Bruzzone, A Novel Approach to Building Change Detection in VHR SAR Images, Proceedings of the SPIE Conference on Image and Signal Processing for Remote Sensing, Edinburg, UK, September 2012. Lorenzo Bruzzone 36
Architecture for Building Change Detection Changes in VHR SAR images implies increase or decrease of backscattering values. Destroyed building Changes in buildings (i.e., new/destroyed buildings) implies simultaneous increase and decrease of backscattering. New building Search for pairs of increase/decrease backscattering pattern. Backscattering decrease Backscattering increase Lorenzo Bruzzone 37
Example: L Aquila Earthquake Multitemporal data set: section (1024 1024 pixels) of two spotlight (CSK ) images acquired before (5 th April 2009) and after (12 th September 2009) the earthquake of L Aquila (Italy, 6 th April 2009). 1m 1m resolution X-band 1-look Amplitude HH-polarization 57-58 degree incidence angle Ascending orbit Right look CSKS1 Calibrated Co-registered Geo-referred Optical image 5 th GeoEye, April 2009 Tele Atlas 2011 RGB 12 multitemporal th September composition 2009 Google (R:09/12/2009, G:04/05/2009, B:09/12/2009) Backscattering decrease Backscattering increase Unchanged areas COSMO-SkyMed Product ASI Agenzia Spaziale Italiana (2009). All Rights Reserved. Lorenzo Bruzzone 38
COSMO-SkyMed Product ASI Agenzia Spaziale Italiana (2009). All Rights Reserved. Example: L Aquila Earthquake Changes in backscattering Candidate building map Backscattering decrease Backscattering increase Destroyed Building Change (no destroyed building) Lorenzo Bruzzone 39
Optical image Pictometry International Corp Microsoft Corporation COSMO-SkyMed Product ASI Agenzia Spaziale Italiana (2009). All Rights Reserved. Example: L Aquila Earthquake Pre-Crisis Reference Image Post-Crisis Reference Image Changed building map Destroyed Building Change (no destroyed building) Lorenzo Bruzzone 40
Remote Sensing Laboratory Dept. of Information Engineering and Computer Science University of Trento Via Sommarive, 14, I-38123 Povo, Trento, Italy 4. Change Detection in Very High Resolution Multisensor Images
Quickbird image before earthquake New challenges: Data Fusion Earthquake of Sichuan province, China, May, 2008 COSMO-Skymed image after earthquake COSMO-SkyMed Product ASI Agenzia Spaziale Italiana (2010). All rights reserved. Lorenzo Bruzzone 42
Top-Down/Bottom-Up Approaches Classification maps, ancillary data, objects maps, cadastrial map Classification map, object map Change detection map at object level Prediction Object extraction Pixel Level Simulated VHR image t 1 VHR image t 2 change detection map Lorenzo Bruzzone 43
CD in Multisource Data: Example Bulding Map at t 1 D. Brunner, G. Lemoine, L. Bruzzone, Earthquake damage assessment of buildings using VHR optical and SAR imagery, IEEE Transactions on Geoscience and Remote Sensing, Vol. 48, No. 5, pp.2403-2420, 2010. t 2 SAR incidence angle Model of the building Simulation of SAR Building Footprint VHR SAR image t 2 Change Detection Map of Buildings Simulated Building footprint at t 1 Lorenzo Bruzzone 44
Conclusion Analysis and exploitation of time series and multitemporal images is a very important topic both from the methodological and the application perspective. Many methodological challenges are related to the properties of new satellite data that require the development of a new generation of processing techniques for the analysis of: VHR multispectral and SAR images. Hyperspectral images. Long time series (data mining). These properties open the possibility to develop also new applications that exploit either the very high geometrical (e.g. analysis of single buildings) or spectral (e.g. detection of subtle changes) resolution and the increased revisit time (e.g. monitoring and surveillance application). Lorenzo Bruzzone 45