Restoration of Missing Data due to Clouds on Optical Satellite Imagery Using Neural

Transcription

1 Restoration of Missing Data due to Clouds on Optical Satellite Imagery Using Neural Sergii Skakun 1, Nataliia Kussul 1, Ruslan Basarab 2 1 Space Research Institute NAS and SSA Ukraine 2 National University of Life and Environmental s of Ukraine Sentinel-2 for Workshop May 22, 2014, ESA/ESRIN, Frascati, Italy

2 Content Objective of the study Methodology Reconstruction of missing data using self-organising Kohonen maps (SOM) Data used Landsat-8 Sich-2 Results Reconstruction of missing data in Landsat-8 and Sich-2 Large scale crop mapping using reconstructed imagery (Landsat-8) Discussion & conclusions

3 Objective of the study Clouds and shadows limiting factor in exploitation of optical satellite imagery cause missing data Existing approaches (filling missing data) inpainting-based (Lorenzi et al. 2011) multispectral-based using MODIS (Roy et al. 2008) multitemporal-based using SOM for MODIS time-series (Latif & Mercier 2010) Objectives to qualitatively assess the use of SOMs for restoring missing data on time-series of high and medium resolution satellite images to provide crop mapping using restored images Landsat-8 images over JECAM Ukraine

4 Methodology: SOM Self-organizing Kohonen maps (SOMs) type of artificial neural network unsupervised learning produces a 2D, discretized representation of the input space SOM architecture SOM training process Neuron winner Training sample Updating weights i( x) arg min l 1, L x w l wl ( n 1) wl ( n) ( n) hl, i( x) ( n)( x w j ( n)), l 1, L

5 Methodology: Restoration Performed for each spectral band separately Training phase: Training samples selected automatically: on a regular grid Only pixels with all valid (i.e. non-missing) values considered Landsat-8 timeseries (Band 4) Restoration of missing values Input Missing X 1 X 2 X 3 Nan X 5 Nan SOM: selection of neuron winner Missing components are taken from neuron winner w i1 w i2 w i3 w i4 w i5 w i6 X 1 X 2 X 3 Nan X 5 Nan w l1 w l2 w l3 w l4 w l5 w l6 Only valid components are considered for finding a neuron winner

6 Data used & experiment setup Landsat-8 Resolution: 30 m Pre-processing: DN -> TOA -> SR Simplified Model for Atmospheric Correction (SMAC) (Rahman & Dedieu 1994) Dates: 16 April; 2 and 18 May; 19 June 2013 Sich-2 Resolution: 8 m DN used Dates: 3 June; 4, 14 and 19 September 2013 Qualitative assessment 2 ROIs selected that were artificially assigned Nan values Metrics: RMSE and Relative RMSE (RRMSE) Landsat-8 (true colour) ROI1 Sich-2 (false colour) ROI2

7 Results (1) Average RMSE error of reconstructing missing values for ROI1 and ROI2 on Landsat-8 images. RMSE values are shown depending on the number of missing values in the time-series (M=1, 2, 3) and fraction of pixels taken for training. X-axis is shown in a logarithmic scale

8 Results (2) Average RRMSE error of reconstructing missing values for ROI1 and ROI2 on Landsat-8 images. RRMSE values are shown depending on the number of missing values in the time-series (M=1, 2, 3) and fraction of pixels taken for training. X-axis is shown in a logarithmic scale

9 Results (3) Average RMSE and RRMSE error of reconstructing missing values for ROI1 and ROI2 on Sich-2 images. RMSE and RRMSE values are shown depending on the number of missing values in the time-series (M=1, 2, 3) and fraction of pixels taken for training. X-axis is shown in a logarithmic scale

10 Results (4) Restoration of ROI1 for Landsat

11 Crop mapping (1) Landsat-8 time-series: Dates (6 images): 16 April, 02 May, 18 May, 19 June, 05 July, 06 August 2013 Path/row: 181/24, 181/25, 181/26 Pre-processing: TOA->DN->SR SR: Simplified Model for Atmospheric Correction (SMAC) (Rahman & Dedieu 1994) Aerosol optical depth: Aeronet station (Kyiv) Clouds & shadows detection Fmask (Zhu & Woodcock 2012) Filling missing values SOMs TOA (top) and SR (bottom) for Landsat-8 acquired on 08 August A true color composition of Landsat-8 bands TOA and SR reflectance are scaled from 0 to

12 Crop mapping (2) Example of restoration for Landsat-8 image acquired on 05 July 2013 (true color composite)

13 Crop mapping (3) Ground observations Along the roads ~ 390 polygons Classes: No. LUCAS Description 1 Axx Artificial 2 B11 Winter wheat (and barley) 3 B32 Winter rapeseed 4 B12, B14 Spring crops (wheat, barley) 5 B16 Maize 6 B22 Sugar beet 7 B31 Sunflower 8 B33 Soybeans 9 B19, B39, B40 Other cereals, other annual crops, temporary grass 10 Cxx, B60 Forest, fruit trees 11 Exx Grassland 12 Fxx Bare land Location of along the roads surveys within Kyiv oblasts Distribution of training (50%) and test (50%) data Gxx Water Test 13 Train

14 Crop mapping (4) Methodology Inputs: Restored SR values (bands 2-7) for 6 multitemporal images Total inputs: 36 Ensemble of 6 neural nets: 30, 40, 50, 60, 70 and 80 hidden units Ensemble: Max average a- posteriori probability PA, % UA, % 1 Artificial Winter wheat Winter rapeseed Spring crops Maize Sugar beet Sunflower Soybeans Other cereals Best single net Ensemble OA, % Kappa Forest Grassland Bare land Water

15 Crop mapping (5) Landsat-8 image of 05 July 2013 (true color composite) Restored Landsat-8 image from 05 July 2013 using SOMs Crop map

16 Conclusions SOMs for high & med resolution satellite images encodes samples with non-missing values during a training phase, and then reconstructs missing values from SOM weights Accuracy of reconstruction Landsat-8: most accurate for NIR bands (11-15%) comparing to visible bands (16-19%) Sich-2: relative error green (4.3%), red (5.8%), and NIR (8%) Efficient for large scale crop mapping

17 Thank you!