European Space Agency (ESA) Landsat MSS/TM/ETM+/OLI Archive: 42 years of our history

Transcription

1 This image cannot currently be displayed. European Space Agency (ESA) Landsat MSS/TM/ETM+/OLI Archive: 42 years of our history Landsat MSS Dataset Improvements and Multi Temporal Analysis Sébastien Saunier, Amy Northrop, Samantha Lavender, Luca Galli, Riccardo Ferrara, Stefano Mica, Roberto Biasutti, Philippe Goryl, Ferran Gascon, Marco Meloni, Bas Altena MultiTemp 2017, Bruges, June 2017

2 Contents From 1975, ESA has used its multi-mission ground systems to acquire, process and archive data from the Third Party Missions (TPM). Landsat data continuity missions are part of ESA s TPM. 2

3 3

4 Landsat 1 - MSS Landsat 2 - MSS Landsat 3 - MSS Landsat 4 - MSS Landsat 5 - MSS Landsat 5 - TM Landsat 6 - ETM Landsat 7 - ETM+ Landat 8 - OLI/TIRS ESA Operation in operational life Operational life 4

5 Landsat Sensor Spectral Bands (µm) RBV (LS-1-3) 1) VNIR 2) VNIR 3) VNIR P) VNIR (available on LS-3) MSS (LS-1-5) 4) VNIR 5) VNIR 6) VNIR 7) VNIR 8) TIR (available on LS-3) TM (LS-4-5) 1) VNIR 2) VNIR 3) VNIR 4) VNIR 5) VNIR 6) TIR 7) SWIR ETM+ (LS-7) 1) VNIR 2) VNIR 3) VNIR 4) VNIR 5) VNIR 6) TIR 7) SWIR P) VNIR Swath Width 185 km 185 km 185 km 185 km Spatial resolution (Across Track / Along Track) 40 m (PAN) 80 m (VNIR) 59x79 m VNIR 240 m TIR 30 m VNIR/SWIR 120 m TIR 15 m PAN 30 m VNIR/SWIR 60 m TIR Radiometric resolution Analogic 6 bit 8 bit 9 bit ( 8 bit transmitted) Band to band registration 0.2 pixel (90 %) 0.2 pixel (90 %) Geodectic Accuracy 500 m (90 %) 400 m (90 %) (without Ground Control Point) Data Rate 15 Mbits/s 85 Mbits/s 2*75 Mbits/s Average Power 50 W 332 W 590 W Telescope Aperture 23 cm 40.6 cm 40.6 cm 5

6 Context The ESA collection of Landsat Historical data observed in Continental Europe, African Region and Arctic has become massive. Major efforts have been undertaken, to improve processing s/w and then produce the most up to date Level 1 products for the community. (As part of the International Ground Station (IGS), ESA is continuously maintaining efforts to acquire, process and distribute new real time Landsat 8 data 6

7 This archive was a black box 7

8 8

9 Status prior improvements / Solutions Item Status Solution Reference Format CEOS Format -Aligned to USGS: GeoTIFF and ancillary files within a zip archive GEO Accuracy Orbit and attitude data are mostly inaccurate preventing operational use of data. Image Geometry is distorted and Panoramic Effect observed -Re evaluate / Reestimate Orbit data -Improve GCP set, DEM, GCP Filtering -Improve Geometric Model (attitude model) -Use Multi Scene Refinement concept LS-DFCB ACS ATBD 9

10 Status prior improvements / Solutions 2/2 Item Status Solution Reference RADIO Accuracy Large variations in pre flight calibration Image Saturation Ground Model Lifetime Calibration Update of calibration Helder, Saunier Image Quality Image Anomalies Detect and Flag with BQA ESA s LANDSAT Product Definition Product generation A lot of products has never been generated Systematic generation and quality control TPM website pmds.eo.esa.int/coll ections/ Helder Absolute Radiometric Calibration of Landsat Using a Pseudo Invariant Calibration Site, IEEE Transactions on Geoscience and Remote Sensing Volume 51, Issue 3 (March 2013). Saunier, Landsat MSS, radiometric calibration processing improvement in Proceeding of ESA Living planet Symposium

11 Difference with USGS products Geometry : Instead of systematically applying scene based ortho-rectification it is in some cases more appropriate to calibrate the geo-referencing model by using data from the whole satellite pass. The distortions for the whole pass are modelled using a weighted, constrained, least squares fit with iterative outlier removal. Also, an accurate ortho rectification is applied for scenes partially occluded with cloud, and for scenes with large portion of water. Radiometric calibration: The analysis of Level 0 data, has revealed that MSS calibration was not optimal, in a way that for bright area target RED / NIR1 bands included a lot of saturation. For these regions, the calibration coefficients have been revised and most data are now almost free from saturations. BQA: Quality information attached to Level 1 MSS pixel has been developed for a better filtering of input pixels. Image Reparation: Pixels flag as corrupted for a certain class of anomaly (sticky bit) are repaired Development / Validation is on going 11

12 Geometric Processing (1/2) Objective is simple maximize number of Terrain Corrected products (L1T). 12

13 Geometric Processing (2/2) Quicklooks for same pass v3.04 (top) and v3.06 (bottom) Note the scenes with large distortions for v

14 Geometric Processing Accuracy Methodology has been validate for TM and Improved for MSS. 14

15 Radiometric Processing 1/2 Intensive work of the SDU / usgs to harmonized MSS calibration. Calibration methodology implemented at ESA. Calibration Stability of MSS data after alignment to TM Data (Helder, 2013) 15

16 Radiometric Calibration 2/2 Objective is simple maximize number of Saturation Free products. Selected Landsat WRS scenes for calibration updates 16

17 17

18 18

19 BQA Pixel based information Scan Line Artefacts (SLA) hot spot, sticky bit Dropped lines Saturation Cloud cover Land / Water mask (ESA CCI product) Bit Value USGS Landsat Collection 1 Description ESA MSS Description 0 1 Designated Fill Designated Fill 1 2 Dropped Pixel Dropped Pixel 2 4 Radiometric 3 8 Saturation Radiometric Saturation 4 16 Cloud Cloud 5 32 Cloud Confidence Cloud Confidence Cloud Shadow Confidence Snow / Ice Confidence Cloud Shadow Confidence Snow / Ice Confidence Scan Line Anomaly (SLA) striping / saturation / hotspot / sticky bit Land / Water mask 19

20

21

22 Insight on the archive Analyze temporal coverage of MSS data for specific countries and considering winter and summer period. EU countries covered since the beginning and almost uniformly. Improvements of the coverage of Eastern Countries (Russian, Kazakstan), Northern Countries (Greenland) and Southern Countries (Western part of Africa ) and Islands. 22

23 23

24 24

25 Proof of concept Outline : Several multi temporal applications for which the use of MSS is valuable have been selected in order to demonstrate that data are fit for application purposes. Thematic Area : Phenology Monitoring, Snow / Ice Monitoring, and urban mapping 25

26 Phenology : Monitoring of Forest (Case 1) / Grass land (Case 2) Approach : Collect a large MSS, TM, LS08 dataset over forest (Case 1), grass land (Case 2). Apply preprocessing, compute NDVI, perform analysis. Case 1 Study site: Bouconne, Buzet Forest around Toulouse (France) Case 2 Study site: The northern coast of East Anglia, (Eastern North Sea coastline of the United Kingdom) 26

27 Phenology Case 1 Results 1/2 27

28 Phenology Case 1 Results 2/2 Agreements between measurements from two different dataset, involving different measurements. Good shape of the average NDVI with more uncertainties at growing season Outliers removed because of In some cases, very small NDI values on MSS1 / MSS2 1985/1987 Long and in late freezed period One LS08 NDVI (Summer 2016) very high compared to previous ones (0.76) 28

29 Phenology Case 2 Results Landsat 5 data, MSS ( )/ TM ( ) Strong Discrepancies between TM and MSS MSS NDVI appears lower than TM and its temporal variability is important. Variability of MSS calibration is an issue, strong changes are mostly due to weather conditions and year to year seasonal shift. 29

30 Snow/Ice : Equilibrium Line Approach : Collect a MSS, Sentinel 2 data acquired (if possible) in Autumn, Segment Equilibrium line, and assess if MSS data radiometry allows to perform this task. 30

31 Snow/Ice : Equilibrium Line 31

32 Snow/Ice : Equilibrium Line Results : MSS2/3/4/5 data allows to search this line at large scale Problem is more the period of data, two observation per year and sometimes cloudy. Dataset, MSS 5, can be densified with TM data. Landsat and S2 Results are in agreement. 32

33 Urban Mapping Approach : Collect a large MSS, TM, LS08 dataset over Toulouse City, more than 100 products, create a movie and assess visually urban dynamic Evaluate if details seen with MSS are sufficiently well defined to characterized changes. Results : No geometric correction applied, except for one product The registration of images are with 1.5 pixels if one consider all the series. See next slides. 33

34 Conclusions The ESA MSS data reprocessing project has been presented. The work brings new knowledge regarding our data stored at ESA. We hope it will benefit to a larger user audience. The same approach has been successfully applied for TM / ETM+ data. However, MSS required very careful / specific treatment in term of product development, product processing and quality control. The reprocessing of MSS data will take place by the end of Summer. 34

35 LS08 Processing and QC at ESA LS08 data Near Real time service proposed by ESA IDEAS+ QC activities performed on a monthly basis: Absolute Geolocation Accuracy Assessment; Multi temporal Geolocation Accuracy Assessment ; Short term geolocation stability data from the same path, observed on the same date are analysed Cross track variability different regions in the image located at several locations in the field view are compared Band to band Geometric Registration Accuracy Assessment; Radiometric Calibration stability over PICS; Statistical comparison between results of the two different ROIs is systematically performed Influence of terrain content is better assessed. BQA band inspection, Image Artefact Analysis and Thermal band inspection. Monthly Performance Report is issued every month and 35

36 Team Introductions expert Product Reprocessing Scalable Service Project management Data collection (L0 & AUX) Bulk processing Repatriation (consolidated L0 & processed L1) Data consolidation L0 QC Processor integration Instrument Data quality Evaluation and Analysis Service Processor development Systematic QC tool development (AMALFI) Tool verification L1 QC Expert support Project management

37 Data Access ESA s TPM online dissemination service: ESA Landsat 8 Web Portal: ESA Landsat 8 RealTime Viewer 37

38 This image cannot currently be displayed. Thank you for your attention

39 MSS v3.05 IPF Enhancements Calibration Coefficients Estimate LS05 MSS Calibration parameters over Libya 4 PICS applying cross calibration methodology; Validation on a small dataset observed over Libya 4 and in Iceland; Validation on a large dataset observed in all available PICS test sites; Define most affected WRS scenes in the archive broken down into Desert / Ice-snow regions; For these scenes apply new calibration parameters depending for L1 product generation depending on the season (Summer / Winter) 39

40 ECMWF Water vapor Ozone MSS Spectral Band Definition MSS Observation date and geometrie SIMULATED Top of Atmosphere Measurement Radiative Transfer Code (6s) Oversampled LANDSAT 8 Surface Reflectance Measurements MSS Top of Atmosphere Measurement Data Collected for all ROIs Statistical Comparison Calibration Parameters 40

41 MSS v3.05 IPF Enhancements Calibration Coefficients Libya 4 dataset, RED band Author: SEBASTIEN 41

42 MSS v3.05 IPF Enhancements Calibration Coefficients Libya 4 dataset, NIR1 band CPF patch with changes on Min / Max Radiance Latest CPF patch with changes on Min / Max Radiance and 42 Radiometric Calibration Gain

43 Timeline of Events IDEAS+ activity X-PReSS activity TM/ETM+ L1 ACTIVITIES QUALITY CONTROL TM MPS TM MTI Processing TM BSK Processing MTI Delta ETM+ KSE & MPS ETM+ MTI Regeneration of TM GCP AUX Mid-orbit gap analysis MSS L1 ACTIVITIES v3.04 development & verification MSS stress test (v3.03) Processing QC of full MSS archive v3.05 specification definition & design CPF definition & preliminary verification QC tool update AUX SV generation IPF, QC Tool & CPF verification 43

44 MSS v3.05 IPF Enhancements Calibration Coefficients Iceland Results (snow ice) Percentage of saturated pixels for 3 L1 scenes: Green and Red bands Little reduction of saturation Percentage of saturated pixels for 3 L1 scenes: NIR1 and NIR2 bands Significant improvement for NIR1 44

45 MSS v3.05 IPF Enhancements Calibration Coefficients LS05 MSS CPF files have been updated (applicable for a restricted number of WRS scenes): LM5CPF _ LM5CPF _ Revised Band 2 and Band 3 scaling parameters improve product quality Minimum Radiance and Maximum Radiance have been changed for the RED and NIR1 bands The NIR1 calibration gain of the first CPF has been changed Validation on a extended PICS dataset is now on going Regarding methodology, improvements have been tested by considering varying AOTs value depending on the location in the scene, it increases the accuracy of the results 45

46 BQA Additional logging To ensure full traceability between L0 and L1 (log files not available to users) Full resolution jpeg QuickLook In addition to existing medium res. png Quality Assurance Band (BQA) 16 bit GeoTIFF file for all MSS L1 products Sticky bit correction (TBC) Updated L1 product quality classification Updated calibration coefficient To reduce L1-introduced saturation Geometry To provide more details on GCPs To give greater confidence on accuracy Bit Value USGS Landsat Collection 1 Description ESA MSS Description 0 1 Designated Fill Designated Fill 1 2 Dropped Pixel Dropped Pixel 2 4 Radiometric 3 8 Saturation Radiometric Saturation 4 16 Cloud Cloud 5 32 Cloud Confidence Cloud Confidence Cloud Shadow Confidence Snow / Ice Confidence Cloud Shadow Confidence Snow / Ice Confidence Scan Line Anomaly (SLA) striping / saturation / hotspot / sticky bit Land / Water mask

47 MSS v3.05 IPF Enhancements Geometry The nominal geometrical correction procedure of the ESA Landsat processor is to: Automatically locate GCPs from a GCP database within a scene by normalized absolute image correlation in orthorectified map geometry. Model the distortion with an affine transform using a weighted, constrained, least squares fit with iterative outlier removal. Apply the correction using DEM to produce L1T product We call this single scene refinement 47

48 MSS v3.05 IPF Enhancements Geometry The overall accuracy of the single scene fit is estimated considering: The number of found GCPs Their correlation SNR Their distribution over the scene The chi squared goodness of fit The number of outliers 48

49 MSS v3.05 IPF Enhancements Geometry If the estimated accuracy obtained by single scene fit is not considered sufficient to produce an optimal quality L1T, several back up possibilities are available: Label the L1T as suboptimal Use multiscene refinement to produce L1T Produce a L1G+ correcting geometry without DEM (not used for ETM) Produce a L1Gt using DEM and nominal geometry (used only for ETM) Produce a L1G with no geometric correction or DEM 49

50 MSS v3.05 IPF Enhancements Geometry In multiscene refinement: The affine transform distortions estimated by single scene fitting are used to estimate 2 distortions at 2 points along track per scene in a common SOM projection geometry. The distortions for the whole pass are modelled using a weighted, constrained, least squares fit with iterative outlier removal. A generic fourier series is now used to model the variation of distortions along track. 50

51 MSS v3.05 IPF Enhancements Geometry The main enhancements w.r.t. v3.04 (delta MSS processor) are: Stronger distortion constraints are used for single scene fitting (this can avoid the production of very distorted scenes and help outlier removal in difficult situations) Use general Fourier series instead of b-splines for multiscene fitting (this works much better in particular when distortions have to be extrapolated for several scenes at start and end of pass) Add accuracy estimates which also consider distribution of used GCPs or multiscene refinement in MTL annotations, for L1G+ products also. Add other statistics to MTL annotations regarding GCP distribution Use another orbit propagator (works better for longer propagations) 51

52 MSS v3.05 IPF Enhancements Geometry Example of multiscene fit with estimated errors (L0 pass with several scenes covered by clouds) Note that for MSS the attitude information are missing 52

53 MSS v3.05 IPF Enhancements Geometry The multiscene refinement accuracy for each scene is estimated considering: The number of used single scene distortion estimates Their input single scene estimated accuracies Their distribution over the scene The chi squared goodness of fit The number of outliers The position of the scene within the pass 53

54 MSS v3.05 IPF Enhancements Geometry New MTL fields per product level Level L1G/L1GT L1G+/L1T L1G+/L1T Fit type None Multi Single MODEL_FIT_TYPE x x x GEOMETRIC_MAX_ERR x x GROUND_CONTROL_POINT_DISCARDED x x NUMBER_EMPTY_GROUND_CONTROL_POINT_WINDOWS x GROUND_CONTROL_POINT_WINDOW<n> x GROUND_CONTROL_POINT_RESIDUALS_SKEW_X x GROUND_CONTROL_POINT_RESIDUALS_SKEW_Y x GROUND_CONTROL_POINT_RESIDUALS_KURTOSIS_X x GROUND_CONTROL_POINT_RESIDUALS_KURTOSIS_Y x 54