Release Notes GAMMA Software,

Transcription

1 Urs Wegmüller, Charles Werner, Andreas Wiesmann, Othmar Frey, Christophe Magnard, Oliver Cartus Gamma Remote Sensing AG Worbstrasse 225, CH-3073 Gümligen 30-Jun-2018 Introduction This information is provided to users of the GAMMA software. It is also available online at This release of the Gamma software includes new programs that provide new capability, additional features to existing programs and bug fixes. Gamma Software on Linux, OSX, and Windows The Gamma software has been compiled and tested on Linux (different distributions), Apple MacOS High Sierra ( ), and Windows 10 (64-bit, should also function on Windows 7). Computationally intensive programs such as used in co-registration and resampling and geocoding have been parallelized using the OPENMP API built into the GCC compiler. Processing speed on Linux, MacOS, and Windows systems is comparable. Linux Distribution: The Gamma software is developed on Ubuntu LTS 64-bit Linux and is tested extensively with this distribution. Hence it is highly recommended to run the software on this distribution. It has also been compiled and tested on Ubuntu LTS and LTS. Announcement: Ubuntu LTS will no longer be supported after the mid 2018 upgrade and LTS after the mid-2019 upgrade. Versions of the Software will also be uploaded for RHEL6, and RHEL7 based on Centos 6, and 7. RHEL5 is no longer supported as some programs cannot be compiled due to outdated Libraries. Announcement: RHEL6 will no longer be supported after the Dec upgrade. For installation instructions for the binary LINUX distributions see the Text file INSTALL_linux.txt (provided with the distribution or found in the main directory of the distribution). Apple MacOS Distribution: The software in this version has been compiled using MacOS High Sierra ( ). You will need to install libraries such as GDAL using MacPorts. Earlier versions of MacOS are no longer supported. For installation instructions for the binary MacOS distributions see the ASCII file INSTALL_macOS_Sierra.txt (provided with the distribution or found in the main directory of the distribution). 1

2 Windows Distribution: The Windows 7, 8, 10 version of the Gamma software is 64-bits and multi-threaded. The software has been compiled under Win10 and is expected to run on Win7, and 8. The build uses the MINGW64 GCC compiler For installation instructions for the binary Windows distributions see the ASCII file INSTALL_WIN64.txt (provided with the distribution or found in the main directory of the distribution). Documentation: The Gamma documentation browser is an HTML based system for viewing the web pages and pdf documents. The documentation browser includes for each module a Contents sidebar on the right side of the screen and a search functionalities. The program gamma_doc facilitates the access to the documentation related to a given module or program: gamma_doc gamma_doc DIFF gamma_doc gc_map Opens the main page of the Gamma documentation browser. Opens the DIFF&GEO documentation. Opens the reference manual web page for gc_map. Further information related to the GAMMA Software is available online: General information: gamma-rs.ch/uploads/media/gamma_software_information.pdf Technical reports, conference and journal papers: gamma-rs.ch/uploads/media/gamma_software_references.pdf Release notes / upgrade information: gamma-rs.ch/uploads/media/gamma_software_upgrade_information.pdf Hardware Recommendations Using multi-core processors (2 or more cores) will bring substantial improvement in processing speed due to parallelization of the code base. There should be at least 4 GB RAM available for each processor core with 8 to 16 GB per core recommended. Disk storage requirements for using the Gamma Software effectively depend on the amount of input data and data products that will be produced. Based on our experience we recommend to consider at least 8 TB space, especially when working with stacks of Sentinel-1 or very high resolution data (TerraSAR-X, Cosmo-Skymed) data. The current trend towards larger data products requires substantially increased storage capacities. GAMMA Software Training Courses GAMMA plans to organize in fall 2018 again training courses at GAMMA (near Bern, Switzerland) for SAR/INSAR (MSP/ISP/DIFF&GEO/LAT) and for PSI (IPTA). See also our web-site under 2

3 Significant Changes in the Gamma Software Modules since the End of 2017 Release Ionospheric phase identification, estimation and mitigation Spatial and temporal variation of the free electron concentration in the ionosphere affects SAR interferograms, in particular at low radar frequencies. The split-spectrum method (e.g. presented in Gomba, G., Parizzi, A., De Zan, F., Eineder, M., Bamler, R., (2016) Toward operational compensation of ionospheric effects in SAR interferograms: the split-spectrum method. IEEE Trans. Geosci. Remote Sens. 54, ) permits separating the ionospheric and the nondispersive phase terms using spectral sub-band images. In this period we spent a significant effort to better support the ionospheric phase identification, estimation and mitigation with a number of dedicated tools. An important aspect in the proposed methods is a reformulation of the split-spectrum method that facilitates the necessary processing steps. Reformulating the split-spectrum method permits determining the ionospheric phase component based on a split-spectrum double-difference interferogram and the full-bandwidth differential interferogram. In this way differing a pair of unwrapped phase images, each one scaled by a large factor, can be avoided, making the method more robust. The applicability of the proposed method is also demonstrated with a new Gamma Software demo example. The following programs are available in support of the ionospheric phase identification, estimation and mitigation (including several new ones): Program ionosphere_check bpf_ssi SLC_freq_shift SSI_INT SSI_ionosphere Burst selection related functionality Calculates for a single SLC an offset map between two azimuth sub-look images. Non-zero azimuth offsets are an indication for significant gradients in the ionospheric path delay. In the presence of such gradients the SLC coregistration procedure needs to consider related offsets. Generates spectral sub-look SLCs used for split-spectrum method. Determines filter parameters, applies filtering, shifts radar frequency to center of extracted sub-band and adjusts SLC parameter file. It also calculates the scaling factors used in the split-spectrum method. Shifts the effective radar carrier frequency of an SLC image by a specified amount. Generates the split-spectrum double difference interferogram, filters and unwraps it. Besides it also generates the full-bandwidth differential interferogram. Calculates the spatially filtered ionospheric phase (float) and the ionosphere-corrected (non-dispersive) differential interferogram (fcomplex), twice the ionospheric phase (fcomplex), twice the non-dispersive phase (fcomplex), and conducts consistency tests between the results For a detailed discussion and references to the literature it is referred to the attached pdf of a manuscript by Wegmüller et al., submitted for publication (please do not distribute this pdf). For a detailed discussion of the processing steps used in a specific example it is referred to the related demo example that was added. Some results are shown in Figure 1. 3

4 Differential interferogram (cpx) Ionospheric phase Ionosphere-corrected differential interferogram (cpx) x ionospheric phase (cpx) 2x non-dispersive phase (cpx) Split-spectrum double-difference interferogram Figure 1 Ionosphere estimation and mitigation example. The main steps used to generate an ionosphere-corrected differential interferogram are the following. After reading the SLC data and selecting the reference we decided on the multi-looking to be used, generate for the reference a multi-look intensity image (MLI) and geocoded it with an available DEM, and resample the DEM height to the MLI slant range geometry. Then we check the two SLCs for ionospheric effects using ionosphere_check. In the case of obvious non-zero azimuth offsets we have to do a careful co-registration of the SLCs considering offsets related to ionospheric effects. Here we also check the range offset field (that should be close to 0.0) for range offset anomalies as they occur for PALSAR-2 IM3 SLC data. Anomalies are identified by one or several strips each with a range offset around plus or minus one SLC pixel. In the presence of such an anomaly the SLC is considered corrupt and the split-spectrum method cannot be applied. After this we determine the refined co-registration look-up table. This is done using the method with rdc_trans followed either by a polynomial refinement (for cases without high ionospheric phase gradients) or a refinement done considering an estimated offset field. Using SSI_INT we calculate then the split-spectrum double difference interferogram, filter and unwrap it. Besides SSI_INT also generates the full-bandwidth differential interferogram. Then we unwrapped and filtered the full-bandwidth differential interferogram and determined the ionospheric phase as well as the ionosphere-corrected (non-dispersive) differential interferogram using SSI_ionosphere. 4

5 Input and output of geocoded data Over the last months some substantial improvements in the I/O of geocoded data were implemented: The program dem_import supports the reading of digital elevation model (DEM) data in various formats and writes out the corresponding x.dem and x.dem_par pair (DEM data format as used within GAMMA Software). In the past at least a partially filled DEM_par file was required to read in a DEM data set (defining the DEM datum, ellipsoid and map projection). Now, in case of a DEM provided as a GeoTIFF or another GDAL supported data, xyz triplets, or ascii grid data (ESRI), the program will automatically interpret the coordinate system if the DEM_par file does not exist yet. In case of xyz triplets, or ascii grid data (ESRI), it will try to find a.prj file with the same root name as the input DEM, and interpret the tag written in that file. The input data can be in any of the map projection types supported by the Gamma Software (EQA, UTM, OMCH, HOM, LCC, LCC2, TM, PS, PC, AEAC, SIN), except OM and SCH. dem_import can also directly transform the DEM height from an altitude above the geoid into an altitude above the ellipsoid. The EGM96 geoid is available in DIFF/scripts both as GeoTIFF and in Gamma format. Note that geoids in GeoTIFF format can also be converted into Gamma format using dem_import by treating them as DEMs. dem_import replaces the previous program dem_conv. par_data_geo generates a DEM parameter file and an image file for the selected band (layer) of a GeoTIFF / GDAL supported geocoded raster file. It is now a Perl script that calls dem_import for the effective data and metadata extraction. All GDAL supported types can be read (Byte, UInt16, Int16, UInt32, Int32, Float32, Float64, CInt16, CInt32, CFloat32 and CFloat64), as well as complex data stored as two separate bands. Output data are written either as float or fcomplex. When using dem_import or par_data_geo, please check if warnings were given by the program and check as well the DEM_par file after its generation, since some GeoTIFF/ESRI tags may be sometimes incomplete or erroneous. data2geotiff can now write GeoTIFF images in any of the map projections supported by the GAMMA software (EQA, UTM, OMCH, HOM, LCC, LCC2, TM, PS, PC, AEAC, SIN), except OM and SCH. The GeoTIFF files can then easily be imported in GIS software. Note that Google Earth is not 100% reliable regarding GeoTIFF import. On the other hand, all generated GeoTIFFs could be reliably imported into QGIS. The ELLIPS, DATUM, and PRO_PAR structures were also slightly modified: - Added EPSG number and keywords to ELLIPS, DATUM, and PRO_PAR structures. - Map projection parameters are now fully included in PRO_PAR structure (previously several parameters were stored in the DEM_PAR structure): - AEAC: "first_std_parallel" and "second_std_parallel" have been moved from DEM_PAR to PRO_PAR structure - HOM: "OM_alpha" and "OM_gamma" have been moved from DEM_PAR to PRO_PAR structure - LCC: the projection is not hard-coded anymore for the 4 zones in France, but works for any input parameters - LCC2: "first_std_parallel" and "second_std_parallel" have been moved from DEM_PAR to PRO_PAR structure - OM: "OM_alpha" and "OM_gamma" have been moved from DEM_PAR to PRO_PAR structure - PS: former "PS_secant_lat" and "PS_central_meridian" are now stored in "lat0" and "lon0" respectively; ll_ps and ps_ll now support a secant plane at the pole as well as a scale factor for that case; now supports false Easting and false Northing parameters 5

6 - SIN: former "SIN_central_meridian" parameter is now stored in "lon0", addition of support for false Easting and false Northing parameters - TM: correction of a minor bug in ll_tm function - UTM: correction of a minor bug in ll_utm function Please notice that DEM_par files created before these modifications are still read correctly. datums.h, ellipsoids.h, and projection_params.h: the databases were updated and completed. Note that some names have been changed to match the usual names of the datums / map projections. create_dem_par and coord_trans: the text interface was updated and made slightly clearer. Available datums / map projections can now be listed by typing "?". If you are using create_dem_par in a script, please check your script! For WGS84 EQA and UTM, there should not be any difference. For the other datums / map projections, there might be some differences and the datum / map projection ID might have changed. Visualization Tools The Python Display Programs and Utilities available as part of the DISP, now include: colormap_list.py Generate text format colormaps (*.cm) from the registered Matplotlib colormaps vis_colormap_bar.py Generate horizontal or vertical colorbar with scale from colormap file or registered colormap viscpx.py Display of complex data intensity, real, imaginary, magnitude, or phase, with linear, logarithmic, or power-law scaling visdt_pwr.py Display of float data such as displacement or unwrapped phase combined with intensity vismph_pwr.py vispwr.py Display the phase of float complex data combined with intensity Display of radar intensity, deformation, unwrapped phase, correlation or other parameter with linear, logarithmic, or power-law scaling visras.py Display raster images (BMP, PNG, JPEG, TIFF formats) visbyte.py Program to display 8-bit byte data in the range 0->255. ras2png.py Program to convert raster image (BMP, JPEG, TIFF,...) to a PNG with the option to make null (black) pixels transparent. Besides the advantage that very large files can be displayed, the new programs offer a wide visualization and rasterfile (quicklooks as well as rasterfiles with one pixel per data file pixel) generation functionality. Color bars can be drawn with the display and in addition to the color scales used already further color maps (as available from Matplotlib) can be selected. Gamma Software Demo examples In this period again some Gamma Software Demo examples were added/modified. Gamma Software Users with a valid license or evaluation license can download the Gamma Software Demo examples here: account: user: gamma_user password: RBS_821 Demo examples which were added/modified to demonstrate the application of methods to identify, estimate and mitigate ionospheric path delay variations. 6

7 Demo example: Gamma_demo_ionosphere_PALSAR1_Fur uya.tar.gz Gamma_demo_ionosphere_PALSAR2_Tok yo.tar.gz Contents Based on a PALSAR-1 SLC pair the identification, estimation and migration of the ionospheric path delay phase is demonstrated. The pair is affected by strong ionospheric effects (same data as used in Furuya M., Suzuki T., Maeda J., and Heki K., (2017). Midlatitude sporadic-e episodes viewed by L-band split-spectrum InSAR, Earth Planets and Space (2017) 69:175; https//doi.org/ /s Based on a PALSAR-2 SLC pair over Tokyo the identification, estimation and migration of the ionospheric path delay phase is demonstrated for a case not affected by particularly strong ionospheric effects. See Figure 1 and Figure 2 to see some of the results of the demo examples. - - /2 0 / /2 0 /2 - - /2 0 /2 a) DInSAR (complex) b) double difference phase c) Method 1: ionosphere d) Method 1: non-dispersive - - /2 0 /2 - - /2 0 /2 - - /2 0 /2 - - /2 0 /2 e) M2: 2 x ionosphere (cpx) f) result (e) 2 x result (c) g) M3: 2 x non-disp. (cpx) h) result (g) 2 x result (d) Fig. 2. Estimation and compensation of ionospheric path delay for PALSAR-2 HH-pol. SM1 mode data acquired on 8-Sep-2016 and 1-Dec-2016 over Tokyo harbor area. Results are shown in slant range geometry. The upper line shows (a) the differential interferogram, (b) the split-spectrum double difference phase, (c) the ionospheric path delay estimated using Eq (3), and (d) the non-dispersive phase calculated by subtracting (c) from (a). The lower line shows (e) twice the ionospheric phase derived using Method 2, (f) the consistency between Methods 1 and 2, (g) twice the non-dispersive phase derived using Method 3 and (h) the consistency between Methods 1 and 3. 7

8 All modules: makefile_static: Modified so that CCFLAGS and CC are defined by the makefile and not by an external environment variables. Users no longer have to define these variables when compiling Gamma software on Linux. CC = gcc CPPFLAGS = -DCPU_LITTLE_END -mtune=generic -ffast-math fopenmp Added /usr/include/mpich-x86_64 to the include files searched to permit compilation on RHEL and CentOS MSP No new program added. ISP multi_look2: Added program to calculate an MLI image from an SLC with separate averaging window dimensions and decimation factors. S1_TOPS_preproc: Modified to function even if some of the swaths have been deleted from the data set. SLC_cat, SLC_cat_all: Addition of a gain correction flag that permits adapting the calibration gain of the second SLC to that of the first SLC. par_s1_slc, par_s1_grd: Addition of a check for the noise values: if the noise values result in a NESZ < -40 db (unrealistically low) or NESZ > -20 db (unrealistically high), they are not subtracted from the data. par_s1_slc: Seconds of the day tags for bursts (in TOPS_par file) are now extended when an acquisition was started at the end of a day and finished in the next day. base_orbit: Improved test for zero baseline when indicated scenes are identical. SWAP_io: New functions cnt_tab_lines and rd_tab_line are introduced for more robust counting of lines in tab files and reading the valid lines. No blank line needed anymore at the end of the file. mosaic_wb, multi_s1_tops, radcal_pwr_stat, SLC_copy_S1_TOPS, SLC_deramp_S1_TOPS, SLC_interp_S1_TOPS, SLC_mosaic_S1_TOPS, split_wb: Introduced more robust routines for counting lines in tab files and reading the valid lines. No blank line needed anymore at the end of the file. bpf_ssi: Added new script to apply band-pass filtering for split-spectrum interferometry. DIFF&GEO gc_map2: Now also supports GPRI MLI parameter files as input. offset_pwrm, offset_pwr_trackingm, offset_pwr_trackingm2: Now checks if signal varies enough within each patch, new option [std_mean] (default value: 0.01) defines the minimum standard deviation/mean ratio required for performing the offset computation. This update helps to avoid getting anomalous offset estimates over open water surfaces. 8

9 gc_map2: Can now use a DIFF_par file as input to generate a refined look-up table and a refined layover-shadow map in radar coordinates. SLC_diff_intf: Modified calculation of the S/C position vectors in the range and azimuth commonband filtering to be robust for spotlight mode when there are few state vectors. DIFF_io, DIFF_lib, typedef_diff.h: Addition of EPSG number and keywords to ELLIPS, DATUM, and PRO_PAR structures. Map projection parameters are now fully included in PRO_PAR structure: several parameters were previously stored in the DEM_PAR structure. (see also page 5 of this document or the history file in the top directory of the software). ellipsoids.h, datums.h, projection_params.h: Extention, update and correction of each database, adaptation to the modified ELLIPS, DATUM, and PRO_PAR structures coord_trans, create_dem_par: Adapted to the modified ELLIPS, DATUM, PRO_PAR, and DEM_PAR structures, updated and clarified the user interface. dem_import: The program dem_conv program has been renamed to dem_import and updated as follows: If the DEM_par file does not exist and the input DEM is a GeoTIFF, dem_import will automatically extract the ellipsoid, datum, and map projection parameters. Other formats still require a DEM_par file as input. If the DEM_par file does not exist and the input DEM is an xyz triplets file or an ascii grid (ESRI) file, dem_import will try to find a prj file with the same root name as the input DEM and automatically extract the ellipsoid, datum, and map projection parameters. dem_import now reads integer as well as complex-valued data stored in GeoTIFF files. par_data_geo: The Perl script par_data_geo replaces and expands the C-program par_data_geo. create_dem_par: Addition of an [EPSG] option that permits defining a geographical coordinate system / map projection by simply entering that number. The interpretation of the EPSG number is performed using GDAL functions. coord_trans: It is now possible to enter an EPSG number for defining the input and/or output geographical coordinate system / map projection. dispmap_enu, multi_mosaic, SLC_interp_lt_S1_TOPS, stacking, WSS_mosaic: Introduced more robust routines for counting lines in tab files and reading the valid lines. No blank line needed anymore at the end of the file. par_s1_slc, par_s1_grd, SLC_cat, SLC_cat_S1_TOPS: Improved support for data acquisitions extending over midnight. SLC_coreg: Added new script to coregister an SLC to a reference SLC. SSI_ionosphere: Added new script to support ionosphere estimation and mitigation based on unwrapped DINSAR and SSI_INT phases. SSI_INT: Updated and moved from ISP to DIFF. Now the spectral band-pass filtering is done for the not-co-registered SLC pair. The filtered and unfiltered slave SLC are then resampled to the the master SLC geometry and used to calculate the complex valued split-spectrum interferogram. Optionally, the complex valued split-spectrum interferogram is unrapped. As input the terrain heights in the reference MLI geometry (used for the phase simulation) and the geocoding lookup table (used for the co-registration) need to be provided as input. pol2rec: Added correction for the electronic range delay of 4.75m when calculating the resampled GPRI2 image. 9

10 DISP ras2bmp.sh: Added program to convert all the Sun raster files in working directory to BMP format. data2geotiff: Now supports conversion of dem_par files into GeoTIFF for all map projections supported in Gamma software (except SCH and OM). kml_plan: Event markers and points on the ground can now be displayed on the KML. LAT histogram: Added capability to either generate statistics and histograms from data regions specified by polygons, or to measure statistics and generate a histogram for points in all the polygons. Specifying - for the polygon list now selects all points in the file. poly_math: Now uses memory blocks to support processing very large images while reducing memory requirements. temp_filt, temp_filt_ad, temp_lin_var, temp_log_var: Introduced more robust routines for counting lines in tab files and reading the valid lines. No blank line needed anymore at the end of the file. IPTA ras_data_pt, rasdt_cmap_pt: Added option to permit drawing small radially weighted spheres rather than rectangular points. ras_data_pt: Added option to specify a colormap. mb, merge_pt, pwr_stat, tpf_2d, ts_rate, vu_disp2d: Introduced more robust routines for counting lines in tab files and reading the valid lines. No blank line needed anymore at the end of the file. pt2geo: Added option to select if checking is performed if a point is within the DEM bounds or not. pol2rec_pt: Added program for polar to rectangular coordinate conversion for GPRI SLC and MLI image data in IPTA vector data format. 10