A Web Application and Subscription Service for Landsat Forest Area Change Tools (LandsatFACT)

Transcription

1 A Web Application and Subscription Service for Landsat Forest Area Change Tools (LandsatFACT) Development Team N.C. Forest Service: David Jones & Brian McLean UNC Asheville s NEMAC: Jim Fox, Derek Morgan, Matt Hutchins, John Frimmel, Mark Phillips, Karin Rogers & Caroline Daughtery 1

2 Presentation Overview 1. Brief background on relevant aspects of Landsat 2. Purpose and scope of LandsatFACT project intended users, use-cases and geographic extent 3. Where we are at and what we have learned thus far in the development of LandsatFACT: server and software architecture user account subscription management forest change WebGIS custom request application 4. Case Study: 2011 Pains Bay Fire 2

3 The Satellite Systems LANDSAT 5 LANDSAT 7 LANDSAT 8 source: USGS (2014) 3

4 A satellite sensor measures radiance at the sensor itself, not surface reflectance from the target. In other words, the sensor is only measuring the intensity of light when it hits the detector surface. Scattering Atmospheric effects Scattering

5 Landsat Bands and Wavelengths source: USGS (2014 5

6 DN to Reflectance Radiance is the amount of radiation coming from an area... measured at the sensor L y = "gain" DN + "offset" Reflectance is the proportion of the radiation striking a surface to the radiation reflected off of it. or L λ LMAX λ LMIN λ = QCALMAX QCALMIN QCAL QCALMIN + LMIN λ source: USGS (2015) 6

7 LandsatFACT Project Area o o 141 World Reference System (WRS-) 2 path/row areas 16 day return per platform offset by 8 days between L7 and L8 7

8 Project Overview o 3 change analysis methods B7 differencing, NDVI, and NDMI o o Performed on the fly as imagery becomes available Built to work with L5, L7, and L8 o o Hosted in the cloud Constructed with free and open source software 8

9 Use-case 9

10 10

11 LandsatFACT User Interaction & Databases 11

12 LandsatFACT/NEMAC Cloud Servers Development Production Cloud0 (CENTOS) Web Development Server Cloud1 (CENTOS) Production Web Server Drupal/MySQL Drupal/MySQL Apache Geoprocessing Libraries (e.g GDAL) MapServer Apache PostGIS RDS Production DB Server PostgreSQL/PostGIS Cloud4 (CENTOS) Geoprocessing Server PostGIS RDS Development DB Server PostgreSQL/PostGIS MapServer Apache Geoprocessing Libraries (e.g GDAL) 12

13 13

14 Scene Geoprocessing Flow Chart New Scene FOR each Quad Find Next Newest Scene Quad Extract Run Fmask Check Comparison Sensor Types Make NDVI, NDMI, B7_Diff, Cloud Mask Make Quads Compute Cloud Cover Percent L8 - L5/ 7 L5/ 7- L5/ 7 L8 - L8 Check IF Quad CC is > or < threshold Resample to 8 - bit > < Check IF Quad is L7 SLC - Off End Quad YE S Make Gap Mask Stitch in regions mosaic and apply overviews.

15 Case Study: 2011 Pains Bay Fire 15

16 Custom Request Validation sent 16

17 Custom Request... Time Range: Data e- mail sent Area-of-Interest: 17

18 Subscription by AIO When new change occurs data e- mail sent 18

19 Case Study: 2011 Pains Bay Fire LE EDC00.tar LE EDC00.tar 19

20 Serving 30 meter LandsatFACT 141 Scenes, 564 individual files Band 7 Differencing NDVI NDMI Quad files are updated as they become available Important to be able to manage overlaps Data Products Dealing with LARGE datasets 20

21 Serving Method 1: Tile Indexing Uses a shapefile with a record for each file Built using gdaltindex: gdaltindex index.shp directory/*.tif MapServer uses index.shp to retrieve images within the requested tiles Overlaps managed by index id Image shows outlines of index.shp file and corresponding images 21

22 Serving Method 2: Virtual Raster Table (VRT) XML format that manages set of raster files, including bands, extents, and metadata Built using gdalbuildvrt: gdalbuildvrt doq_index.vrt doq/*.tif MapServer uses file.vrt to retrieve images in the requested extents Overlaps managed by sort order of files 22

23 Serving Method 3: Single Raster Individual files combined or stitched together to create a single raster Built using gdal_merge.py Mosaic gdal_merge.py o out.tif directory/*.tif Overlaps managed by controlling sort order of files 23

24 Serving Method 4: Combined use of single raster mosaic and tile indexing MapServer layer GROUP used for multiple layers (allows them to be referenced by one name) SCALE limits were set on each MapServer Layer so each would be used at their most optimal extents Switch from Mosaic to Tile Index happens automatically based on requested zoom level Mosaic Tile Index 24

25 Optimization Methods Converting 16bit files to 8bit Reduced from ~16GB to ~8GB files Mosaic Overviews GDAL Overviews using gdaladdo to build overview images in existing TIF Files Specified overview levels: 2, 4, 8, 16 Overview level resolution= 30 * n Uses nearest neighbor downsampling Increases TIF file size MapServer will use the most optimized level, which decreases on-the-fly processing 25

26 Performance Comparison 8bit 30m Files Entire Southeast State (~1:10,000,0 00) Sub-State (~1:1,000,000) Local/County (~1:200,000) Tile Index 50s 45s 3.1s 527ms VRT 1m 1m 594ms 577ms Single Raster Mosaic (no Overviews) Single Raster Mosaic with Overviews (levels 2, 4, 8, 16) 17s 17s 14s 3.9s 468ms 1.2s 823ms 497ms 26

27 Development Time Line of LandsatFACT 27

28 The Landsat Forest Area Change Tools (LandsatFACT) project was made possible by a grant from the USDA Forest Service Region 8 State and Private Forestry Redesign competitive allocation. Partners:

29 Other Landsat Projects to be Aware of EROS Science Processing Architecture LEDAPS Fmask CFmask Global Forest Change Global Forest Watch Global Forest Watch Fires ESRI Landsat Imagery Services ESRI Landsat 8 Dynamic Services Change Matters

30 References Chambers, S.D. (2002). Application of spectral change detection techniques to identify forest harvesting using Landsat TM data. Retrieved from Virginia Tech Digital Library and Archives. Hargrove, W. W., Spruce, J. P., Christie, W. M., & Schroeder, S. W. (2013). Highlights of satellite-based forest change recognition and tracking using the ForWarn System. US Department of Agriculture, Forest Service, Research & Development, Southern Research Station. Irish, R. R. (2000). Landsat 7 science data users handbook. National Aeronautics and Space Administration, Report,