GTC Todd Bacastow, DigitalGlobe Radiant Todd Stavish, In-Q-Tel CosmiQ Works

Transcription

1 GTC 2017 Todd Bacastow, DigitalGlobe Radiant Todd Stavish, In-Q-Tel CosmiQ Works

2 SpaceNet Overview Inspiration Components Datasets Competitions Inspired by ImageNet 1. Datasets Publicly available satellite imagery & labeled data 2. Competition Public challenges against remote sensing problems 1 st Release (8/16) 50cm 8-band over Rio de Janeiro 2 nd Release (1/17) Points of Interest (POI) over Rio 3 rd Release (2/17) 30cm 8-band over Las Vegas, Paris, Shanghai & Khartoum 1 st Competition Completed 12/16 2 nd Competition Launched on 3/20

3 Source:

4 Source: DigitalGlobe, Inc.

5 Source:

6

7

8

9

10 The data management challenge 10

11 How do we get 100 PB into the cloud? Home broadband: 300 years DirectConnect: 6-18 months ($$$) X 1,400 11

12 or a bigger snowball a Snowmobile 12

13 13

14 SpaceNet Datasets SpaceNet on AWS is an open repository of 5,700+ km 2 of satellite imagery and 520,000+ vectors made available to developers to enable geospatial machine learning. Rio de Janeiro Buildings Released August 2016 Rio de Janeiro Points of Interest (POIs) Released January 2017 Las Vegas, Paris, Khartoum, and Shanghai Buildings Released February Imagery: 50 cm WV-2 mosaic and 8-band MSI covering 1900 km 2 Building Footprints: 220,594 covering 252 km 2 Imagery: 50 cm WV-2 mosaic POIs: 120,155 individual POIs from 460 feature classes Released with NGA support Imagery: 30 cm WV-3 image strips and 8-band MSI covering 3,880 km² Building Footprints: 221,376

15 Rio Public Data Set Rio de Janeiro, Brazil Imagery: 50cm WV-2 mosaic + 8-band MSI covering 1900 km 2 Building Footprints: 220,594 covering a 252 km 2 AOI

16 Rio Points of Interest Dataset 12 datasets with 35 unique layers containing more than 120,000 individual points of interest POI Datast Includes Subset of 11,114 points across 139 features that have been identified as discernable in the provided satellite imagery Public Facilities Released in GIS (geodatabase) and machine learning friendly formats (parsable JSON) Provides quality estimation attributes (e.g. confirmation and resolution) Utilities Introduces the concept of an object hierarchy akin to ImageNet s use of WordNet (e.g. infrastructure- >buildings->apartments) Transportation

17 Rio POI Dataset

18 Newly Released Public Data Sets Imagery: 30cm WV-3 single strip images + 8-band MSI Total Building Footprints: 221,376 covering a 3,880 km 2 AOI across for 4 additional cities: Las Vegas, Paris, Shanghai, and Khartoum Las Vegas Paris Shanghai Khartoum 270 km 2 1,560 km 2 1,170 km km 2 109,807 Footprints 16,663 Footprints 69,433 Footprints 25,463 Footprints 69GB Raster Data 402GB Raster Data 302GB Raster Data 373GB Raster Data SpaceNet March

19 Lowering the Barrier of Entry for SpaceNet SpaceNet contains a massive amount of labeled data in GeoJSON files, an unfamiliar format for most data scientists. We released code to transform these labels into a multitude of other formats (NumPy arrays, image masks, etc.) more conducive to machine learning. * Naïve approach yields F1=0.57 Imagery Courtesy of DigitalGlobe Imagery Imagery Courtesy Courtesy of DigitalGlobe of 19

20 crowdsourcing.topcoder.com/spacenet 20

21 SpaceNet Challenges The SpaceNet Challenge is a series of coding competitions with cash prizes that make use of SpaceNet on AWS datasets to accelerate geospatial machine learning. Automated Mapping Challenge - Round 1 Nov. Dec Automated Mapping Challenge - Round 2 March May 2017 High Revisit Activity Detection Challenge Mid-2017 Rio de Janeiro Building extraction $35,000 in prizes Las Vegas, Paris, Khartoum, Shanghai Building extraction w/ 2x performance $15,500 in prizes Imagery will show places with economic indicators and focus on activity-based analytics

22 SpaceNet Challenge Metric and Scoring Metric was an IoU comparison with a threshold o IoU(A,B) = area(a intersection B) / area(a union B) Top public leaderboard F1 score was o precision = TP / (TP + FP) o recall = TP / (TP + FN) o F1= 2 * precision * recall / (precision + recall) Source: Walber (Own work) [CC BY-SA 4.0 ( via Wikimedia Commons. 22

23 SpaceNet Challenge - Round 1 Challenge Competition focused on automated feature extraction Evaluation Results were evaluated with scientifically grounded metrics (F1 Score) Cash Prizes $35,000 in prizes were paid to the top performing teams Competitors 42 competitors worldwide Submissions 242 submissions Winning Result F1 Score of from Brazil International Top 5 submissions were international Competition Timeline 10/24 10/31 11/7 11/14 12/8 12/22 Pre- Registration Training Data + Visualizer Released Google OnAir Hangout w/ SpaceNet Experts Match Began (3-Week Competition) Competition Ends Winners Announced The relatively low F1 scores of the winning submissions indicate that automated building footprint extraction remains a challenging problem that warrants further research 23

24 SpaceNet Challenge Round 1: Winning Solution The winning implementation was developed by a Brazilian Topcoder Implementation was custom and used random forests with brute force polygon search Results of the first challenge were promising given limited time and use of an early training dataset More information CosmiQ Works blog SpaceNet: Winning Implementations and New Imagery Release Summary of approach: 1. Classify pixels into 3 categories: border, inside a building, and other. 2. Based on individual pixel classification, generate candidate polygons that may contain buildings 3. Evaluate polygon candidates to select those with a confidence above a given threshold; discard remaining polygons 24

25 Round 1 Winning Solution

26 SpaceNet Challenge - Round 2 Challenge Competition on footprint extraction over four diverse cities Evaluation Highest F1 per city and averaged across all cities Cash Prizes Up to $15,500 in prizes to be paid to the top performing teams Competition Timeline (Estimated) 2/17 3/20 4/1 5/23 5/31 Training Data Released Match Began (9-Week Competition) Early Incentive Awarded Competition Ends Winners Announced 26

27 SpaceNet Challenge Round 2 Early Results F-score: ~0.6, average of all four cities Improvements in imagery resolution and vector labels Higher F-scores in Round 2 initially seems to be directly related to better training data - imagery and labels

28 How to Get Involved 1. Utilize SpaceNet on AWS data for research Use the data to train models for research or commercial uses Publish open source code, blog posts, and research papers 2. Participate in current/future SpaceNet Challenges SpaceNet Challenge Round 2 is live Tell your friends 3. Contribute/sponsor future open data releases Looking for new participants to contribute to the release of additional data sets The data must have an open license and come prepared 28

29 Thank You