5/14/2018. About Your Hosts. Mapping Accessibility Through Google Street View. Captioning

Transcription

1 arrow points to the "cc" icon in the audio and video panel Transc en logo and NIDILR R logo 5/14/2018 Mapping Accessibility Through Google Street View will begin at 12:30 pm ET Audio and Visual are provided through the on-line webinar system. This session is closed captioned. Individuals may also listen via telephone by dialing Access Code: This is not a toll-free number. Captioning Real-time captioning is provided; open the window by selecting the CC icon in the AUDIO & VIDEO panel You can move and re-size the captioning window. Within the window you change the font size, and save the transcript About Your Hosts TransCen, Inc. Mission Statement: Improving lives of people with disabilities through meaningful work and community inclusion Mid-Atlantic ADA Center, a project of TransCen, Inc. Funded by National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR), Administration for Community Living, U.S. Department of Health and Human Services 1

2 arrow points to microphone icon on audio and video panel Resizing dropdown box 5/14/2018 Listening to the Webinar Online: Please make sure your computer speakers are turned on or your headphones are plugged in Control the audio broadcast via the AUDIO & VIDEO panel If you have sound quality problems, please go through the AUDIO WIZARD by selecting the microphone icon within the AUDIO & VIDEO panel Listening to the Webinar (cont.) To connect by telephone: Pass Code: This is not a toll-free number Customizing Your View Resize the whiteboard where the presentation slides are shown to make it smaller or larger by choosing from the drop down menu located above and to the left of the whiteboard; the default is fit page 2

3 Customize Your View continued Resize/Reposition the CHAT, PARTICIPANT, and AUDIO & VIDEO panels by detaching and using your mouse to reposition or stretch/shrink Each panel may be detached using the icon in the upper right corner of each panel Technical Assistance If you experience technical difficulties Use the CHAT panel to let us know Call Archive This webinar is being recorded and can be accessed within a few weeks You will receive an with information on accessing the archive 3

4 Mapping Accessibility Through Google Street View will begin at 12:30 pm ET Audio and Visual are provided through the on-line webinar system. This session is closed captioned. Individuals may also listen via telephone by dialing Access Code: This is not a toll-free number. Mapping accessibility via Google Street Associate Professor Computer Science University of Washington 4

5 Project Sidewalk The Team Professors Grad students Jon Froehlich David Jacobs Kotaro Hara Manaswi Saha Michael Jin Sun Soheil Saugstad Behnezhad Undergraduate Students Maria Furman Vicki Le Robert Moore Christine Chan Daniil Zadorozhnyy High School Students Jonah Chazan Anthony Li Niles Rogoff Zach Lawrence And more and more Alex Zhang Improving access to the physical world Our overarching research Question How can we develop scalable solutions that map the accessibility of urban infrastructure? Project Sidewalk [ASSETS 12, CHI 13, HCOMP 13, ASSETS 13 Best Paper, UIST 14, TACCESS 15, SIGACCESS 15, CHI 16, ASSETS 17] 30.6 million U.S. adults have a mobility impairment Source: US Census, 210 5

6 15.2 million use an assistive aid Source: US Census, 210 No Curb Ramps 6

7 Physical Obstacles Incomplete Sidewalks Surface Problems 7

8 Physical Obstacles No Curb Ramp Surface Degradation Accessible infrastructure has a significant impact on the independence and mobility of citizens [Thapar et al., 2004 ; Nuernberger, 2008] I usually don t go where I don t know [about accessible routes] -P3, congenital polyneuropathy 8

9 A key challenge is that there are few mechanisms to determine accessible areas of a city a priori The National Council on Disability noted that there is no comprehensive information on the degree to which sidewalks are accessible in cities. National Council on Disability, 2007 The impact of the Americans with Disabilities Act: Assessing the progress toward achieving the goals of the ADA There are many approaches for data collection but they typically require onsite reporting, which limits scalability 9

10 Accessibility data collection Traditional accessibility audits Walkability Audit Wake County, North Carolina Walkability Audit Wake County, North Carolina Safe Routes to School Walkability Audit Rock Hill, South Carolina Accessibility Data Collection 311 Systems Accessibility data collection Mobile reporting solutions 10

11 Accessibility data collection Mobile reporting solutions The NYC311 app has a specific option for broken sidewalks Accessibility data collection Reporting on accessibility of places g g 11

12 Accessibility data collection Reporting on accessibility of places rg org These are fantastic and important tools but they focus on the accessibility of places rather than the accessibility of how to get to places. Accessibility data collection Reporting on accessibility of places rg org Moreover, they require onsite access of a place to submit a report. We are pursuing a complementary two-fold approach 12

13 Surface Problems 1 To develop scalable methods that mine massive repositories of online map imagery to identify accessibility problems semiautomatically No Curb Ramps Physical Obstacles 13

14 Physical Obstacles 2 To create new accessibility-aware mapping tools that support people with disabilities and provide unprecedented views of urban accessibility Mouse over neighborhoods to view access score 14

15 Personalize score based on mobility level Upping severity of surface problems 15

16 Mapping The Accessibility of the World Two Focus Areas 1 Scalable Data Collection Methods [ASSETS 12, CHI 13, HCOMP 13, ASSETS 13, UIST 14, TACCESS 15, ASSETS 17] 2 New accessibility-aware mapping tools [SIGACCESS 15, CHI 16] Mapping The Accessibility of the World Key Research Questions 1 Scalable Data Collection Methods [ASSETS 12, CHI 13, HCOMP 13, ASSETS 13, UIST 14, TACCESS 15, ASSETS 17] Is online map imagery a good source for accessibility data? Can we create interactive tools that enable crowd workers to find accessibility problems? How can we leverage computational techniques to scale our approach? Mapping The Accessibility of the World Key Research Questions 1 Scalable Data Collection Methods [ASSETS 12, CHI 13, HCOMP 13, ASSETS 13, UIST 14, TACCESS 15, ASSETS 17] Is online map imagery a good source for accessibility data? Can we create interactive tools that enable crowd workers to find accessibility problems? How can we leverage computational techniques to scale our approach? 16

17 Google Streetview Real World Put another way: How well do accessibility problems found in Google Street View correspond with the real world? Is Google Street view a reasonable Dataset for accessibility audits? Physical Audits vs. Google Street View 179 Bus stops Washington DC & Seattle 42 km surveyed 273 intersections Washington DC & Baltimore 34 km surveyed 17

18 Is Google Street view a reasonable Dataset for accessibility audits? Comparison Results: Spearman rank coefficients Bus Stops Intersections vs. vs. Physical Audit Data GSV Audit Data ρ= 0.88 Physical Audit Data GSV Audit Data ρ= 0.98 All results statistically significant at p < Is Google Street view a reasonable Dataset for accessibility audits? Consistent With Findings in literature See: Odgers et al., 2012; Wilson et al., 2013; Kelly et al., 2013; Bader, et al., 2017 Is Google Street view a reasonable Dataset for accessibility audits? City infrastructure changes slowly Avg image age in Bus Stop Dataset 1.7 yrs (SD=0.7) Avg image age in Intersection Dataset 1.5 yrs (SD=0.7) 18

19 Google Street View is a reasonable proxy for studying the state of street-level accessibility 19

20 Mapping The Accessibility of the World Key Research Questions 1 Scalable Data Collection Methods [ASSETS 12, CHI 13, HCOMP 13, ASSETS 13, UIST 14, TACCESS 15, ASSETS 17] Is online map imagery a good source for accessibility data? Can we create interactive tools that enable crowd workers to find accessibility problems? How can we leverage computational techniques to scale our approach? Mapping The Accessibility of the World Key Research Questions 1 Scalable Data Collection Methods [ASSETS 12, CHI 13, HCOMP 13, ASSETS 13, UIST 14, TACCESS 15, ASSETS 17] Is online map imagery a good source for accessibility data? Can we create interactive tools that enable crowd workers to find accessibility problems? How can we leverage computational techniques to scale our approach? Crowdsourcing Accessibility Audits Initial Crowdsourcing System Labeling Interface Verification Interface 20

21 Crowdsourcing Accessibility Audits Web-based Labeling Interface 4-Step Process 1. Find & label problem Crowdsourcing Accessibility Audits Web-based Labeling Interface 4-Step Process 1. Find & label problem Crowdsourcing Accessibility Audits Web-based Labeling Interface 4-Step Process 1. Find & label problem 2. Categorize problem 21

22 Crowdsourcing Accessibility Audits Web-based Labeling Interface 4-Step Process 1. Find & label problem 2. Categorize problem Crowdsourcing Accessibility Audits Web-based Labeling Interface 4-Step Process 1. Find & label problem 2. Categorize problem 3. Rate problem severity Crowdsourcing Accessibility Audits Web-based Labeling Interface 4-Step Process 1. Find & label problem 2. Categorize problem 3. Rate problem severity 22

23 Crowdsourcing Accessibility Audits Web-based Labeling Interface 4-Step Process 1. Find & label problem 2. Categorize problem 3. Rate problem severity 4. Submit work Crowdsourcing Accessibility Audits Web-based Labeling Interface 4-Step Process 1. Find & label problem 2. Categorize problem 3. Rate problem severity 4. Submit work Receive another image to label & process repeats. Crowdsourcing Accessibility Audits Web-Based Verification Interface 3-Step Process 1. Verify label 23

24 Crowdsourcing Accessibility Audits Web-Based Verification Interface 3-Step Process 1. Verify label Crowdsourcing Accessibility Audits Web-Based Verification Interface 3-Step Process 1. Verify label 2. Verify rating Crowdsourcing Accessibility Audits Web-Based Verification Interface 3-Step Process 1. Verify label 2. Verify rating 3. Provide details 24

25 Crowdsourcing Accessibility Audits Web-Based Verification Interface 3-Step Process 1. Verify label 2. Verify rating 3. Provide details Check for false negatives Crowdsourcing Accessibility Audits Study Method 1. Create image dataset 2. Generate ground truth labels 3. Deploy our tools to crowd 4. Compare performance to ground truth 1 Crowdsourcing Accessibility Study Method Downloaded 229 GSV Images New York Baltimore Baltimore Washington DC Los Angeles 25

26 50 images Sidewalk Ending 66 images Object in Path 67 images Surface Problems 47 images Missing Curb Ramps 50 images No Problems Crowdsourcing Accessibility Audits Study Method 1. Create image dataset 2. Generate ground truth labels 26

27 2 Crowdsourcing Accessibility Study Method Create Ground Truth Labels Object in Path Bob Sue Alice Object in Path No Curb Ramp Bob s Labels Object in Path Object in Path Sue s Labels Object in Path Object in Path No Curb Ramp Alice s Labels Majority Vote } Object in Path Object in Path No Curb Ramp Researcher Ground Truth Crowdsourcing Accessibility Audits Study Method 1. Create image dataset 2. Generate ground truth labels 3. Deploy our tools to crowd 3 Crowdsourcing Accessibility Study Method Deploy Tools to Mechanical Turk 27

28 Crowdsourcing Accessibility Study Results Mturk Study Statistics 185 Labelers 7,517 Labels 35.2s Label An Image 273 Verifiers 19,189 Verifications 10.5s Verify An Image Crowdsourcing Accessibility Study Results Mturk Study Statistics 185 Labelers 7,517 Labels 35.2s Label An Image 273 Verifiers 19,189 Verifications 10.5s Verify An Image Crowdsourcing Accessibility Audits Study Method 1. Create image dataset 2. Generate ground truth labels 3. Deploy our tools to crowd 4. Compare performance to ground truth 28

29 Are crowd workers capable of finding accessibility problems in online map imagery? Crowdsourcing Accessibility Study Results Overall Labeling Accuracy With one labeler per image Sidewalk Ending 85% Crowdsourcing Accessibility Study Results Overall Labeling Accuracy With one labeler per image Sidewalk Ending Missing Curb Ramps Surface Problem Object in Path 85% 79% 77% 73% 29

30 Crowdsourcing Accessibility Study Results Overall Labeling Accuracy With one labeler per image Average Overall Accuracy 78% 81% Sidewalk Ending 85% Missing Curb Ramps Surface Problem Object in Path 79% 77% 73% Multiclass Overall Sidewalk Ending No Curb Ramp Surface Problem Object in Path No Problem Binary Overall Problem No Problem Crowdsourcing Accessibility Study Results Common Labeler Mistakes Over labeling (i.e., tendency towards false positives) Crowdsourcing Accessibility Study Results Common Labeler Mistakes Over labeling (i.e., tendency towards false positives) Random Labels Category errors (e.g., misunderstanding, malevolence) (i.e., ambiguous problem category) 30

31 Average Accuracy (%) Average Accuracy (%) Average Accuracy (%) 5/14/2018 Crowdsourcing Accessibility Study Results Accuracy as a function of Labelers Per Image 100% 90% 80% 70% 60% 50% 1 turker 1 labeler (N=28) 3 turkers 3 labelers (N=9) 5 turkers 5 labelers (N=5) 7 turkers 7 labelers (N=4) 9 turkers 9 labelers (N=3) (majority vote) (majority vote) (majority vote) (majority vote) Error bars: standard error Crowdsourcing Accessibility Study Results Accuracy as a function of Labelers Per Image 100% 90% 80% Multiclas s 78% 84% 87% 87% 88% 70% 60% 50% 1 turker 1 labeler (N=28) 3 turkers 3 labelers (N=9) 5 turkers 5 labelers (N=5) 7 turkers 7 labelers (N=4) 9 turkers 9 labelers (N=3) (majority vote) (majority vote) (majority vote) (majority vote) Error bars: standard error Crowdsourcing Accessibility Study Results Accuracy as a function of Labelers Per Image 100% 90% 80% Multiclas s 81% 78% Binar y 87% 84% 90% 91% 90% 87% 87% 88% 70% 60% 50% 1 turker 1 labeler (N=28) 3 turkers 3 labelers (N=9) 5 turkers 5 labelers (N=5) 7 turkers 7 labelers (N=4) 9 turkers 9 labelers (N=3) (majority vote) (majority vote) (majority vote) (majority vote) Error bars: standard error 31

32 Average Accuracy (%) 5/14/2018 Crowdsourcing Accessibility Study Results Accuracy With Crowd verification 100% 90% 80% 75% 81% 78% 88% 89% 80% 93% Multiclas s 82% 82% Binar y 91% 70% 60% 50% 1 labeler 1 labeler, 3 verifiers Time COST Error bars: standard error; experiments run on subset of data 3 labelers 3 labelers, 3 verifiers 5 labelers With basic quality control measures, minimally trained crowd workers can find accessibility problems with an accuracy of ~93% But this approach relied purely on manual labor. Can we do better? 32

33 Mapping The Accessibility of the World Key Research Questions 1 Scalable Data Collection Methods [ASSETS 12, CHI 13, HCOMP 13, ASSETS 13, UIST 14, TACCESS 15, ASSETS 17] Is online map imagery a good source for accessibility data? Can we create interactive tools that enable crowd workers to find accessibility problems? How can we leverage computational techniques to scale our approach? Tohme 遠目 Remote Eye Tohme 遠目 Remote Eye 1 svcrawl Web Scraper 33

34 Tohme 遠目 Remote Eye 1 svcrawl Web Scraper Google Street View Panoramas 3D Point-cloud Data Top-down Google Maps Imagery 2 Street View images 3D-depth maps GIS Metadata Top-down map images <Latitude & longitude/> GIS metadata <GSV image age/> Street Dataset <Street & city names/> <Intersection topology/> Scraped Area: 11.3 km 2 Tohme 遠目 Remote Eye Scraped Area: 11.3 km 2 Urban Residenti al 1 svcrawl Web Scraper D.C. Baltimore Los Angeles Saskatoon Dataset Statistics 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 1,086 intersections 2,877 curb ramps 647 missing curb ramps 2.2 yrs (SD=1.3) average GSV image age Tohme 遠目 Remote Eye 1 svcrawl 3 svdetect Web Scraper Automatic Curb Ramp Detection 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 34

35 Tohme 遠目 Remote Eye 1 svcrawl 3 svdetect Web Scraper Automatic Curb Ramp Detection 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset Tohme 遠目 Remote Eye 1 svcrawl 3 svdetect Web Scraper Automatic Curb Ramp Detection 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset True Positive Tohme 遠目 Remote Eye 1 svcrawl 3 svdetect Web Scraper Automatic Curb Ramp Detection False Positive 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset True Positive 35

36 Tohme 遠目 Remote Eye 1 svcrawl 3 svdetect Web Scraper Automatic Curb Ramp Detection False Negative False Positive 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset True Positive Tohme 遠目 Remote Eye 1 svcrawl 3 Web Scraper svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification Predicted CV success 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 svcontrol Automatic Task Allocation Tohme 遠目 Remote Eye 1 svcrawl 3 Web Scraper svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification Predicted CV success 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 Predicted CV failure svcontrol Automatic Task Allocation svlabel 6 Crowd Labeling 36

37 Tohme 遠目 Remote Eye 1 svcrawl 3 Web Scraper svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 svcontrol Automatic Task Allocation Tohme 遠目 Remote Eye 1 svcrawl 3 Web Scraper svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification Predicted CV success 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 svcontrol Automatic Task Allocation Tohme 遠目 Remote Eye 1 svcrawl 3 Web Scraper svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification Predicted CV success 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 svcontrol Automatic Task Allocation 37

38 Tohme 遠目 Remote Eye 1 svcrawl 3 Web Scraper svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 svcontrol Automatic Task Allocation svlabel 6 Crowd Labeling Tohme 遠目 Remote Eye 1 svcrawl 3 Web Scraper svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 Predicted CV failure svcontrol Automatic Task Allocation svlabel 6 Crowd Labeling Tohme 遠目 Remote Eye 1 svcrawl 3 Web Scraper svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 Predicted CV failure svcontrol Automatic Task Allocation svlabel 6 Crowd Labeling 38

39 Tohme 遠目 Remote Eye 1 svcrawl 3 Web Scraper svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 Predicted CV failure svcontrol Automatic Task Allocation svlabel 6 Crowd Labeling Tohme 遠目 Remote Eye 1 svcrawl 3 Web Scraper svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 svcontrol Automatic Task Allocation svlabel 6 Crowd Labeling Verifiers cannot fix false negatives (i.e., they cannot add new labels) es ages 39

40 Tohme 遠目 Remote Eye 1 svcrawl Web Scraper 3 svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 svcontrol Automatic Task Allocation svlabel 6 Crowd Labeling Tohme 遠目 Remote Eye 3 svdetect Automatic Curb Ramp Detection 1. Deformable part model (DPM) 2. Post-processing DPM 3. SVM-based classifier 1 Automatic curb ramp detector Deformable Part Model Root filter Parts filter Displacement cost Root filter Parts filter Displacement cost Felzenszwalb et al., CVPR 08, CVPR 10 40

41 1 Automatic curb ramp detector Deformable Part Model Root filter Parts filter Displacement cost 1 Automatic curb ramp detector Deformable Part Model True Positives 1 False Positives 12 False Negatives 0 1 Automatic curb ramp detector Deformable Part Model Curb ramps Detected In Sky & ON Roofs Multiple redundant detection boxes True Positives 1 False Positives 12 False Negatives 0 41

42 Automatic curb ramp detector 2 Post-Process DPM Output 3d-Point Cloud to remove curb ramps above ground 2 Automatic curb ramp detector Post-Process DPM Output Non-Maximum Suppression to remove Overlapping detections True Positives 1 False Positives 12 False Negatives 0 2 Automatic curb ramp detector Post-Process DPM Output True Positives 1 False Positives 5 False Negatives 0 42

43 3 Automatic curb ramp detector SVM-Based Refinement SVM Filters detections based on size, color, & Position in scene True Positives 1 False Positives 5 False Negatives 0 3 Automatic curb ramp detector Final Output True Positives 1 False Positives 3 False Negatives 0 1 Automatic curb ramp detector DPM Output True Positives 6 False Positives 11 False Negatives 1 43

44 3 Automatic curb ramp detector Final Output True Positives 6 False Positives 4 False Negatives 1 3 Automatic curb ramp detector Final Output False Negative (Hard to correct) True Positives 6 False Positives 4 False Negatives 1 False Positives (EASY TO CORRECT) Tohme 遠目 Remote Eye 1 svcrawl Web Scraper 3 svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 svcontrol Automatic Task Allocation svlabel 6 Crowd Labeling 44

45 Smart Task Allocator SVM trained with 23 input features Binary classifier trained to predict occurrence of false negatives from svdetect stage Curb Ramp Detector Output (16 Features) Raw # of bounding boxes Descriptive stats of confidence scores Descriptive stats of XYcoordinates 4 svcontrol Automatic Task Allocation 3D-Point Cloud Data (5 Features) Intersection Complexity (2 Features) Descriptive stats of depth information (e.g., average, median, variance) of pixel depth Cardinality (# of connected streets) Amount of road Tohme 遠目 Remote Eye 1 svcrawl Web Scraper 3 svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification Predicted CV success 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 Predicted CV failure svcontrol Automatic Task Allocation svlabel 6 Crowd Labeling Verification tool Crowd Interfaces Correct false positives from computer vision 45

46 Crowd Interfaces Verification tool Correct false positives from computer vision Crowd Interfaces Verification tool Correct false positives from computer vision Tohme 遠目 Remote Eye 1 svcrawl Web Scraper 3 svdetect Automatic Curb Ramp Detection 5 svverify Crowd Verification Predicted CV success 2 Street View images 3D-depth maps Top-down map images GIS metadata Street Dataset 4 Predicted CV failure svcontrol Automatic Task Allocation svlabel 6 Crowd Labeling 46

47 Crowd Interfaces Labeling Tool Crowd Interfaces Labeling Tool Crowd Interfaces Labeling Tool 47

48 Accuracy (%) Accuracy (%) Manual Labeling Automatic Detection & Manual Verification Automatic Task Allocation Manual Labeling Automatic Detection & Manual Verification Automatic Task Allocation 5/14/2018 Tohme Study Method 1. Generate ground truth labels 2. Train computer vision & task controller 3. Deploy Tohme to Mechanical Turk 4. Compare Tohme to baseline Tohme Evaluation Overall Results 100% Precision Recall F-measure 80% 60% 40% 20% 0% Manual Labeling CV + Verification Tohme System 100% bottom workflow 100% top workflow full tohme system Tohme Evaluation Overall Results 100% 80% 60% 84% 88% 86% 68% 58% Precision Recall F-measure 83% 86% 84% 63% 40% 20% 0% Manual Labeling CV + Verification Tohme System 100% bottom workflow 100% top workflow full tohme system 48

49 Accuracy (%) Manual Labeling Automatic Detection & Manual Verification Automatic Task Allocation 5/14/2018 Tohme Evaluation Overall Results 100% 80% 60% 40% 20% 0% 84% 88% 86% Manual Labeling 68% 58% 63% CV + Verification Precision Recall F-measure 83% 86% 84% Tohme System 100% bottom workflow 94s Per Scene 100% top workflow 42s Per Scene full tohme system 81s Per Scene Tohme Evaluation Task Controller Performance 1 svcrawl 3 Web Scraper Street View images 3D-depth maps Top-down map images GIS metadata 2 Street Dataset svdetect Automatic Curb Ramp Detection 4 svcontrol Automatic Task Allocation 5 svverify Crowd Verification svlabel 6 Crowd Labeling 80% Scenes Correctly routed 50% Scenes Correctly routed Tohme Evaluation Simulated Perfect Task Controller rb on 5 svverify Crowd Verification Simulated perfect task controller 100% Scenes Correctly routed 100% Scenes Correctly routed Overall Speedup Increases Over Manual Baseline 14% 27% Speedup Speedup 4 svcontrol Automatic Task Allocation svlabel 6 Crowd Labeling 49

50 Improving Detection algorithms Automatic detection is hard Occlusion Illumination Viewpoint Variation Structures Similar to Curb Ramps Scale Curb Ramp Design Variation Improving Detection algorithms Applying Convolutional Neural Networks Recently published at CVPR 17 Input image Context map 50

51 51

52 52

53 53

54 Project Sidewalk Sidewalk Contributions Users 1,075 miles 253,414 Labels Project sidewalk Preliminary accuracy analysis Project sidewalk Preliminary accuracy analysis: By label type 90% 76% 50% 34% 56% 54

55 Project sidewalk Preliminary accuracy analysis: By label type 90% 76% 50% 34% 56% 8.1s 12.8s 10.1s 13.7s Project sidewalk Preliminary accuracy analysis: By Zone Project sidewalk Preliminary accuracy analysis: By Labeling speed 55

56 Current & Future work New Hybrid Workflows & interfaces Current & Future work New Hybrid Workflows & interfaces Future Work Tracking Accessibility infrastructure over time Sept 2007 Jul 2009 May 2011 June 2011 May 2014 Aug 2014 Nov

57 Future Work Tracking Accessibility infrastructure over time Sept 2007 Jul 2009 June 2011 May 2014 July 2015 Manual Label Project Sidewalk Novel Assistive Technology Applications DC vs NYC New models & viz of city accessibility Smart routing for people with impairments Cross-city comparison tools 57

58 ACknowledgements Funding Sources NSF # , Google, IBM PI Froehlich, Co-PI David Jacobs Mapping accessibility via Google Associate Professor Computer Science University of Washington View Thank You! Mid-Atlantic ADA Center Toll Free: (DC, DE, MD, PA, VA, WV) Telephone: