A GI Science Perspective on Geocoding:

Transcription

1 A GI Science Perspective on Geocoding: Accuracy, Repeatability and Implications for Geospatial Privacy Paul A Zandbergen Department of Geography University of New Mexico

2 Geocoding as an Example of Applied GI Science Why geocoding? Arguably the most successful application of GISc In very widespread use Fundamental step in many types of spatial analysis Often considered as relatively easy and largely error-free Big questions: How good is current geocoding? What are the minimum quality expectations for geocoding? What are the effects of errors in geocoding on spatial analysis? What are the (unintended) consequences of widespread, highquality geocoding? How can geocoding be improved?

3

4 Types of Geocoding County ZIP code (ZIP and ZIP+2/4) Street Parcel Address point Routes Types of geocoding will vary by country and region, but in the United States street geocoding is by far the most widely used, both for research and commercial applications

5 What s Behind the Geocoding Process? Probabilistic Record Linkage Standardization Soundex Well-established algorithms, dating back to early efforts by US Census (DIME, TIGER) Ongoing research into modifications, including Markov-chain models

6 How Does Street Geocoding Work? End Offset (%) 747 Main St 701 Side Offset Find the zone (ZIP, City, etc.) 2. Match the street ( by Name, Type, Dir, etc.) 3. Match the segment with the proper range 4. Linear interpolation along segment 5. Apply offsets

7 Typical Errors Spelling issues Incomplete street address Prefix, suffix, direction conflicts Apartment and unit numbers Ambiguous street names Outdated/incomplete street reference data Well documented types of errors

8 Framework for Geocoding quality 1. Completeness The percentage of records that could be reliably matched 2. Positional accuracy The difference between the geocoded location and the true location 3. Repeatability/robustness Agreement between results from repeated geocoding

9 What do we know? Completeness Surprisingly little research Most research makes ad-hoc decisions on what is acceptable Positional accuracy Errors can be substantial, but distribution not well characterized Larger error in rural areas Repeatability/robustness Limited research Effect of different geocoding algorithms appears limited Quality of street reference data the determining factor Potential trade- offs: e.g. increase match rate while sacrificing positional accuracy

10 Research Case Studies Data sets School children and schools in Orlando County Sex offenders in Florida Banks and grocery stores in Florida Fishing and boat license holders in Florida Analyses Positional accuracy Repeatability Effects of positional error on spatial analysis Comparison of alternatives

11 Maple Street NE

12

13 Left range: Right range:

14

15

16 441 Maple St NE

17 GoogleMaps: 441 Maple St NE

18 MapQuest: 441 Maple St NE

19 Error Measurement Tool Fishbone Tool

20 Results get messy sometimes

21 The Squeeze Effect

22 Typical Error Distribution Cumulative Frequency (%) N =104,865 school children in Orange County, FL ,000 Positional Error (m) Zandbergen, P.A Influence of geocoding quality on environmental exposure assessment of children living near high traffic roads. BMC Public Health, 7:37.

23 Typical Error Estimates Statistic Value (m) Log-normalized Q-Q plot Min ~0 4 Max 32,356 Mean 66 2 SD Median 90 th % 95 th % Expected Normal th % N , Observed Value Based on the 90 th percentile, typical street geocoding does not meet the accuracy standards for a 1:100,000 scale map based on National Map Accuracy Standards! Zandbergen, P.A Influence of geocoding quality on environmental exposure assessment of children living near high traffic roads. BMC Public Health, 7:37.

24 Quality of Street Reference Data

25 Use of Offsets Small offsets are commonly used Very minor effect on positional accuracy Optimum ~25 meters

26 Effect of Density and Parcel Size Cumulative Number of Locations (%) Urban Residential - Parcel < 1 acre Urban Residential - Parcel > 1 acre Rural Residential - Parcel < 1 acre Rural Residential - Parcel > 1 acre Positional Error (feet) Zandbergen, P.A. Spatial variability in the positional accuracy of street geocoding. International Journal of Geographical Information Science (under review).

27 Repeatability How do geocoding results (match rate and positional accuracy) vary by: Geocoding algorithms Street reference data Quality of street reference data is the most significant factor Local street centerlines and/or E-911data are typically superior How do commercial firms compare to results from GIS Analysts? Again, quality of street reference data dominates, which for most commercial firms is good (unless they use TIGER 2000 data) Substantial variability among providers No relationship between cost and quality

28 Volusia County, FL 4 land use types 3 street datasets Zandbergen, P.A. Repeatability of street geocoding. Computers and Geosciences (under review).

29 Example: Sex Offender Residency Restrictions Zandbergen, P.A. and T.C. Hart Reducing housing options for convicted sex offenders: Investigating the impact of residency restriction laws using GIS. Justice Research and Policy, 8:1-24.

30

31 Errors in Classification Street geocoding of schools and offenders In Out Total Parcel geocoding of schools and offenders In Out Total False positives False negatives

32 Mapping Residency Restrictions: Parcel-level Zandbergen, P.A. and T.C. Hart Reducing housing options for convicted sex offenders: Investigating the impact of residency restriction laws using GIS. Justice Research and Policy, 8:1-24.

33 Example: Schools and School Children in Proximity to High Traffic Roads

34

35 Error and Bias in Analysis Results Zandbergen, P.A Influence of geocoding quality on environmental exposure assessment of children living near high traffic roads. BMC Public Health, 7:37.

36

37 Alternatives to Street Geocoding Street geocoding is dominant in the US Techniques used in other jurisdictions: Postal code (Canada) Address points (Australia and UK) Two alternatives for the US: Parcel boundaries Address points Both produce exceptional positional accuracy, but application currently limited by: Data availability / consistency Unproven match rates

38

39

40

41

42 Match rate results Match rates for different databases for Bay County, Fl Address Parcel Roads Commercial Banks Daycares Elevators Fishing Licenses Grocery Stores Sex Offenders Zandbergen, P.A. A comparison of address point, parcel and street geocoding techniques. Computers, Environment and Urban Systems (under review).

43 Error Propagation Modeling Not very well developed, for something as simple and common as street geocoding Monte-Carlo simulation of variability in: Match rate Positional accuracy Determine effects of error/bias on: Clustering Proximity analysis Neighborhood assignment Contribute to standards and procedures for geocoding In development.

44 Geospatial Privacy Collection of individual level data is growing rapidly in public and private sector Analysis of the individual level is very attractive for researches (overcomes MAUP and ecological fallacy issues) Spatial identifiers have not received the same level of concern as individual identifiers (like name) As geocoding becomes easier, cheaper and more accurate, so does the ability to reverse geocode spatial data Geospatial privacy has been recognized in the literature, but limited formal requirements or guidelines exist that recognize the increasing availability and capabilities of geospatial tools

45 Reverse Geocoding

46 Geographic Masking Traditional approach to protect privacy is through geographic masking typically a random perturbation Masking may alter data in undesirable ways effects of masking on spatial analysis has not received a lot of attention How much masking is needed to adequately protect geospatial privacy is poorly understood

47

48 Tracking Technologies Old-fashioned radio telemetry Traditional GPS GPS enabled cellphones Hybrid GPS / Wi-Fi Widespread adoption of (real-time) tracking is eminent

49 GPS-enabled Cellphones

50 Real-time Tracking

51

52 Google Earth Plug-ins

53

54

55

56 Real-time Online Tracking

57 Future Developments Address point geocoding will grow in importance and become the standard in the US Individual-level data will become a more widespread unit of analysis Tracking technologies will start to make a major impact Geospatial privacy protection needs to catch up to current trends

58 Elements of my research agenda Ongoing: Geocoding quality (completeness, positional accuracy, repeatability) Error propagation modeling Reverse geocoding and geographic masking Reliability of tracking technologies (i.e. cellphones vs. WiFi vs. GPS) and implications for geocoding Planned for: Spatial-temporal geocoding

59 Spatial-temporal Geocoding Where people spend their time not just where their mailing address is. Short-term: Where people live, work, shop etc. How people spend their day/week Long-term: Migration over years/decades How people spend their life Applications: transportation, emergency management, urban planning, environmental health How do we do this technically, practically and ethically?