What to do with 500M Location Requests a Day?

Size: px

Start display at page:

Download "What to do with 500M Location Requests a Day?"

Beverly Reeves
6 years ago
Views:

1 What to do with 500M Location Requests a Day? OGC Workshop Expanding GeoWeb to an Internet of Things May COM.Geo 2011 Kipp Jones Chief Architect Skyhook

2 Overview System Background Research Goals Activity SpotRank Live SpotRank Demographics Tile Based Motion Analysis Privacy Preserving Demograph

3 Skyhook Skyhook s Core Location combines signals from Wi-Fi, GPS and cell towers to quickly and accurately determine device location Over 250m beacons worldwide Deployed on over 100m devices 500m location request per day Select Customers Skyhook s Wi-Fi Database in Manhattan Select Customers

4 Research Background Phase I Initial Questions: What does aggregate user behavior tell us? About users? Groups? Activities? Events? Locations? Can we discern patterns in the data? How can this information be leveraged? Operational? Applications? Scale? Project Deliverables: Produce Baseline Activity Measure Examine Realtime Activity Behavior Explore business use cases

5 Activity Based Analysis Time / space based analysis 'heat' or aggregate request analysis based on different scales in 9me and space Frequency / phase domain (. etc find temporal pulses (hourly, weekly, daily, group like frequency and phase ac9vity areas can be applied at different spa9al dimensions Baseline / anomaly detec9on use training dataset to compute baseline and noise threshold for spa9al region run data against baseline and detect anomalies classify anomalies using above methods

6 Large Scale Events Control Sample Event Viewing and Impedance Clustering Recurring Affinity Cluster 'Control' day versus St. Patrick's Day Parade > 2.7x baseline control average over the area

7 Local Intensity Boston Sunday Yankees Stadium, No Game vs. Game Magenta = 1000 requests per square KM per day 10 monday games over 30 week sample

8 SpotRank Boston Marathon Real-time population density based on location requests Sample data from 1pm the day of the Boston Marathon and one week prior shows the day-to-day difference in population density

9 User Behavior

10 User Behavior

11 potrank Packaged Tile Size ~400m ~1000 Tiles in Manhattan ~4B Tiles cover the Earth

12 SATURDAY 2 AM MONDAY Daily Comparison: Red squares measure ac9vity. Comparison of 2AM ManhaUan on Saturday versus Monday.

13 equest Intensity Versus Variance Ac9vity and Consistency: Red squares measure ac9vity (request counts per hour per 9le); yellow circles measure variance. Where red dominates, usage is consistent for 9le- hour Monday 3AM Monday 6PM

14 Classification Emergence bursts South Sta9on aggregated use pattern where users emerge into a context of uncertainty Impedance clustering Accident on Mass Pike pattern where multiple users are trying to navigate around traffic blockages or unanticipated impediments Social affinity/tribal clustering Presiden9al Inaugura9on groups of users have gathered together voluntarily around or in anticipation of a cultural event Arterial accumula9on Commonwealth Ave. commuting pathways or pedestrian routes, generally occurring in temporal pulses Ins9tu9onal nuclea9on Longwood Medical usage clusters which have been identified occurring within the confines of academic campuses or hospital facilities

15 Frequency & Phase Analysis Midtown (48 th & 8 th ) Washington Square / Greenwich Village Houston

16 Tile Ac3vity Counts Midtown Houston Greenwich Village Feb 24, 2009 Sep 12, 2009 Raw request logs Bin requests by hour Discretize into 'tiles' 400m square Hourly requests per tile for 3 sample tiles

17 Frequency Analysis Frequency of.5 represents the maximum period we can analyze (½ the maximum sample of 1 hour) is 1/12 th the maximum frequency == 24 hours. Intensity in frequency domain Identify periodic patterns of activity Project future based on past dependable patterns Largest spike shows at the 24 hour cycle Fits intuition regarding daily cycle Not the only cycles that can be found

18 24- hour Frequency and Phase Left shows intensity at the 24 hour cycle All sample tiles show strong affinity for daily activity Right shows phase of peak Green tile peaks approximately 5 hours before the red tile every day Can help partition and classify tiles based on their phase

19 Tile Spectra Only one of our example tiles shows strong monthly periodicity Greenwich Village tile Pattern is consistent month after month

20 Business Use Cases Retail site selection Out of Home advertising (e.g. billboards) Civic planning Network planning Mobile & location based advertising Temporal location context Social guidance Traffic planning/adjusting

21 Research Background Phase II Initial Questions: What demographic information can be gleaned from aggregate anonymous location requests? Are there additional methods to enhance anonymous demographic inference? Project Deliverables: Design and Validate Privacy Preserving Demographic Model Test Model for Accuracy and Anonymity Identify system level data privacy mechanisms required to scale

22 Privacy Preserving Personalized Demographics Question: How to build up time based active demographics for individuals without violating privacy and anonymity? Goals: Preserve privacy and anonymity Derive demographics based on time and location Provide demographic information for personalization Produce SpotRank Demograph

23 Privacy Preserving Personalized Demographics Idea Never store unique ID with location Use tile based demographics with m-anonymity Aggregate time-based tile demographics per ID per time bucket Create active demographic cookie Can be stored client-side or server-side Produce usage based Spotrank Demograph based on aggregated cookie collection Plus Location is never stored for an individual Individual locations cannot be reversed out of active demographics Active time-based buckets can vary in size and number

24 Privacy Preserving Personalized Demographics Minus Derived demographics based on usage Base demographics are limiting factor Have to maintain ID:Demographic relationship Questions How to handle opt-in/out capability How to handle 3rd party access How to handle data retention Other concerns?

25 Contact Kipp Jones

LOCATION PRIVACY & TRAJECTORY PRIVACY. Elham Naghizade COMP20008 Elements of Data Processing 20 rd May 2016

LOCATION PRIVACY & TRAJECTORY PRIVACY Elham Naghizade COMP20008 Elements of Data Processing 20 rd May 2016 Part I TRAJECTORY DATA: BENEFITS & CONCERNS Ubiquity of Trajectory Data Location data being collected