Pervasive Indoor Localization and Tracking Based on Fingerprinting. Gary Chan Professor, CSE HKUST

Transcription

1

2 Pervasive Indoor Localization and Tracking Based on Fingerprinting Gary Chan Professor, CSE HKUST 2

3 Catchphrase: Location, Location, Location! 3

4 Outdoor Location-Based Services (LBS) Based on GPS (Global Positioning System) with satellites Widely and successfully commercialized Navigation, places of interest, hotels, parks, gas stations, etc. 4

5 Indoor Location-Based Services (LBS) Even larger and more lucrative market: more than 70% of our time is spent indoors Don Dodge, developer advocate at Google 5

6 Market Opportunities A multi-billions market Malls, super-marts or exhibition centers Help customers to find items, shops or booths Location-based recommendations, mobile advertisement and marketing Enable O2O (online-to-offline) business Track people and crowd/flow analysis Customer behavior Airport Help tourists to find their gates for flights Shop while waiting for flights Social applications: finding nearby people sharing the same interest Location-based games (e.g., treasure hunt) Track assets Health Track people, patients, etc.

7 Indoor Positioning: Localization and Tracking Localization: Where am I? Tracking: Where are you? User experience depends largely on the accuracy of the technique! 7

8 Challenges of Indoor GPS Global Positioning System (GPS) simply does not work indoor! Line-of-sight not easily achievable indoor Complicated indoor environment leads to complex signal fading, shadowing and interference High requirements Cost-effective to deploy High accuracy (~5m) Computationally light-weighted Power and memory efficient Privacy, etc. 8

9 Application Venues Malls are the most mapped venues around the world Airports come second Growing works on advertising and analytics to target customers in these venues Airports 13% Other 8% 4th Qtr 0% Commerce 79% 9

10 Wherami: The HKUST Indoor Positioning Technology 10

11 Turning Research to Commercialization Wherami Tech. Transfer Government ITC LSCM Innovation and Technology Funds (ITF) Technology Transfer Center, Department and School of Engineering at HKUST Advices and support on contractual agreements R&D facilities and human resources Industry sponsors and collaborators Funding, technical support, trial sites, equipment, etc 11

12 Use of Wi-Fi for Positioning Pervasive deployment In HKUST atrium, we can detect more than 400 AP hot spots (HKUST ) The dominant localization technology by 2015 Easy and cost-effective to deploy (source: ABI research indoor location report) Other (LED...) 10% Bluetooth % Wi-Fi (Possily combined with sensor fusion, Cell or other) 75% 12

13 Wherami: Pervasive Indoor Positioning Supported by ITC LSCM and industries Smart guide on smart-phones Beyond just digital map Indoor navigation: How can I get there? Are any good promotions being offered around me? Are my friends around me? Where are my kids? Where are my customers? What are their behaviors? 13

14 Fingerprint Concept Each person has a unique fingerprint Stored in a database Given a fingerprint Match it to the database Find the person Wi-Fi fingerprinting Fingerprint = Wi-Fi signals Person = Physical location 14

15 Wi-Fi Signals Collected at a Location 15

16 The Wi-Fi Signals Form a Unique Fingerprint for the Location

17 Site Survey and Fingerprint Collection

18 User Location Calculation

19 Research Challenge: Signal Measurement Noise

20 Stochastic Optimization under Noisy Signals Minimize the overall expected signal difference in signal map Subject to constraints on possible regions the target may be in minimize i N m Δ p, q i ω i subject to x = i ω i r i, r i C m, x x l d l, l Λ i N m ω i = 1, 0 ω i 1

21 Client-based Localization App-based Android + ios Non-intrusive and transparent to existing WiFi infrastructure No string attached besides your phones

22 Integration with Other Signals Bluetooth (BLE)/iBeacon Dead reckoning/ins (inertial navigation system) Step counter, gyroscope, accelerometer, etc. Magnetic field GPS/AGPS/NLP (Network location provider) Barometer Ultrasound RFID Etc. Beacon 1 Beacon 2 Beacon 3 Beacon 4 22

23 BLE (Bluetooth Low Energy) integration Good improvements on the area where WiFi signals are weak Case I do nothing 23 Case II Restrict the estimation

24 Automatic Fingerprint Updating for Survey Reduction Using Implicit crowdsourcing and Gaussian process regression (GPR) The samples has a joint Gaussian distribution Regression function v = f l + ε Specified by mean and covariance function Joint Gaussian v N(m L, cov v ) f l ~ GP(m l, k(l, l )) cov v = K + σ n 2 I ε N(0, σ n 2 ) f v N( m(l ) m L, k(l, l ) k l, L T k(l, L) K + σ n 2 I ) f L, v, l N(μ, σ 2 ) 24

25 AP-Based Solution for User Tracking 25

26 Design Dimensions Without infrastructure installation Infrastructure outlay (APs and ibeacons) No client app installation User tracking and analytics Client-based apps User localization and navigation Better user positioning accuracy and analytics 26

27 AP-based Tracking vs Clientbased Localization Device Capability AP-based Any Wi-Fi devices (including ios, Andriod, Wi-Fi tag, Windows, etc) Client-based Require Wi-Fi API Need ibeacon to localize iphones Tracking Transparency Transparency to client side Client app interacts with the users Accuracy Improvement Installation of more APs Antenna improvement Fusion of different signals Computation Server Client, hence the device can be offline Positioning interval Seconds Real time (less than a second)

28 Wherami Technology Features Driven by system research and advanced development Full solution suite for user positioning AP-based tracking Client-based localization Joint AP-client localization to further improve accuracy Indoor-outdoor and multi-storey roaming (3D routing) Low cost, easy and fast to deploy Accurate and low computation Precision 2m ~ 10m Low site survey cost Adaptive to signal changes Ready to be integrated into other applications Location API is provided 28

29 Main Functions Real-time positioning Real-time navigation Destination search Best path selection Moving trajectory Panoramic visual clue Targeted services Nearby facility search Program search Event/Ad prompt 29

30 Shopping Mart or Superstore Aim for better shopping experience in large area with many aisles Location-based promotions For bargain hunters For shoppers with a shopping list Targeted advertisement (behavior analysis) 30

31 Shopping Mall: Pushed Ads 31

32 Hospital, Out-Patients, Kids and Elderly home Smart Wrist Band Bluetooth or Wi-Fi enabled Detect the heart rate, step count, sleep status, etc Tracking by SmartAP Track and trace the location of patient/elderly Real time monitor and report the health condition

33 Context-aware Computing or Smart Space: Museum and Theme Parks 33

34 Location-based Games and Treasure Hunting 34

35 Intelligent Transportation

36 Deployment Trials: HKIA, Cyberport, Olympian City, Union Hospital, HKCEC, HKUST, etc. 36

37 Hong Kong International Airport Where is the check-in counter for Airline X? How to get to Gate 24? The closest restaurant? Push ads: there is a discount in the coffee shop nearby! 37

38 HKIA trials Checkin asile K, J, H Gate 20~26 Meeting Hall 38

39 Trials in HKIA 39

40 How we conduct trials 3. WiFi data analysis and pre-process 1. Identify reference points 2. Collect WiFi fingerprinting 40

41 Demonstration (youtube video) 41

42 Cyberport Trial Trial area: 1. The Arcade whole building 2. Cyberport 1 floor L 3. Cyberport 2 floor L 4. Cyperport 3 floor L3 and L4 42

43 LBS in Cyberport Location game: treasure hunt Panoramic Visual Clue Location advertisement 43

44 Trial at Olympian City 44

45 Hospital Where is the Pathology Centre? Where is the nearest washroom or food court? 45

46 Hong Kong Convention and Exhibition Centre Where is the xyz company booth? Are there any forum discussions this afternoon and where are they held? Where is the 3D Printing Solutions zone? Where is my friend? 46

47 HKUST Campus Trial area: 1. Academic Building 2. Lee Shau Kee Business Building 3. Institute of Advanced Study Building 47

48 HKUST Deployment Where is classroom for Course X or Seminar Y? How to get to Room Z? My friend shares his position to me I am going for a discussion with him! 48

49 Wherami main functions Localization Navigation Content Search 49

50 Offline Map Navigation Target Search Augmented Panoramic View Location-based Ads Course/Flight Search Seminar/Promotion Events Promotion games (e.g. treasure hunt) Outpatient Procedure Flow Real Time Queue Status Patient Identity Verification Real Time Flow Control Wherami Trial Venues Core Features Value Added Features Wherami Trial Venue University Campus HKUST Shopping Mall Cyberport and Olympian City Airport HKIA Hospital Union Hospital Exhibition HKCEC 50

51 We are Happy to Demonstrate Our Technology at Your Sites Visit us at: