Pervasive Indoor Localization and Tracking Based on Fingerprinting Gary Chan Professor, CSE HKUST 2
Catchphrase: Location, Location, Location! 3
Outdoor Location-Based Services (LBS) Based on GPS (Global Positioning System) with satellites Widely and successfully commercialized Navigation, places of interest, hotels, parks, gas stations, etc. 4
Indoor Location-Based Services (LBS) Even larger and more lucrative market: more than 70% of our time is spent indoors Don Dodge, developer advocate at Google 5
Market Opportunities A multi-billions market Malls, super-marts or exhibition centers Help customers to find items, shops or booths Location-based recommendations, mobile advertisement and marketing Enable O2O (online-to-offline) business Track people and crowd/flow analysis Customer behavior Airport Help tourists to find their gates for flights Shop while waiting for flights Social applications: finding nearby people sharing the same interest Location-based games (e.g., treasure hunt) Track assets Health Track people, patients, etc.
Indoor Positioning: Localization and Tracking Localization: Where am I? Tracking: Where are you? User experience depends largely on the accuracy of the technique! 7
Challenges of Indoor GPS Global Positioning System (GPS) simply does not work indoor! Line-of-sight not easily achievable indoor Complicated indoor environment leads to complex signal fading, shadowing and interference High requirements Cost-effective to deploy High accuracy (~5m) Computationally light-weighted Power and memory efficient Privacy, etc. 8
Application Venues Malls are the most mapped venues around the world Airports come second Growing works on advertising and analytics to target customers in these venues Airports 13% Other 8% 4th Qtr 0% Commerce 79% 9
Wherami: The HKUST Indoor Positioning Technology 10
Turning Research to Commercialization Wherami Tech. Transfer Government ITC LSCM Innovation and Technology Funds (ITF) Technology Transfer Center, Department and School of Engineering at HKUST Advices and support on contractual agreements R&D facilities and human resources Industry sponsors and collaborators Funding, technical support, trial sites, equipment, etc 11
Use of Wi-Fi for Positioning Pervasive deployment In HKUST atrium, we can detect more than 400 AP hot spots (HKUST 2013-08-08) The dominant localization technology by 2015 Easy and cost-effective to deploy (source: ABI research indoor location report) Other (LED...) 10% Bluetooth 4.0 15% Wi-Fi (Possily combined with sensor fusion, Cell or other) 75% 12
Wherami: Pervasive Indoor Positioning Supported by ITC LSCM and industries Smart guide on smart-phones Beyond just digital map Indoor navigation: How can I get there? Are any good promotions being offered around me? Are my friends around me? Where are my kids? Where are my customers? What are their behaviors? 13
Fingerprint Concept Each person has a unique fingerprint Stored in a database Given a fingerprint Match it to the database Find the person Wi-Fi fingerprinting Fingerprint = Wi-Fi signals Person = Physical location 14
Wi-Fi Signals Collected at a Location 15
The Wi-Fi Signals Form a Unique Fingerprint for the Location
Site Survey and Fingerprint Collection
User Location Calculation
Research Challenge: Signal Measurement Noise
Stochastic Optimization under Noisy Signals Minimize the overall expected signal difference in signal map Subject to constraints on possible regions the target may be in minimize i N m Δ p, q i ω i subject to x = i ω i r i, r i C m, x x l d l, l Λ i N m ω i = 1, 0 ω i 1
Client-based Localization App-based Android + ios Non-intrusive and transparent to existing WiFi infrastructure No string attached besides your phones
Integration with Other Signals Bluetooth (BLE)/iBeacon Dead reckoning/ins (inertial navigation system) Step counter, gyroscope, accelerometer, etc. Magnetic field GPS/AGPS/NLP (Network location provider) Barometer Ultrasound RFID Etc. Beacon 1 Beacon 2 Beacon 3 Beacon 4 22
BLE (Bluetooth Low Energy) integration Good improvements on the area where WiFi signals are weak Case I do nothing 23 Case II Restrict the estimation
Automatic Fingerprint Updating for Survey Reduction Using Implicit crowdsourcing and Gaussian process regression (GPR) The samples has a joint Gaussian distribution Regression function v = f l + ε Specified by mean and covariance function Joint Gaussian v N(m L, cov v ) f l ~ GP(m l, k(l, l )) cov v = K + σ n 2 I ε N(0, σ n 2 ) f v N( m(l ) m L, k(l, l ) k l, L T k(l, L) K + σ n 2 I ) f L, v, l N(μ, σ 2 ) 24
AP-Based Solution for User Tracking 25
Design Dimensions Without infrastructure installation Infrastructure outlay (APs and ibeacons) No client app installation User tracking and analytics Client-based apps User localization and navigation Better user positioning accuracy and analytics 26
AP-based Tracking vs Clientbased Localization Device Capability AP-based Any Wi-Fi devices (including ios, Andriod, Wi-Fi tag, Windows, etc) Client-based Require Wi-Fi API Need ibeacon to localize iphones Tracking Transparency Transparency to client side Client app interacts with the users Accuracy Improvement Installation of more APs Antenna improvement Fusion of different signals Computation Server Client, hence the device can be offline Positioning interval Seconds Real time (less than a second)
Wherami Technology Features Driven by system research and advanced development Full solution suite for user positioning AP-based tracking Client-based localization Joint AP-client localization to further improve accuracy Indoor-outdoor and multi-storey roaming (3D routing) Low cost, easy and fast to deploy Accurate and low computation Precision 2m ~ 10m Low site survey cost Adaptive to signal changes Ready to be integrated into other applications Location API is provided 28
Main Functions Real-time positioning Real-time navigation Destination search Best path selection Moving trajectory Panoramic visual clue Targeted services Nearby facility search Program search Event/Ad prompt 29
Shopping Mart or Superstore Aim for better shopping experience in large area with many aisles Location-based promotions For bargain hunters For shoppers with a shopping list Targeted advertisement (behavior analysis) 30
Shopping Mall: Pushed Ads 31
Hospital, Out-Patients, Kids and Elderly home Smart Wrist Band Bluetooth or Wi-Fi enabled Detect the heart rate, step count, sleep status, etc Tracking by SmartAP Track and trace the location of patient/elderly Real time monitor and report the health condition
Context-aware Computing or Smart Space: Museum and Theme Parks 33
Location-based Games and Treasure Hunting 34
Intelligent Transportation
Deployment Trials: HKIA, Cyberport, Olympian City, Union Hospital, HKCEC, HKUST, etc. 36
Hong Kong International Airport Where is the check-in counter for Airline X? How to get to Gate 24? The closest restaurant? Push ads: there is a discount in the coffee shop nearby! 37
HKIA trials Checkin asile K, J, H Gate 20~26 Meeting Hall 38
Trials in HKIA 39
How we conduct trials 3. WiFi data analysis and pre-process 1. Identify reference points 2. Collect WiFi fingerprinting 40
Demonstration (youtube video) 41
Cyberport Trial Trial area: 1. The Arcade whole building 2. Cyberport 1 floor L 3. Cyberport 2 floor L 4. Cyperport 3 floor L3 and L4 42
LBS in Cyberport Location game: treasure hunt Panoramic Visual Clue Location advertisement 43
Trial at Olympian City 44
Hospital Where is the Pathology Centre? Where is the nearest washroom or food court? 45
Hong Kong Convention and Exhibition Centre Where is the xyz company booth? Are there any forum discussions this afternoon and where are they held? Where is the 3D Printing Solutions zone? Where is my friend? 46
HKUST Campus Trial area: 1. Academic Building 2. Lee Shau Kee Business Building 3. Institute of Advanced Study Building 47
HKUST Deployment Where is classroom for Course X or Seminar Y? How to get to Room Z? My friend shares his position to me I am going for a discussion with him! 48
Wherami main functions Localization Navigation Content Search 49
Offline Map Navigation Target Search Augmented Panoramic View Location-based Ads Course/Flight Search Seminar/Promotion Events Promotion games (e.g. treasure hunt) Outpatient Procedure Flow Real Time Queue Status Patient Identity Verification Real Time Flow Control Wherami Trial Venues Core Features Value Added Features Wherami Trial Venue University Campus HKUST Shopping Mall Cyberport and Olympian City Airport HKIA Hospital Union Hospital Exhibition HKCEC 50
We are Happy to Demonstrate Our Technology at Your Sites Visit us at: http://mwnet.cse.ust.hk/wherami http://www.cse.ust.hk/~gchan 51