Indoor Localization. Qian Zhang

Transcription

1 Indoor Localization Qian Zhang

2 Applications of Indoor Localization Targeted Location Based Advertising Indoor Navigation (e.g. Airport Terminals) Real Life Analytics (Gym, Office, etc..) Indoor localization platform providing high accuracy could enable a host of applications

3 Lots of Technologies! WiFi Beacons Ad hoc signal strength Ultrasound Floor pressure Ultrasonic time of flight Infrared proximity Laser range-finding Stereo camera Array microphone Physical contact

4 Technologies to be covered in this Chapter: Wireless-based solution VLC-based solution Multi-source based solution

5 Agenda 01 Wireless-based Solutions 02 VLC-based solutions 03 Multi-source based solutions

6 Wireless Technologies for Localization Name Effective Range Pros Cons GSM 35km Long range Very low accuracy LTE 30km-100km Wi-Fi 50m-100m Readily available; Medium range Low accuracy Ultra Wideband 70m High accuracy High cost Bluetooth 10m Readily Available; Medium accuracy Short range Ultrasound 6-9m High accuracy High cost, not scalable RFID & IR 1m Moderate to high accuracy Short range, Line-Of-Sight (LOS) NFC <4cm High accuracy Very short range

7 Fingerprinting: Radar Fingerprinting: PinLoc SpotFi: Decimeter Level Localization using WiFi Push the Limit of WiFi based Localization for Smartphones Accurate RFID Positioning in Multipath Environments

8 Fingerprinting Mapping solution Address problems with multipath Better than modeling complex RF propagation pattern

9 Fingerprinting SSID (Name) BSSID (MAC address) Signal Strength (RSSI) linksys 00:0F:66:2A:61:00 18 starbucks 00:0F:C8:00:15:13 15 newark wifi 00:06:25:98:7A:0C 23

10 Fingerprinting Easier than modeling Requires a dense site survey Usually better for symbolic localization Spatial differentiability Temporal stability

11 Received Signal Strength (RSS) Profiling Measurements Construct a form of map of the signal strength behavior in the coverage area The map is obtained: Offline by a priori measurements Online using sniffing devices deployed at known locations They have been mainly used for location estimation in WLANs

12 Received Signal Strength (RSS) Profiling Measurements Different nodes: Anchor nodes Non-anchor nodes A large number of sample points (e.g., sniffing devices) At each sample point, a vector of signal strengths is obtained jth entry corresponding to the jth anchor s transmitted signal The collection of all these vectors provides a map of the whole region The collection constitutes the RSS model It is unique with respect to the anchor locations and the environment The model is stored in a central location A non-anchor node can estimate its location using the RSS measurements from anchors

13 RADAR: An In-Building RF-Based User Location and Tracking system Functional Components Base Stations (Access Points) Mobile Users Fundamental Idea in RADAR Signal Strength is a function of the receiver s location Road Maps Techniques to build the Road Maps Empirical Method Radio Propagation Model Search Techniques Nearest Neighbor in Signal Space (NNSS) NNSS Avg. Viterbi-like Algorithm Paramvir Bahl and Venkata N. Padmanabhan

14 Data Collection Key Step in the proposed approach Records the Radio Signal as a function of the user location Off-Line Phase Construct/validate models for signal propagation Real-Time Phase (Infer location of user) Every packet received by the base station, the WiLIB extracts Signal Strength Noise floor at the transmitter Noise floor at the receiver MAC address of the transmitter

15 Data Processing Traces collected from the off-line phase are unified into a table consisting of tuples of the format [ x, y, d, ss(i), snr(i) ] I {1,2,3} Search Algorithm NNSS NNSS Avg. Viterbi-like Algorithm Layout Information

16 Algorithm and Experimental Analysis

17 Empirical Method 280 combinations of user location and orientation (70 distinct points, 4 orientations on each point) Uses the above empirical data recorded in the off-line phase to construct the search space for the NNSS Algorithm Algorithm (Emulates the user location problem) Picks one location and orientation randomly Searches for a corresponding match in the rest of the 69 points and orientations Comparison with Strongest Base Station Random Selection

18 Error Distance Values

19 Empirical Method (Cntd. ) Multiple Nearest Neighbor Increases the accuracy of the Location Estimation N1 T N2 G N3 Figure : Multiple Nearest Neighbors T True Location G Guess N1,N2,N3 - Neighbors

20 Empirical Method (Cntd. ) Impact of Number of Number of Samples Accuracy obtained by all the samples can be obtained if only a few samples are taken No. Of Real-Time Samples Error Distance degradation 1 30% 2 11% 3 4% Impact of User Orientation Off-line readings for all orientations is not feasible Work around is to calculate the error distance for all combinations

21 Empirical Method (Cntd. ) Tracking a Mobile User Analogous to the user location problem New Signal Strength data set Window size of 10 samples 4 Signal Strength Samples every second Limitation of Empirical Method To start off with needs an initial signal strength data set Relocation requires re-initialization of the initial data set

22 Radio Propagation Model Introduction Alternative method for extracting signal strength information Based on a mathematical model of indoor signal propagation Issues Reflection, scattering and diffraction of radio waves Needs some model to compensate for attenuation due to obstructions Models Rayleigh Fading Model : Infeasible Rician Distribution Model : Complex Wall Attenuation Factor

23 Wall Attenuation Factor

24 Radio Propagation Model (Cntd. ) Advantages: Cost Effective Easily Relocated

25 Conclusion RF-based user location and tracking algorithm is based on Empirically measured signal strength model Accurate Radio Propagation Model Easily relocated RADAR could locate users with high degree of accuracy Median resolution is 2-3 meters, which is fairly good Used to build Location Services Printing to the nearest printer Navigating through a building

26 Fingerprinting: Radar Fingerprinting: PinLoc SpotFi: Decimeter Level Localization using WiFi Push the Limit of WiFi based Localization for Smartphones Accurate RFID Positioning in Multipath Environments

27 While most WiFi based localization schemes operate with signal strength based information at the MAC layer, PinLoc recognizes the possibility of leveraging detailed physical (PHY) layer information

28 Fingerprinting Wireless Channel a/g/n implements OFDM Wideband channel divided into subcarriers Frequency subcarriers Intel 5300 card exports frequency response per subcarrier phase and magnitude over 30 subcarriers richly capture the scattering in the environment

29 Is WiFi Channel Amenable to Localization? Two key hypotheses need to hold: 1. Temporal Channel responses at a given location may vary over time However, variations must exhibit a pattern a signature 2. Spatial Channel responses at different locations need to be different channel responses from multiple OFDM subcarriers can be a promising location signature

30 Measured channel response at different times Using Intel cards Variation over Time cluster1 cluster2 cluster1 cluster2 Observe: Frequency responses often clustered at a location But not necessarily one cluster per location

31 Measured channel response at different times Using Intel cards Variation over Time cluster2 2 clusters with different cluster1 mean and variance cluster1 cluster2 But not necessarily one cluster per location

32 How Many Clusters per Location? Do all 19 clusters occur with same frequency? Unique clusters per location

33 Cluster Occurrence Frequency Most frequent cluster Others 4th 3rd 2nd most Unique clusters per location 3 to 4 clusters heavily dominate, need to learn these signatures

34 Is WiFi Channel Amenable to Localization? Temporal Channel responses at a given location may vary over time However, variations must exhibit a pattern a signature Spatial Clusters with different Location Signature mean and variance Channel responses at different locations need to be different

35 What is the Size of a Location? Localization granularity depends on size RSSI changes in orders of several meters (hence, unsuitable)

36 What is the Size of a Location? Localization granularity depends on size RSSI changes in orders of several meters (hence, unsuitable) 3 cm apart 2 cm apart Cross correlation with signature at reference location Channel Define response location changes as 2cm every x 2cm 2-3cm area, call them pixels

37 Im (H(f)) But Will all pixels have unique signatures? Self Similarity > Max ( Cross Similarity Pixel 1 Pixel 2 Pixel 3 Real (H(f))

38 For correct pixel localization Self Similarity AP1 > Max ( Cross ) 0 Similarity - AP2 AP1 and AP2 Self Max (Cross) Self Max (Cross) Self Max (Cross) 67% pixel accuracy even with multiple APs

39 67% accuracy inadequate can we improve accuracy? Opportunity: - Humans exhibit natural (micro) movements - Likely to hit several nearby pixels - Combine pixel fingerprints into super-fingerprint

40 From Pixels to Spots 2cm Pixel Spot Combine pixel fingerprints from a 1m x 1m box. Intuition: low probability that a set of pixels will all match well with an incorrect spot

41 PinLoc: Architecture and Modeling Parameters: (w K, U K, V K ) Variational Inference (Infer.NET) PinLoc measures the CFRs at spots of interest during the training phase and tries to identify as many of the unique clusters as possible during a war-driving period Test Data

42 Im (H(f)) Per pixel signature Real (H(f))

43 Im (H(f)) Per spot signature Real (H(f))

44 Evaluated PinLoc (with existing building WiFi) at: Duke museum ECE building Café (during lunch) Roomba calibrates 4m each spot Testing next day PinLoc Evaluation Compare with Horus (best RSSI based scheme)

45 Performance 90% mean accuracy, 6% false positives WiFi RSSI is not rich enough, performs poorly - 20% accuracy Accuracy per spot False positive per spot

46 Fingerprinting: Radar Fingerprinting: PinLoc SpotFi: Decimeter Level Localization using WiFi Push the Limit of WiFi based Localization for Smartphones Accurate RFID Positioning in Multipath Environments

47 SpotFi: Decimeter Level Localization using WiFi Manikanta Kotaru, Kiran Joshi, Dinesh Bharadia, Sachin Katti Stanford University

48 Requirement for Ideal Localization System

49 1. Easily Deployable Commercial WiFi chips

50 1. Easily Deployable Commercial WiFi chips No hardware or firmware change 4

51 1. Easily Deployable Commercial WiFi chips No hardware or firmware change No User Intervention 5

52 2. Universal Localize any WiFi device No specialized sensors

53 3. Accurate Error of few tens of centimeters 1 m

54 State-of-the-art System Deployable Universal Accurate RADAR, Bahl et al, 00 HORUS, Youssef et al, 05 ArrayTrack, Xiong et al, 13 PinPoint, Joshi et al, 13 CUPID, Sen et al, 13 LTEye, Kumar et al, 14 Phaser, Gjengset et al, 14 Ubicarse, Kumar et al, 14 SpotFi, Kotaru et al, 15

55 System Overview

56 Localization - Overview

57 Localization - Overview

58 Challenge - Multipath

59 Subcarriers Solving The Multipath Problem State-of-the-art Model signal on antennas alone SpotFi Model signal on both antennas and subcarriers Antennas f 1 f 2 f 3 f 4

60 Overall Architecture SpotFi collects CSI and RSSI measurements from all the APs that can hear the packet transmitted by the target SpotFi calculates the ToF and AoA of all the propagation paths from the target to each of the APs SpotFi then identifies the direct path between the target and the AP that did not undergo any reflections SpotFi estimates the location of the target by using the direct path AoA estimates and RSSI measurements from all the APs

61 θ 1, τ 1 θ 2, τ 2 Step 1: Resolve Multipath

62 Signal Modeling Equal Distance Line

63 Phase Phase Path Difference = 2π (Phase Difference) wave length 1 / frequency 0 Distance travelled by the WiFi signal

64 Signal Modeling AoA (Angle of Arrival) Equal Phase Line

65 Signal Modeling AoA (Angle of Arrival) Uniform linear array consisting of M antennas: For AoA of θ, the target s signal travels an additional distance of d*sin(θ) to the second antenna in the array compared to the first antenna This results in an additional phase of -2π*d*sin(θ)*f/c at the second antenna

66 Signal Modeling - AoA Define Φ 1 = e j2πdsinθ 1f c Phase at the antenna 1: x 1 = Γ 1 θ 1 Phase at the antenna 2: x 2 = Γ 1 Φ Phase at the antenna 3: x 3 = Γ 1 Φ 1 2 Γ 1 is complex attenuation of the path. Φ 1 depends on AoA

67 Say There Are Two Paths

68 Say There Are Two Paths x 1 = Γ 1 x 2 = Γ 1 Φ 1 x 3 = Γ 1 Φ 1 2

69 Say There Are Two Paths x 1 = Γ 1 + Γ 2 x 2 = Γ 1 Φ 1 + Γ 2 Φ 2 x 3 = Γ 1 Φ Γ 2 Φ 2 2

70 Problem Statement x 1 = Γ 1 + Γ 2 x 2 = Γ 1 Φ 1 + Γ 2 Φ 2 x 3 = Γ 1 Φ Γ 2 Φ 2 2 CSI - Known

71 Problem Statement x 1 = Γ 1 + Γ 2 x 2 = Γ 1 Φ 1 + Γ 2 Φ 2 x 3 = Γ 1 Φ Γ 2 Φ 2 2 Parameters - Unknown

72 Problem Statement x 1 = Γ 1 + Γ 2 x 2 = Γ 1 Φ 1 + Γ 2 Φ 2 x 3 = Γ 1 Φ Γ 2 Φ 2 2 Number of paths (or AoAs) < Number of antennas (or equations)

73 Typical Indoor Multipath

74 That s A Problem State-of-the-art Commodity WiFi chips Number of antennas/equations should be at least 5

75 Subcarriers How To Obtain More Equations? Model signal on both antennas and subcarriers Antennas f 1 f 2 f 3 f 4

76 Each Subcarrier Gives New Equations f 2 f 1

77 Signal Modeling ToF (Time of Flight) Define Ω 1 = e j2π f 2 f 1 τ 1 Phase at first subcarrier: x 1 = Γ 1 Phase at second subcarrier: x 2 = Γ 1 Ω 1 Γ 1 is complex attenuation of the path. Ω 1 depends on incoming signal ToF

78 Estimate both AoA and ToF More number of equations in terms of parameter of our interest

79 Say There Are Two Paths At first subcarrier, for 3 antennas x 1 = Γ 1 x 2 = Γ 1 Φ 1 2 x 3 = Γ 1 Φ 1 At second subcarrier, for 3 antennas y 1 = Γ 1 Ω 1 y 2 = Γ 1 Φ 1 Ω 1 y 3 = Γ 1 Φ 2 1 Ω 1

80 Say There Are Two Paths At first subcarrier, for 3 antennas x 1 = Γ 1 + Γ 2 x 2 = Γ 1 Φ 1 + Γ 2 Φ x 3 = Γ 1 Φ 1 + Γ 2 Φ 2 At second subcarrier, for 3 antennas y 1 = Γ 1 Ω 1 + Γ 2 y 2 = Γ 1 Φ 1 Ω 1 + Γ 2 Φ 2 Ω 2 y 3 = Γ 1 Φ 2 1 Ω 1 + Γ 2 Φ 2 2 Ω 2

81 Subcarrier 2 Subcarrier 1 Problem Statement CSI - Known x 1 = Γ 1 + Γ 2 x 2 = Γ 1 Φ 1 + Γ 2 Φ 2 x 3 = Γ 1 Φ Γ 2 Φ 2 2 y 1 = Γ 1 Ω 1 + Γ 2 y 2 = Γ 1 Φ 1 Ω 1 + Γ 2 Φ 2 Ω 2 y 3 = Γ 1 Φ 1 2 Ω 1 + Γ 2 Φ 2 2 Ω 2

82 Subcarrier 2 Subcarrier 1 Problem Statement Parameters - Unknown y 1 = Γ 1 + Γ 2 y 2 = Γ 1 Φ 1 + Γ 2 Φ 2 y 3 = Γ 1 Φ Γ 2 Φ 2 2 y 1 = Γ 1 Ω 1 + Γ 2 y 2 = Γ 1 Φ 1 Ω 1 + Γ 2 Φ 2 Ω 2 y 3 = Γ 1 Φ 1 2 Ω 1 + Γ 2 Φ 2 2 Ω 2

83 Subcarrier 2 Subcarrier 1 Problem Statement x 1 = Γ 1 + Γ 2 Number of equations = Number of Subcarriers x Number of Antennas x 2 = Γ 1 Φ 1 + Γ 2 Φ x 3 = Γ 1 Φ 1 + Γ 2 Φ 2 y 1 = Γ 1 Ω 1 + Γ 2 y 2 = Γ 1 Φ 1 Ω 1 + Γ 2 Φ 2 Ω 2 y 3 = Γ 1 Φ 2 1 Ω 1 + Γ 2 Φ 2 2 Ω 2

84 AoA, ToF Estimates θ 1, τ 1 θ 2, τ 2

85 θ 1, τ 1 θ 1, τ 1 θ 2, τ 2 Step 2: Identify Direct Path

86 AoA, ToF Estimates θ 1, τ 1 θ 2, τ 2

87 Use Multiple Packets θ 1, τ 1 θ 2, τ 2

88 Use Multiple Packets

89 Use Multiple Packets

90 Use Multiple Packets

91 Direct Path Likelihood Higher weight Smaller ToF Higher weight Lower weight Lower weight Higher weight

92 Direct Path Likelihood Lower weight Higher weight Smaller ToF Tighter Cluster Lower weight Lower weight Lower weight

93 Direct Path Likelihood Lower weight Higher weight Higher weight Lower weight Smaller ToF Tighter Cluster More Packets Lower weight

94 Highest Direct Path Likelihood

95 θ 1, τ 1 θ 1, τ 1 θ 2, τ 2 Step 3: Localize The Target

96 Use Multiple APs Direct Path AoA = 45 degrees Signal Strength = 10 db Direct Path AoA = -45 degrees Signal Strength = 20 db Direct Path AoA = 10 degrees Signal Strength = 30 db Find location that best explains the AoA and Signal Strength at all the APs

97 Use Different Weights Direct Path AoA = 45 degrees Signal Strength = 10 db Direct Path Likelihood Direct Path AoA = -45 degrees Signal Strength = 20 db Direct Path Likelihood Direct Path AoA = 10 degrees Signal Strength = 30 db Direct Path Likelihood Use different weights for different APs

98 Evaluation

99 Testbed Access point 52 m Target Locations AP Locations Target

100 Empirical CDF Indoor Office Deployment 16 m ArrayTrack Ubicarse SpotFi 0.3 m 0.4 m 0.4 m m m Target Locations AP Locations Localization Error (m)

101 Stress Test Obstacles Blocking The Direct Path 52 m Target Locations AP Locations

102 Stress Test Obstacles Blocking The Direct Path Empirical CDF m m Target Locations AP Locations Localization Error (m)

103 Empirical CDF Effect of WiFi AP Deployment Density APs 4 APs 5 APs m Localization Error (m)

104 Conclusion Deployable: Indoor Localization with commercial WiFi chips Accurate: Accuracy comparable to state-of-the-art localization systems which are not suitable for wide deployments Universal: Simple localization targets with only a WiFi chip

105 Fingerprinting: Radar Fingerprinting: PinLoc SpotFi: Decimeter Level Localization using WiFi Push the Limit of WiFi based Localization for Smartphones Accurate RFID Positioning in Multipath Environments

106 Push the Limit of WiFi based Localization for Smartphones Hongbo Liu, Yu Gan, Jie Yang, Simon Sidhom, Yan Wang, Yingying Chen Department of Electrical and Computer Engineering Stevens Institute of Technology Fan Ye IBM T. J. Watson Research Center

107 The Need for High Accuracy Smartphone Localization Help users navigation inside large and complex indoor environment, e.g., airport, train station, shopping mall. Understand customers visit and stay patterns for business Train Station Shopping Mall Airport

108 Smartphone Indoor Localization - What has been done? Contributions in academic research WiFi indoor localization High accuracy indoor localization RADAR [INFOCOM 00], Horus [MobiSys 05], Chen et.al[percom 08] Cricket [Mobicom 00], WALRUS [Mobisys 05], DOLPHIN [Ubicomp 04], Gayathri et.al [SECON 09] WiFi enabled smartphone indoor localization SurroundSense [MobiCom 09], Escort [MobiCom 10], WILL[INFOCOM 12], Virtual Compass [Pervasive 10] Commercial products Is it possible to achieve high accuracy localization using most prevalent WiFi infrastructure? Google Map Localization error up to 10 meters Shopkick Locate at the granularity of stores

109 Received Signal Strenth (dbm) Root Cause of Large Localization Errors Am I here? I am around here ~ 2 meters WiFi as-is is not a suitable candidate for high accurate localization due to large errors Is it possible to address this fundamental limit without the need of additional hardware or infrastructure? 6-8 meters AP 1 AP 2 AP 3 AP 4 32: [ Permanent -22dB, -36dB, environmental -29dB, -43dB settings, ] such as furniture Physically placement distant and locations walls. share similar WiFi 48: [ Transient -24dB, -35dB, factors, -27dB, such as -40dB] dynamic obstacles and interference. Received Signal Strength! Orientation, holding position, time of day, number of samples

110 Inspiration from Abundant Peer Phones in Public Place Increasing density of smartphones in public spaces How to capture the physical constraints? Provide physical constraints from nearby peer phones Peer 3 Peer 1 Peer 2 Target

111 Basic Idea Peer 2 Peer 1 Peer 3 Target Exploit acoustic signal/ranging to construct peer constraints Interpolated Received Signal Strength Fingerprint Map WiFi Position Estimation Acoustic Ranging

112 System Design Goals and Challenges Peer assisted localization Exactly what is the algorithm to search for the best fit position and quantify the signal similarity so that to reduce large errors? Fast and concurrent acoustic ranging of multiple phones How to design and detect acoustic signals? Ease of use Need to complete in short time. Not annoy or distract users from their regular activities.

113 System Work Flow WiFi position estimation Peer recruiting & ranging Rigid graph construction Peer assisted localization Peer recruiting & ranging Minimizing Identify nearby the impact peers on users regular activities HTC EVO Only phones close enough can detect recruiting signal Sound signal design Peer phones willing to help send their IDs to the server Beep emission strategy Deploy extra timing buffers to accommodate variations in the reception of the schedule at different phones, e.g., 20 ms Acoustic signal detection ADP2 Fast ranging KHz Unobtrusive to human ears Robust to noise Lab Employ virtual synchronization Train Station scheme based Shopping on time-multiplexting Mall Airport Change point detection Correlation method

114 System Work Flow WiFi position estimation Rigid graph Peer recruiting & ranging construction Peer assisted localization Rigid graph construction Construct the graph G and G based on initial WiFi position estimation and the acoustic ranging measurements. Graph G based on WiFi position estimation Rigid Graph G based on acoustic ranging

115 System Work Flow WiFi position estimation Peer recruiting & ranging Peer assisted localization Rigid graph construction Peer assisted localization Acoustic ranging graph WiFi based graph Graph Translational Orientation Movement Estimation

116 Prototype and Experimental Evaluation Prototype Devices HTC EVO ADP 2 Trace-driven statistical test Feed the training data as WiFi samples Perturb distances with errors following the same distribution in real environments

117 Localization Accuracy Localization performance across different real-world environments (5 peers) Median error 90% error Lab Train Station Shopping Mall Airport Peer assisted method is robust to noises in different environments

118 Overall Latency and Energy Consumption Overall Latency Pose little more latency than required in the original WiFi localization about 1.5 ~ 2 sec Energy Consumption Negligible impact on the battery life e.g., with additional power consumption at about 320mW on HTC EVO - lasts 12.7 hours with average power of 450mW

119 Discussion Peer Involvement Use incentive mechanism to encourage and compensate peers that help a target s localization Movements of users Do not pose more constraints on movements than existing WiFi methods Affect the accuracy only during sound-emitting period Happens concurrently and shorter than WiFi scanning Triggering peer assistance Provides the technology for peer assistance Up to users to decide when they desire such help

120 Conclusion Leverage abundant peer phones in public spaces to reduce large localization errors Aim at the most prevalent WiFi infrastructure Do not require any special hardware Exploit minimum auxiliary COTS sound hardware readily available on smartphones Utilize much more accurate distance estimate through acoustic ranging to capture unique physical constraints Demonstrate our approach successfully pushes further the limit of WiFi localization accuracy Lightweight in computation on smartphones In time not much longer than original WiFi scanning With negligible impact on smartphone s battery life time

121 Fingerprinting: Radar Fingerprinting: PinLoc SpotFi: Decimeter Level Localization using WiFi Push the Limit of WiFi based Localization for Smartphones Accurate RFID Positioning in Multipath Environments

122 Accurate RFID Positioning in Multipath Environments Jue Wang & Dina Katabi ACM Sigcomm 2013

123 RFIDs Battery-free RF stickers with unique IDs

124 RFIDs 5-cent stickers to tag any and every object Reader s range is ~15m Imagine you can localize RFIDs to within 10 to 15 cm!

125 If we can locate RFID to within 10 to 15cm No more customer checkout lines

126 If we can locate RFID to within 10 to 15cm No more customer checkout lines

127 The Challenge: Multipath Effect Localization uses RSSI or Angle-of-Arrival (AoA) But, signal bounces off objects in the environment Angle of signal is not the direction of the RFID Multipath propagation limits the Accuracy of RFID localizations

128 PinIt Accurate RFID localization (e.g., 10 to 15cm) even in multipath and non-line-of-sight settings Focuses on proximity to reference RFIDS Exploits multipath effects to increase accuracy

129 PinIt Exploits Multipath Signals from nearby RFIDs propagate along closer paths and experience similar reflections

130 PinIt Exploits Multipath Signals from nearby RFIDs propagate along closer paths and experience similar reflections

131 PinIt Exploits Multipath Signals from nearby RFIDs propagate along closer paths and experience similar reflections

132 PinIt Exploits Multipath Signals from nearby RFIDs propagate along closer paths and experience similar reflections Nearby RFIDs have similar profiles with smaller shifts in the peaks

133 Implementation & Evaluation Implemented a PinIt Reader in USRP Commercial off-the-shelf RFIDs Mounted the antenna on an irobot that slides back and forth

134 Positioning Accuracy 200 RFIDs deployed on the shelves in the library spaced by 15 cm PinIt improve the accuracy by 6x in comparison to AoA and 10x in comparison to RSSI

135 Automatic Checkout

136 Five items in two adjacent baskets at checkout

137 Which Items Belong to Which Basket?

138 Is the Cookie Bag in the Orange or Blue Basket?

139 Why Dynamic Time Warping (DTW)? i i i time i i+2 time Any distance (Euclidean, Manhattan, ) which aligns the i-th point on one time series with the i-th point on the other will produce a poor similarity s core. A non-linear (elastic) alignment prod uces a more intuitive similarity meas ure, allowing similar shapes to match even if they are out of phase in the ti me axis.

140 How is DTW Calculated? C Q Every possible warping between two time series, is a path though the matrix. We want the best one DTW( Q, C) K min w K k 1 k C Q This recursive function gives us the minimum cost path (i,j) = d(q i,c j ) + min{ (i-1,j-1), (i-1,j ), (i,j-1) } Warping path w

141 One more note The time series can be of different lengths.. Q C C Q Warping path w

142 Is the Noodle in the Orange or Blue Basket?

143 Brief Summary PinIt provides accurate RFID positioning even in multipath and NLOS settings It uses DTW to compare RFID multipath profiles It enables new applications including eliminating checkout lines, object tracking in libraries and pharmacies, smart homes,

144 Agenda 01 Wireless-based Solutions 02 VLC-based solutions 03 Multi-source based solutions

145 Wearables Can Afford: Light-weight Indoor Po sitioning with Visible Light Zeyu Wang, Zhice Yang, Jiansong Zhang, Chenyu Huang,Qian Zhang Hong Kong University of Science and Technology

146 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map

147 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map

148 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map

149 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map

150 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map Dead reckoning: Use inertial sensors to calculate moving path

151 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map Dead reckoning: Use inertial sensors to calculate moving path

152 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map Dead reckoning: Use inertial sensors to calculate moving path

153 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map Dead reckoning: Use inertial sensors to calculate moving path

154 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map Dead reckoning: Use inertial sensors to calculate moving path Accuracy is not enough (~several meters)

155 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map Dead reckoning: Use inertial sensors to calculate moving path Dedicated localization infrastructure

156 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map Dead reckoning: Use inertial sensors to calculate moving path Dedicated localization infrastructure

157 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map Dead reckoning: Use inertial sensors to calculate moving path Dedicated localization infrastructure

158 Indoor Localization Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map Dead reckoning: Use inertial sensors to calculate moving path Dedicated localization infrastructure Complex and high-cost to handle RF multipath

159 Visible Light Positioning

160 Visible Light Positioning

161 Visible Light Positioning

162 Visible Light Positioning Visible Light Positioning (VLP) is an emerging positioning technique that based Visible Light Communication (VLC) Light bulbs are densely deployed Location anchors are ubiquitous Light beam is very directional No multipath, localization is simple and accurate More Light is free of radio wave Positioning through light bulbs is green in energy

163 How VLC generally works? Modulate Light Intensity Normal Light Modulated Light Time

164 Problem in VLC: Flickering 10Hz 100Hz >1000Hz

165 Consequence: Overhead in Client Additional Receiving Device Using customized light sensor that requires cumbersome calibration[1] High Computational Overhead Using very high resolution camera to extract the roller shuttering patterns[2] >1000Hz These overhead can hardly be afforded in wearables. Can they be eliminated? Must be LED [1] L. Li etc. Epsilon: A visible light based positioning system in NSDI 14 [2] Y.-S. Kuo etc. Luxapose: Indoor positioning with mobile phones and visible light in Mobicom 14

166 Idea: Flickering-free Modulation Instead of changing the intensity, we modulate information by changing the polarization of light Human eyes CANNOT perceive changes in polarization Therefore low baud rate in transmitters Therefore low decoding overhead in clients

167 PIXEL Review the display mechanism of LCD!

168 PIXEL: One Pixel from LCD Back Light Polarizing Film Polarizing Film Eyes

169 PIXEL: One Pixel from LCD 0V Voltage Back Light Polarizing Film Liquid Crystal Polarizing Film Eyes

170 PIXEL: One Pixel from LCD 5V Back Light Polarizing Film Liquid Crystal Polarizing Film Eyes

171 PIXEL: VLC Transmitter Voltage Eyes Camera Back Light Polarizing Film Liquid Crystal Polarizing Film Eyes VLC Transmitter

172 PIXEL: VLP Architecture VLC Transmitter Su n Location Location Polarizing Film Location Location

173 Challenge: User Mobility

174 Challenge: User Mobility (Cont.) Received Light Intensity Voltage Low Voltage High Receiving Direction SNR

175 Solution: Dispersion

176 Solution: Dispersion (Cont.) Disperse the Polarization of Different Colors in to Different Directions Dispersor

177 Solution: Dispersion (Cont.) Received Color Voltage Low Voltage High Receiving Direction SNR

178 Positioning Method

179 Positioning Method

180 Challenges: Less Beacons Existing methods for camera-based VLC localization require multiple beacon lamps(3 or more) being captured in a single image Field Test: 2 or less beacon lamps can be captured by the front camera in normal holding position Portable cameras do not have wide Field of View The ceiling of buildings is normal limited to several meters. Example: 3m below, camera of iphone 6 can only cover 3*3m 2 of the ceiling.

181 Challenges: Less Beacons (Cont.)

182 Challenges: Less Beacons (Cont.) 1 2 Location Ambiguity

183 Solution: Sensor Assisted Localization The position of the receiver has 6 degrees of freedom: 3 in location and 3 in 3D orientation. Each received beacon adds 2 AoA constraints to the position and orientation. The gravity sensor adds 2 constraints to the 3D orientation. Two beacons are enough

184 Solution: Sensor Assisted Localization 1 2 Gravity Location Ambiguity

185 Implementation VLC Transmitter Polarizing film ($0.001/cm 2 ) LCD with only one pixel ($0.03/cm 2 ) Glass box with optical rotation liquid 14Hz Baud Rate Location Beacon 5bit Preamble + 8bit Location ID + 4bit CRC Client Polarizing film ($0.001/cm 2 ) Android App with VLC decoding and VLP algorithm Smart phone: Galaxy S II (1.2GHz CPU, 8 Megapixel Camera) Wearable: Google Glass

186 SNR (db) Evaluation-VLC θ VLC Transmitter w/o dispersor with dispersor Receiver's Orientation θ (degree)

187 SNR (db) Evaluation-VLC VLC Transmitter d Distance (m)

188 CDF Evaluation-VLP m m Positioning Error (cm)

189 CDF Evaluation-VLP Samsung Galaxy SII 1200MHz Google Glass 300MHz Google Glass 600MHz Google Glass 800MHz VLP Processing Time Cost (ms)

190 Conclusion We introduce a light weight VLC method that based on modulating light s polarization We propose to use optical rotation material/dispersor to hand users mobility We implement and evaluate the VLP system, and results show submeter accuracy can be achieved in both smart phone and wearables.

191 Agenda 01 Wireless-based Solutions 02 VLC-based solutions 03 Multi-source based solutions

192 SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache, Martin Azizyan and Romit Roy Choudhury

193 Context Pervasive wireless connectivity + Localization technology = Location-based applications

194 Location-Based Applications (LBAs) For Example: GeoLife shows grocery list when near Walmart MicroBlog queries users at a museum Location-based ad: Phone gets coupon at Starbucks iphone AppStore: 3000 LBAs, Android: 500 LBAs

195 Location-Based Applications (LBAs) For Example: GeoLife shows grocery list when near Walmart MicroBlog queries users at a museum Location-based ad: Phone gets coupon at Starbucks iphone AppStore: 3000 LBAs, Android: 500 LBAs Location expresses context of user Facilitates content delivery

196 As if Location is an IP address for content delivery

197 Thinking about Localization from an application perspective

198 Emerging location based apps need place of user, not physical location Starbucks, RadioShack, Museum, Library Latitude, Longitude We call this Logical Localization

199 Can we convert from Physical to Logical Localization?

200 Can we convert from Physical to Logical Localization? State of the Art in Physical Localization: 1. GPS Accuracy: 10m 2. GSM Accuracy: 100m 3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-100m

201 Can we convert from Physical to Logical Localization? State of the Art in Physical Localization: 1. GPS Accuracy: 10m 2. GSM Accuracy: 100m 3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-100m Widely-deployable localization technologies have errors in the range of several meters

202 Several meters of error is inadequate to logically localize a phone Physical Location Error

203 Several meters of error is inadequate to logically localize a phone Starbucks RadioShack Physical Location Error The dividing-wall problem

204 Contents SurroundSense Evaluation Limitations and Future Work Conclusion

205 Hypothesis It is possible to localize phones by sensing the ambience such as sound, light, color, movement, WiFi

206 Sensing over multiple dimensions extracts more information from the ambience Each dimension may not be unique, but put together, they may provide a unique fingerprint

207 Should Ambiences be Unique Worldwide? Q R H J I K C O B A E D F G P Q L M N

208 SurroundSense Multi-dimensional fingerprint Based on ambient sound/light/color/movement/wifi Starbucks RadioShack Wall

209 GSM provides macro location (strip mall) SurroundSense refines to Starbucks Should Ambiences be Unique Worldwide? Q R H J I K C O B A E D F G P Q L M N

210 SurroundSense Architecture Ambience Fingerprinting Matching Sound Color/Light Acc. WiFi + Test Fingerprint = GSM Macro Location Fingerprint Database Logical Location Candidate Fingerprints

211 Lightn ess Normalized Count Fingerprints 0.14 Acoustic fingerprint (amplitude distribution) Sound: (via phone microphone) Amplitude Values 0 Color and light fingerprints on HSL space Color: (via phone camera) Hue Saturation 1

212 Fingerprints Movement: (via phone accelerometer) Moving Cafeteria Clothes Store Grocery Store Static

213 Fingerprints Movement: (via phone accelerometer) Moving Cafeteria Clothes Store Grocery Store Static Queuing

214 Fingerprints Movement: (via phone accelerometer) Moving Cafeteria Clothes Store Grocery Store Static Queuing Seated

215 Fingerprints Movement: (via phone accelerometer) Moving Cafeteria Clothes Store Grocery Store Static Pause for product browsing

216 Fingerprints Movement: (via phone accelerometer) Moving Cafeteria Clothes Store Grocery Store Static Pause for product bro wsing Short walks between pro duct browsing

217 Fingerprints Movement: (via phone accelerometer) Moving Cafeteria Clothes Store Grocery Store Static Walk more

218 Fingerprints Movement: (via phone accelerometer) Moving Cafeteria Clothes Store Grocery Store Static Walk more Quicker stops

219 Fingerprints Movement: (via phone accelerometer) Cafeteria Clothes Store Grocery Store Moving Static WiFi: (via phone wireless card) ƒ(overheard WiFi APs)

220 Discussion Time varying ambience Collect ambience fingerprints over different time windows What if phones are in pockets? Use sound/wifi/movement Opportunistically take pictures Fingerprint Database War-sensing

221 Contents SurroundSense Evaluation Limitations and Future Work Conclusion

222 Evaluation Methodology 51 business locations 46 in Durham, NC 5 in India Data collected by 4 people 12 tests per location Mimicked customer behavior

223 Accuracy (%) Cluster Evaluation: Per-Cluster Accuracy Cluster No. of Shops Localization accuracy per cluster

224 Accuracy (%) Cluster Evaluation: Per-Cluster Accuracy Cluster No. of Shops Localization accuracy per cluster Multidimensional sensing

225 Accuracy (%) Cluster Evaluation: Per-Cluster Accuracy Cluster No. of Shops Localization accuracy per cluster Fault tolerance

226 Accuracy (%) Cluster Evaluation: Per-Cluster Accuracy Cluster No. of Shops Localization accuracy Sparse WiFi per APs cluster

227 Accuracy (%) Cluster Evaluation: Per-Cluster Accuracy Cluster No. of Shops Localization accuracy per cluster No WiFi APs

228 Evaluation: Per-Scheme Accuracy Mode WiFi Snd-Acc-WiFi Snd-Acc-Lt-Clr SS Accuracy 70% 74% 76% 87%

229 CDF Evaluation: User Experience WiFI Snd-Acc-WiFi Snd-Acc-Clr-Lt SurroundSense Random Person Accuracy Average Accuracy (%)

230 Why does it work? The Intuition: Economics forces nearby businesses to be different Not profitable to have 3 coffee shops with same lighting, music, color, layout, etc. SurroundSense exploits this ambience diversity

231 Contents SurroundSense Evaluation Limitations and Future Work Conclusion

232 Limitations and Future Work Energy-Efficiency Localization in Real Time Non-business locations

233 Limitations and Future Work Energy-Efficiency Continuous sensing likely to have a large energy draw Localization in Real Time Non-business locations

234 Limitations and Future Work Energy-Efficiency Continuous sensing likely to have a large energy draw Localization in Real Time User s movement requires time to converge Non-business locations

235 Limitations and Future Work Energy-Efficiency Continuous sensing likely to have a large energy draw Localization in Real Time User s movement requires time to converge Non-business locations Ambiences may be less diverse

236 Contents SurroundSense Evaluation Limitations and Future Work Conclusion

237 SurroundSense Today s technologies cannot provide logical localization Ambience contains information for logical localization Mobile Phones can harness the ambience through sensors Evaluation results: 51 business locations, 87% accuracy SurroundSense can scale to any part of the world

238 End of This Chapter