Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance

Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian April 11, 2013

Outline 1 Introduction Device-Free Localization Limits of Previous Methods 2 Methods Histogram Difference Imaging and Tracking via Kernel Distance 3 Experiments and Results Experiments Results 4 Conclusion

Device-Free Localization Device and Device Free Localization of People 5 6 7 8 9 10 11 12 13 5 6 7 8 9 10 11 12 13 4 3 2 1 T 4 3 2 1 7 5 14 15 16 17 18 19 20 21 7 5 14 15 16 17 18 19 20 21 Radio device localization: RFID tag, Mobile Phone Device free localization (DFL): Idea: directly use human motion/presence as signal Applications: emergency response, smart homes, context-aware computing, etc.

Device-Free Localization Received Signal Strength (RSS)-Based DFL RSS: available in standard wireless devices Fingerprint-based DFL: training is needed for each environment, not applicable for emergency situations Model-based DFL: [Patwari and Agrawal, 2008] models shadowing effects of people/objects on RSS, and proposes the idea of radio tomographic imaging (RTI) Patwari and Agrawal. Effects of Correlated Shadowing: Connectivity, Localization, and RF Tomography. In Proc. of the 7th ACM/IEEE IPSN, April 2008.

Limits of Previous Methods Previous DFL Methods Shadowing-based RTI: first RTI method to image shadowing loss due to human presence, not robust to non-line-of-sight (NLOS) environments; Variance-based RTI (VRTI) [Wilson and Patwari, 2011]: works in NLOS environments, but cannot locate stationary people; Sequential Monte Carlo (SMC) [Wilson and Patwari, 2012]: can locate stationary and moving people, but requires empty-area calibration, and is not a real-time method. Wilson and Patwari. See-Through Walls: Motion Tracking Using Variance-Based Radio Tomography Networks. IEEE Trans. Mobile Computing, May 2011, pp.612-621. Wilson and Patwari. A Fade Level Skew-Laplace Signal Strength Model for Device-Free Localization with Wireless Networks. IEEE Trans. Mobile Computing, June 2012, pp. 947-958.

Limits of Previous Methods Goal: Break These Limits Features RTI VRTI SMC Through-wall? No Yes Yes Stationary people? Yes No Yes Real-time? Yes Yes No Empty-area calibration? Yes No Yes Table: Features of previous DFL methods. Goal: a real-time method capable of imaging both stationary and moving people in both LOS and NLOS environments without training or empty-area calibration.

Histogram Difference Idea: Use RSS Histogram Instead of RSS Mean or Variance Notation of RSS histogram at time n: h n = i w n,i I y i (1) where y i : RSS at time i (discrete value, quantization 1 dbm) N = y max y min + 1: range of RSS histogram I: N-length indicator vector w n,i : weight for I y i

Histogram Difference Two Weighting Schemes Uniform weight (FIR filter) w n,i = { 1 T 0 i T 1 0 otherwise (2) Exponentially weighted moving average (EWMA) w n,i = { β(1 β) n i i n 0 otherwise (3) where 0 < β < 1 is the forgetting factor

Histogram Difference Two Types of RSS Histograms EWMA: an IIR filter with lower computational and memory complexity h n = (1 β)h n 1 + βi y n (4) only requires current RSS y n and previous histogram h n 1. Long-term histogram (LTH) q and short-term histogram (STH) p FIR scheme: LTH q with a high T value IIR scheme: LTH q with a low β value (β = 0.05 for LTH; β = 0.9 for STH)

Histogram Difference Observations of RSS Histograms Online LTH (IIR using online data) is similar to empty-area LTH (FIR using offline data) STH with people is different from LTH STH w/o people is similar to LTH Idea: quantify the difference between online LTH and STH Frequency 1 0.8 0.6 Online LTH Empty room LTH STH with person STH w/o person 43 45 0.4 0.2 0 4 3 2 44 42 41 1 48 47 46 RSS (dbm) Figure: RSS histograms (same link).

Histogram Difference Histogram Difference Metrics Many ways to measure the difference D(p, q) between p and q: Earth mover s distance (very computationally expensive) Kullback-Leibler divergence (D KL (p, q) = k p k log p k q k ) Kernel distance (symmetric distance metric) [Joshi, et al. 2011] Joshi, et al. Comparing Distributions and Shapes Using the Kernel Distance. In Proc. of the 27th ACM symposium on computational geometry, June 2011.

Histogram Difference Kernel Distance Definition: D K (p, q) = p T Kp + q T Kq 2p T Kq, (5) where K is an N by N kernel matrix from a 2-D kernel function. Gaussian kernel: K(y j, y k ) = exp ( y j y k 2 σ 2 G ), where y j and y k are the jth and kth elements, and σg 2 is the kernel width parameter. Epanechnikov kernel: ) K(y j, y k ) = 3 4 (1 y j y k 2 I σe 2 yj y k σe 2, where I a is the indicator function.

Histogram Difference Efficient Computation: O (N) Instead of O ( N 2) Rewrite (5) as: D(p, q) = (K 1 2 p) T K 1 2 p + (K 1 2 q) T K 1 2 q 2(K 1 2 p) T K 1 2 q Let u = K 1 2 p, v = K 1 2 q: Updating u n and v n : D(p, q) = u v l2 u n = (1 β p )u n 1 + β p K 1 2 Iy n v n = (1 β q )v n 1 + β q K 1 2 Iy n where the term K 1 2 I y n is simply the y n th column of matrix K 1 2.

Histogram Difference Example of Kernel Distance RSS (dbm) 40 50 60 70 80 90 100 0 50 100 150 Time (n) 4.0 3.5 3.0 2.5 2.0 1.5 1.0 Kernel distance 0.5 0.0 Figure: RSS ( ) and kernel distance (+) time series for a link which a person crosses at n = 23 and n = 120.

Imaging and Tracking via Kernel Distance RTI Idea Divide the area into P pixels Estimate pixel values x = [x 1, x 2,, x P ] T, which represent human presence, from RSS link measurements and model W [Patwari and Agrawal, 2008] Figure: One example of RTI image estimate.

Imaging and Tracking via Kernel Distance Kernel Distance-Based Radio Tomographic Imaging (KRTI) Notation: Let d = [d 0,..., d L 1 ] T denote a vector with L kernel distances, d l = D(p l, q l ) Problem: Estimate image ˆx from d and model W (ill-posed inverse problem L < P) Regularized least squares solution: ˆx = Π K d where Π K = (W T W + αc 1 x ) 1 W T. where α is the regularization parameter, and C x is the covariance of the image used in regularization.

Imaging and Tracking via Kernel Distance Localization and Tracking The location of a person is estimated as: ẑ = r q where q = arg max ˆx p p where ˆx p is the pth element of vector ˆx. For a moving person: Use a Kalman filter on the localization estimates to track locations over time. Localization and tracking of multiple people can be achieved from KRTI images.

Experiments Experiment Testbed DE C B A ZigBee radio node: 2.4 GHz, IEEE 802.15.4 Spin: token passing protocol; when one transmits, others measure RSS DE C B A Packet data: node ID and measured RSS values Laptop-connected note listens all traffic to record all pairwise RSS

Experiments Experiments 34 radio nodes are deployed around a 9 m by 8 m (Exp. 1-4) or a 12 m by 5 m (Exp. 5) area to locate a stationary person (Exp. 1) or a moving person (Exp. 2-5). Name Task Description Exp.1 stationary person calm day through-wall Exp.2 moving person calm day through-wall Exp.3 moving person windy day with fans Exp.4 moving person windy day with fans Exp.5 moving person at a cluttered bookstore Table: Experimental datasets (new data Exp. 3 and 4 reported in this paper is available at http://span.ece.utah.edu/data-and-tools).

Experiments Experimental Layout (through-wall) 8 Kitchen Door 6 Door Y (m) 4 2 Living room Tree 0 2 Nodes on stands Camera Nodes not on stands 0 2 4 6 8 10 X (m) Figure: Experiment layout of Exp. 1-4.

Experiments Experimental Layout (cluttered bookstore) Y (m) 6 5 4 3 2 1 0 A D RF sensors Shelves Path 1 2 0 2 4 6 8 10 12 X (m) B C Figure: Experiment layout and picture of Exp. 5.

Results Imaging of a Stationary Person (Exp. 1) m 8 7 6 5 4 3 2 1 0 0 2 4 6 8 m 0.20 0.18 0.16 0.14 0.12 0.10 0.08 0.06 0.04 0.02 0.00 m 8 7 6 5 4 3 2 1 0 0 2 4 6 8 m 0.20 0.18 0.16 0.14 0.12 0.10 0.08 0.06 0.04 0.02 0.00 Figure: Imaging results of a stationary person (true location shown as ) from KRTI (left) and VRTI (right).

Results Localization of a Stationary Person (Exp. 1) 8 8 6 6 Y (m) 4 Y (m) 4 2 0 Known locations Estimates RF sensors 2 0 Known locations Estimates RF sensors 0 2 4 6 8 10 X (m) 0 2 4 6 8 10 X (m) Figure: Location estimates of a person standing at twenty locations from KRTI (left, average error 0.71 m, 0.03 sec per estimate) and SMC (right, average error 0.83 m, 3 to 4 minutes per estimate).

Results Localization of a Moving Person (Exp. 2-5) Figure: RMSE comparison with variance-based methods VRTI and subspace variance-based radio tomography (Zhao and Patwari. Noise Reduction for Variance-Based Device-Free Localization and Tracking. In Proc. of the 8th IEEE SECON, June 2011.)

Results Tracking of a Moving Person Y (m) 6 5 4 3 2 1 0 1 2 0 2 4 6 X (m) 8 10 12 Figure: Kalman filter tracking result of Exp. 5 (true path shown as dash line).

Conclusion Using kernel distance in radio tomography allows us to locate both stationary and moving people in an LOS or non-los environment in real-time; Instead of using empty-area calibration, KRTI uses online calibration, which enables applications in emergency situations; Real-world experiments show that KRTI outperforms previous DFL methods in localization accuracy and computational efficiency.