One Decade of Sensorless Sensing: Wireless Networks as Human Context Sensors

Transcription

1 One Decade of Sensorless Sensing: Wireless Networks as Human Context Sensors SPAWC 2015

2 Outline 1 Introduction 2 RSS Device-Free Localization 3 Context Beyond Location 4 Conclusion

3 Outline 1 Introduction 2 RSS Device-Free Localization 3 Context Beyond Location 4 Conclusion

4 My Talk: In Between Space Wireless Communication Devices and Systems "Sensorless" Sensing Radar Devices and Systems Radar research has produced amazing monitoring systems (Low cost) Comms devices perform channel estimation The gap is wide

5 Sensor Network Story (circa 2000) Image Credit: The Sensor Wireless sensors were predicted to: Cost.05 USD Be everywhere: walls, air, etc. Use ambient energy sources We forgot about the cost/energy of the sensor Network Museum

6 Sensor Network Re-imagined The radio itself, provided that it can measure the strength of the incoming signal, is the only sensor we use; with this sensorless sensing approach, any wireless network becomes a sensor network. From Kristen Woyach, Daniele Puccinelli, Martin Haenggi, Sensorless sensing in wireless networks: implementation and measurements, IEEE WiOpt 2006

7 Radio as a Sensor According to Woyach et al., received signal strength (RSS) can: Detect a person crossing a link line Classify the environment as changed or unchanged Detect very small changes in position of TX or RX Estimate rotational velocity of the TX or RX

8 Key Characteristics: Radio as a Sensor Use wireless communication devices (low cost) Re-purpose existing channel estimates Wireless network becomes multistatic radar Applications in context awareness

9 Radio Sensor Measurements Wireless comms devices do estimate the channel, but most don t allow access. 1 Received signal strength (RSS) 2 MIMO channel state information (CSI) 3 Phase measurement unit (PMU) N transceivers O ( N 2) links

10 Context Sensing What/who is around us? What are people doing? Humans & computers should act appropriately for the context Image Credit: xkcd.com/138/

11 Application: Tracking People Image: Not all people will wear tags Efficiency: Smart buildings Safety: Evacuations, factories, aging-in-place Security: Monitoring, surveillance, health care articles/view?11615

12 Device-free Localization (DFL): Features of RF Alternatives: Video, Audio, Thermal, Infrared Radio waves penetrate (non-metal) walls, furniture, smoke Works in the dark, quiet Not as privacy-invasive as audio or video surveillance

13 Outline 1 Introduction 2 RSS Device-Free Localization 3 Context Beyond Location 4 Conclusion

14 Zigbee RSS Measurement 40 Link 1 Link 2 45 RSS (dbm) CC2531 USB dongle 2.4 GHz, IEEE , 15 channels RSS for each packet Time index Person changes RSS Two identical links: different changes

15 Problem Statement: RSS Device-free Localization RSS changes most due to people in environment near link One person / object affects multiple links Mesh network of N nodes O ( N 2) RSS measurements Find: Count, locations of people

16 RSS-DFL: Survey of Current Capabilities Experimental tests report 10 cm - 2 m avg. error using 5-35 nodes in m 2, and can track 1-4 people.

17 Radio Tomographic Imaging (RTI) 1 Quantify presence on each link 2 Presume it is linear combination of presence in pixels 3 Pick regularization method 4 Solve inverse problem

18 History: Shadowing as Linear Spatial Filter shadowing field px ( ) x i x k link a link b x j x l Two nearby links shadowing is correlated Model: shadow loss is a line integral of a spatially correlated field 1 1 N. Patwari and P. Agrawal, Effects of correlated shadowing: connectivity, localization, and RF tomography, IPSN 2008.

19 Discrete-space Model Consider simultaneously all M pair-wise links: y = W x + n y = [y 1,... y M ] T = measured change in RSS x = [x 1,... x N ] T = discretized presence field (e.g., db/voxel) W = [[w i,j ]] i,j = weights; n = noise

20 Shadowing Field Estimation Problems Measure y, change in RSS from empty period Assume known W. Estimate x. Ill-posed! Pixels links, other issues Linear model isn t true physics; W is unknown.

21 Real-time Approaches to Image Estimation Real-time requirement: linear estimator ˆx = Πy Projection Π needs only be calculated once Complexity: Order of # Links # pixels Regularization: e.g., Tikanov, Least-squares

22 Regularized Image Estimation Algorithms 1 Regularized inverse: minimize penalized squared error 2 f (x) = W x y 2 + α Qx 2 when Q is the derivative: [ ] 1 Π Tik = W T W + α(dx T D X + DY T D Y ) W T 2 Assume correlated image x and use regularized least squares. ( ) 1 Π RLS = W T W + αcx 1 W T 2 J. Wilson and N. Patwari, Radio tomographic imaging with wireless networks, IEEE TMC, 2010.

23 Shadowing RTI Experiment: Open deployment in atrium y is decrease in RSS compared to no person present

24 Shadowing RTI with Passive Tags Reader: 2 TX, 2 RX; 40 passive tags on floor of 16 m 2 area 3 30 cm average error B. Wagner, B. Striebing, D. Timmermann, A system for live localization in smart environments, IEEE ICNSC,

25 Variance RTI Problem: Through-wall RSS changes don t fit attenuation model Use short-term RSS variance for y Average error: cm, in 72 m 2 area 4 RSS (dbm) RSS (dbm) RSS (dbm) Link (27,0) to (15.45,26.4) Link (6,0) to (20,26.4) Vacant network area Stationary human obstructing link Moving human obstructing link Time (samples) 4 J. Wilson and N. Patwari, See through walls: motion tracking using variance-based radio tomography networks, IEEE TMC, 2011.

26 Multiple Channel RTI Fading condition diversity. Anti-fade links are most informative Spatial model (ellipse width) should be a function of fade level and sign of RSS change 5 Auto-update calibration for long-term apartment (23 cm error) 6 5 O. Kaltiokallio, M. Bocca, N. Patwari, A fade level-based spatial model for radio tomographic imaging, IEEE TMC, M. Bocca, O. Kaltiokallio, and N. Patwari, Radio tomographic imaging for ambient assisted living, Evaluating AAL Systems Through Competitive Benchmarking, 2013.

27 Multiple Person Tracking Particle filtering, m error 7 RTI-based, real-time, 1-4 people, < 55 cm error 8 7 F. Thouin, S. Nannuru and M. Coates, Multi-target tracking for measurement models with additive contributions, ICIF M. Bocca et al., Multiple target tracking with RF sensor networks, IEEE TMC, 2013.

28 RTI in 3-D Detect, classify vehicles on road 9 Classify person s pose 10 9 C.R. Anderson, R.K. Martin, T.O. Walker, R.W. Thomas, Radio tomography for roadside surveillance, IEEE JSTSP, B. Mager, N. Patwari, M. Bocca, Fall detection using RF sensor networks, PIMRC 2013.

29 RSS Fingerprint Attenuation/variance/histogram on each link forms high dimensional vector Train w/ person at each grid location Learn map from RSS vector to coordinate 2 m median error in hallways of 1500 m 2 area m avg. error in 150 m 2 area, tracking four people M. Seifeldin et al., Nuzzer: a large-scale device-free passive localization system for wireless environments, IEEE TMC C. Xu et al., SCPL: indoor device-free multi-subject counting and localization using RSS, IPSN 2013.

30 RSS Fingerprint: Pros and Cons Need training w/ person on each grid point No need for sensor coords Increased complexity in # people Database degrades as other things move

31 RSS Fingerprint: Degradation Original state: 8 Y coordinate (m) TV console Boxes of Books House plant Dining set Coat rack Bedroom door Sink Bags of groceries Washing machine Bathroom door 2 Couch 1 Node locations 0 0 Final state: X coordinate (m) Y coordinate (m) Filing cabinet Ironing board Node locations X coordinate (m) Random change, retest, repeat Error rate doubles each six changes, regardless of method

32 Statistical Inversion Method I Joint person tracking and sensor location 13 Expectation Maximization (EM)-based algorithm 30 cm error (open field, 49 m 2 ) 13 Xi Chen et al., Sequential Monte Carlo for simultaneous passive device-free tracking and sensor localization using received signal strength measurements, IPSN 2011.

33 Statistical Inversion Method II Learning of distribution of each link 14 Gaussian mixture model cm error (open field) 14 A. Edelstein, M. Rabbat, Background subtraction for online calibration of baseline RSS in RF sensing networks, IEEE TMC Y. Zheng and A. Men, Through-wall tracking with radio tomography networks using foreground detection, WCNC 2012.

34 Robust Line Crossing Location Estimation s1 s2 s3 State (a) j=1 j= N Short Segments Border Link Lines Nodes (b) Given: Link RSS measurements Nodes Person Location 0 Time Person's Track Short Segment Problem: Find between which nodes a person crossed. Each link is unreliable. Use redundant (longer links). How? Error correction coding 16, hidden Markov model P. Hillyard et al., You re crossing the line, IEEE SPW P. Hillyard et al., Demo: Detecting and Localizing Border Crossings Using RF Links IPSN 2015.

35 Spatial Model for Variance Need model for: What is the variance vs. person position? Measurement at Bookstore, nodes on shelves Normalize link, person position s.t. x r = (-1, 0), x t = (1,0) Find average variance by human position w.r.t. RX, TX

36 Spatial Model: Setup Human = tall cylinder diameter D Reflectors in a plane. TX, RX, in plane z above 18 Propagation via single reflection, path loss d n 18 N. Patwari and J. Wilson, Spatial models for human motion-induced signal strength variance on static links", IEEE Trans. Info. Forensics & Security, 2011.

37 Spatial Model: Results Mean variance spatial functions: Y Coordinate X Coordinate Matches with our, others results

38 Outline 1 Introduction 2 RSS Device-Free Localization 3 Context Beyond Location 4 Conclusion

39 Breathing Rate Estimation Norm. Avg. PSD Actual Breathing Rate Normalized Average PSD Frequency (Hz) Breathing causes periodic change in RSS Measure many channels RSS over time (30 s) 19 Peak of avg. PSD. Error: about 0.4 breaths/min 19 O. Kaltiokallio et al., Catch a breath: non-invasive respiration rate monitoring via wireless communication, IPSN 2014.

40 Channel State Information (CSI) Measurement Hacked driver for Intel WiFi n 5300 NIC 20 Gives channel gain (amplitude and phase) For 30 subcarriers from among OFDM subcarriers For each antenna pair in 3x3 MIMO 20 D. Halperin, Tool Release: Gathering n Traces with Channel State Information, ACM SIGCOMM CCR, 2011.

41 Introduction RSS Device-Free Localization Context Beyond Location Conclusion Activity Recognition Activities vary in temporal, frequency, dist n characteristics Using training DB, can classify w/ machine learning S. Sigg, M. Scholz, et al., RF-sensing of activities from non-cooperative subjects..., IEEE TMC B. Wei, W. Hu, M. Yang, C.T. Chou, Radio-based Device-free Activity Recognition with Radio Frequency Interference, IPSN 2015.

42 Person Counting Problem: Estimate the # of people moving randomly in area Solution: Match RSS hist to analytical pdf S. Depatla, A. Muralidharan and Y. Mostofi, "Occupancy Estimation Using Only WiFi Power Measurements," IEEE JSAC 2015.

43 Gesture Recognition from Micro-Doppler Gestures (left) vary in time-doppler characteristics Small (<20 Hz) Doppler can be estimated from OFDM packets (using an SDR RX) 24 Can be measured from AM at a passive RFID tag Q. Pu, S. Gupta, S. Gollakota, and S. Patel, Whole-Home Gesture Recognition Using Wireless Signals, MobiCom B. Kellogg, V. Talla, and S. Gollakota Bringing Gesture Recognition to All Devices, NSDI 2014.

44 Keystroke Recognition Typing on a keyboard near a TX changes MIMO channel 26 Phase of changes can be measured and used to estimate key typed Typing a few known words allows training Implemented with SDRs, but could be done with n CSI 26 B. Chen, V. Yenamandra and K. Srinivasan, Tracking Keystrokes Using Wireless Signals, MobiSys 2015.

45 Outline 1 Introduction 2 RSS Device-Free Localization 3 Context Beyond Location 4 Conclusion

46 Open Research Area: Modeling How to model human position effect on channel Statistical, temporal Function of link length, environment, fade level

47 Open Research: Context Awareness Need for fundamental temporal, Doppler, statistical features from gestures and activities to reduce training req ts. Interface to channel estimates made on COTS RFICs Compared to radar devices, incomplete data Find estimators, bounds for estimators, from such data

48 Commercialization: Xandem RSS-based motion detection system

49 Security Pain Point

50 Conclusion Channel estimation in wireless comms enables context awareness sensing Significant commercial needs Eg: localization, monitoring, activity, gesture recognition Many estimation, detection, classification problems yet to be solved

51 Introduction RSS Device-Free Localization Context Beyond Location Conclusion Questions and Comments More info on