HeadScan: A Wearable System for Radio-based Sensing of Head and Mouth-related Activities

Transcription

1 HeadScan: A Wearable System for Radio-based Sensing of Head and Mouth-related Activities Biyi Fang Department of Electrical and Computer Engineering Michigan State University Biyi Fang Nicholas D. Lane Mi Zhang Fahim Kawsar 1 IPSN Apr. 13, 2016

2 The age of wearables is upon us Accelerometer Mic Vision Physiology Wearables provide possibilities to understand oneself as well as the world. 2

3 Limitation of Current Sensors Accelerometer Complex human activities need multiple Accelerometers. 3

4 Limitation of Current Sensors Audio + Video Audio and Video are privacy intrusive. 4

5 Limitation of Current Sensors Physiology Most Physiology sensors require firm skin contact. 5

6 We need a novel sensing modality for wearables that is non-contact, privacypreserving while still providing rich information. 6

7 Radio as a Sensing Modality Radio sensing has attracted considerable attention: Indoor Activity [1] Track Indoor Movement [2] Crowd Counting [3] All of these applications use radios/wifi deployed in the ambient environment such as homes and public places. 7

8 What will radio sensing mean for wearables? Can it sense at a distance? People? Object? Material? Can it sense complex movements? Transportation mode? Can it sense physiological signals? Breath? Heart? Muscle? 8

9 Comparison of Wearable Sensing Modalities Multiple Privacy Contact Accelerometer Audio + Video Physiology Radio 9 We envision radio has a significant potential to provide new sensing capabilities for next-gen wearables.

10 This work represents our first step to realize our vision. 10

11 HeadScan Overview HeadScan is a wearable system that uses radio to sense head and mouth-related activities Rationale: Eat, Drink, Cough, Speak Important, e.g., healthcare, social computing etc. Existing wearable technologies have their limitations multiple privacy contact diet monitoring audio sociometer acoustic sensing 11 HeadScan solves all the problems.

12 Key Principle Radio Signal Tx Rx HeadScan Wearable Unit Static Moving 12

13 Observations Eat, Drink, Cough, Speak have distinctive shapes and periodicities. eat drink One Bite One Sip cough speak One Cough One Consonant Radio signal contains rich information of mouth/head movements. 13

14 Hardware Prototype Hardware contains two parts: Two omni-directional antennas as transmitter (1) and receiver (2) antenna. One wearable unit that contains two HummingBoard Pro. Each board connects with one Intel WiFi card for measuring CSI (3)

15 Radio Signal Processing Pipeline Wearable Unit Raw CSI Data Segmentation Segmented Data Low Pass Filter Filtered Data PCA Principal Components Recognized Activity Classifier Sparse Coefficient Vector Sparse Representation Feature Vector Feature Extraction 15

16 Noise Removal Low Pass Filter: Radio signal is noisy. Remove noise and outliers that are not caused by the targeted activities. We selected 20 Hz as our cut-off frequency. 16

17 Principle Component Analysis Principal Component Analysis (PCA) on subcarriers: CSI has 30 subcarriers and they are highly correlated. Select the first projection of CSI data Remove redundant information 17

18 Sparse Representation Feature Extraction: Similar to knn with difference in the distance definition Extract features based on previous observations eat drink cough speak 18

19 Sparse Representation Residual Determination (Distance): Construct overcomplete dictionary A: Sparse Representation of new sample Residual (distance) calculate 19

20 Sparse Representation Compare the distance: 20

21 Evaluation Setup Methodology: Compare Radio vs. Audio (2) [4] Examine different factors (5) Data Collection: Collected data from 7 participants Radio and Audio collected simultaneously Lab Setting Scripted Activities A total of 7.2 hours 5,171 samples -- collected 21

22 Result Comparison between Radio and Audio-based Sensing Wearable Experiments conducted in clean environment Leave-one-subjectout Cross Validation 22 Radio performs better with average accuracy 86.3% compared to 81.7% of Audio

23 Result Impact of Radio Signal Transmission Rate Configured at multiple sampling rate % 85.7% 80.6% 67.4% 63.7% The average accuracy of 100, 50, 10, 8 and 5 Hz are 86.2%, 85.7%, 80.6%, 67.4%, and 63.7% respectively. Minimum requirement to maintain above 80% of sampling rate is 10 Hz, which takes tiny part of wireless bandwidth.

24 Result Impact of Radio Transmitter/Receiver On-body Locations 92.0% 82.7% 69.2% 68.0% Abbreviation: C, Collar; S, Shoulder; Left/Right symbol Collar represents receiver/ transmitter antenna location. This is because Rx is much more sensitive to movements Shoulder that occur at a closer location to Rx. Shoulder and Collar (SC) are the best deploy locations of receiver and transmitter antenna, respectively. 24

25 Result Impact of Interference Caused by Nearby People Operated when there are 0, 1 and 2 people nearby the subject % 93% 90% Average accuracy are 90%, 93% and 90% when 0, 1 and 2 people nearby, respectively,. HeadScan wearable system is robust to the interference caused by nearby people.

26 Limitation & Future Work All the data collected are in the lab environment. We will conduct in-the-wild experiments in the future. Our prototype is a little bit bulky. We are designing a new prototype to make it more wearable. Our radio signal processing pipeline is heavyweight and thus runs offline. We will design an lightweight pipeline to make it online. 26

27 Conclusion We designed and implemented a radio based system with competitive performance, with features of privacypreserving and non-contact. We believe our research opens up a new direction. We have discovered more possibilities along this direction. Welcome to join us at MobiSys

28 Reference 1. Wei, Bo, et al. "Radio-based device-free activity recognition with radio frequency interference." Proceedings of the 14th International Conference on Information Processing in Sensor Networks. ACM, Adib, Fadel, Zachary Kabelac, and Dina Katabi. "Multi-person localization via rf body reflections." 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15) K. Yatani et al. BodyScope: A wearable acoustic sensor for activity recognition. In ACM Conference on Ubiquitous Computing,

29 Q & A Thank You 29