Introduction to Mobile Sensing Technology

Size: px

Start display at page:

Download "Introduction to Mobile Sensing Technology"

William Carpenter
5 years ago
Views:

1 Introduction to Mobile Sensing Technology Kleomenis Katevas

2 Image by CRCA / CNRS / University of Toulouse

5 In this talk What is Mobile Sensing? Sensor data, but what can we actually sense? Sensor Fusion. Mobile Sensing on Android and ios. Continuous Sensing Platforms. Mobile Sensing in Research.

6 NFC WiFi GPS Ambient Light Sensor Barometer Camera(s) Bluetooth Magnetometer Microphone(s) Water Sensor Proximity Sensor Gyroscope Accelerometer Motion Coprocessor

7 Satellite Navigation Systems It calculates the location (latitude and longitude), altitude, and speed of the device based on the distance from at least four satellites (trilateration). Average accuracy is ~3m. Not suitable for indoor positioning. GPS is the most popular one, but others are available or coming in the future (GLONASS, COMPASS, Galileo, IRNSS).

Wireless networking technology. Can been used as an indoor positioning system (IPS) by triangulating the Received Signal Strength Indication (RSSI) of other Wi-Fi hot-spots.

8 Wireless networking technology. Can been used as an indoor positioning system (IPS) by triangulating the Received Signal Strength Indication (RSSI) of other Wi-Fi hot-spots. Some mobile devices combine GPS, cell tower triangulation and WiFibased location to improve accuracy. Read Survey of Wireless Indoor Positioning Techniques and Systems by Hui Liu et al.

9 Wireless networking technology for exchanging data over short distances. (~10 meters for Class 2 radios) Can also be used for indoor positioning. Very useful for spatial-temporal data gathering (participant mobility and social interactions). Bluetooth LE (BLE), marketed as Bluetooth Smart, has very low battery consumption.

All devices can be ibeacon transmitters, receivers (or both), but the technology

10 ibeacon Apple s implementation of Bluetooth Smart, used as an indoor positioning system. Classifies the location of the device as Immediate, Near or Far. All devices can be ibeacon transmitters, receivers (or both), but the technology is also open to third-party hardware. Also available in Android using thirdparty libraries.

11 ibeacon Data Format Company ID (Apple: 0x004C) UUID (16 byte hex) Minor (uint ) Type (always 0x02) Major (uint ) Measured Power (rssi in 1m distance)

12 Estimote Indoor Location

13 Eddystone Google s answer to ibeacon. Fully open-source specification (Apache v2.0) with source code available for Android and ios. Three frames: Eddystone-UID (similar to ibeacon ) Eddystone-URL (broadcasts a URL) Eddystone-TLM (broadcasts telemetry information)

14 Some terminology Six degrees of freedom (3D Translation & 3D Rotation)

15 Magnetometer Magnetic field sensor. Tells you the actual orientation of the device relative to the magnetic north (not the true north). Three axis magnetometer: x, y, z (East, North, Up)

16 Magnetometer Magnetic field sensor. Tells you the actual orientation of the device relative to the magnetic north (not the true north). Three axis magnetometer: x, y, z (East, North, Up)

and Roll (but not Yaw!) using trigonometric calculations.

17 Accelerometer A sensor that measures the force of acceleration of the device. 3 axis accelerometer: x, y, z (Surge, Heave, Sway) Can also calculate Pitch and Roll (but not Yaw!) using trigonometric calculations. Also used to understand the device orientation, by measuring the acceleration caused by the gravity.

18 Accelerometer A sensor that measures the force of acceleration of the device. 3 axis accelerometer: x, y, z (Surge, Heave, Sway) Can also calculate Pitch and Roll (but not Yaw!) using trigonometric calculations. Also used to understand the device orientation, by measuring the acceleration caused by the gravity.

19 Gyroscope 3 axis gyroscope: x, y, z (Pitch, Roll, Yaw) Tells you how much your device is being rotated over time. Less computationally expensive Pitch and Roll. Also provides Yaw.

20 Demo time

21 Comparing the three sensors + - Magnetometer No drift problem (depend on magnetic north) Noisy. Hard to calibrate. Not accurate for rotation detection. Tilt compensation. Accelerometer No drift problem (depend on earths gravity) Noisy, especially in high frequencies. Mixes linear acceleration and gravity. Gyroscope Smooth reading even in high frequencies. Great dynamic response. Large errors due to the drift problem.

22 What is sensor fusion? The use of multiple sensors so that they compensate each other s weaknesses. Users (developers) do not want to know about sensors. What they really want is accurate and responsive 3D translation, 3D rotation and orientation.

23 Accelerometer and Gyro Fusion (Sachs, 2010)

24 Accelerometer and Gyro Fusion Integration (Sachs, 2010)

25 Sensor Fusion (Sachs, 2010) Gyroscope Orientation 3D Rotation Accelerometer Gravity

26 Sensor Fusion (Sachs, 2010) Integration Gyroscope Orientation 3D Rotation Accelerometer Gravity

27 Magnetometer and Gyro Fusion (Sachs, 2010)

28 Sensor Fusion (Sachs, 2010) Gyroscope Orientation 3D Rotation Magnetometer Direction Accelerometer Gravity

29 Gravity and Linear Acceleration Linear Acceleration = Accelerometer Data - Gravity In most cases, it is very hard to separate Gravity from Linear Acceleration as gravity changes really fast. High pass filters do not work very well.

30 Sensor Fusion (Sachs, 2010) Gyroscope Orientation 3D Rotation Magnetometer Direction Accelerometer Gravity Linear Acceleration 3D Translation

31 Sensor Fusion (Sachs, 2010) Gyroscope Orientation 3D Rotation Magnetometer Accelerometer Direction Gravity Double Integration Linear Acceleration 3D Translation

32 Motion Coprocessors Apple s M7 / M8 / M9 chip. Continues measures motion data from Magnetometer, Accelerometer, Gyroscope and Barometer sensors. Automatically performs activity detection: stationary, walking, running, automotive, unknown. M8 also provides distance and elevation using the Barometer sensor. M9 can also track Pace and Cadence. Apps can query data of the past 7 days when loaded. Does not provide the raw data.

33 Sensing Motion in ios and Android Fused data of: Raw and Calibrated data of: - Accelerometer - Gyroscope - Magnetometer - User Acceleration - Gravity - Attitude - Rotation rate - Magnetic field

34 Mobile Sensing on Android Sampling Rate in motion sensors is not consistent. Timestamp is not consistent. Lack of a motion coprocessor, only Pedometers. (this will change very soon ) Android fragmentation problem. Bluetooth Smart (BLE) is only fully supported in Lollipop.

35 Sampling rate on Android Samsung Galaxy S2 Google Nexus 4 FASTEST 100 Hz 200 Hz GAME 50 Hz 50 Hz UI 15 Hz 30 Hz NORMAL 5 Hz 15 Hz

36 Mobile Sensing on ios Running a process in the background is (almost) not possible. Access to some sensors is limited (e.g. WiFi, NFC). Restrictions in the App Store.

SensingKit: A Multi-Platform Mobile Sensing Framework Platform Characteristics SensingKit Server Works in Android and ios mobile systems. Captures Motion, Location, Proximity, Environmental data.

37 SensingKit: A Multi-Platform Mobile Sensing Framework Platform Characteristics SensingKit Server Works in Android and ios mobile systems. Captures Motion, Location, Proximity, Environmental data. Power efficient using Bluetooth Smart (4.0). Easily extensible using a modular design. Automated time sync and data processing on the server. Available in open-source under the GNU LGPL v3.0. Sensor Modules Model Manager Data Communication Module Internet Web Services Data Plug-in System Data Extraction For more info, check

38 SensingKit: Battery Consumption Power (%) iphone 5S Idle Accelerometer (100Hz) Gyroscope (100Hz) Magnetometer (100Hz) Device Motion (100Hz) Location (Best Accuracy) ibeacon Broadcast (1Hz) ibeacon Scan (1Hz) ibeacon Scan + Broadcast (1Hz) Microphone ( Hz) Time (hours) Battery consumption of SensingKit running on an iphone 5S.

39 Detecting Group Formations

41 } P1 P1 P1 P2 P3 P4 P5 P6 P1 P1 P1 P2 P P2 P2 P2 P3 P P3 P3 P3 P4 P P4 P4 P4 P5 P P5 P5 P5 P6 P P6 P6 P P1 P2 P3 P4 P5 P6 P P P P P P time (sec) (a) Weighed Adjacency Matrices (b) Binary Adjacency Matrix (c) Graph

42 Results Confusion Matrix

43 Thank you! David Sachs, Sensor Fusion on Android Devices: A Revolution in Motion Processing, Google Tech Talk, February Retrieved from Kleomenis Katevas, Hamed Haddadi, Laurissa Tokarchuk, Richard G. Clegg, "Detecting Group Formations using ibeacon Technology", 4th International Workshop on Human Activity Sensing Corpus and Application (HASCA2016) in conjunction with UbiComp 2016, September 2016, Heidelberg, Germany. Kleomenis Katevas, Hamed Haddadi and Laurissa Tokarchuk, SensingKit: Evaluating the Sensor Power Consumption in ios devices, 12th International Conference on Intelligent Environments (IE'16), September 2016, London, UK. Kleomenis Katevas, Hamed Haddadi and Laurissa Tokarchuk, Poster: SensingKit -- A Multi-Platform Mobile Sensing Framework for Large-Scale Experiments, Extended abstract, ACM MobiCom 2014, Maui, Hawaii, September 2014.

IoT. Indoor Positioning with BLE Beacons. Author: Uday Agarwal

IoT. Indoor Positioning with BLE Beacons. Author: Uday Agarwal IoT Indoor Positioning with BLE Beacons Author: Uday Agarwal Contents Introduction 1 Bluetooth Low Energy and RSSI 2 Factors Affecting RSSI 3 Distance Calculation 4 Approach to Indoor Positioning 5 Zone