A Novel Tracking System for AAL Based on Smartphone Technology. DIEEI University of Catania, Italy

Transcription

1 A Novel Tracking System for AAL Based on Smartphone Technology DIEEI University of Catania, Italy

2 Outline AAL research activities at DIEEI The proposed methodology for user tracking The proposed methodology for the Monitoring of ADL & Fall events Experimental results

3 Whatis AAL? Ambient Assisted Living: smart use of technology to support well-being in the preferred living environment for people who might otherwise find this difficult (e.g. infirm or elderly people who want to continue living in their own homes) Quality of life Social interaction Active aging Well-being Safety Synergy between competencies to improve life quality of weak(impaired and elderly) people Why research on AAL & Assistive technologies? Social problem Disabled people represent 50 million persons in the EU (about 10% of the entire population) According to ISTAT ( ), 4.8% of the Italian population (2,800,000 persons) suffer from some kind of disability, the 75% of which are over-65 years old. New innovative solutions can have a major effect on quality of life, thus easing the pressure of increasing costs in European social and care systems Attractive potential market Fundings AAL Joint Programme The European Commission has formally proposed a continuation of the AAL Joint Programme for the period HORIZON 2020 Accessible and Assistive ICT, Embedded Accessibility of Future ICT.

4 Associazione Italiana Ambient Assisted Living DIEEI is a supporter of the AALOpenAssociation focusing on scientific and technological challenges posed by AAL. Monitoring of a tactile path Image enhancement by noise modulation Multisensor Haptic cane Sensors Methodologies Color sensor based on fuzzy logic Tactile perception

5 The proposed user tracking methodology Objective: to develop an User-Centered Designed (UCD) Assistive platform for mobility and assistance of weak users Approach: sensing and processing features of SmartPhones Methodology: multi-sensor data fusion and advanced signal processing Features: user-friendly device, high reliable algorithms for data mining Smartphone: Accelerometer Compass Gyroscope ADL + Fall detection + Feedback Server (PC) Signal Processing User-Environment Contextualization/ Interaction User tracking: Walking Distance Estimation (Step-Size) Heading Environment info + Smartphone Android Communication App LabviewDevelopment Environment User-Environment Interaction (UEI)

6 The proposed user tracking methodology Objective: to develop an User-Centered Designed (UCD) Assistive platform for mobility and assistance of weak users Approach: sensing and processing features of SmartPhones Methodology: multi-sensor data fusion and advanced signal processing Features: user-friendly device, high reliable algorithms for data mining UCD is a type of design and a process in which the needs, wants, and limitations of end users of a product are given extensive attention at each stage of the design process. Smartphone: Accelerometer Compass Gyroscope ADL + Fall detection + Feedback Server (PC) Signal Processing User-Environment Contextualization/ Interaction User tracking: Walking Distance Estimation (Step-Size) Heading Environment info + Smartphone Android Communication App LabviewDevelopment Environment User-Environment Interaction (UEI)

7 The proposed user tracking methodology Objective: to develop an User-Centered Designed (UCD) Assistive platform for mobility and assistance of weak users Approach: sensing and processing features of SmartPhones Methodology: multi-sensor App developed data fusion for Android andos advanced signal processing Features: user-friendly device, high reliable algorithms for data mining Smartphone: Accelerometer Smartphone: Compass Accelerometer Gyroscope Compass Gyroscope ADL + Fall detection + - Open Source Development Environment - Access to all sensor embedded into the smartphone Server (PC) - Wireless communication Signal (WiFi, Processing Server Bluetooth) (PC) Signal Processing App features: User-Environment - Read inertial and orientation sensors available Contextualization/ on the smartphone User-Environment Interaction - Refresh Rate fully configurable Feedback Contextualization/ (used 50Hz) - Sensors Data are transferred to Interaction PC through TCP/IP protocol Feedback User tracking: Walking Distance Estimation (Step-Size) Heading Environment info + Smartphone Smartphone Android Communication App Android Communication App LabviewDevelopment Environment LabviewDevelopment Environment User-Environment Interaction (UEI)

8 The proposed user tracking methodology Objective: to develop an Activities User-Centered of Daily Designed Living (UCD) Assistive platform for mobility and assistance of weak users Basic routine activities that people Approach: sensing and processing features Fall tenddetection do everyday without needing assistance. of SmartPhones Methodology: There are sixmulti-sensor basic ADLs: data fusion and advanced signal processing The fall in the elderly is a major public health problem as it causes many disabling Features: eating user-friendly device, high reliable algorithms for data mining fractures but also has dramatic psychological consequences which reduce the bathing independence of the person. dressing Smartphone toileting Step up Server (PC) It was established that the earlier the fall is reported, the lower is the rate of transferring(walking) Signal Processing morbidity-mortality. Step down continence Sitting Smartphone: Accelerometer Compass Gyroscope ADL + Fall detection Feedback User-Environment Contextualization/ Interaction Android Communication App An individual's ability to perform ADLs is important for determining what type of long-term care (e.g. nursing-home care or home care) and coverage the individual needs(i.e. Medicare, Medicaid or long-term care insurance). + User tracking: Walking Distance Estimation (Step-Size) Heading Environment info + LabviewDevelopment Environment User-Environment Interaction (UEI)

9 Dead Reckoning Techniques Reconstruction of the single step is performed by the combination of the Headingθ i and the Step Size S i. The absolute position is obtained by cumulating step by step the relative positions. y Y N Y = ΔS *cos(θ) X = ΔS *sinθ) Si Walking Path Reconstruction ΔS1 ΔS2 ΔS3 ΔS4 θi X x E The position error increases step by step. It needs periodic correction through absolute measurement of position. Useful to evaluate short distance path Useful to estimate physical capability of the user

10 Schematization of the proposed approach Methodology: multi-sensor data fusion and advanced signal processing Accelerometer Compass Gyroscope Θ Compass Compensation ʃ Compensation Θ Gyro CKF Θ Data Fusion θ Acceleration Step Size (SS) & Walking Distance (WD) Dynamic SS Static SS DFA SS Δs XY coordinates

11 Accelerometer An accelerometer is a device that measures static and dynamic accelerations. What are accelerometers useful for? - By sensing the amount of dynamic acceleration, you can analyze the way the device (and thereforethebodytowhichitisattached)ismoving. - By measuring the amount of static acceleration due to gravity, you can find out the angle thedeviceistiltedatwithrespecttotheearth.

12 Accelerometer Modeling Readout strategy Technology Output interface Analog Digital (I 2 C, SPI)

13 I 2 C & SPI Serial Peripheral Interface Is a synchronous serial data standard, named by Motorola, that operates in full duplex mode. SPI is often referred to as SSI(Synchronous Serial Interface). Devices communicate in master/slave mode where the master device initiates the data frame. Multiple slave devices are allowed with individual slave select lines. SCLK: serial clock (output from master); MOSI: master output, slave input (output from master); MISO: master input, slave output (output from slave); SS: slave select (active low, output from master). Alternative naming conventions are also widely used Frequencies: 40 MHz to 100 MHz Inter-Integrated Circuit Is a multimaster serial single-ended computer bus invented by Philips used for attaching low-speed peripherals to a motherboard, embedded system, cellphone, or other electronic device. I²C uses only two bidirectional lines: Serial Data Line (SDA) Serial Clock (SCL) I²C defines basic types of messages, each of which begins with a START and ends with a STOP Frequencies: 10 kbit/s (LSm) to 5 Mbit/s (UFm)

14 Magnetic Compass The magnetic compass consists of a magnetized pointer free to align itself with Earth's magnetic field. A compass is a magnetically sensitive device capable of indicating the direction of the magnetic North The strength of the earth's magnetic field is about 0.5 to 0.6 gauss and has a component parallel to the earth's surface that always points toward the magnetic north pole It is an absolute heading sensor. It is affected by environmental magnetic fields and ferromagnetic bodies which alter the Earth's magnetic field. The tilt problem Problem: The reading will be more inaccurate the further the compass is tilted. This problem occurs because the compass is only using the X and Y axis of the earths magnetic field. When the compass is not parallel to these axis the amount of magnetism felt by the needle will change based on how out of alignment the compass is to these axis. X h = X M * cos(pitch) + Z M * sin(pitch) y h = X M *sin(roll)*sin(pitch) + Y M *cos(roll) -Z M *sin(roll)*cos(pitch) heading = arctan(y h /x h )

15 Gyroscope A gyroscope is a device for measuring or maintaining orientation, based on the principles of angular momentum. Mechanically,agyroscopeisaspinningwheelordiscinwhichtheaxleisfreetoassumeanyorientation. Gyroscopes measure angular velocity, how fast something is spinning about an axis. It is not affected by gravity and magnetic fields.

16 Step Size Estimation Accelerometer A y, A z A z Step detection Step Size estimation Dynamic SS Static SS DFA SS Methodology: - Step Detection: Processing of the vertical axis A z of the accelerometer - Step Size Estimation : Three types of algorithms have been used: Static: exploit a constant value of the Step Size, evaluated in a calibration phase in case of a MEDIUM step size and a NORMAL walking speed. Dynamic: computes an adaptive estimation of the Step Size, using the frequency f i and the variance ν i of the forward acceleration: Dss = N α step= 1 f i + β ν + γ i α, β evaluated during the calibration phase (10 m for 3 times with 3 walking speed) Adaptive : a DF algorithm provides a combination of both the static and dynamic estimation depending on the acceleration variance.

17 Step recognition Smartphone is placed on the right hip Right Step Twodifferentheuristicthresholdsto recognize the step: about9 and 11 m/s 2 Left Step Twosetof values (right/leftstep)of α, β and γ parameters for the three walking speed: low, normal and fast.

18 Static and Dynamic Algorithms Test Results Three Test Speed Mode: SLOW, NORMAL, FAST Test Pattern Walked using : SMALL step sizes, MEDIUM step sizes.

19 Static and Dynamic Algorithms Test Results Three Test Speed Mode: SLOW, NORMAL, FAST Test Pattern Walked using : SMALL step sizes, MEDIUM step sizes. J wd Nom Estim WD WD % = 100 Nom WD J wd % LOW Speed NORMAL Speed FAST Speed SMALL MEDIUM SMALL MEDIUM SMALL MEDIUM WD dyn 3,9 7,2 4,8 0,3 8,5 8,1 WD stat 40,6 3,8 25,8 3,8 40,6 3,8

20 Heading Determination Accelerometer Compass Gyroscope Θ Compass Compensation ʃ Compensation Θ Gyro CKF Θ Methodology: Compass: the absolute Heading estimation is obtained by computing the data from the Tilt Compensated 3-axis Compass (Roll and Pitch) Gyroscope : the relative Heading estimation is given by integration of the 3-axis Gyroscope Constrained Kalman Filter (CKF): Adaptive Heading estimation by processing the Compass and Gyroscope by taking into account the variance of the two signals (noise).

21 The Kalman Filter The KF addresses the general problem of trying to estimate the state x ϵr n of a discrete-time controlled process governed by the linear stochastic difference equation with a measurement : w k and v k represent the process and measurement noise. They are assumed to be indipendent, white and with normal probability distributions : Target: to findthe estimate of the signal x p(w) ~ N(0, Q) p(v) ~ N(0, R)

22 The Kalman Filter The KF addresses the general problem of trying to estimate the state x ϵr n of a discrete-time controlled process governed by the linear stochastic difference equation with a measurement : w k and v k represent the process and measurement noise. They are assumed to be indipendent, white and with normal probability distributions : Target: to findthe estimate of the signal x p(w) ~ N(0, Q) p(v) ~ N(0, R) Kalmanfilter finds the most optimum averaging factor for each consequent state. Also somehow remembers a little bit about the past states.

23 The KF Step-by-Step Step 1 Build a Model A, B and H are in general form matrices, but in most cases can be considered just numeric values. Moreovertheycanchangebetweenstatesbutmostofthetimewecanassumethatthey reconstant Required: estimation of the mean and standard deviation of the noise functions w k-1 and v k. Step 2 Start the Process The next step is to determine the necessary parameters and initial values. Two distinct set of equations applied at each k th state : Time Update (prediction) Measurement Update (correction To start the process, we need to know the estimate of x 0, and P 0.

24 The KF Step-by-Step Step 3 Iterate The previous estimates will be the input for the current state. prior estimate Posterior estimate prior error covariance

25 Experimental results and test Users in good health have been requested to perform five repetition of a 20 m long square walking path Gyroscope heading information is affected by a drift while compass is influenced by magnetic noise. The CKF estimation is strongly anchored to the compass heading estimation while rejecting magnetic noise in the compass signal by following the gyroscope trend.

26 Experimental results and test Users in good health have been requested to perform five repetition of a 20 m long square walking path J θ Nom Estim θ θ % = 100 Nom θ Total Measured Heading (target 1710 ) COMPASS GYRO CKF J θ % Euclidean Distance(m) Gyroscope heading information is affected by a drift while compass is influenced by magnetic noise. The CKF estimation is strongly anchored to the compass heading estimation while rejecting magnetic noise in the compass signal by following the gyroscope trend.

27 4 Compass Heading - Dynamic StepSize 2 Gyroscope Heading - Dynamic StepSize 2 Kalman Heading - Dynamic StepSize Y axis [m] 0-2 Y axis [m] -2-4 Y axis [m] X axis [m] X axis [m] X axis [m] WALKING DISTANCE (nominal 100m): Walking distance STATIC 101,38 DYNAMIC 99,60 TOTAL ROUNDS (NOMINAL 1710 ): COMPASS GYRO KALMAN HEADING ROUNDS

28 Nominal walking distance = 357 m