Interpreting streaming biosignals: in search of best approaches to augmenting mobile health monitoring with machine learning for adaptive clinical decision support V.M. Jones, R. Batista, R.G.A. Bults, H. op den Akker, I. Widya, H. Hermens, R. Huis in t Veld, T. Tonis, M. Vollenbroek Hutten Presenter: Val Jones Telemedicine Group University of Twente v.m.jones@utwente.nl LEMEDS workshop: Learning from Medical Data Streams 13th Conference on Artificial Intelligence in Medicine (AIME'11), July 2 6, 2011, Bled, Slovenia /
Outline Brief overview of our research with Body Area Networks in healthcare Two illustrative applications of health BANs The need to extend with Machine Learning and Data Mining Discussion on best approaches and future directions
Outline Brief overview of our research with Body Area Networks in healthcare Two illustrative applications of health BANs The need to extend with Machine Learning and Data Mining Discussion on best approaches and future directions
The Telemedicine Group at the University of Twente (UT) Part of Biomedical Signals and Systems Investigate smart and ambulatory systems for remote monitoring and treatment Technologies Body Area Networks (BANs), wearable devices, mobile & wireless technologies http://telemedicine.ewi.utwente.nl/
LEMEDS workshop goal to bring together experts in data stream mining interested in medical applications and medical domain experts interested in timely analysis of their data streams for clinical decision support We are Computer Science researchers who work with clinicians (problem owners) offering an application area and looking to this workshop for input from experts in data stream mining
Health BANs at the University of Twente (UT) Since 1999 researching mobile applications for healthcare. In 2001, while in WWRF, developed concept: Body Area Networks (BANs) + wireless communications + wearable devices (sensors, actuators...) > remote monitoring and treatment services for patients Since 2002 (IST Mobihealth) developed and trialled various health BANs
BAN based m health systems BAN network of communicating devices worn on, in or around the body which provide mobile services to the wearer BAN networking technology which can be applied in healthcare domain to provide telemedicine services: Health BAN BAN = (MBU, set(ban device)) BAN device = sensor actuator MM device... (MBU local processing, storage and comms)
IntraBAN communications Wired or Wireless eg BlueTooth ExtraBAN communications Wireless eg. GPRS UMTS WiFi
Starting in 2002 with MobiHealth, 3 generations of health BAN projects Mobihealth BANs 2.5/3G comms m-health trials X-MOTION teleambulance HealthService24 Business models Preparation for commercialisation Awareness Context aware Location-based Interpretation of biosignals MOSAIC/ Ami@Work Future vision: AmI-suits, AmIEs U-CARE COPD.com FOVEA Lifestyle advice Myotel myofeedback based teletreatment services Telemedicine Group, UT: health BAN projects CLEAR Telecare @ home Mobiguide (COPD) PGS
Pregnancy COPD Epilepsy Chronic pain PDA screen MobiHealth BV
BAN devices used to date SENSORS electrodes for measuring ECG and EMG pulse oxymeter motion sensors (step counters, 3D accelerometers), temperature and respiration OTHER DEVICES positioning devices alarm buttons a multi modal biofeedback device
Outline Brief overview of our research with Body Area Networks in healthcare Two illustrative applications of health BANs The need to extend with Machine Learning and Data Mining Discussion on best approaches and future directions
FREEBAND AWARENESS PROJECT: Epilepsy Application to detect in real time (or before onset) an epileptic seizure, by interpreting changes in biosignals (temporal lobe epilepsy) location detected by GPS appropriate action can be taken, such as: patient can be warned ( stop driving, or lie down ) help can be dispatched
Epilepsy BAN Activity sensor Xsens MT9-B inertial sensor, sensing 3D acceleration Electrodes Ag/AgCl contact electrodes (ECG -> heart rate) Mobi-8 Sensor front end * MBU (HTC P3600) + GPS
Seizure Detection experimental algorithm based on changes in HR Backend: algorithm based on Fourier analysis of R top intervals of the ECG signal BAN: less sophisticated (and less processing power intensive) algorithm using data fusion (heart rate and 3D accelerometer data) experimental algorithm ECG signals sampled at 1024 Hz. Heart rate increase calculated using two moving time windows 3D accelerometer data sampled at 128 Hz used to calculate patient s activity level and posture (lying or not) by making use of the earth s gravitational field. By fusing heart rate increase, activity level and posture information the algorithm is designed to distinguish between heart rate increases due to physical activity and due to seizure
m-health portal - ECG, activity, HR
Epilepsy application: map display
Awareness project: Epilepsy and Innovations: chronic pain applications Context awareness Interpretation of biosignals Location based services Chonic pain trials on real patients Epilepsy trials on healthy volunteers only, to evaluate technical performance of the system
FOVEA PROJECT: Health and Wellbeing Application Targeting support of sustainable behavioural change in relation to healthy lifestyle Sensing, decision support and feedback One application weight management Goal: promote health and prevent chronic illnesses associated with overweight/obesity Ongoing project
FOVEA weight management application real time personalised feedback and advice at the point of decision making Real time automatic registration of exercise and consumption > EE, EI peer to peer connects the user s personal device (BAN) to a food database (eg in a restaurant or supermarket) Prototype implemented in one restaurant Restaurant of the Future, Wageningen
Some of the research issues Applying : behavioural change theory (TTM, SoC models, target: inclined abstainer) Nutrition education theory Developing: Req s engineering methodology Investigating: How to provide effective decision support to motivate adherence to healthy lifestyle programme through medium and long term
Smartphone app: some screenshots User profile System knows user s weight management goals personal diet plan (worked out with dietician) agreed meal compositions (balanced nutrition)..
Mobile device Connects to Restaurant database And detects the buffets (using BT or UMTS) FOVEA system knows layout of buffets content of buffets System guides the user through the restaurant and gives advice at the point of decision making
List of buffets FOVEA system knows food and drink items on offer today at the restaurant (and their calorific values)
Compliant items Options for warm beverages "compliant" and "noncompliant" items (for this user) highlighted in green and orange Compliant means this item is part of the lunch composition selected by the user on this occasion
Energy expenditure and intake * Registration of food and drink consumption on the phone enables real time estimation of energy intake and helps the user to manage their daily energy budget Real time measurement of physical activity (using the smart phone s on board 3 axis accelerometer) is used to estimate energy expenditure
View an item The user can see the impact of consuming one item (here a mueslibol) on energy balance ( it will send their energy budget negative) This screen shows A mueslibol has150 KiloCalories A mueslibol is not compliant with current lunch composition The user is free to ignore advice *
CHECKOUT Food and drink items purchased are registered automatically Weight is captured and sent to the smartphone
There was only time to show a few features of the (mobile part of) the FOVEA system The FOVEA system, including the mobile system, is being trialled in 2011 at the RoF on 60 trial subjects selected from regular visitors to the RoF with BMI 25.00 29.99
Request to LEMEDS/AIME participants Your area is very relevant to FOVEA. Would you kindly fill in a short survey? (10 12 mins). We would value your response very much. Respondents enter a raffle to win an IPod Shuffle. https://fs6.formsite.com/noldus/form31/secure_index. html The questionnaire will be closed on August 1 st 2011.
Outline Brief overview of our research with Body Area Networks in healthcare Two illustrative applications of health BANs The need to extend with Machine Learning and Data Mining Discussion on best approaches and future directions
Two contrasting applications Medical VS health and wellbeing (prevention) Emergency scenarios VS non emergency Both involve mobile monitoring and feedback analysis and interpretation of (streaming) biosignals in combination with other knowledge and context sources in (near) real time
FOVEA could benefit from ML Example of mobile monitoring and feedback application which could be augmented with more intelligent decision support More effective if could monitor and learn from effectiveness of various behavioural change strategies re adherence to weight management guidelines > adapt better to individual user, improve chances of success in reaching and maintaining personal healthy lifestyle goals
Data mining Epilepsy one of many examples of mobile monitoring and feedback applications which could soon be continuously streaming huge quantities of physiologial and context data from many patients > possible discovery of new clinical knowledge DM + ML to accumulated biosignal and context data from many patients > potential to improve detection of ongoing seizures and ability to predict seizures
Classical KBS ESS approach to CDSSs Condition-specific Knowledge Base Condition specific clinical knowledge Best practice eg. clinical practice guidelines CDSS Reasoning component Conclusions Advice Explanation Patientspecific Patientspecific Patient- data data BAN specific data - data - biosignals BAN data - - biosignals - context info - context info Patient data Patient Patient data data Treatment Treatment Treatment plan plan plan EMR * (Van Melle, Shortliffe, and Buchanan)
Data mining over large population: Knowledge discovery Condition-specific Knowledge Base Task knowledge re biosignal analysis and interpretation Condition specific clinical knowledge Best practice eg. clinical practice guidelines CDSS Reasoning component Conclusions Advice Explanation RT Feedback RT Intervention Patientspecific Patientspecific Patient- data data BAN specific data - data - biosignals BAN data - - biosignals BAN data - - context -Biosignals info - context info Patient - context data info Patient data Treatment Patient data Treatment plan Treatment plan plan EMR Machine learning: Adaptation to Individual Patient BAN Augmented with medical data streams, ML + DM
Outline Brief overview of our research with Body Area Networks in healthcare Two illustrative applications of health BANs The need to extend with Machine Learning and Data Mining Discussion on best approaches and future directions
data stream processing from heterogeneous sources. Can we get inspiration from: 1) OS models, multi users, OS consumes and outputs multiple infinite IO streams, input streams sporadic, not predictable; issues like fairness, deadlock 2) FP work on stream modelling and processing (UST) 3) speech processing multi level parallel processing of some prefix of an infinite stream NLP/language engineering community
Some of our early ML experiments ML approach to improving patient compliance to feedback [Akker et al [16]] Classical ML techniques applied to physical activity and behavioural data from mobile monitoring to predict appropriate timing for feedback messages Extraction of contextual features Testing of supervised classifier schemes Feature selection analysis done using Genetic algorithms
[Wieringa et al [17] AIME 11] Extends approach with a structured message ontology Selection from alternative expressions of the same motivational message (intention) based on learning of individual preferences based on past compliance Ontology pruned according to patient context at the time of use
Some immediate challenges and opportunities Incorporation of real time input and automated analysis of streaming biosignals and context data into CDSSs Selection of best technical approaches and mechanisms for implementing adaptive CDS on a mobile platform Distribution of coherent CDS functionality across a fixed+mobile distributed environment Maintenance of consistency of knowledge and beliefs across the distributed environment (IST MobiGuide)
Thank you for your attention Survey URL https://fs6.formsite.com/noldus/form31/secure_index.html Telemedicine Group http://telemedicine.ewi.utwente.nl/ Biomedical Signals and Systems http://bss.ewi.utwente.nl/
*** Epilepsy application algorithms HR and activity level algos implemented on BAN BAN seizure detection algo could only be tested offline (healthy subjects data ) due to computational limitations of the PDA (2004 8) Seizure detection algorithm still needs to be tested on data from epileptic patients, before specificity, sensitivity etc. can be established The experiments with healthy subjects showed some false positives