Labels - Quantified Self App for Human Activity Sensing. Christian Meurisch, Benedikt Schmidt, Michael Scholz, Immanuel Schweizer, Max Mühlhäuser

Transcription

1 Labels - Quantified Self App for Human Activity Sensing Christian Meurisch, Benedikt Schmidt, Michael Scholz, Immanuel Schweizer, Max Mühlhäuser

2 MOTIVATION Personal Assistance Systems (e.g., Google Now) Source: giga.de Human Behavior Dataset User study / Tools This talk: Quantified Self Instrumentation for Human Activity Sensing 2

3 CHALLENGES Tracking of Personal Data User s point of view Willingness Effort Privacy Releasing of data Technical view Continuous tracking of sensor data Invisible tracking (e.g., energy-efficient) Annotation of sensor data Handling of heterogenous data Multiple devices/platforms support 3

4 Kraken.me OUR APPROACH Previous Work Multi-device user tracking suite [1] mobile (Android, ios) desktop (Windows, Linux, Mac) social (Facebook, Twitter, ) Sensors [2] physical (e.g., acceleration, location) virtual (e.g., calendar, call logs) social connector (e.g., friends list) Automatic tracking of sensor data [1] UbiComp 14 Workshop: HASCA [2] MobiCase 14 4

5 OUR APPROACH Previous Work Labels App Annotation feature (UI) 5

6 Labels OUR APPROACH Labels App Quantified Self App for Human Activity Sensing Providing pre-defined time slots of daily routines based on location changes User need to annotate his activities Optional: Tagging Optional: Adjustment of time slots Integrated into Kraken.me infrastructure Manual annotation of sensor data (ground-truth) 6

7 OUR APPROACH Labels App: Annotation Support Time slot detection based on location changes (DBSCAN) adjustable (merging, splitting, removing) 7

8 OUR APPROACH Labels App: Annotation Support Time slot detection based on location changes (DBSCAN) adjustable (merging, splitting, removing) Pre-defined labels Plausibility check: rule-based selection e.g., Transition label is not available at places 8

9 OUR APPROACH Labels App: Annotation Support Time slot detection based on location changes (DBSCAN) adjustable (merging, splitting, removing) Pre-defined labels Plausibility check: rule-based selection e.g., Transition label is not available at places Reducing annotation effort Increasing annotation quality 9

10 OUR APPROACH Labels App: Metrics Capture user s annotation behavior Quantity assurance covered time labels per time slots Traffic light principle Quality assurance tagging behavior human reaction time powers of recollection 10

11 OUR APPROACH Labels App: in recent months Labels 2.0 (Android) considering user feedback Android Wear (Smartwatch) simpler annotation process 11

12 OUR APPROACH Labels App: in recent months ios App larger (potential) target group 12

13 FIELD STUDY 195 students of TU Darmstadt were recruited 163 students completed the four-week study Gender: 76% male, 24% female Age: ø 25.4 years Course of study: 90% Master, 10% Bachelor Practical phase of seminar course Requirements: Android 4.x phone, Facebook account ~16,4 % dropout rate # Quantified Self 24/7 over 4 weeks " Questionnaires

14 FIELD STUDY Quantified Self Dataset TIME PERIOD PARTICIPANTS PRE-DEFINED ACTIVITIES 28d ANNOTATED TIME SLOTS SENSOR DATA Ø FACEBOOK FRIENDS / USER 43K 277M

15 FIELD STUDY Quantified Self Dataset 24/7 self-tracking over 4 weeks ( ) TIME PERIOD PARTICIPANTS PRE-DEFINED ACTIVITIES 28d ANNOTATED TIME SLOTS SENSOR DATA Ø FACEBOOK FRIENDS / USER 43K 277M

16 FIELD STUDY Quantified Self Dataset Students of TU Darmstadt TIME PERIOD PARTICIPANTS PRE-DEFINED ACTIVITIES 28d ANNOTATED TIME SLOTS SENSOR DATA Ø FACEBOOK FRIENDS / USER 43K 277M

17 FIELD STUDY Quantified Self Dataset Student Activities Lecture (L) Exercise Group (E) Learning (LN) Teamwork (TW) Meeting (M) Break (B) TIME PERIOD PARTICIPANTS PRE-DEFINED ACTIVITIES 28d ANNOTATED TIME SLOTS SENSOR DATA Ø FACEBOOK FRIENDS / USER 43K 277M 7.2 Other Activities Sleeping (S) Transition (T) Job (J) Leisure (LS) Custom (C) Unknown (U) 17

18 FIELD STUDY Quantified Self Dataset TIME PERIOD PARTICIPANTS PRE-DEFINED ACTIVITIES 28d ANNOTATED TIME SLOTS SENSOR DATA Ø FACEBOOK FRIENDS / USER 43K 277M 7.2 Count (raw) Count (filtered) Time slots (total) 52,609 35,416 - auto-created 46,440 32,096 - user-created 6,169 3,320 - stationary 38,891 26,877 - transition 13,718 8,539 Time slots w/ annotation 43,393 35,416 - w/ tag(s) 31,998 31,998 Tags (total) 39,908 39,908 18

19 FIELD STUDY Quantified Self Dataset TIME PERIOD PARTICIPANTS PRE-DEFINED ACTIVITIES 28d ANNOTATED TIME SLOTS SENSOR DATA Ø FACEBOOK FRIENDS / USER 43K 277M 7.2 ~82.5 % (!) annotated time slots Count (raw) Count (filtered) Time slots (total) 52,609 35,416 - auto-created 46,440 32,096 - user-created 6,169 3,320 - stationary 38,891 26,877 - transition 13,718 8,539 Time slots w/ annotation 43,393 35,416 - w/ tag(s) 31,998 31,998 Tags (total) 39,908 39,908 19

20 FIELD STUDY Quantified Self Dataset TIME PERIOD PARTICIPANTS PRE-DEFINED ACTIVITIES 28d ANNOTATED TIME SLOTS SENSOR DATA Ø FACEBOOK FRIENDS / USER Count (raw) 43K 277M Other J 7.2 Count (filtered) Time slots (total) 52,609 35,416 - auto-created 46,440 32,096 - user-created 6,169 3,320 - stationary 38,891 26,877 - transition 13,718 8,539 Time slots w/ annotation 43,393 35,416 - w/ tag(s) 31,998 31,998 Tags (total) 39,908 39,908 Group Abbr. Activity Count Ratio Student Activities Activities L Lecture 2, % E Exercise % LN Group Learning 6, % TW Teamwork 1, % M Meeting % B Break 6, % S Sleeping 4, % T Transition 10, % Job 1, % LS Leisure 6, % C Custom 2, % U Unknown % Total 43, % 20

21 FIELD STUDY Quantified Self Dataset TIME PERIOD PARTICIPANTS PRE-DEFINED ACTIVITIES 28d ANNOTATED TIME SLOTS SENSOR DATA Ø FACEBOOK FRIENDS / USER 43K 277M 7.2 Sensor type Sensor data Sampling method (trigger) Count Physical sensor Virtual sensor Acceleration Periodic (system) 15,655,893 Location Periodic (system) 24,679,844 Light On changes (system) 148,382,777 Network connections On changes (system) 1,120,029 Ringtone mode Periodic (manual) 9,005,776 Browsing history Periodic (manual) 3,060,571 Application usage On changes (system) 4,832,596 Call logs On changes (system) 910,614 Calendar events On changes (system) 78,559 Phone contacts On changes (system) 68,470 Google activities Periodic / on changes (system) ( 69,236,646 Additional sensors Power status Network traffic WiFi Scans Total 277,031,775 21

22 FIELD STUDY Quantified Self Dataset TIME PERIOD PARTICIPANTS PRE-DEFINED ACTIVITIES 28d Other Social Networks Twitter Google+ Foursquare Microsoft Live LinkedIn Xing ANNOTATED TIME SLOTS SENSOR DATA Ø FACEBOOK FRIENDS / USER 43K 277M Distribution Count of participating friends 22

23 FIELD STUDY Questionnaires Questionnaire I Demographic information Labels app rating (SUS, UEQ) 163 students Questionnaire II Scientific tests Personality (BFI-44) Social Skills (SSI-90) Academic Motivation (AMS-28) 52 students 23

24 STUDY RESULTS App Evaluation: Rating Please rate the Labels app on a five-star scale 3.37 ±

25 STUDY RESULTS App Evaluation: Rating Please rate the Labels app on a five-star scale 3.37 ± 0.98 Specific app features 2# 1# 0#!1#!2# Auto#detec-on# Manual#crea-on# Merging#/#spli8ng# Labeling# Tagging# 25

26 System Usability Scale (SUS) STUDY RESULTS App Evaluation: SUS 10-item scale to measure subjective usability Likert-scale (alternating questions) ± okay - ( good ) 26

27 STUDY RESULTS App Evaluation: UEQ User Experience Questionnaire (UEQ) 26-item questionnaire for measuring user experience captures 6 areas 3# 2# 1# 0#!1#!2#!3# A(rac,veness# Perspicuity# Efficiency# Dependability# S,mula,on# Novelty# 27

28 CONCLUSION User-supportive Quantified Self app: Labels user-annotated data physical + virtual sensor data social connector Overcome technical challenges Field test 163 students, 4 weeks (24/7) 43K annotated time slots (~82.5%) Usability: okay (SUS: ~70); User experience (UEQ): positive - Minimal effort - User-supportive UI - Invisible tracking Tools Dataset Analysis (Human Behavior) Providing Assistance / Intervention 28

29 FUTURE WORK Enhancement of our tracking suite (2015) Modular platform Mobile (Android, ios), Desktop Field studies: Student Assistant ( ) Sharing our tools Analyses of collected dataset ( ) Human behavior User routines Complex activity recognition User characterization Personality Social skills Good ideas? Open for cooperations Anonymization of our dataset for publication (~2016) 29

30 ' THE END Thanks for Your Attention!