PROJECT FINAL REPORT Grant Agreement number: 299408 Project acronym: MACAS Project title: Multi-Modal and Cognition-Aware Systems Funding Scheme: FP7-PEOPLE-2011-IEF Period covered: from 04/2012 to 01/2013 Name of the scientific representative of the project's co-ordinator, Title and Organisation: Prof. Peter Robinson University of Cambridge, UK Tel: +44 1223 334637 Fax: - E-mail: pr10@cl.cam.ac.uk Project website address: N/A
1 Final publishable summary report Our eyes are involved in nearly everything that we do in our daily lives. In addition to physical activity, eye movements are also strongly linked to several processes of visual cognition and attention. This suggests that eye movements carry additional information on a user s context that are difficult if not impossible to infer using common modalities, such as body movements or physiological measurements. The first objective of the MACAS project was to develop new knowledge and technology for automatic analysis and recognition of cognitive processes from human visual behaviour a challenge at the core of ambient intelligence toward systems and services that are proactive and adaptive to human needs. A second objective was to push the state-of-the-art in using gaze for natural and efficient, i.e. calibration-free, human-computer interaction. MACAS finished early due to the fellow taking up a new research position abroad. Nevertheless, the project made a number of contributions torwards the above research objectives. Core contributions of the project during the reporting period include, based on earlier work of the fellow, further investigations on automatic inference of visual memory recall from visual behaviour, a (so far unpublished) study on inferring user expertise as well as another study on inferring document types from visual behaviour, a robust method for smooth pursuit detection as well as for using smooth pursuit movements for interaction and eye tracker calibration, a novel wearable EOG head cap particularly geared for long-term eye movement recordings, as well as two prototype computer vision systems for eye gesture recognition and model-based gaze estimation on unmodified handheld devices. Through close collaborations with researchers in the UK,, Germany and Japan, MACAS further contributed to closely related research efforts on calibration-free gaze-based interaction with situated displays and gaze-based object transfer between ambient and hand-held devices, as well as physical behaviour modelling and qualitative activity recognition. Finally, through the MACAS project, we organised two workshops on pervasive eye tracking and mobile eye-based human-computer interaction at the leading conferences in ubiquitous computing (UbiComp) and eye movement research (ECEM). 2 Use and dissemination of foreground This project spawned 11 international conference and two journal publications, one of which is in the prestigious ACM Surveys. Two international workshops in pervasive eye tracking and mobile eye-based interaction were co-organised by the researcher. Four separate funding proposals have been submitted by the fellow during the reporting period: one DFG Emmy Noether starting grant, one FET Young Explorer grant, one FET Open grant, and one Marie Curie ITN grant.
Section A (public) TEMPLATE A1: LIST OF SCIENTIFIC (PEER REVIEWED) PUBLICATIONS, STARTING WITH THE MOST IMPORTANT ONES N O. Title 1 A Tutorial on Human Activity Recognition Using Body- Worn Inertial Sensors 2 MotionMA: Motion Modelling and Analysis by Demonstration 3 SideWays: A Gaze Interface for Spontaneous Interaction with Situated Displays 4 EyeContext: Recognition of High-level Contextual Cues from Human Visual Behaviour 5 Eye Pull, Eye Push: Moving Objects between Large Screens and Personal Devices with Gaze and Touch 6 Qualitative Activity Recognition of Weight Lifting Exercises 7 Pursuits: Spontaneous Interaction with Displays based on Smooth Pursuit Eye Movement and Moving Targets 8 I know what you are reading Recognition of document types using mobile eye tracking 9 Eye Gesture Recognition on Portable Devices 10 AutoBAP: Automatic Coding of Body Action and Postures Units from Wearable Sensors Main author Andreas Bulling Eduardo Velloso Yanxia Zhang Andreas Bulling Jayson Turner Eduardo Velloso Melodie Vidal Kai Kunze Vytautas Vaitukaiti s Eduardo Velloso Title of the periodical or the series Number, date or frequency Publisher Place of publication Year of publicati on Relevant pages Permanent identifiers (if available) ACM Surveys Monthly ACM Press journal 2013 no CHI 2013 Yearly ACM Press Paris, France 2013 1309-1318 10.1145/2470654.2466171 no CHI 2013 Yearly ACM Press Paris, France 2013 851-860 10.1145/2470654.2470775 no CHI 2013 Yearly ACM Press Paris, France 2013 305-308 10.1145/2470654.2470697 no Interact 2013 Yearly Springer Cape Town, South Africa AugmentedH uman 2013 UbiComp 2013 Yearly ACM Press Stuttgart, Germany Yearly ACM Press Zurich, ISWC 2013 Yearly ACM Press Zurich, PETMEI 2012 Yearly ACM Press Pittsburgh, USA ACII 2013 Yearly IEEE Geneva, 2013 170-186 10.1007/978-3-642-40480- 1_11 2013 116-123 10.1145/2459236.2459256 no 2013 439-448 10.1145/2493432.2493477 no 2013 113-116 10.1145/2493988.2494354 no 2012 711-714 10.1145/2370216.2370370 no 2013 no Is/Will open access provide d to this publica tion? no
11 Pursuit Calibration: Making Gaze Calibration Less Tedious and More Flexible 12 Wearable Eye Tracking for Mental Health Monitoring 13 EyeTab: Model-based gaze estimation on unmodified tablets Ken Pfeuffer Melodie Vidal Erroll Wood UIST 2013 Yearly ACM Press St. Andrews, UK 2013 261-270 10.1145/2501988.2501998 no Computer Communicati ons 35(11) Elsevier journal 2012 1306-1311 10.1016/j.comcom.2011.11.002 no ETRA 2014 Biannual ACM Press Safety Harbor, 2014 no Florida TEMPLATE A2: LIST OF DISSEMINATION ACTIVITIES NO. Type of activities Main leader Title Date/Period Place Type of audience Size of audience Countries addressed 1 Publications Andreas Bulling ACM Surveys 2 Publications Eduardo Velloso 31st SIGCHI Human Factors in Systems (CHI 3 Publications Yanxia Zhang 31st SIGCHI Human Factors in Systems (CHI 4 Publications Andreas Bulling 31st SIGCHI Human Factors in Systems (CHI 5 To appear Journal April/May 2013 Paris, France April/May 2013 Paris, France April/May 2013 Paris, France Publications Jayson Turner 14th IFIP TC13 Human-Computer Interaction (INTERACT Cape Town, South Africa
6 Publications Eduardo Velloso 4th Augmented Human Conference (AugmentedHum 7 Publications Melodie Vidal Joint Pervasive and Ubiquitous (UbiComp an March 2013 Stuttgart, Germany Zurich, 8 Publications Kai Kunze 17th Symposium on Wearable Computers (ISWC Zurich, 9 Publications Vytautas Vaitukaitis 2nd Workshop on Pervasive Eye Tracking and Mobile Eye-based Interaction (PETMEI 2012) September 2012 Pittsburgh, USA 10 Publications Eduardo Velloso 5th biannual Humaine Association Affective and Intelligent Interaction (ACII 11 Publications Ken Pfeuffer 26th ACM Symposium on User Interface Software and Technology (UIST 12 Publications Melodie Vidal Computer Communications October 2013 Geneva, St. Andrews, UK June 2012 journal 13 Publications Erroll Wood 8th Symposium on Eye Tracking Research and Applications To appear Safety Harbor, USA
14 Workshops Andreas Bulling Ubiquitous September 2012 15 Workshops Andreas Bulling European Eye Movements August 2013 Pittsburgh, USA Lund, Sweden