T-Labs Series in Telecommunication Services Series editors Sebastian Möller, Berlin, Germany Axel Küpper, Berlin, Germany Alexander Raake, Berlin, Germany
More information about this series at http://www.springer.com/series/10013
Robert Walter Whole Body Interaction with Public Displays 123
Robert Walter Telekom Innovation Laboratories Quality and Usability Lab Technische Universität Berlin Berlin Germany ISSN 2192-2810 ISSN 2192-2829 (electronic) T-Labs Series in Telecommunication Services ISBN 978-981-10-4456-4 ISBN 978-981-10-4457-1 (ebook) DOI 10.1007/978-981-10-4457-1 Library of Congress Control Number: 2017935559 Springer Nature Singapore Pte Ltd. 2017 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Printed on acid-free paper This Springer imprint is published by Springer Nature The registered company is Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore
Acknowledgements First, I want to thank my advisor Jörg Müller. He inspired and shepherded my work on the topic since my related diploma thesis on Using depth cameras to enable novel interaction techniques on public displays up to this day. I especially thank my co-authors David Lindlbauer, Gilles Bailly, Nina Valkanova, Michael Nischt, Andreas Bulling, and Florian Alt for a vivid collaboration and lots of great input during our studies. It was an honor and a lot of fun to work with you guys! A warm Thank you! goes to all the students that supported our studies, namely Ines Ben Said, Constantin Schmidt, Dieter Eberle, Viktor Miruchna, Martin Schuessler, Andreas Fender, Tiare Feuchtner, Tongyan Ning, Hossein Maktoufi, Maurice ten Koppel, Thor Bossuyt, and Niklas Hillgren. They carried on a significant workload, made over-hours when necessary and always offered their help. Moreover, I want to acknowledge the Quality and Usability Lab of Telekom Innovation Laboratories, chaired by Sebastian Möller for providing a great environment, an excellent working atmosphere, as well as access to shared expertise of an unparalleled team of outstanding international researchers across different fields throughout my entire thesis work. I thank Albrecht Schmidt, Daniel Michelis, Uta Hinrichs, Anne Roudaut, Céline Coutrix, and Christina Dicke for valuable feedback during writing of the papers. Big thanks also goes to the staff of Mensa TU Hardenbergstrasse, Telekom Shop 4010 in Berlin, betahaus Berlin, Factory Berlin, Telekom Shop Berlin-City at Kurfürstendamm, and the Alexander von Humboldt Institut für Internet und Gesellschaft (HIIG) for offering their facilities and providing valuable support during the field studies! Without their trust, openness to innovative technology and support, it would not have been possible to conduct studies in the field. The work presented in this thesis was funded by the European Institute of Innovation and Technology (EIT) and the German Federal Ministry of Education and Research (BMBF) in the framework for the Software Campus program (grant number 01IS12056). It was additionally supported by the Alexander von Humboldt Foundation, the Quaero program, the Max Planck Center for Visual Computing v
vi Acknowledgements and Communication (MPC-VCC), the Cluster of Excellence Multimodal Computing and Interaction of Saarland University (EXC-MMCI), the European Research Council (ERC), and the Deutsche Telekom AG (DTAG).
Contents Part I Introduction and Motivation 1 Introduction... 3 1.1 Context and Motivation... 4 1.2 Goals and Problem Statement... 5 1.2.1 Display Blindness... 6 1.2.2 Interaction Blindness... 6 1.2.3 Immediate Usability... 6 1.2.4 Motivation.... 7 1.3 Research Questions.... 7 2 Methodology... 9 2.1 Ethics... 10 Part II Background 3 State of the Art... 13 3.1 Interactive Public Displays and Digital Signage... 13 3.1.1 Stages of Interaction... 13 3.1.2 Overcoming Display Blindness... 15 3.2 Communicating Interactivity.... 15 3.2.1 Immediate Usability... 16 3.3 Visual Attention... 17 3.4 Revealing Gestures... 18 3.4.1 Gestures... 18 3.4.2 Gesture Revelation and Guidance... 20 3.4.3 Mid-Air Gestures... 21 3.5 Item Selection.... 22 3.5.1 Menu Techniques... 22 3.5.2 From Multi-touch to Mid-Air... 22 vii
viii Contents 4 Technology.... 25 4.1 Range Imaging... 25 4.1.1 Structured Light... 25 4.1.2 Time-of-Flight.... 27 Part III Studies 5 A Field Study on Communicating Interactivity.... 31 5.1 Introduction... 31 5.2 Psychological Cues and Interactivity... 32 5.2.1 Representation: Recognizing Oneself... 33 5.2.2 Abstraction, Biological Motion, and Body Schema... 34 5.2.3 Perceptual Causality and Animacy... 34 5.2.4 Relevance for this Study... 35 5.3 Studies... 35 5.3.1 Hardware and Implementation.... 36 5.3.2 Study Design.... 36 5.4 Preliminary Study... 36 5.5 Laboratory Study... 37 5.5.1 Conditions.... 37 5.5.2 Task and Stimulus.... 38 5.5.3 Apparatus and Design... 38 5.5.4 Results... 38 5.5.5 Discussion.... 39 5.6 Field Study... 40 5.6.1 Deployment.... 40 5.6.2 Conditions.... 41 5.6.3 Data Analysis... 42 5.6.4 Findings... 43 5.7 Conclusion... 51 6 Visual Attention Analysis... 53 6.1 Introduction... 53 6.2 Experiment... 54 6.2.1 Interface Elements.... 55 6.2.2 Participants and Apparatus... 56 6.2.3 Tasks.... 57 6.2.4 Procedure and Methodology... 58 6.3 Results... 59 6.3.1 Visual Attention... 59 6.3.2 Recognition and Recall... 60 6.3.3 Visual Behavior over Time... 61 6.3.4 Qualitative Findings... 61 6.4 Discussion... 62
Contents ix 6.4.1 Limitations... 63 6.5 Conclusion... 63 7 A Field Study on Visualizing Gesture Hints... 65 7.1 Introduction... 65 7.2 StrikeAPose... 67 7.2.1 Game... 67 7.2.2 Teapot Gesture... 68 7.2.3 Revelation Strategies... 69 7.3 Laboratory Study... 70 7.3.1 Experimental Design... 70 7.3.2 Results... 72 7.3.3 Optimizing Strategies... 73 7.4 Field Study... 74 7.4.1 Quantitative Results... 75 7.4.2 Qualitative Results... 76 7.5 Generalizability and Limitations... 82 7.6 Conclusion... 83 8 A Field Study on Mid-Air Item Selection... 85 8.1 Introduction... 85 8.2 Design Space... 87 8.3 Pilot Study... 93 8.3.1 Results... 93 8.4 Iterative Design Study... 94 8.4.1 Selection Techniques.... 95 8.4.2 Apparatus and Participants... 95 8.4.3 Instructions and Task... 96 8.5 Field Study... 99 8.5.1 System... 100 8.5.2 Techniques... 100 8.5.3 Data Analysis... 100 8.5.4 Results... 101 8.6 Recommendations.... 104 8.7 Conclusion... 105 Part IV Conclusion 9 Conclusion... 109 10 Future Work... 113 Colophon... 115 References... 117
Acronyms 1D 2D 3D ANOVA API ATM CLP CMS CV DOF DOOH FOV HCI IDE IR LCD LIDAR NUI OLED OOH PIP RGB RGB-D TOF VR WIMP One-dimensional Two-dimensional Three-dimensional Analysis of variance Application Programming Interface Automated Teller Machine City-Light-Poster Content Management System Computer Vision Degree of Freedom Digital Out-of-home Field of View Human Computer Interaction Integrated Development Environment Infrared Liquid Crystal Display Light Detection and Ranging Natural User Interface Organic Light-emitting Diode Out-of-home Picture-in-Picture Red, Green, Blue Red, Green, Blue, Depth Time-of-flight Virtual Reality Windows, Icons, Menus, Pointer xi