Embodied Interaction Research at University of Otago Holger Regenbrecht
Outline A theory of the body is already a theory of perception Merleau-Ponty, 1945 1. Interface Design 2. First thoughts towards a theory 3. Humble steps 1. Living Gallery 2. Presence Robot 3. Social Presence 4. Future work
The Human-Computer Interface office desk Win desktop
3DUI: Interfaces for virtual environments
The Interface Design Issue To date interface design mainly deals with the phenomena only (e.g. WIMP interface design); not well-grounded Metaphors are successfully used: seems to be reasonable to rely on in further R&D BUT Metaphors are often used in wrong context (e.g. desktop) AND Metaphors are chosen accidentally We see a strong need for founding principles to be developed on a theoretical and empirical basis
First thoughts towards a theory
First Thoughts::Embodied Interaction Paradigm Approach In search for guidance for interaction design in virtual and real environments To date new interface development mainly based on guess, trial, and error Embodied Interaction paradigm could be one promising approach for improved grounding Basis: Heidegger (Geworfenheit / Handeln), Glenberg (embodied cognition), J.J. Gibson (Affordances), Dourish (definition, examples)... igroup discussion in progress
First Thoughts::What is Embodied Interaction? Definition (preliminary) Dourish (2004): Embodiment 1: Embodiment means possessing and acting through a physical manifestation in the world. (p.100) Embodiment 2: Embodied phenomena are those that by their very nature occur in real time and real space. (p. 101) Tangible Computing + Social Computing > Embodied Interaction Embodied Interaction is a form of a human-information interaction which emphasises the human body (part) 1. as the main spatial and temporal frame of reference 2. as the central instrument of perception and action
First Thoughts::EI Types Types of Embodied Interactions (regarding spatial relationship) EI-1: Interaction with the (virtual) environment (room) reference of own body to environment defines inside / outside relation (void and solid) serves as an indicator for being in the world affected by navigation interface EI-2: Interaction with objects (within environment) reference body object affected by most 3DUI techniques EI-3: Interaction with subjects (if shared environment) constituted by social presence co-presence (being together in one space) psychological involvement (e.g. mutual understanding) behavioural engagement (e.g. dependent action)
Humble Steps
Humble steps::empirical Studies Embodied Interaction with environment (EI-1) Navigation 1. Self in local environment (Living Gallery; Burrows, Hauber, & Regenbrecht) 2. Self in distant environment (Presence Robot; Kerse, Regenbrecht, & Purvis) 3. Self in remote virtual environment (Social Presence; Hauber, Regenbrecht, & Hills)
Self in local environment Living Gallery
Humble steps 1::Living Gallery Using proximity of body to display as an invisible interface Project together with Dunedin Public Art Gallery Display of morphing artwork in a real picture frame Proximity of user s body in front of display alters content Validation of interface in laboratory and field (public space, gallery) Is this a suitable form of interaction? (e.g. compared to video sequence or buttons ) work in progress
Humble steps 1::Living Gallery Very simple setup: standard display (TFT monitor or projector) web camera OpenCV for face recognition Content: video sequence with morph scene (using single frames in program) various laboratory tests and tweaking
Humble steps 1::Living Gallery laboratory setup
Self in distant environment Presence Robot
Humble steps 2::Presence Robot Telepresence: being virtually present at a remote, real location Can telepresence be achieved with an affordable and easy to use system? ( I am there ; She/he is here ) the device remote site local site
Humble steps 2::Presence Robot Personal Digital Assistant (PDA) with a camera represents the remote self at the local site PDA is mounted to a pan and tilt unit which can be controlled from the remote location (keyboard) Use of COTS components (e.g. Lego MindStorms)
Humble steps 2::Presence Robot
Humble steps 2::Presence Robot In a pilot user study we could show, that (a) even this simple setup can be used as a telepresence system (b) the control of the field of view within the remote environment is important, and (c) some social presence properties are positively influenced by the user-initiated control. dimensions assessed: self-reported co-presence[5] perceived other s co-presence[5] telepresence[5, 1] mutual understanding[3] satisfaction with the group process[3] interpersonal attraction[3] social presence[8] usability =>paper at ICAT 2005
Self in remote, virtual environment car/pe! and Social Presence
Humble steps 3::Social Presence Videoconferencing V/C is a beneficial technology: can save time and money and potentially allows for the support of some non-verbal cues (compared to letter, email, phone) Compared to Face-To-Face meeting still artificial experience Social Presence as an indicator SVE try to overcome artificial character of traditional V/C mainly with the (re-) introduction of three-dimensionality MyPhone daviko AccessGrid ichat
Humble steps 3::Social Presence Shared Virtual Environments Alicestreet Active Worlds UCL: research on SVE SmartMeeting
Humble steps 3::Social Presence SVE car/pe! Originally developed at DaimlerChrysler Research in Germany Ongoing R&D in Germany and New Zealand 3 participants represented as video avatars Self-movement with mouse or SpaceMouse Standard computer screen (PC version) Display of pictures, PowerPoint slides, 3D models, and remote interactive computer screens in virtual room 3D sound
Humble steps 3::Social Presence SVE car/pe!
Humble steps 3::Social Presence Experiments car/pe! used in this study without any documents involved, just a timer screen via remote computer screen Modified version with fixed position in front of screen (no avatar movement) for condition 2D Interface limited to small set of functions Use of computer mouse only Medium quality audio (16 Bit stereo) and video (CIF, 15 Hz)
Humble steps 3::Social Presence Experiments How do two- and three-dimensional videoconferencing interfaces support social presence? Use of the same videoconferencing system for both conditions for minimisation of bias (in particular A/V codecs). Task chosen, which is not tailored to three-dimensional interface (desert survival game) and was used before in a pilot study Communication between three participating parties as a standard case in CMC. Comparison of 2D (referring to traditional V/C), 3D (referring to SVE) and Face-To-Face (the gold standard ) 14 one-hour sessions with 3 participants each in 3 conditions (126 trials in total) Two instruments applied to measure Social Presence (questionnaires, Networked Minds and Semantic Differential) General Hypotheses: Social Presence: Face-To-Face > 3D > 2D
Humble steps 3::Social Presence Experiments Task: Desert Survival Game condition I 2D fixed view, WYSIWIS, mono sound condition II 3D free navigation, individual views, 3d-sound condition III Face-To-Face natural communication gold standard
Humble steps 3::Social Presence 6.5 6 5.5 5 4.5 4 3.5 Experiments: Main Result 3 2D 3D FtF Social Presence (measure based on Short et al., 1976)
Humble steps 3::Social Presence Conclusions Social Presence increases from 2D to 3D to FTF Introduction of three-dimensionality in computermediated communication is a well-founded step Semantic Differential Measure (Short et al.) is an elegant and reliable instrument Improvement of performance of interface needed Inclusion of more real-world metaphors promising Inclusion of more non-verbal cues (like gestures) Hypotheses: Higher Embodiment results in higher performance and communication quality papers at Presence 2005 and ICAT 2005
Future Work
Future Work::Communication Space (3D-COMM)
Future Work::Cybertherapy Collaborative Cybertherapy: Goal: VR based system for the treatment of various disorders and investigations on Embodied Interaction (mainly EI-2 and EI-3) Joint work of Psychology and Information Science based on car/pe! concept
Future Work::Cybertherapy Virtual World Client Therapist
Future Work::Collaborative Augmented Reality Environment (CARE) tracking cameras users w/ hmd s real and virtual objects (AR) autostreoscopic screen real room virtual participants
Future Work::...more Further development of Embodied Interaction Theory Micro Affordances in (shared) virtual environments (EI-2 and EI-3) Video-Mediated Communication Research (Social Presence and Interface Quality) Development and Validation of interfaces for the support of non-verbal cues (gestures, eye-to-eye, navigation) Continuation of Presence Robot project (improved hardand software, distance tests) Continuation of Living Gallery projects (setup in Dunedin Public Art Gallery with conservation content) Work on collaboration and communication tools Research on 3DUI car/pe! (desktop, immersive, (mobile)) Establishment of an usability test-bed for VMC
Summary To date our humble Embodied Interaction approaches look promising Mainly EI-1 (self in room) considered so far, continuation with other dimensions needed Development of theory and practise in parallel seams to be reasonable (in incremental steps) Stronger co-operation with other partners in research and industry needed Interns, Master- and PhD students wanted
Contact holger@infoscience.otago.ac.nz regenbre@igroup.org PhD study at Otago/NZ (regulations and scholarships) www.otago.ac.nz/international/postgraduate/
References Merleau-Ponty, M. (1945). The Phenomenology of Perception. English translation, 1962. London: Routledge. Dourish, O. (2004). Where the Action Is The Foundations of Embodied Interaction. The MIT Press, Cambridge/MA, London/UK.