Universidade de Aveiro Departamento de Electrónica, Telecomunicações e Informática. Interaction in Virtual and Augmented Reality 3DUIs

Transcription

1 Universidade de Aveiro Departamento de Electrónica, Telecomunicações e Informática Interaction in Virtual and Augmented Reality 3DUIs Realidade Virtual e Aumentada 2017/2018 Beatriz Sousa Santos

2 Interaction is the communication that occurs between a user and the VR application that is mediated through the use of input and output devices. (Jerald, 2016) 2

3 VE versus Immersive VE Virtual environment (VE): a 3D computer generated world seen from a first-person point of view, where the viewpoint is under the real-time control of the user Immersive VE: has more realistic sensory outputs than typical desktop 3D graphics; usually appears to surround the user in space 3

4 3DUIs let users interact with virtual environments, objects, or information using direct 3D input in the physical and/or virtual world Isn t the 3D interface obvious? Naturalism vs. Magic Naturalism: make VE work exactly like real world Magic: give user new abilities - Perceptual - Physical - Cognitive 4

5 Naturalism vs Magic Spatial input technologies allow high levels of interaction fidelity 3D UIs must be designed carefully, no UI can replicate the real-world experience High levels of naturalism can enhance performance and the overall UX Moderately natural 3D UIs can be unfamiliar and decrease performance Traditional interaction styles can provide good performance, but result in lower presence and engagement Bowman, D. A., McMahan, R. P., & Ragan, E. D., Questioning naturalism in 3D user interfaces. Communications of the ACM, 55(9), 78 88,

6 What makes 3D interaction difficult? Spatial input Lack of constraints Lack of standards Lack of tools Lack of precision Fatigue Layout more complex Perception, 6

7 Goals of interaction design in VEs: Performance (efficiency, accuracy, productivity) Usability (ease of use, ease of learning, user comfort and satisfaction) Usefulness (users focus on tasks, interaction helps users meet goals) 7

8 Universal interaction tasks for VEs Navigation Travel motor component Wayfinding cognitive component Selection Manipulation System control Symbolic input (Bowman et al., 2005) 8

9 Example: Navigation + selection + manipulation <- walking and gestures 9

10 Example: Imaginary Musem setup The user walks in a real room and navigates in the virtual room (similar to the real one) Paulo Dias, João Pinto, Sérgio Eliseu, Beatriz Sousa Santos, Gesture interactions for Virtual Immersive Environments: navigation, selection and manipulation, N. Streitz and P. Markopoulos (Eds.), Distributed, Ambient, and Pervasive Interactions DAPI 2016, Lecture Notes in Computer Science,LNCS 9740, pp

11 Another example: Same tasks; very different interaction techniques input device Navigation Selection 11

12 Interaction Patterns for VR Selection Patterns Manipulation Patterns Viewpoint Control Patterns Indirect Control Patterns Compound Patterns (Jerald, 2016) 14

13 Travel taxonomy of techniques (Bowman et al., 2005)

14 Selection/ Manipulation Taxonomy of techniques (Bowman et al., 2005)

15 Classification of system control techniques (Bowman et al., 2008) 17

16 System control design guidelines: Don t disturb flow of action Use consistent spatial reference Allow multimodal input Structure available functions Prevent mode errors by giving feedback 18

17 Symbolic input Communication of symbols (text, numbers, and other symbols/marks) to the system Is this an important task for 3D UIs? Why is symbolic input in 3D UIs different from symbolic input elsewhere? 19

18 Symbolic input techniques Keyboard-based (Miniature, Chord, Soft keyboards, ) Gesture-based (Sign language gestures, other gestures) Speech-based (Single-character, whole-word recognition) /technology/vr-headsets/ 20

19 Keyboard-based - Miniature keyboards - Chord keyboards - Soft keyboards Gesture-based Sign language gestures Numeric gestures Instantaneous gestures Speech-based Single-character speech recognition Whole-word speech recognition Unrecognized speech input 21

20 2D interaction in 3D VEs Useful for appropriate tasks (match task and input DOFs) - Can integrate seamlessly with 3D - If presence is important, the 2D interface should be embedded, not overlaid Examples: - Interaction on the projection surface or viewplane - Using a tablet for VE input 22

21 Constraints Artificial limitations designed to help users interact more precisely or efficiently Examples: - Snap-to grid - Intelligent objects - Single DOF controls 23

22 Passive haptic Feedback Props or near-field haptics Examples: - Flight simulator controls - Steering wheel Increase presence, improve interaction %20Graspable%20UIs/Thesis.gf.html 24

23 Two-handed interaction Symmetric vs. Asymmetric Dominant vs. non-dominant hand Manipulation initiated by ND hand Simulator for learning cataract surgery Guiard s principles - ND hand provides frame of reference - ND hand used for coarse tasks, D hand for fine grained tasks 25

24 Guidelines for Easy-to-Use 3D Interaction Techniques Floating objects are the exception Objects don t interpenetrate Interaction should be only with Visible Objects Perspective and occlusion are the strongest depth cues People see the object, not the cursor. 27

25 What future to 3DUI? The design domain of 3D UI is rapidly expanding due to recent technology advancements and new interaction techniques No single configuration is right for all conditions The definition of 3D interaction isn t limited to interaction with VR, but Traditionally 3DUI has been related to VR; not any more: Games Large displays Offer excellent opportunities to 3DUI 28

26 Evaluation How to evaluate of 3D 3DUIs? User Interfaces Evaluation is the analysis, assessment, and testing of the entire UI or part of it (e.g. an input device or interaction technique) The main purpose of evaluation is the identification of usability problems or issues, leading to changes in the UI design Design and evaluation should be performed iteratively design evaluation redesign evaluation, until the UI is good enough, based on some metrics 29

27 Most common goals of evaluation: problem identification redesign of a UI Other important purposes: development of performance models more general understanding of the usability of a technique, device, or metaphor leading to design guidelines 30

28 Distinctive Characteristics of 3D User Interface Evaluation Many of the fundamental evaluation concepts and goals are similar to traditional 2D UIs, but there are issues specific to 3D UIs concerning: Physical Environment Evaluator User Evaluation type... 31

29 Evaluation methods A classification of methods according to three characteristics: involvement of users requiring users (empirical methods) not requiring users (analytical methods) context of evaluation generic application specific types of results produced quantitative (e.g. times and n. of errors) qualitative (e.g. preferences) 32

30 Main evaluation methods applicable in VEs observation Requiring users questionnaire (empirical) interview controlled experiments observation Requiring Not requiring users (analytical) heuristic evaluation cognitive walkthrough 33

31 General evaluation guidelines Begin with informal evaluation Plan for the differences between traditional UI and 3D UI evaluation Choose an evaluation approach that meets your requirements Use a wide range of metrics In formal evaluation: Design experiments with general applicability Use pilot studies to select variables to be tested in the main experiment Look for variable interaction rarely will a single technique be always best 38

32 Conclusions Usability is one of the most important issues in VE applications Safety and comfort are crucial Adapting 2D interfaces is not enough Implementation details are critical to ensure usability All VR/AR applications should be carefully tested 39

33 Main bibliography - Jerald, J., The VR Book: Human-Centered Design for Virtual Reality, ACM and Morgan & Claypool, D. A. Bowman, 3D User Interfaces,In: Soegaard, Mads and Dam, Rikke Friis (eds.). The Encyclopedia of Human-Computer Interaction, 2nd Ed., The Interaction Design Foundation. (online Oct, 2017) - Bowman, D., E. Kruijff, I. Poupyrev, J. LaViola, 3D User Interfaces: Theory and Practice, Addison Wesley, Craig, A., Sherman, W., Will, J., Developing Virtual Reality Applications: Foundations of Effective Design, Morgan Kaufmann,

34 Other bibliography J. Chen and D. A. Bowman. Domain-specific design of 3d interaction techniques: An approach for designing useful virtual environment applications. Presence: Teleoper. Virtual Environ. Vol. 18, no. 5, p , D. A. Bowman, R. P. McMahan, and E. D. Ragan, Questioning naturalism in 3D user interfaces, Commun. ACM, vol. 55, no. 9, p. 78, D. A. Bowman and M. Coquillart, Sabine, Froehlich, Bernd, Hirose, 3D User Interfaces : New Directions and Perspectives, IEEE Comput. Graph. Appl., vol. 28, no. 6, pp , F. Argelaguet and C. Andujar, A survey of 3D object selection techniques for virtual environments, Comput. Graph., vol. 37, no. 3, pp ,

35 Aknowledgement The author of these slides is greteful to: Gregory Burdea and Philipe Coiffet Doug Bowman for making available the slides supporting their books All colleagues and students that have contributed in any way 42