VE Input Devices. Doug Bowman Virginia Tech

Size: px

Start display at page:

Download "VE Input Devices. Doug Bowman Virginia Tech"

Alexandra Fletcher
6 years ago
Views:

1 VE Input Devices Doug Bowman Virginia Tech

2 Goals and Motivation Provide practical introduction to the input devices used in VEs Examine common and state of the art input devices look for general trends spark creativity Advantages and disadvantages Discuss how different input devices affect interface design (C) 2006 Doug Bowman, Virginia Tech 2 In this lecture we will discuss the various input and output devices that are used in 3D user interfaces and virtual environment applications.

5 Human-VE interface Env. model Tracking system Simulation loop: -render -check for events -respond to events -iterate simulation -get new tracker data Display(s) Input device(s) (C) 2006 Doug Bowman, Virginia Tech 5

6 Input device characteristics Degrees of Freedom (DOFs) & DOF composition (integral vs. separable) Type of electronics: Digital vs. analog Range of reported values: discrete/continuous/hybrid Data type of reported values: Boolean vs. integer vs. floating point (C) 2006 Doug Bowman, Virginia Tech 6

7 More input device characteristics User action required: active/passive/hybrid Method of providing information: push vs. pull Intended use: locator, valuator, choice, Frame of reference: relative vs. absolute Properties sensed: position, motion, force, (C) 2006 Doug Bowman, Virginia Tech 7

Desktop devices: 6-DOF devices 6 DOFs without tracking Often isometric

11 Tracking devices: position trackers Measure position and/or orientation of a sensor Degrees of freedom (DOFs) Most VEs track the head motion parallax natural viewing (C) 2006 Doug Bowman, Virginia Tech 11

12 Other uses for trackers Track hands, feet, etc. whole body interaction motion capture application Correspondence between physical/virtual objects Props 5,6 spatial input devices (C) 2006 Doug Bowman, Virginia Tech 12

Electromagnetic trackers Exs: Polhemus Fastrak, Ascension Flock of Birds Most common (?

15 Optical/vision-based trackers Exs: Vicon, HiBall, ARToolkit Advantages accurate can capture a large volume allow for untethered tracking Disadvantages image processing techniques occlusion problem (C) 2006 Doug Bowman, Virginia Tech 15

Inertial trackers Exs: Intersense IS-300, Intertrax2 Less noise, lag

Tracking devices: eye tracking (C) 2006 Doug Bowman, Virginia Tech 18 Eye tracking systems provide applications with knowledge of the user s gaze

The figure on the left shows the Eyegaze system, a non-intrusive approach which uses an infra-red source that reflects off of the pupil, developed by

18 Tracking devices: eye tracking (C) 2006 Doug Bowman, Virginia Tech 18 Eye tracking systems provide applications with knowledge of the user s gaze direction. This information opens the door to a number of interesting interaction techniques such as eye directed selection and manipulation. The figure on the left shows the Eyegaze system, a non-intrusive approach which uses an infra-red source that reflects off of the pupil, developed by LC Technologies. The figure on the right shows iview, a head-mounted eye tracking device developed by SensoMotoric Instruments. References:

Tracking devices: bend-sensing gloves CyberGlove 7, 5DT Reports hand posture Gesture: single

Tracking devices: pinch gloves Conductive cloth at fingertips Any gesture of 2 to 10

21 Case study: Pinch Gloves Pinch gloves are designed to be a combination device (add a position tracker) Very little has been done with Pinch Gloves in VEs - usually 1 or 2 gestures for: Object selection Tool selection Travel (C) 2006 Doug Bowman, Virginia Tech 21

22 Characteristics of Pinch Gloves Relatively low cost Very light User s hand becomes the device User s hand posture can change Allow two-handed interaction Huge number of possible gestures (C) 2006 Doug Bowman, Virginia Tech 22

23 Characteristics of Pinch Gloves II Much more reliable than data gloves Support eyes-off input Can diminish Heisenberg effect Support context-sensitive gesture interpretation (C) 2006 Doug Bowman, Virginia Tech 23

Limited to comfortable gestures Visual feedback on

25 Pinch Gloves for menus TULIP system 14 ND hand selects menu, D hand selects item within menu Limited to comfortable gestures Visual feedback on virtual hands rapmenu (C) 2006 Doug Bowman, Virginia Tech 25

26 Pinch Gloves for text input Pinch Keyboard 14 Emulate QWERTY Pinch finger to thumb to type letter under that finger Move/rotate hands to change active letters Visual feedback (C) 2006 Doug Bowman, Virginia Tech 26

3D mice Ring Mouse Fly Mouse Wand Cubic Mouse Dragonfly (C) 2006 Doug Bowman, Virginia Tech 27 The Ring Mouse (top picture) is a

The main advantages of this device is that it is wireless and inexpensive.

This device has five buttons instead of two and also can be used as a microphone.

27 3D mice Ring Mouse Fly Mouse Wand Cubic Mouse Dragonfly (C) 2006 Doug Bowman, Virginia Tech 27 The Ring Mouse (top picture) is a small device worn on the user s finger which uses ultrasonic tracking. It also has two buttons for generating discrete events. The main advantages of this device is that it is wireless and inexpensive. The Fly Mouse is a 3D mouse that also uses ultrasonic tracking. This device has five buttons instead of two and also can be used as a microphone. The Cubic Mouse (shown in the figure on the right) is an input device developed at GMD that allows users to intuitively specify three-dimensional coordinates in graphics applications. The device consists of a box with three perpendicular rods passing through the center and buttons for additional input.

28 Special-purpose devices: using conductive cloth Virtual toolbelt Used to select virtual tools Good use of proprioceptive cues Interaction slippers 3 Step on displayed options Click heels to go home (C) 2006 Doug Bowman, Virginia Tech 28

Special-purpose devices: Painting Table4 (C) 2006 Doug Bowman, Virginia Tech 29 The Painting Table is another example of a special-purpose input device that is used in the CavePainting application, a

29 Special-purpose devices: Painting Table4 (C) 2006 Doug Bowman, Virginia Tech 29 The Painting Table is another example of a special-purpose input device that is used in the CavePainting application, a system for painting 3D scenes in a virtual environment. The device uses a set of conductive cloth contacts as well as traditional buttons and digital sliders. Users can dip the paint brush prop into the colored cups to change brush strokes. The bucket is used to throw paint around the virtual canvas. References: Keefe, D., Acevedo, D., Moscovich, T., Laidlaw, D., and LaViola, J. CavePainting: A Fully Immersive 3D Artistic Medium and Interactive Experience, Proceedings of the 2001 Symposium on Interactive 3D Graphics, 8593,

Special-purpose devices: ShapeTape 11 (C) 2006 Doug Bowman, Virginia Tech 30 ShapeTape is a continuous bend and twist sensitive strip which encourages twohanded manipulation.

30 Special-purpose devices: ShapeTape 11 (C) 2006 Doug Bowman, Virginia Tech 30 ShapeTape is a continuous bend and twist sensitive strip which encourages twohanded manipulation. A BAT is attached and the tool (shown in the figure on the right) is used for creating and editing curves and surfaces along with cameral control and command access. ShapeTape senses bend and twist with two fiber optic sensors at 6cm intervals. References: Balakrishnan, Ravin, George Fitzmaurice, Gordon Kurtenbach, and Karan Singh. Exploring Interactive Curve and Surface Manipulation Using a Bend and Twist Sensitive Input Strip Proceedings of the 1999 Symposium on Interactive 3D Graphics, , 1999.

31 Human input: speech Frees hands Allows multimodal input No special hardware Specialized software Issues: recognition, ambient noise, training, false positives, (C) 2006 Doug Bowman, Virginia Tech 31

Human input: Bioelectric Control (C) 2006 Doug Bowman, Virginia Tech 32 A recent development at NASA Ames Research Center is a bioelectric input device which reads muscle nerve signals emanating

The nerve signals are analyzed using pattern recognition software and then routed through a computer to issue relevant interface commands.

32 Human input: Bioelectric Control (C) 2006 Doug Bowman, Virginia Tech 32 A recent development at NASA Ames Research Center is a bioelectric input device which reads muscle nerve signals emanating from the forearm. These nerve signals are captured by a dry electrode array on the arm. The nerve signals are analyzed using pattern recognition software and then routed through a computer to issue relevant interface commands. The figure on the left shows a user entering numbers on a virtual numeric keypad while the figure on the right shows a user controlling a virtual 757 aircraft. References: Jorgensen, Charles, Kevin Wheeler, and Slawomir Stepniewski. Bioelectric Control of a 757 Class High Fidelity Aircraft Simulation,

Human input: Body Sensing Devices (C) 2006 Doug Bowman, Virginia Tech 33 The MIT Media Lab s affective computing group has developed a Prototype Physiological Sensing System which includes a Galvanic

33 Human input: Body Sensing Devices (C) 2006 Doug Bowman, Virginia Tech 33 The MIT Media Lab s affective computing group has developed a Prototype Physiological Sensing System which includes a Galvanic Skin Response sensor, a Blood Volume Pulse sensor, a Respiration sensor, and an Electromyogram. By using this prototype, interface developers can monitor a user s emotional state to dynamically modify an application s interface to better fit the user s needs. References:

More human input Breathing device - OSMOSE Brain-body actuated

36 UNIPORT First Locomotion Device For U.S. Army (1994) Proof-of-concept demonstration Developed in six weeks Difficult to change direction of travel Small motions such as side-stepping are impossible (C) 2006 Doug Bowman, Virginia Tech 36

Treadport Developed in 1995 Based on a standard treadmill with the user being monitored and constrained by mechanical attachment to the user s

37 Treadport Developed in 1995 Based on a standard treadmill with the user being monitored and constrained by mechanical attachment to the user s waist User actually walks or jogs instead of pedaling Physical movement is constrained to one direction (C) 2006 Doug Bowman, Virginia Tech 37

38 Individual Soldier Mobility Simulator (Biport) Most sophisticated locomotion device Designed for the conduct of locomotion studies Hydraulic-based locomotion driven w/ force sensors at the feet Safeguards limited responsiveness Too awkward to operate (C) 2006 Doug Bowman, Virginia Tech 38

39 Omni-Directional Treadmill 15,16 Most recently developed locomotion device for U.S. Army Revolutionary device that enables bipedal locomotion in any direction of travel Consists of two perpendicular treadmills Two fundamental types of movement User initiated movement System initiated movement (C) 2006 Doug Bowman, Virginia Tech 39

Virtual Motion Controller 17 Weight sensors in platform sense user s position over platform Step

43 Walking in place 18,19 Analyze tracker information from head, body, feet Neural network (Slater) GAITER project (Templeman) Shown to be better than purely virtual movement, but worse than real walking 20 (C) 2006 Doug Bowman, Virginia Tech 43

44 Classification of locomotion devices/techniques Virtual motion Virtual turning Desktop VEs Vehicle simulators CAVE wand Real turning Most HMD systems Walking in place VMC Real motion Stationary cycles Treadport Biport Wide-area tracking UNIPORT ODT (C) 2006 Doug Bowman, Virginia Tech 44

46 PDA as ideal VE device? 22 Offers both input and output Has on-board memory Wireless communication Portable, light, robust Allows text / number input Can be tracked to allow spatial input (C) 2006 Doug Bowman, Virginia Tech 46

49 References [1] Matias, E., MacKenzie, I., & Buxton, W. (1993). Half-QWERTY: A One-handed Keyboard Facilitating Skill Transfer from QWERTY. Proceedings of ACM INTERCHI, [2] Thomas, B., Tyerman, S., & Grimmer, K. (1998). Evaluation of Text Input Mechanisms for Wearable Computers. Virtual Reality: Research, Development, and Applications, 3, [3] LaViola, J., Acevedo, D., Keefe, D., & Zeleznik, R. (2001). Hands-Free Multi-Scale Navigation in Virtual Environments. Proceedings of ACM Symposium on Interactive 3D Graphics, Research Triangle Park, North Carolina, [4] Keefe, D., Feliz, D., Moscovich, T., Laidlaw, D., & LaViola, J. (2001). CavePainting: A Fully Immersive 3D Artistic Medium and Interactive Experience. Proceedings of ACM Symposium on Interactive 3D Graphics, Research Triangle Park, North Carolina, [5] Bowman, D., Wineman, J., Hodges, L., & Allison, D. (1998). Designing Animal Habitats Within an Immersive VE. IEEE Computer Graphics & Applications, 18(5), [6] Hinckley, K., Pausch, R., Goble, J., & Kassell, N. (1994). Passive Real-World Interface Props for Neurosurgical Visualization. Proceedings of CHI: Human Factors in Computing Systems, [7] Kessler, G., Hodges, L., & Walker, N. (1995). Evaluation of the CyberGlove(TM) as a Whole Hand Input Device. ACM Transactions on Computer-Human Interaction, 2(4), [8] LaViola, J., & Zeleznik, R. (1999). Flex and Pinch: A Case Study of Whole-Hand Input Design for Virtual Environment Interaction. Proceedings of the International Conference on Computer Graphics and Imaging, [9] Ware, C., & Jessome, D. (1988). Using the Bat: a Six-Dimensional Mouse for Object Placement. IEEE Computer Graphics and Applications, 8(6), [10] Zeleznik, R. C., Herndon, K. P., Robbins, D. C., Huang, N., Meyer, T., Parker, N., & Hughes, J. F. (1993). An Interactive 3D Toolkit for Constructing 3D Widgets. Proceedings of ACM SIGGRAPH, Anaheim, CA, USA, (C) 2006 Doug Bowman, Virginia Tech 49

50 References (2) [11] Balakrishnan, R., Fitzmaurice, G., Kurtenbach, G., & Singh, K. (1999). Exploring Interactive Curve and Surface Manipulation Using a Bend and Twist Sensitive Input Strip. Proceedings of the ACM Symposium on Interactive 3D Graphics, [12] Froehlich, B., & Plate, J. (2000). The Cubic Mouse: A New Device for Three-Dimensional Input. Proceedings of ACM CHI. [13] Mapes, D., & Moshell, J. (1995). A Two-Handed Interface for Object Manipulation in Virtual Environments. Presence: Teleoperators and Virtual Environments, 4(4), [14] Bowman, D., Wingrave, C., Campbell, J., & Ly, V. (2001). Using Pinch Gloves for both Natural and Abstract Interaction Techniques in Virtual Environments. Proceedings of HCI International, New Orleans, Louisiana. [15] Darken, R., Cockayne, W., & Carmein, D. (1997). The Omni-directional Treadmill: A Locomotion Device for Virtual Worlds. Proceedings of ACM Symposium on User Interface Software and Technology, [16] Iwata, H. (1999). Walking About Virtual Environments on an Infinite Floor. Proceedings of IEEE Virtual Reality, Houston, Texas, [17] Wells, M., Peterson, B., & Aten, J. (1996). The Virtual Motion Controller: A Sufficient-Motion Walking Simulator. Proceedings of IEEE Virtual Reality Annual International Symposium, 1-8. [18] Slater, M., Usoh, M., & Steed, A. (1995). Taking Steps: The Influence of a Walking Technique on Presence in Virtual Reality. ACM Transactions on Computer-Human Interaction, 2(3), [19] Slater, M., Steed, A., & Usoh, M. (1995). The Virtual Treadmill: A Naturalistic Metaphor for Navigation in Immersive Virtual Environments, Virtual Environments '95: Selected Papers of the Eurographics Workshops (pp ). New York: SpringerWien. [20] Usoh, M., Arthur, K., Whitton, M., Bastos, R., Steed, A., Slater, M., & Brooks, F. (1999). Walking > Walking-in- Place > Flying, in Virtual Environments. Proceedings of ACM SIGGRAPH, (C) 2006 Doug Bowman, Virginia Tech 50

51 References (3) [21] Iwata, H., Yano, H., Nakaizumi, F., & Kawamura, R. (2001). Project FEELEX: adding haptic surface to graphics. Proceedings of ACM SIGGRAPH, Los Angeles, [22] Watsen, K., Darken, R., & Capps, M. (1999). A Handheld Computer as an Interaction Device to a Virtual Environment. Proceedings of the Third Immersive Projection Technology Workshop. [23] Zhai, S. (1998). User Performance in Relation to 3D Input Device Design. Computer Graphics, 32(4), (C) 2006 Doug Bowman, Virginia Tech 51

3D UIs 101 Doug Bowman

3D UIs 101 Doug Bowman Welcome, Introduction, & Roadmap 3D UIs 101 3D UIs 201 User Studies and 3D UIs Guidelines for Developing 3D UIs Video Games: 3D UIs for the Masses The Wii Remote and You 3D UI and