Realtime 3D Computer Graphics Virtual Reality Virtual Reality Input Devices Special input devices are required for interaction,navigation and motion tracking (e.g., for depth cue calculation): 1 WIMP: Windows, Icons, Menu, Pointer
Special input devices are required for interaction,navigation and motion tracking (e.g., for depth cue calculation): 1. Motion Trackers: Position and orientation of a reference system in 3D requires to measure 6 Degrees of Freedom (DOFs). 1 WIMP: Windows, Icons, Menu, Pointer Special input devices are required for interaction,navigation and motion tracking (e.g., for depth cue calculation): 1. Motion Trackers: Position and orientation of a reference system in 3D requires to measure 6 Degrees of Freedom (DOFs). 2. 3D Mice/Wands etc. : Specialized devices for point and click WIMP 1 -style metaphors have to account for additional DOFs. 1 WIMP: Windows, Icons, Menu, Pointer
Special input devices are required for interaction,navigation and motion tracking (e.g., for depth cue calculation): 1. Motion Trackers: Position and orientation of a reference system in 3D requires to measure 6 Degrees of Freedom (DOFs). 2. 3D Mice/Wands etc. : Specialized devices for point and click WIMP 1 -style metaphors have to account for additional DOFs. 3. Joint sensors: Sensors which measure movement of user s joints (Also possible with trackers and inverse kinematics). 1 WIMP: Windows, Icons, Menu, Pointer Special input devices are required for interaction,navigation and motion tracking (e.g., for depth cue calculation): 1. Motion Trackers: Position and orientation of a reference system in 3D requires to measure 6 Degrees of Freedom (DOFs). 2. 3D Mice/Wands etc. : Specialized devices for point and click WIMP 1 -style metaphors have to account for additional DOFs. 3. Joint sensors: Sensors which measure movement of user s joints (Also possible with trackers and inverse kinematics). 4. Props: Real placeholders for virtual objects. 1 WIMP: Windows, Icons, Menu, Pointer
Special input devices are required for interaction,navigation and motion tracking (e.g., for depth cue calculation): 1. Motion Trackers: Position and orientation of a reference system in 3D requires to measure 6 Degrees of Freedom (DOFs). 2. 3D Mice/Wands etc. : Specialized devices for point and click WIMP 1 -style metaphors have to account for additional DOFs. 3. Joint sensors: Sensors which measure movement of user s joints (Also possible with trackers and inverse kinematics). 4. Props: Real placeholders for virtual objects. 5. Movement effect sensors: Measure the effect user movement has to the surrounding (no kinematics involved). 1 WIMP: Windows, Icons, Menu, Pointer Special input devices are required for interaction,navigation and motion tracking (e.g., for depth cue calculation): 1. Motion Trackers: Position and orientation of a reference system in 3D requires to measure 6 Degrees of Freedom (DOFs). 2. 3D Mice/Wands etc. : Specialized devices for point and click WIMP 1 -style metaphors have to account for additional DOFs. 3. Joint sensors: Sensors which measure movement of user s joints (Also possible with trackers and inverse kinematics). 4. Props: Real placeholders for virtual objects. 5. Movement effect sensors: Measure the effect user movement has to the surrounding (no kinematics involved). 6. Skin sensors, neural interfaces, bio-sensors: Measure skin resistance, brain activity and other body related data. 1 WIMP: Windows, Icons, Menu, Pointer
Special input devices are required for interaction,navigation and motion tracking (e.g., for depth cue calculation): 1. Motion Trackers: Position and orientation of a reference system in 3D requires to measure 6 Degrees of Freedom (DOFs). 2. 3D Mice/Wands etc. : Specialized devices for point and click WIMP 1 -style metaphors have to account for additional DOFs. 3. Joint sensors: Sensors which measure movement of user s joints (Also possible with trackers and inverse kinematics). 4. Props: Real placeholders for virtual objects. 5. Movement effect sensors: Measure the effect user movement has to the surrounding (no kinematics involved). 6. Skin sensors, neural interfaces, bio-sensors: Measure skin resistance, brain activity and other body related data. and hybrid devices. 1 WIMP: Windows, Icons, Menu, Pointer
Input is measured by a multitude of physical and biological principles, e.g., Input is measured by a multitude of physical and biological principles, e.g., 1.electro-magnetism
Input is measured by a multitude of physical and biological principles, e.g., 1.electro-magnetism 2.optics (marker/marker less, visible spectrum/infrared) Input is measured by a multitude of physical and biological principles, e.g., 1.electro-magnetism 2.optics (marker/marker less, visible spectrum/infrared) 3.electrics (voltage, impedance, electrical flow, )
Input is measured by a multitude of physical and biological principles, e.g., 1.electro-magnetism 2.optics (marker/marker less, visible spectrum/infrared) 3.electrics (voltage, impedance, electrical flow, ) 4.acoustics (ultrasound, ) Input is measured by a multitude of physical and biological principles, e.g., 1.electro-magnetism 2.optics (marker/marker less, visible spectrum/infrared) 3.electrics (voltage, impedance, electrical flow, ) 4.acoustics (ultrasound, ) 5.inertia
Input is measured by a multitude of physical and biological principles, e.g., 1.electro-magnetism 2.optics (marker/marker less, visible spectrum/infrared) 3.electrics (voltage, impedance, electrical flow, ) 4.acoustics (ultrasound, ) 5.inertia Input devices produce data... Input is measured by a multitude of physical and biological principles, e.g., 1.electro-magnetism 2.optics (marker/marker less, visible spectrum/infrared) 3.electrics (voltage, impedance, electrical flow, ) 4.acoustics (ultrasound, ) 5.inertia Input devices produce data......discrete event based (buttons, state changers).
Input is measured by a multitude of physical and biological principles, e.g., 1.electro-magnetism 2.optics (marker/marker less, visible spectrum/infrared) 3.electrics (voltage, impedance, electrical flow, ) 4.acoustics (ultrasound, ) 5.inertia Input devices produce data......discrete event based (buttons, state changers)....continuously (discrete but continuously sampled). Electromagnetic tracker used to be most common see: put-that-there (Bolt, 1980) Transmitter Creates three orthogonal lowfrequency magnetic fields Short range version: < 1m Long range version: < 3m Receiver(s) Three perpendicular antennas. Distance is inferred from the currents induced in the antennas. Receiver Transmitter - Noisy requires filtering. - Affected by metal requires non-linear calibration. - Wireless versions expensive. 6DOF Magnetic tracker & DataGlove
Acoustic trackers Uses ultrasound Typical setup for 3 DOF: 3 microphones and1 speaker Distance is inferred from the travel time of the sound + No interference with metal + Relatively inexpensive - Line of sight issues - Sensitive to air temperature and certain noises Logitech Fly Mouse Hybrid trackers (e.g.,intersense IS-600/900) inertial (orientation) acoustic (position) Inertial trackers (Intersense IS-300) + Less noise, lag - Only 3 DOFs (orientation) Use gyroscopes and accelerometers Optical marker based tracker marker reflects IR light Combined to unique spatial configuration per tracked position + No interference with metal + Low latency + High resolution - Line of sight issues (more cameras help) 6DOF optical tracker by ART
3D mice/wands Several buttons and sensors for selection of binary states and/or continuous state changes (e.g., potentiometers). Often hybrid devices for additional position/orientation. tracked wand ring mouse space orb CubicMouseTM First 12 DOF input device Tracks position and rotation of rods using potentiometers Other shapes and implementations possible Mini Cubic Mouse pictures courtesy of IMK Fraunhofer Gesellschaft
Data Gloves Used to track the user s finger movements. For posture and gesture detection. Almost always used with a tracker sensor mounted on the wrist Common types: 5DT Glove (left) 5/16 sensors CyberGlove (right) 18/22 sensors here hybrid modification for flexion and pinch Data Gloves Used to track the user s finger movements. For posture and gesture detection. Almost always used with a tracker sensor mounted on the wrist Common types: Sensors: 20/suit 100 updates/sec 3 meters range from base unit Resolution<2 mm and <.2 degrees Electronic unit (2 hours battery life) 5DT Glove (left) 5/16 sensors Wireless suit (Ascension Technology) CyberGlove (right) 18/22 sensors here hybrid modification for flexion and pinch Body suites Used to track the overall body movement Angles measured by resistance or by inverse kinematics based on certain body points
Head-prop Courtesy of Hinkley et al. Cyberglove with haptics ShapeTape-prop courtesy of Balakrishnan et al. Treadmill types (e.g. bicycles) Head-prop Courtesy of Hinkley et al. Treadmill types (e.g. bicycles) Cyberglove with haptics ShapeTape-prop courtesy of Balakrishnan et al. all the preceding and/or Speech Input continuous vs. one-time recognition choice and placement of microphone training vs. no training handling of false positive recognition surrounding noise interference Can complement other modes of interaction!" multi-modal interaction (by, e.g., additionally including gesture processing which benefits from the VR sensory equipment)
Fiktion: Interaktion The Ultimate Display
The Ultimate Display The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such room would be fatal. With appropriate programming such a display could literally be the Wonderland into which Alice walked. The Ultimate Display The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such room would be fatal. With appropriate programming such a display could literally be the Wonderland into which Alice walked. (Sutherland 1965)
Fiktion: Interface für das Ultimate Display? Fiktion: Results of physical contact
References Bolt, R. A. (1980): Put That There: Voice and Gesture at the Graphics Interface. In: Computer Graphics 14/3, (pp. 262-270) Bühl, Achim (1997): Die virtuelle Gesellschaft. Politik, Ökonomie und Kultur im Zeichen des Cyberspace. In: Gräf, Lorenz/ Krajewski, Markus (Hrsg.): Soziologie des Internet. Handeln im elektronischen Web- Werk, Frankfurt/M. /New York: Campus, 39-59. Gibson, William (1984): Neuromancer (first print) Gibson, William (1999): Neuromancer, 9. Aufl., München: Heyne 1999 Okoshi, T. (1976): Three-Dimensional Imaging Techniques, Academic Press, New York. Peters, G. (2000). Theories of Three-Dimensional Object Perception - A Survey. Recent Research Developments in Pattern Recognition, Transworld Research Network. Sutherland, I.E. (1968): A Head-Mounted Three-Dimensional Display. In: AFIPS Conference Proceedings, Vol. 33, Part I, pp. 757-764.