User Interfaces in Panoramic Augmented Reality Environments Stephen Peterson Department of Science and Technology (ITN) Linköping University, Sweden Supervisors: Anders Ynnerman Linköping University, Sweden François Jouen EPHE, France European 1 Organisation for the Safety of Air Navigation
Presentation Overview Context Background Current Work Related experiments Proposed experimentation 2
Context Objective: To investigate the feasibility of using see-through display systems in ATC towers Advantage: potential performance benefits for visual tasks Visually complex scenarios reduce head-up / head-down switch Poor visibility conditions maintain/enhance visual input Methodology Technology-centred design Empirical experimentation Technology-centred Design Empirical Experimentation 3
4 Context
Presentation Overview Context Background Current Work Related experiments Proposed experimentation 5
Background Augmented Reality (AR) Computer visualization on a semi-transparent display device, situated between the user and observed physical objects The real object can be overlaid with computer graphics positioning and orientation of the user, screen and observed objects is critical Panoramic environment An abstraction of environments like the Air Traffic Control tower Egocentric space user is fixed in confined volume view of the far field surrounding the user Closest objects are 30 m from user beyond optical infinity (6 m) 6
Background Display systems for panoramic AR Attached Video VST-HMD (video cameras and miniature display) Optical HMD (miniature display, retinal laser display) Non-attached Computer monitor Spatial display (transparent screen) 7 Classification of AR displays [Bimber05] We will investigate optically combining non-attached (spatial) displays Pros: Cons: non-intrusive more accurate registration infinite real world resolution unlimited peripheral FoV gaze-tracking possible collaboration is a technical challenge limited working volume
Background Spatial display technology Light-reflecting + Image distance bigger than screen distance, bright Screen is space consuming Light-reflecting Light-diffracting + Flexible screen size Projectors are space consuming Light-emitting + Very bright, very compact Experimental Light-diffracting 8 Light-emitting
Background Head-up display (HUD) Related technology applied to (mostly military) aircraft cockpits for decades, more recently in cars Panoramic environment Light-reflecting spatial display Mostly only superimposing symbology at fixed screen locations, not perfect overlays as in AR 9 Boeing 737 HUD. Courtesy Boeing
Background Image overlay AR display scene-fixed symbology Aim to register (spatially align) the real and virtual objects in the view of the user. Vertical and horizontal registration of objects in screen is important, but so is also depth. HUD screen-fixed symbology Used in aircraft cockpits, an example of panoramic environment (visual input from far field). Focal plane is at a fixed distance (usually infinity) from the user. This is referred to as collimating the image at a particular distance. Research shows unclear or contradictory need for infinity collimation, hints that it depends on how symbols and data is presented -> the level of realsuperimposed object connectivity. 10
Presentation Overview Context Background Current Work Related experiments Proposed experimentation 11
Current Work Collimation in Transparent Projection Screens for Panoramic Augmented Reality Environments In the range between screen-fixed symbology overlay in traditional HUDs and scene-fixed symbology in traditional AR, the need for collimation should be further examined. 12
Presentation Overview Context Background Current Work Related experiments Proposed experimentation 13
Related experiments Fürstenau (DLR), 2004: On the use of transparent rear projection screens to reduce head-down time in the air traffic control tower [Fürstenau 2004] 14
Presentation Overview Context Background Current Work Related experiments Proposed experimentation 15
Proposed experimentation A. When is biocular overlay sufficient? Hypothesis: Stereoscopy is not required and can be replaced by biocular overlay in panoramic environments. Setup: Compare results in a visual search task when screen is placed at 6 and 25 m from the user, and the display mode is biocular and stereoscopic. Independent variables: display mode, userscreen distance Dependent variables: accuracy 16
Proposed experimentation B. Is infinity collimation possible in transparent projection screens? Hypothesis: Collimation can be achieved in transparent projection screens. Setup: Compare results in a visual search task between display and background when a) the foreground image is placed on the transparent projection screen which is collimated through biocular overlay and the screen is placed at optical infinity, and b) the foreground image is placed directly on the background (simulated collimation). Independent variables: collimation technique Dependent variables: accuracy 17
Proposed experimentation C. How is collimation linked to object superimposition mode? Hypothesis: Collimation in a transparent projection screen does not have to be at optical infinity for certain object superimposition modes. Setup: Compare results in a visual search task at different collimation distances (ex. 1, 2 and 6 m) and different real-superimposed object connectivity (ex screen-fixed superimposition, scene-linked symbology and scene augmentations). Independent variables: user-screen distance, real-superimposed object connectivity Dependent variables: accuracy 18
Proposed experimentation Equipment The proposed experimentation will be performed on a 100x75 cm transparent projection screen The screen is non-depolarizing which allows for biocular or stereoscopic display modes with passive stereo through polarization multiplexing The user is required to wear polarized glasses The screen is rear-projected from 38 below by two 6500 ANSI lumen projectors at XGA resolution The user s head and eye positions are given in real time by a camerabased eye tracking system 19
For further information or questions: stephen.peterson@eurocontrol.int 20