Perceptual Characters of Photorealistic See-through Vision in Handheld Augmented Reality Arindam Dey PhD Student Magic Vision Lab University of South Australia Supervised by: Dr Christian Sandor and Prof. Bruce H. Thomas
Problems with most AR browsers: Pieces of isolated information instead of one integrated visualization Small screen problem becomes even worse Extremely limited visualizations How to display occluded points of interest (POIs)?
Melt Vision Sandor et al. 2010. Egocentric space-distorting visualizations for rapid environment exploration in mobile mixed reality. In the Proceedings of IEEE Conference on Virtual Reality, pages 47 50, Waltham, USA, March 2010.
Edge-overlay X-ray Avery et al. 2009. Improving spatial perception for augmented reality x-ray vision. In VR 09: Proceedings of the IEEE Conference on Virtual Reality, IEEE, 79 82.
Saliency-based X-ray Sandor et al. 2010. An Augmented Reality X-Ray System Based on Visual Saliency. In the Proceedings of IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 27 36, Seoul, South Korea, October 2010.
Experiments Melt Edge-overlay X-ray Saliency-based X-ray Experiment I Depth Perception Dey et al. 2010. Evaluating Depth Perception of Photorealistic Mixed Reality Visualizations for Occluded Objects in Outdoor Environments. In Proceedings of ACM Symposium on Virtual Reality Software and Technology, pages 211-218, Hong Kong, China, November, 2010.
Setup & Task 20 participants verbally reported distances of target cubes Arindam Dey
Conditions Cue-off Between Participant Cue-on X-ray Within Participant Melt
Results Signed Error Distances mostly underestimated Cue-on significantly reduced error for both Melt and X-ray (p<.001) Main effect of distance
Results Absolute Error Melt + Cue-on significantly (p<.001) better than all other including X-ray + Cue-on
Results Accuracy Cue-on was more accurate for both Melt and X-ray (p<.001) With increasing distance in Cue-on condition: Melt consistent X-ray lost accuracy due to visual noise
Results Response Time Depth cue significantly increased response time Cue-on: Melt was significantly faster than X-ray
Results Learning Effect Cue-on in later trials: Response Time decreased Accuracy consistent
Contradiction Livingston et al. 2009 Dey et al. 2010 Livingston et al. 2009. Indoor vs. outdoor depth perception for mobile augmented reality. In VR 09: Proceedings of the IEEE Conference on Virtual Reality, IEEE Computer Society, Washington, DC, USA, 55 62.
Contradiction (cont.) Possible Reasons Use of a handheld display (vs HMD) Use of a video see-through setup (vs optical see-through) Use of our MR system (more akin to virtual reality) Longer experimental distances (vs 4.83m to 38.64m) Dense edge-overlay of X-ray Effect of Parking marks? Livingston et al. 2009 Arindam Dey
Core Findings Contradiction outdoor depth underestimation Melt is better than X-ray - visual noise negatively effects depth perception Future Work Further investigation of the reasons HMD vs Handheld With vs without - See-through visualizations - Additional depth cue Distances ranged between 30 80 meters
Experiments Melt Edge-overlay X-ray Saliency-based X-ray Experiment II Target Acquisition Online Survey
Experimental Setup Target Acquisition 16 participants Participants had to select a red circle 4 different foregrounds were used 2 different size of target circles (9 pixel, 16 pixel) Variables X-ray: within participant Surface: within participant Target size: between participant We measured Selection Time (msec) Edge-overlay Saliency-based Arindam Dey
Results No significant difference We excluded Mixed Surface
Mixed Surface had a positively skewed data set
Effect of Brightness on Saliency-based X-ray Sunny Day Cloudy Day
Brightness Low Medium High Experimental Setup Online Survey (see-perceive-mark) Edge Low Medium High Edge Low Medium High Edge-overlay Saliency-based
Results Online Survey (see-perceive-mark) Foreground Background Total
Core Findings Saliency-based X-ray provides richer foreground information - without impeding background legibility significantly Bright foregrounds are evil for Saliency-based X-ray Future Work Improve prototype to deal with bright foregrounds Refine and evaluate motion saliency Port to mobile phone and perform more realistic evaluation
Acknowledgments Everyone in Magic Vision Lab, especially Dr Christian Sandor Andrew Cunningham Helen Ftanos Brad Cameron Nokia Research Center, Finland Anonymous participants and reviewers
Thank You Arindam Dey Arindam.Dey@unisa.edu.au We hire PhD Students, Post Docs, and Interns For more details - Visit: www.magicvisionlab.com Contact: Dr Christian Sandor christian@sandor.com