Virtual Reality and Natural Interactions Jackson Rushing Game Development and Entrepreneurship Faculty of Business and Information Technology j@jacksonrushing.com 2/23/2018
Introduction Virtual Reality strives to bring users closer to applications Realism is paramount in many applications Natural interactions are needed for realism Presence, and flow
Evolution of Input Devices First we had NES in the mid 1980s Computing hardware was fairly weak, 8 bit processor, 2kb of ram Games were fairly simple Number of inputs were limited
Evolution of Input Devices Early 1990s the SNES came out Computing hardware getting stronger, 16 bit processor, 128kb of ram Games became more complicated, possible inputs and interactions evolved as well
Evolution of Input Devices 1996 the Nintendo 64 came out Computing hardware even stronger, 64 bit processor, 8mb ram Games could now utilize a polygonal 3D world Analogue joystick allowed 3D movement, and 3D games needed dedicated buttons for camera movement
Evolution of Input Devices Now we re going to skip forward 20 years, to today Current VR setups consist of a 6DOF tracked headset, controllers, and sensors. Available inputs are gaze, hand movements, head movements, buttons on controllers and finger positions
Evolution of Input Devices In VR, controllers that can be tracked can show physically impossible UI, such as floating UI elements that instruct the user on what certain buttons/gestures do It s also possible to use traditional input methods, but many don t work, especially when the user isn t familiar with them, because they aren t visible in the virtual environment.
Problem Developers today are settling for old user interface standards Pointing and clicking isn t something that s done in every day life, especially off of a floating plane Can be thought of as using a huge, high definition TV to watch a low resolution, black and white version of a movie It can be difficult to become immersed in a piece of software while performing such unnatural interactions http://www.portugal-tech.pt/showthread.php?t=2611
Presence Combination of many factors On the most fundamental level, the feeling of being present in a digital world Related to flow Things like UIs not optimized for VR, as well as uncomfortable, unnatural interactions can make presence difficult to achieve Natural interactions are required for presence
Unnatural Interactions Unnatural interactions come about because there isn t a standard for VR interactions, developers have to build their own solutions using the few best practices
Natural Interactions Low level gestures Actions that represent daily real human actions Physical interfaces that are also represented in the virtual environment Joystick, steering wheel, tracked controllers Helps ground the user physically Can feel something that s in the virtual environment
Future of VR interactions Life-like interactions Less intermediary devices Eventually, no intermediary devices Directly interfacing with software Hand gestures, body movement, eye movement, speech Complete immersion, no obstacles User interface is completely software Data can be collected, interactions improved Transparency Retinal projection (Magic Leap) AR
Conclusion More forward-facing solutions Reaching the boundaries of a format leads to the next format More natural interactions Preparation for AR Future of VR isn t VR, it s AR Natural, transparent interactions are going to be even more important when proper AR comes around
Sources [1] J. Durbin, Oculus Touch vs. HTC Vive - Which Is The Better VR Controller?, UploadVR, 16-Jun-2016. [Online]. Available: https://uploadvr.com/oculus-touch-vs-htcvive-better-controller/. [Accessed: 09-Feb-2018]. [2] N. E. Seymour, A. G. Gallagher, S. A. Roman, M. K. O'Brien, V. K. Bansal, D. K. Anderson, and R. M. Satava, Virtual Reality Training Improves Operating Room Performance, Annals of Surgery, pp. 458 464, Oct. 2002. [3] M. McCoy-Thompson, How to Make VR Experiences That Actually Matter, Singularity Hub, 05-Feb-2018. [Online]. Available: https://singularityhub.com/2017/05/18/how-to-make-vr-experiences-that-actually-matter/#sm.00000u8vgqjybcfbcvjznxbis06qb. [Accessed: 09-Feb-2018]. [4] M. J. Schuemie, P. Van Der Straaten, M. Krijn, and C. A. P. G. Van Der Mast, Research on Presence in Virtual Reality: A Survey, CyberPsychology and Behavior, vol. 4, no. 2, 2001. [5] X. Zeng, X. Sang, D. Chen, P. Wang, N. Guo, B. Yan, and K. Wang, An interactive VR system based on full-body tracking and gesture recognition, Optical Design and Testing VII, 2016. [6] K. Olszewski, J. J. Lim, S. Saito, and H. Li, High-fidelity facial and speech animation for VR HMDs, ACM Transactions on Graphics, vol. 35, no. 6, pp. 1 14, Nov. 2016. [7] Sam Machkovech - Apr 8, 2016 1:02 pm UTC - https://arstechnica.com/tech-policy/2016/04/oculus-brings-real-and-pervasive-data-mining-to-virtual-reality/ https://www.gamasutra.com/view/news/312469/game_design_deep_dive_creating_comfortable_ui_for_vr_strategy_in_skyworld.php https://en.wikipedia.org/wiki/super_nintendo_entertainment_system#technical_specifications https://en.wikipedia.org/wiki/nintendo_entertainment_system https://en.wikipedia.org/wiki/gamecube https://en.wikipedia.org/wiki/wii https://en.wikipedia.org/wiki/wii_u https://en.wikipedia.org/wiki/nintendo_switch