CONTENTS. ENGINEERS GUIDE TO VIRTUAL REALITY & AUGMENTED REALITY Special Features IN IN THIS ISSUE

Size: px

Start display at page:

Download "CONTENTS. ENGINEERS GUIDE TO VIRTUAL REALITY & AUGMENTED REALITY Special Features IN IN THIS ISSUE"

Damon Patterson
5 years ago
Views:

2 IN IN THIS ISSUE CONTENTS ENGINEERS GUIDE TO VIRTUAL REALITY & AUGMENTED REALITY Vice President & Publisher Clair Bright Special Features Time for Visionary VR Tech to Focus on Vision By Lynnette Reese, Editor-in-Chief, Embedded Systems Engineering 4 VR/AR (XAR) Makes its Way through a Fragmented Landscape By Lynnette Reese, Editor-in-Chief, Embedded System Engineering 9 Editorial Editor-in-Chief Lynnette Reese lreese@extensionmedia.com Managing Editor Anne Fisher afisher@extensionmedia.com Senior Editors Caroline Hayes chayes@extensionmedia.com Dave Bursky dbursky@extensionmedia.com Pete Singer psinger@extensionmedia.com John Blyler jblyler@extensionmedia.com Creative / Production Production Traffic Coordinator Darla Rovetti Production Manager Nicky Jacobson Graphic Designer Simone Bradley Senior Web Developer Slava Dotsenko Advertising / Reprint Sales Vice President, Sales Embedded Electronics Media Group Clair Bright cbright@extensionmedia.com (415) ext. 15 Marketing/Circulation Jenna Johnson jjohnson@extensionmedia.com To Subscribe Extension Media, LLC Corporate Office President and Publisher Vince Ridley vridley@extensionmedia.com (415) ext. 18 Vice President & Publisher Clair Bright cbright@extensionmedia.com Human Resources / Administration Darla Rovetti Special Thanks to Our Sponsors Embedded Systems Engineering is published by Extension Media LLC, th Street, San Francisco, CA Copyright 2018 by Extension Media LLC. All rights reserved. Printed in the U.S. 2 December 2018

4 Time for Visionary VR Tech to Focus on Vision Virtual Reality has immersed us in captivating alternate worlds and intense gaming experiences. However, VR has much to look into, including VR-induced cybersickness, why children should not use VR headsets, and the contradictions VR displays present to our eyes. By Lynnette Reese, Editor-in-Chief, Embedded Systems Engineering The global market for Virtual Reality (VR) and Augmented Reality (AR) is expected to reach $94.4 billion by 2023, according to ReportLinker, a technology company that provides industry data on the topic. Of the two areas (VR and AR), VR held 60% of the market share in VR can create an immersive experience that can fascinate users who are transported to another world. Time spent in a VR headset is an issue for youngsters, however. VR is a new technology, so the effect of the long-term use on eyesight is unknown. VR is not recommended for kids age 12 and younger, as VR headsets are made to fit the pupillary distance of an adult, not a child. Excessive use of VR may also affect the way the eyes grow and lead to myopia or near-sightedness. Another side effect that VR is working to overcome is cybersickness, where nausea akin to motion sickness can be induced when using VR headsets, although not all VR headsets and not all people using them experience the same intensity of cybersickness. What challenges does VR face, and how can the VR industry overcome them? THE EFFECT OF VR HEADSETS ON EYESIGHT The Canadian Association of Optometrists states Most VR headset manufacturers have put in place warnings for children. This is impor- Figure 1: Commuters on a Chiltern Railways train to London experience a taste of Western Australia in fully immersive virtual reality (VR) headsets. The 360-degree interactive format gives passengers the chance to meet the wildlife of Western Australia. (Image: Joe Pepler/PinPep) 4 December 2018

6 a couple of hours without stopping is the norm, not the exception. But VR induces an eyesight-focusing situation that does not often occur in the real world, mainly because a VR headset fixes the display close to the eyes. No one knows if this is an actual health problem yet, but it s important to be at least aware of what s happening. Figure 2: Samsung VR demonstration. (Image: Maurizio Pesce, CC2.0) tant because a child s visual system continues to develop throughout childhood, and VR systems are set to the pupillary distance of the average adult, not child. Extended exposure to the awkward visual posture created by VR headsets can alter the development of focusing, tracking, and depth perception. 2 An increase in the incidence of myopia in the general population has been linked to the advent of the extensive use of smartphones, but no studies have definitively shown that using VR increases the propensity to develop myopia or any other permanent damage. Nevertheless, makers of VR headsets include warnings of risk to eyesight. Myopia has long been associated with near-work tasks that prevent one s eyes from naturally focusing at different distances in natural light. Microscopes are a lot like VR headsets, in that they involve close-up viewing through binocular lenses. A study associated the occupational use of microscopes with myopia in adults. 3 VR headsets are not often used in an occupational setting, however, which implies near constant use for 40 hours a week or more. AR headsets do allow natural vision but provide an overlay of virtual objects or information, much like a heads-up display in a cockpit. Presently, known optical issues specific to VR headsets include eye strain and dry eyes, since people caught up in an active or tense situation in a game do not blink as often as they should. Dry eyes can lead to extreme pain. Over a prolonged period of VR immersion, many users forget to take a break to rest their eyes to avoid eye strain. VR is fun, engaging, and immersive. It s understandable that playing for For thousands of years, our eyes have naturally converged to focus on objects located at different distances. However, the visual presentation in a VR headset forces an unnatural action for the eyes. In a VR headset, a separate image for each eye is projected on fixed, bright displays physically located close to the eyes. VR displays induce a sense of depth and a 3D quality by introducing a lateral relative offset of the objects in each image, which is like creating a slight double vision effect if you were to overlap the two images on one display. The higher the lateral disparity between the images, the nearer the object seems. In the real world, our eyes both converge and focus on the same point. In VR, objects that appear to be moving around at a distance are just a few centimeters from the eyes, which messes up the natural convergence action. VERGENCE AND ACCOMMODATION COUPLING Eyesight is complicated. To perceive an object, you first point your eyes in the direction of that object (convergence) and then your eyes lenses focus on it (accommodation). In industry lingo, vergence and accommodation coupling work together to create a meaningful image for you. In a VR headset, the eyes remain focused at a fixed distance even though they converge on objects that are seemingly located further away. In a VR headset, your eyes adjust to changes in convergence as objects move around, but unlike natural vision, your eyes never change focus, or accommodation. As a result, VR headsets force users to make exaggerated convergent eye movements when they look at virtual near objects. The eyes are forced to accommodate a situation that is not the normal mode of operation in the real world and can result in discomfort for minutes or hours. The long-term effects of VR headset use, especially on children, are not known. In a paper presented in a scientific report from the U.S. National Institutes of Health, Ocular effects of virtual reality headset wear in young adults, Turnbull and Phillips describe how the focal distance in VR is different than in natural environments, which affects vergence eye movements. To prevent double vision as gaze shifts 1. Global Industry Analysts. Market Research. Reportlinker Insight, Global Industry Analysts, June 2018, Augmented-Reality-and-Virtual-Reality-Market-by-Devices-by-Component-by-Application-by-Geography-Global-Market-Size-Share- Development-Growth-and-Demand-Forecast.html. 2. Are Virtual Reality Headsets Dangerous for Our Eyes? The Canadian Association of Optometrists, 23 May 2017, opto.ca/health-library/arevirtual-reality-headsets-dangerous-for-our-eyes. 3. McBrien, N A, and D W Adams. A Longitudinal Investigation of Adult-Onset and Adult-Progression of Myopia in an Occupational Group. Refractive and Biometric Findings. Current Neurology and Neuroscience Reports., U.S. National Library of Medicine, Feb. 1997, nlm.nih.gov/pubmed/ /. 6 December 2018

8 between objects, users make both version and vergence eye movements, which minimise retinal disparity of the object between eyes, and permit the object of interest to be perceived binocularly. In realworld viewing of objects, vergence eye movements are associated with changes in accommodation to focus the eyes at the depth of the object. However, in VR, the focal distance of all objects on the screen is constant, and the eyes must converge without changing accommodation to maintain a clear retinal image. Thus, wearing a VR HMD creates a dissociation between convergence and accommodative demands, which may contribute to visual discomfort. 4 Twirling around in a swivel chair also induces dizziness and an offkilter center of balance that can be uncomfortable if you spin too fast or for too long. There aren t any studies that tell us whether spinning around in a swivel chair for hours at a time is detrimental to one s health in the long term, but it is certain that such sustained activity is unnatural in the evolution of the human race. CYBERSICKNESS Users who experience discomfort while using VR headsets have also experienced cybersickness, which is a feeling of nausea much like motion sickness. Cybersickness occurs when visual information doesn t match the user s body position, sending false signals to the senses, such as when the VR headset immerses the user in a moving landscape when in reality the user is sitting still. A VR experience full of sharp turns, rapid acceleration, and falling would be sufficient for most to induce cybersickness. A time-lag between physical movement and perceived movement can also create sickness. Cybersickness results in a short-term (a few minutes) to long-term (a few hours) illness characterized by dizziness, vertigo, disorientation, headache, and nausea. Cybersickness that lasts for hours can potentially be a real-life problem, for instance, leading to hours of impaired balance or driving. Game developers have found that adding a virtual stationary object in the game can help reduce cybersickness. Stationary objects in the frame can be as simple as adding a front-seat bar in a roller coaster or a virtual nose. 5 VR games that do not require a lot of head movement contribute less to cybersickness, as well. Displays with images that move with you as they do in the natural world are an improvement. VR headset displays that move the entire visual field when the wearer moves their head can induce sickness, because the experience is little more than strapping a television to your face. Light-field technology can reduce cybersickness since it creates images where people can look around as the image stays fixed while the wearer moves through the virtual world. Light-field technology imparts imagery that is more true-to-life in terms of what influences depth perception, shadows, and motion parallax (i.e., objects closer to you move faster than those farther away from you). CONCLUSION VR is still in its technological infancy. Any technology that makes people feel sick needs improvement. VR makers are well aware of the problem, although no one knows the exact mechanics of the cause as to why some are more prone to cybersickness than others. We can mitigate cybersickness with high-performance hardware and with VR games that avoid sending false or conflicting sensory signals about physical movement to the brain. Eliminating the headset in VR is not a possibility yet, although a real-life holodeck as imagined by Star Trek Next Generation creators would solve the vergence and accommodation coupling issue discussed above. Displays that are less exhausting for the eyes as backlit LCDs might also improve the VR experience. Without information on the effects of long-term use of VR headsets, the best action for VR use today is to limit the use of VR to 30 minutes per session, take regular breaks, and avoid any VR use for children under the age of twelve unless the VR is specifically designed and intended for children. Lynnette Reese is Editor-in-Chief, Embedded Intel Solutions and Embedded Systems Engineering, and has been working in various roles as an electrical engineer for over two decades. She is interested in open source software and hardware, the maker movement, and in increasing the number of women working in STEM so she has a greater chance of talking about something other than football at the water cooler. 4. Turnbull, Philip R. K., and John R. Phillips. Ocular Effects of Virtual Reality Headset Wear in Young Adults. Current Neurology and Neuroscience Reports., U.S. National Library of Medicine, Nov , 5. Kanarbik, Kevin, and Al William Tammsaar. Best Ways of Producing Cybersickness in VR. Introduction to Computational Neuroscience, University of Tartu, Estonia, 2015, interview with Madis Vasser. 8 December 2018

VR/AR (XAR) Makes its Way through a Fragmented Landscape VR/AR developers continue to strive for a fully immersive experience and additional and innovative content.

9 VR/AR (XAR) Makes its Way through a Fragmented Landscape VR/AR developers continue to strive for a fully immersive experience and additional and innovative content. But why is consumer VR/AR (XAR) adoption so slow? By Lynnette Reese, Editor-in-Chief, Embedded Systems Engineering Figure 1: Ultrahaptics creates the sensation of touch in midair. Combined with VR, one can pick and place virtual objects in VR. (Image: ultrahaptics.com) XAR is a term that industry experts use to refer to some mixture of Augmented Reality (AR) and Virtual Reality (VR). VR is at one end of the spectrum and AR is at the other extreme. In between you have some virtual and some augmented reality, but it is a mixture, which is XAR. VR is a stepping stone to something even better; eventually there will be more complex devices than what we have today. XAR devices are made up of three main areas: visual, audio, and interaction with the virtual environment. First, developers want to create an immersive 360 experience equivalent to reality. To do that, the visuals must be of very high density with a lot of pixels to make sure that whatever the user sees in a virtual world is equivalent to what they would see in real life. However, the VR industry is not there yet. Second, the user needs immersive, realistic sound such that the experience is intuitive and natural, exactly emulating a situational environment in every sense. Again, the industry is not there yet. The third area for any XAR business is interaction that is intuitive, or like a natural interruption. Many children experiencing VR for the first time want to touch or push what they are seeing. A natural response to a virtual entity is confirmation that the virtual environment is doing something right. Ideally, the most immersive experience would have all three areas: visual, spatial audio, and interaction through touch. The industry is seeing products like UltraHaptics for interaction, but touching virtual objects in mid-air using ultrasonic vibration is nascent technology. XAR can bridge gaps and connect people to experiences where they can access places that are otherwise inaccessible. With XAR, people can immerse themselves in a new environment that they couldn t otherwise experience. The future of XAR is an immersive medium where people can experience something directly through their senses, as opposed to having that experience interrupted by a headset that blocks the view. How will XAR technology evolve? Visually, the industry is making progress towards the desired pixel density. The goal is a very high resolution that equals the resolution of the human eye while also delivering a seamless sensation with a high number of frames per second (fps), although many would say that a 1K by 1K pixel quality per eye is good enough. Processors that can deliver a high rate of fps, a high resolution in the density of pixels, and very low latency are the holy grail for VR. Frame rates less than 90 fps tend to cause simulation sickness in most people. Eliminating the headset requires the same performance delivered to wall-sized displays. Rendering data that meet such demanding requirements provides a quality immersive experience but implies the need for potent processors. OVERCOMING PLATFORM FRAGMENTATION VR gaming, unlike traditional gaming platforms, is taking a long time to make its way into the mainstream. One difficulty in attracting interest is that it s difficult to portray the richness of VR in a two-dimensional video when advertising to the uninitiated. Another challenge facing VR is that technology is changing fast in a very fragmented VR platform market. For example, in VR gaming there are no standards for a User Interface (UI). Each platform creates a different way for users to navigate. Without standards, a fragmented realm of VR gaming platforms means that VR games and other content must be ported to several different platforms. 9

Yet much of what drives the gaming industry is the content, not the platform.

inhabit. Augmented Reality (AR) on mobile devices like smartphones and tablets is experiencing uptake, ever since Pokémon Go was released in mid-2016.

10 Yet much of what drives the gaming industry is the content, not the platform. Developers want to maximize the content experience on every form factor, yet they have a lower performance margin to work with when altering games and user experiences in VR because they cannot risk dropping framerate. It s difficult for VR game developers to ensure maximum saturation, the maximum amount of immersive or detailed content, while also accounting for all of the user devices that games or applications inhabit. Augmented Reality (AR) on mobile devices like smartphones and tablets is experiencing uptake, ever since Pokémon Go was released in mid For example, with an AR app on a smartphone, a customer can hold their phone up to a shelf and see a product information overlay. However, people do not want to hold their phone at arm s length for a long time. Figure 3: Farmers Insurance uses Oculus Rift to train employees for real-world scenarios before they go out into the field, cutting travel costs to send new hires to training facilities. (Image: Oculusforbusiness.com) captured memory of an event, such as sharing the kids opening Christmas presents with a parent stationed in the military overseas. As a solitary experience, VR for the moment is still, for the most part, socially isolating. Currently, the price for a decent VR system is as high as a laptop. A genuinely good system for a top-of-the-line VR experience costs in the range of a thousand dollars or more, which is out of reach of the mainstream consumer. Three areas need to improve before VR enters the mainstream consumer market: an affordable price for quality VR, the ability for people to share or socialize with the VR experience, and additional and innovative content. Figure 2: Walmart partnered with STRIVR to deploy an immersive training program for Oculus Rift in 200 Walmart academies across the U.S. (Image: Oculusforbusiness.com) WHY IS VR SO SLOW TO BE TAKEN UP? VR has not hit the tipping point yet. One analogy to consumer VR adoption is the PC market of the 80s and 90s, when at first PCs were too expensive while also having too little memory, storage, and processor speed to be good enough for the market that we have in gaming today. Simple gaming platforms like Atari were the limit of the gaming experience. Today we expect fast-moving, immersive gaming experiences that can also carry multi-player communities with a social aspect. XAR does open up new experiences. One can watch video on a personal screen and avoid disturbing others because the headset is immersive and confines the screen to the user s eyes only. For training purposes, XAR cannot be beat, since trainees are not as easily distracted as they would be watching a typical 2D training video. The headset forces the wearer to be in the training environment. Surreptitiously looking at one s phone is not possible without removing the headset. Future consumer-oriented devices using VR would include cameras that will allow families to take stereoscopic videos of children or pets that can be shared with family members on another continent. The family VR video would be an immersive The social aspect of VR is improving. Often, parents will buy a gaming platform with the excuse of kids or families playing it together. But unlike bowling with a Wii console at a party, VR has been slow to create socially engaging experiences. Presently, VR is only experienced by the individual who is wearing a headset. When PCs first enabled chat facilities on the internet, the social aspect of engaging with real people made services like AOL take off. VR has a capability similar to chat in that an immersive conversation, albeit using an avatar (one s physical representation to others in VR), seems more real than text flowing across a screen. Supported on Oculus Rift and HTC Vive, Facebook Spaces (still in beta) offers a VR Figure 4: Bigscreen offers a VR space where remote teams can collaborate together in virtual offices. Other use cases of Bigscreen include a virtual living room to watch movies, play video games, browse the web, and hang out with friends. Bigscreen beta launch was in March December 2018

11 experience much like a chat room but with the physical experience of seeing other VR users in the room, also gesturing and talking. Facebook recently updated the capability for user s avatars to look more like the user, although they still resemble cartoon characters. Bigscreen is a VR room (or lounge ) for sharing experiences with others around the world. According to the Bigscreen site, Use cases of Bigscreen include both entertainment and productivity. It s used as a virtual living room to watch movies, play video games, browse the web, and hang out with friends. It s also used for productivity as a tool for remote teams to collaborate together in virtual offices. This use case for XAR enables collaborating with physical movements possible, such as arranging dance choreography or rehearsing a theater act. Bigscreen requires a quality VR headset as well as a PC with a minimum of 16 GB RAM, a powerful graphics card, and an Intel Core i5/i7 processor or AMD Ryzen (or higher) core for a comfortable performance. Another such VR lounge is the Oculus Rec Room, where users can play games together with virtual others. Many users claim that in a VR meeting space, they immediately feel like the person they re talking to is right next to them. The audio experience correlates to the visual experience. Users on a quality system like the Oculus can hear a colleague on the left and another on the right in these virtual meeting rooms, which makes all the difference versus a flat-screen video conference in terms of meeting others. XAR is a growing yet challenging industry. However, rapidly improving technology is enabling XAR to incrementally improve quality for a more immersive experience. Lynnette Reese is Editor-in-Chief, Embedded Intel Solutions and Embedded Systems Engineering, and has been working in various roles as an electrical engineer for over two decades. She is interested in open source software and hardware, the maker movement, and in increasing the number of women working in STEM so she has a greater chance of talking about something other than football at the water cooler. Virtual Reality & Augmented Reality ONLINE eecatalog.com/virtual-augmented-reality Explore... Top Stories and News White Papers Expert Opinions (Blogs) Exclusive Videos Valuable Articles Virtual Reality & Augmented Reality Quarterly Report newsletter

Bring Imagination to Life with Virtual Reality: Everything You Need to Know About VR for Events

Bring Imagination to Life with Virtual Reality: Everything You Need to Know About VR for Events 2017 Freeman. All Rights Reserved. 2 The explosive development of virtual reality (VR) technology in recent