Depth Imaging the engine of the renaissance of VR/AR

Transcription

1 Depth Imaging the engine of the renaissance of VR/AR OR VR/AR is hear but are we ready? DR. Giora Yahav 1

2 Image: Mashable, Bob Al-Greene 2

3 Introduction 3

4 The Beginning 1957 Morton Hellig SENSORAMA Morton Heilig built a single user console called Sensorama that included a stereoscopic display, fans, or emitters, stereo speakers and a moving chair. This enabled the user watch television in three dimensional ways There were clearly elements of AR involved in this device the fact that the environment itself was viewed in a real time situation - even if recorded. 4

5 HMD the ultimate display Philco Corporation 1965 The Headsight Ivan Sutherland 1965 Damocles sword This project desired to produces visual stimulation together with a tracking system. It also had been used for military purposes. The VR in this invention would look like the world the person lived in. It focuses on 3D objects and 5 also provides tactile stimuli

6 The Eye Phone 1989 Two people demonstrate the EyePhone system which uses special goggles and a à DataGloveÃ, which allows them to see and move objects around in a computer created environment. The EyePhone, developed by VPL Research, was on display at the Texpo Telecommunications Show held in San Francisco June 7,

7 AR is Born 1992 Augmented Realityis supposed to have been coined by Professor Tom Caudell while working in Boeing s Computer Services' Adaptive Neural Systems Research and Development project in Seattle LB Rosenberg creates what s widely recognized as the first functioning AR system for the US Air Force known as VIRTUAL FIXTURES Steven Feiner, Blair Maclntyre and Doree Seligmann - all of whom now lead in the field of AR - submitted a paper on a prototype system they called KARMA (Knowledge-based Augmented Reality for Maintenance Assistance) 7

8 The mobile era 2008 Mobilizy, Austrian based company was among the pioneers as it brought its Wikitude app to the T-Mobile G1 allowing Android users to take in the world through their mobile phone cameras and see augmentations on the screen 8

9 Today Oculus Rift Google HoloLens by Microsoft "the first really professional PC-based VR headset". HoloLens is the first fully untethered, see-through holographic computer. 9

10 The Key Elements for a Consumer Grade VR/AR Display We see that AR/VR is not new than what will make it this time to stay?? Computer 3 D Camera 10

11 3D Imaging 11

12 Depth Extraction Methods Stereoscopic vision Time Of Flight Pulsed TOF Structured light Phase Modulation From: Using 3D sensors to bring depth discernment to embedded vision apps, 4/13/2013 Michael Brading, Kenneth Salsman and Manjunath Somayaji, Aptina Imaging; Tim Droz and Daniël Van Nieuwenhove, SoftKinetic; Pedro Gelabert, TI; and Brian Dipert, BDTI and Embedded Vision Alliance 12

13 Triangulation Structured Light (Primesense) 13

14 Signal ToF Sensor Pulsed Time of Flight 1. Pulse of light is sent by the Camera 2. Pulse of light is returned from the target 3. Returning light is truncated by the shutter 4. Pixel response corresponds to imaged depth R(D) 14 Pixel #

15 ToF Sensor Pulsed Time of Flight Target reflectance adds another variable to the pixel response: R(ρ, D) Therefore, the depth is traditionally calculated by taking to responses and canceling out the reflectance: 1. Gated response: G(ρ, D) 2. Ungated response: U(ρ) D~G/U 15

16 3D Imaging for Mobile Devices 16

17 Why do we need the 3D camera in AR? Mapping and positioning Interfacing 17

18 The Key Considerations for Mobile Devices Size & Efficiency Power Cost 18

19 Depth Extraction Pyramid 19

20 Total Potential for Photonic Enhancement Contrast Ratio Noise Level Illumination Saturation level Si Fill Factor Quantum Efficiency Ambient illummination 20

21 The results: Pulsed TOF 21

22 and others ST TOF Assisted AF Oppo R7 Plus Nexus? ASUS Zenphone 22

23 Microsoft HoloLens 23

24 Q&A 24