Virtual Reality I. Visual Imaging in the Electronic Age. Donald P. Greenberg November 9, 2017 Lecture #21

Size: px

Start display at page:

Download "Virtual Reality I. Visual Imaging in the Electronic Age. Donald P. Greenberg November 9, 2017 Lecture #21"

Evan Rogers
5 years ago
Views:

1 Virtual Reality I Visual Imaging in the Electronic Age Donald P. Greenberg November 9, 2017 Lecture #21

2 1968: Ivan Sutherland 1990s: HMDs, Henry Fuchs 2013: Google Glass History of Virtual Reality 2016: HTC & Valve s SteamVR Vive 25 Years

3 Types of Virtual Reality (VR) by Viewer Constraints Static location of viewer Movable location of viewer Augmented Reality

4 Static Location Known (or not considered)xx ee, yy ee, zz ee Unknown (Ɵ ee, ΘΘ ee, φφ ee ) How measured- (accelerometers) cameras

5 Movable Location e.g., within a room Unknown (xx ee, yy ee, zz ee ) Unknown (Ɵ ee, ΘΘ ee, φφ ee ) How measured- triangulation signals, light, timing I saw you cameras in goggles or cameras on wall

6 Augmented Reality No limitations Need GPS position relative to an outside environment > Not yet available- not accurate enough Current accuracy is unclear Can put virtual image in real space (MSFT Hololens)

7 Augmented Reality Augmented reality is the interpretation of digital graphical information with live video or the user s physical environment in real time. UNC

8 Microsoft s Hololens 3/30/16

9 Types of VR by Data and Image Generation Algorithms Environment Maps and Texturing Full 3D Geometric Environments Combination

10 Environment Maps and Texturing Can t get close to virtual surface Stationary position? Input- Photography Rendered image Both methods would be placed as a texture on the inside of a hemispherical dome (e.g., New York Times)

11 Full 3D Environment Required detail models (3D) Can move within constrained environment Can be tracked (Gaze Directions) Rendering- not fast enough Currently limited to diffuse environments Sometimes with shadow maps Limited light sources

12 Combination 2D + 3D Environment Maps & 3D Environments Can move within 3D virtual environments Need models and limitations Accuracy? Where do we need the 3D information?

13 Virtual Reality A term used to describe a computer generated environment which can simulate the perception of PRESENCE.

14 Virtual Reality A person immersed within this virtual world can manipulate objects, interact with the environment, and explore the virtual world in the same perceptual way as one interacts with the physical world.

15 Human in the Loop Abstract Interpretation Viewing a Picture on Television Cinema Viewing Presence

20 Microsoft s Hololens

21 Microsoft buys Minecraft

22 Microsoft s Hololens

23 Magic Leap

24 Oculus Rift DK2 2014

Oculus Rift March 28, 2016 Oculus VR PenTile OLED 2160x1200 (1080x1200 per eye) at 90 Hz Integrated 3D audio headphones (user removable/exchangeable)

25 Oculus Rift March 28, 2016 Oculus VR PenTile OLED 2160x1200 (1080x1200 per eye) at 90 Hz Integrated 3D audio headphones (user removable/exchangeable) Wikipedia 6DOF (3-axis rotational tracking + 3- axis positional tracking) through USBconnected IR LED sensor, which tracks via the constellation method.

26 HTC and Valve s SteamVR Vive 2016 Available Tomorrow

27 Virtual Reality Speed (Refresh and update rates) Field of view and resolution Color and dynamic range Bandwidth

28 Refresh Rates 2016 Note that an insufficient refresh rate will cause flicker. For most cases the displays must be refreshed at least 50 cycles/second. Current virtual reality displays (2016) operate at 90 Hz.

29 Update Rates 2016 The human eye can process approximately separate images per second (maybe even less). To simulate motion, update rates must be greater than this. The movie industry has historically used 24 fps and the television industry 30 fps. Since in VR/AR continuous motion is paramount, then rendering must be about 11 milliseconds (1000/90) or less.

30 Field of view Horizontal Vertical Wikipedia

31 Field of View Foundations of Sensation and Perception. Mather, George

32 Field of View for Humans Humans have an almost 180 degree frontal horizontal field of view The vertical range of the visual field is approximately 135 degrees The resolution, color discrimination, and reaction times is not uniform across the field of view

33 Stereoscopic & Monoscopic Depth Information Note that only when the two fields overlap do we have the ability to determine depth stereoscopically Otherwise most of our depth cues are received monoscopically

34 Binocular Vision Binocular Vision, which is the basis for stereopsis is important for depth perception and covers 114 degrees (horizontally) of the human visual field. The remaining sixty to seventy degrees have no binocular vision (because only one eye can see those portions of the visual field)

35 Human Depth Perception Depth Perception Oculomotor Visual Binocular Monocular Binocular Monocular Convergence Accommodation Stereopsis Static Cues Motion Parallax Perspective Familiarity, Relative Size Motion, Position Occlusion Texture Gradient Shading, Shadows, Highlights Atmospheric Blur

36 Saccadic Motion The eye jumps, comes to rest momentarily (producing a small dot on the record), then jumps to a new locus of interest. - David H. Hubel. EYE, BRAIN, AND VISION, 1988 Scientific American Books, Inc. p. 80.

37 Resolution of the Human Eye Humans can tell visual details at distances larger than 0.3 arc minutes The Field of View (FOV) of the human eye can be generously estimated as 120 by 90 degrees

38 Resolution of the Human Eye (120 degrees x 60 arcminutes / degree x 1 pixel / 0.3 arcminutes) x (90 degrees x 60 arcminutes / degree x 1 pixel / 0.3 arcminutes) 431,568,000 pixels; 432 MegaPixels. A 1080p display is 2.1 megapixels.

39 Image from Eye, Brain, and Vision, David Hubel, 19

40 Opponent Color Theory Analog Computing Figure 12.9 Foundations of Sensation and Perception, George Mathe

41 Dynamic Range of Displays luminance (log cd/m 2 ) range of illumination displays starlight moonlight indoor lighting sunlight scotopic mesopic photopic poor contrast no color low acuity good contrast good color high acuity

42 Samsung AMOLED Displays Standard LCD RGB Display PenTile Display

43 Distorted Images

44 Truck Art

45 Distorted Images DORSEY 1993

46 Distorted Images DORSEY 1993

47 Distorted Images DORSEY 1993

48 Technology of Current VR Systems

49 Oculus Rift

50 Oculus Rift Components Accelerometers and logic board Lenses Flat 1080P AMOLED Display

51 Oculus Rift Angular Rotation

52 Distortion Strategy

53 Oculus Rift Distortion Strategy

54 Oculus Rift Distorted Image 1080x1200 pixels per eye

55 Compensating for Lens Distortion The lenses in Head Mounted Displays (HMD s) apply a pincushion distortion to the image To correct the image to the viewer, the pipeline applies a barrel distortion as a post rendering process, to yield an undistorted image Since this operation requires a resampling, the image becomes slightly blurred

56 Compensating for Lens Distortion To overcome these deficiencies, frequently the image is rendered at a higher pixel resolution, but this requires more processing time Another method is to render the scene normally, then stretch it over a 3D polygon mesh as a template, and use a GPU to display it. (e.g. photographing a curved virtual screen)

57 End..

Virtual Reality. Lecture #11 NBA 6120 Donald P. Greenberg September 30, 2015

Virtual Reality. Lecture #11 NBA 6120 Donald P. Greenberg September 30, 2015 Virtual Reality Lecture #11 NBA 6120 Donald P. Greenberg September 30, 2015 Virtual Reality What is Virtual Reality? Virtual Reality A term used to describe a computer generated environment which can simulate