LOOKING AHEAD: UE4 VR Roadmap. Nick Whiting Technical Director VR / AR

LOOKING AHEAD: UE4 VR Roadmap Nick Whiting Technical Director VR / AR

HEADLINE AND IMAGE LAYOUT RECENT DEVELOPMENTS

RECENT DEVELOPMENTS At Epic, we drive our engine development by creating content. We use these as real-world test cases to drive our framework and optimizations to the engine.

RECENT DEVELOPMENTS INSTANCED STEREO RENDERING Lets us use a single draw call to draw both the left and right eyes, saving CPU (and some GPU) time. In Bullet Train, we saved about 1.75ms on the CPU, and 0.75ms on the GPU Currently works on PC, and coming soon on PS4 and mobile platforms Enable it in your project s settings

RECENT DEVELOPMENTS Traditional Stereo Rendering, where each eye is drawn one at a time and sequentially.

RECENT DEVELOPMENTS Instanced Stereo Rendering, where each eye is drawn at the same time, with one draw call.

RECENT DEVELOPMENTS INSTANCED STEREO RENDERING Instanced stereo is available now for the deferred (desktop) renderer and the PS4. We re also implementing multi-view extension support for the forward (mobile) renderer, and improving PS4 support to better utilize the hardware.

RECENT DEVELOPMENTS HIDDEN AND VISIBLE AREA MESH We draw meshes to cull out geometry that you can t see, and only apply post processing where you can. For the deferred renderer, the visible area mesh is a bigger optimization! This is specific per-platform

RECENT DEVELOPMENTS The Hidden Area Mask uses a mesh to early-out on pixels that aren t visible in the final image.

RECENT DEVELOPMENTS The Hidden Area Mask uses a mesh to early-out on pixels that aren t visible in the final image.

RECENT DEVELOPMENTS The Visible Area Mask uses a mesh to constrain our post processing to the visible pixels only.

RECENT DEVELOPMENTS HIDDEN AND VISIBLE AREA MESH With the two combined, we saw the following gains on actual game content: Oculus Vive PSVR ~0.25-0.3 ms ~0.3-0.4 ms ~1.0 ms

RECENT DEVELOPMENTS CAMERA REFACTOR As of 4.11, we ve completely rewritten the camera system in order to make developing much easier! Camera Components now move exactly as the real HMD is moving You can attach components (meshes, UI, etc) directly to the camera component!

RECENT DEVELOPMENTS LATE UPDATING Latency is super important, so we update cameras, attachments, and motion controllers right before we kick off GPU thread work, to make sure they feel as snappy as possible. This makes a huge difference on 60Hz platforms.

RECENT DEVELOPMENTS Above are the game and render threads, as well as the display flip. The red line represents the latency previously seen by Motion Controllers and camera-attached components, while the green is the improved latency.

HEADLINE AND IMAGE LAYOUT PLATFORM SUPPORT

PLATFORM SUPPORT As of 4.12, we support the following platforms out of the box: Oculus Rift Steam VR (including the HTC Vive) PSVR OSVR (preview) Samsung Gear VR Google Daydream Leap Motion

PLATFORM SUPPORT As of 4.12, we support the following platforms out of the box: Oculus Rift Steam VR (including the HTC Vive) PSVR OSVR (preview) Samsung Gear VR Google Daydream Leap Motion

BUILT-IN PLATFORM SUPPORT

PLATFORM SUPPORT Create once, and deploy anywhere. Desktop / Console Oculus Rift HTC Vive / Steam VR PSVR OSVR Mobile Samsung Gear VR Google Daydream

PLATFORM SUPPORT All of these platforms go through UE4 s common VR interfaces, so you can make your content once, and deploy it anywhere. Unified Camera System Motion Controller System Optimized rendering paths Low-latency optimizations Oculus Vive PSVR OSV R

PLATFORM SUPPORT What about Augmented Reality?

PLATFORM SUPPORT Download it now, for free!

PLATFORM SUPPORT We ll have more announcements this year though, so stay tuned ^-^

HEADLINE AND IMAGE LAYOUT ENGINE AND FRAMEWORK FEATURES

ENGINE AND FRAMEWORK FEATURES Performance is key in all of this. Not only GPU, but CPU. C++ In 4.12, you can begin experimenting using the Blueprint to C++ Conversion for extra performance on console and mobile platforms. We ll be improving support in the future!

ENGINE AND FRAMEWORK FEATURES MIXED REALITY MOVIE OUTPUT We love what people have done to mix real and VR footage! We re adding support for this right into the engine Support for static or tracked cameras Near and far cameras, so you can properly composite the scene Plan to support capturing video into the engine directly in the future

ENGINE AND FRAMEWORK FEATURES PERFORMANCE BASED SCALABILITY Adding hooks to get timing information from the platform, to let you scale your performance up and down accordingly We ll add the hooks for you to scale how you see fit Each game is different, and may choose to scale in different ways! Screen Percentage 90% - 150% + MSAA (4x, 8x, 16x)

HEADLINE AND IMAGE LAYOUT RENDERING

RENDERING For our development of Bullet Train, we used the desktop deferred renderer. Because we have access to a lot of information about the scene in the GBuffer, we can cheat and create a lot of optimizations and effects leveraging the information.

RENDERING To achieve the reflections in Bullet Train, we reflected the geometry on to billboards below ground, and then used the scene normals to distort the reflections realistically.

RENDERING To achieve the reflections in Bullet Train, we reflected the geometry on to billboards below ground, and then used the scene normals to distort the reflections realistically.

RENDERING The same technique was used on the characters, but we also blended between a few poses in order to handle rotation.

RENDERING We also utilized the GBuffer in order to fake the lighting in the train scene. This scene originally used 70+ dynamic lights!

RENDERING However, since we know scene normals and depth, we can project a pre-made light / shadow texture on to the world.

RENDERING We believe in the power of leveraging the deferred renderer to create visually rich experiences. However, there are some trade-offs that might not be right for all experiences in VR.

RENDERING DESKTOP FORWARD RENDERER

RENDERING Comparison of Renderer Types Deferred Renderer Temporal anti-aliasing Access to GBuffer for effects Full access to all rendering features Maximum versatility Forward Renderer Multi-sample anti-aliasing (MSAA) Faster overall rendering pass Specialized rendering features Maximum scalability

RENDERING How does forward help you scale more effectively? Deferred Renderer Forward Renderer Screen Percentage 110% - 140% Screen Percentage 90% - 150% * See Alex Vlachos GDC 2016 for more detail! + MSAA (4x, 8x, 16x)

RENDERING There are advantages to both! Deferred is a full-featured workhorse, but takes a bit of skill to fully leverage. Temporal AA limits how sharp your image can be. Forward is a feature specialized renderer, with less features, but faster baseline. MSAA is the sharpest solution for anti-aliasing.

RENDERING There are advantages to both! We re working closely with the team at Oculus in order to leverage the work and research they ve done into VR Forward Rendering in UE4.

RENDERING DESKTOP FORWARD RENDERER

RENDERING DESKTOP FORWARD RENDERER Coming this Fall*

HEADLINE AND IMAGE LAYOUT VR EDITOR

VR EDITOR Epic has a long history of innovation in how we create content in games, going all the way back to 1998

VR EDITOR The VR Editor began its life as a collaboration between Epic and students at Stanford University. Two teams took very different approaches to investigating what editing VR content from in VR would be like.

VR EDITOR TEAM KARAAGE This team focused on trying out various novel input devices, like the Novint Falcon and a 3D CAD mouse. The importance of haptics! Novel inputs took significant time to master UI was treated as a ring

VR EDITOR TEAM JARVIS This team focused on trying out smartphone style interfaces using two motion controllers Very intuitive for most people, easy to pick up! Tried voice controls to get around UI limitations

VR EDITOR Target Audience 1: Ourselves! Layout Scripting and Iteration Polish As a first target, we d like to improve iteration time on the first and the last 20% of our level creation process.

VR EDITOR Target Audience 2: Non-Professional Game Developers! By leveraging intuitive interfaces, we can make a more approachable, natural experience for those new to 3D tools

RENDERING We released the VR Editor Preview at GDC, and put it in 4.12 We put it out there to get as much feedback from real users as possible. We re also beginning to use it to develop our own content internally for the first time in production. With that in mind, here s where we re going.

VR EDITOR IMPROVED EDITING GIZMO Based on feedback, we needed to make the editing gizmo easier to use, yet more exact. Bigger locations to grab at Better snapping to locations and rotations when manipulating the objects

VR EDITOR MORE WORKFLOWS While we have all of our Slate UI available in the VR Editor, we re beginning to investigate other workflows that might benefit from specialized UI. Animation Posing Quick Level Scripting Material Editing

HEADLINE AND IMAGE LAYOUT DOCUMENTATION AND LEARNING

DOCUMENTATION AND LEARNING VR CONTENT EXAMPLES We want to give you a jump start on common tasks that you have to do in VR, by providing content examples you can use in your projects Example Camera Setups Movement Interaction

DOCUMENTATION AND LEARNING TEMPLATE PROJECT A great base for you to start creating content that is performant on your target device. Includes settings for optimal rendering and performance in VR, so you can build your projects right from them.

DOCUMENTATION AND LEARNING Getting off to a quick start is key to success! These templates and samples will be put up as soon as they re ready for you to use. Documentation is currently being updated to reflect all current devices and their latest SDKs.

THANK YOU! HEADLINE AND IMAGE LAYOUT