VR with Metal 2 Session 603

Transcription

1 Graphics and Games #WWDC17 VR with Metal 2 Session 603 Rav Dhiraj, GPU Software 2017 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.

2 VR Support in macos Developing VR Apps External GPU support

3 What is Virtual Reality? Immersive 360 3D experience Direct object manipulation Interactive environment Motion tracking

4 Enabled with Metal 2

5 Enabled with Metal 2 Direct to display capability for VR Headsets

6 Enabled with Metal 2 Direct to display capability for VR Headsets Targeted features for VR

7 Enabled with Metal 2 Direct to display capability for VR Headsets Targeted features for VR Foundational support for External GPUs

8 Platform Support NEW HTC Vive Head Mounted Display Valve SteamVR runtime Valve OpenVR Framework

9 VR Compositor Image warping for HMD optics

10 VR Compositor Image warping for HMD optics

11 Building a VR App

12 Building a VR App Two options

13 Building a VR App Two options Game engine with VR support Hides VR compositor complexity Familiar toolchain

14 Building a VR App Two options Game engine with VR support Hides VR compositor complexity Familiar toolchain Build your own native VR app Full control of rendering and synchronization

15

16

17

18 Overall, the porting of Space Pirate Trainer to macos with Unity went very smooth. We had it running on macos under a couple of hours Dirk Van Welden, I-Illusions

19 Native SteamVR App Custom app built using the OpenVR Framework Binaries and documentation available on GitHub macos sample code available soon Application OpenVR Framework SteamVR Runtime HMD and Controllers

20 VR App Building 101 Overview of VR development macos platform specifics Anatomy of a VR frame VR best practices

21 VR App Building 101 Overview of VR development macos platform specifics Anatomy of a VR frame VR best practices

22 Traditional Workload Non-VR VBL VBL VBL GPU

23 Traditional Workload Non-VR 60 fps target VBL VBL VBL GPU

24 Traditional Workload Non-VR 60 fps target 16.7 ms available frame time VBL VBL VBL GPU 16.7 ms

25 Traditional Workload Non-VR 60 fps target 16.7 ms available frame time Entire frame time available for GPU work VBL VBL VBL GPU Frame 0 Frame ms

26 VR Workload Frame time differences VBL VBL VBL GPU

27 VR Workload Frame time differences 90 fps target Reduces frame time to 11.1 ms VBL VBL VBL VBL GPU 11.1 ms

28 VR Workload Frame time differences 90 fps target Reduces frame time to 11.1 ms Additional GPU work by VR compositor VBL VBL VBL VBL GPU App 0 0 App 1 1 App ms

29 VR Workload Frame time differences 90 fps target Reduces frame time to 11.1 ms Additional GPU work by VR compositor VBL VBL VBL VBL GPU App 0 0 App 1 1 App ms 1 ms

30 VR Workload Frame time differences 90 fps target Reduces frame time to 11.1 ms Additional GPU work by VR compositor ~10ms frame time budget for your app VBL VBL VBL VBL GPU App 0 0 App 1 1 App ms 10 ms 1 ms

31 VR Workload More work every frame

32 VR Workload More work every frame Rendering the scene twice Left and right eye

33 VR Workload More work every frame Rendering the scene twice Left and right eye Rendering at higher resolution HTC Vive headset: 2160x1200 Commonly supersample at 1.2 to 1.4x when rendering

34 VR App Building 101 Overview of VR development macos platform specifics Anatomy of a VR frame VR best practices

35 Direct to Display NEW Low latency path that bypasses the Window Server VR Compositor presents directly to the HMD

36 Direct to Display NEW Low latency path that bypasses the Window Server VR Compositor presents directly to the HMD VR headsets not exposed as regular displays

37 Present to Display macos App macos Window Server Metal Display

38 Present to Headset macos with Metal 2 App VR App macos Window Server VR Compositor (eg. SteamVR) Metal 2 Metal 2 Display HMD

39 Selecting a Metal Device

40 Selecting a Metal Device SteamVR selects the device attached to the VR Headset

41 Selecting a Metal Device SteamVR selects the device attached to the VR Headset Apps should select the same Metal device

42 Selecting a Metal Device SteamVR selects the device attached to the VR Headset Apps should select the same Metal device id<mtldevice> vrdevice = nil; vrsystem->getoutputdevice((uint64_t*)&vrdevice, vr::texturetype_iosurface);

43 Managing Surfaces App VR Compositor App L L R R VR Compositor L R

44 Managing Surfaces App VR Compositor App Render L L R R VR Compositor L R

45 Managing Surfaces App VR Compositor App Render L L R R Submit VR Compositor L R

46 Managing Surfaces App VR Compositor App Render L L R R Submit VR Compositor Warp L R

47 Managing Surfaces App VR Compositor App Render L L R R Submit VR Compositor Warp L R Present

48 Managing Surfaces App VR Compositor IOSurfaces App Render L L R R Submit VR Compositor Warp L R Present

49 // Creating Metal Textures MTLTextureDescriptor *texturedesc = [MTLTextureDescriptor new]; texturedesc.width = vrwidth; texturedesc.height = vrheight; texturedesc.pixelformat = MTLPixelFormatRGBA8Unorm_sRGB; texturedesc.storagemode = MTLStorageModeManaged; texturedesc.usage = MTLTextureUsageRenderTarget MTLTextureUsageShaderRead; id <MTLTexture> right_tex = [device newtexturewithdescriptor:texturedesc iosurface:rightiosurface plane:0]; id <MTLTexture> left_tex = [device newtexturewithdescriptor:texturedesc iosurface:leftiosurface plane:0];

50 // Creating Metal Textures MTLTextureDescriptor *texturedesc = [MTLTextureDescriptor new]; texturedesc.width = vrwidth; texturedesc.height = vrheight; texturedesc.pixelformat = MTLPixelFormatRGBA8Unorm_sRGB; texturedesc.storagemode = MTLStorageModeManaged; texturedesc.usage = MTLTextureUsageRenderTarget MTLTextureUsageShaderRead; id <MTLTexture> right_tex = [device newtexturewithdescriptor:texturedesc iosurface:rightiosurface plane:0]; id <MTLTexture> left_tex = [device newtexturewithdescriptor:texturedesc iosurface:leftiosurface plane:0];

51 // Creating Metal Textures MTLTextureDescriptor *texturedesc = [MTLTextureDescriptor new]; texturedesc.width = vrwidth; texturedesc.height = vrheight; texturedesc.pixelformat = MTLPixelFormatRGBA8Unorm_sRGB; texturedesc.storagemode = MTLStorageModeManaged; texturedesc.usage = MTLTextureUsageRenderTarget MTLTextureUsageShaderRead; id <MTLTexture> right_tex = [device newtexturewithdescriptor:texturedesc iosurface:rightiosurface plane:0]; id <MTLTexture> left_tex = [device newtexturewithdescriptor:texturedesc iosurface:leftiosurface plane:0];

52 // Creating Metal Textures MTLTextureDescriptor *texturedesc = [MTLTextureDescriptor new]; texturedesc.width = vrwidth; texturedesc.height = vrheight; texturedesc.pixelformat = MTLPixelFormatRGBA8Unorm_sRGB; texturedesc.storagemode = MTLStorageModeManaged; texturedesc.usage = MTLTextureUsageRenderTarget MTLTextureUsageShaderRead; id <MTLTexture> right_tex = [device newtexturewithdescriptor:texturedesc iosurface:rightiosurface plane:0]; id <MTLTexture> left_tex = [device newtexturewithdescriptor:texturedesc iosurface:leftiosurface plane:0];

53 VR App Building 101 Overview of VR development macos platform specifics Anatomy of a VR frame VR best practices

54 VR Compositor Synchronization VR compositor and App work in lock step Results of App rendering submitted to VR compositor GPU is a shared resource Submission order matters

55 VR Compositor Synchronization Start of frame VBL VBL App VR Compositor CPU Metal GPU 11.1 ms

56 VR Compositor Synchronization Start of frame WaitGetPoses() VBL VBL App VR Compositor CPU Metal GPU 11.1 ms

57 VR Compositor Synchronization Start of frame WaitGetPoses() VBL VBL App Encode VR Compositor CPU Metal GPU 11.1 ms

58 VR Compositor Synchronization Submit to VR compositor VBL VBL App Encode VR Compositor CPU Metal GPU 11.1 ms

59 VR Compositor Synchronization Submit to VR compositor VBL VBL App Encode Commit VR Compositor CPU Metal GPU App Render 11.1 ms

60 VR Compositor Synchronization Submit to VR compositor VBL VBL App Encode Commit Submit x2 VR Compositor Encode CPU Metal GPU App Render 11.1 ms

61 VR Compositor Synchronization Guaranteeing submission order VBL VBL App Encode/ VR Compositor Encode CPU Metal GPU App Render 11.1 ms

62 VR Compositor Synchronization Guaranteeing submission order VBL VBL App Encode/ Scheduled VR Compositor Encode CPU Metal GPU App Render 11.1 ms

63 VR Compositor Synchronization Guaranteeing submission order VBL VBL App Encode/ Scheduled PostPresentHandoff() VR Compositor Encode CPU Metal GPU App Render 11.1 ms

64 VR Compositor Synchronization Guaranteeing submission order VBL VBL App Encode/ Scheduled PostPresentHandoff() VR Compositor Encode Commit CPU Metal GPU App Render 11.1 ms

65 VR Compositor Synchronization Guaranteeing submission order VBL VBL App Encode/ Scheduled PostPresentHandoff() VR Compositor Encode Commit CPU Metal GPU App Render Warp 11.1 ms

66 vr.vrcompositor()->waitgetposes(vrtrackeddeviceposes, vr::k_unmaxtrackeddevicecount, nullptr, 0); id<mtlcommandbuffer> commandbuffer = [scenecommandqueue commandbuffer]; // render left and right eye images into eyetextures [self _drawscenewithcommandbuffer:commandbuffer]; [commandbuffer commit]; vr.vrcompositor()->submit(vr::eye_left, &eyetextures[0], &vreyebounds[0]); vr.vrcompositor()->submit(vr::eye_right, &eyetextures[1], &vreyebounds[1]); // wait until the GPU work is scheduled [commandbuffer waituntilscheduled]; // signal to the compositor that it can submit work to the GPU vr.vrcompositor()->postpresenthandoff();

67 vr.vrcompositor()->waitgetposes(vrtrackeddeviceposes, vr::k_unmaxtrackeddevicecount, nullptr, 0); id<mtlcommandbuffer> commandbuffer = [scenecommandqueue commandbuffer]; // render left and right eye images into eyetextures [self _drawscenewithcommandbuffer:commandbuffer]; [commandbuffer commit]; vr.vrcompositor()->submit(vr::eye_left, &eyetextures[0], &vreyebounds[0]); vr.vrcompositor()->submit(vr::eye_right, &eyetextures[1], &vreyebounds[1]); // wait until the GPU work is scheduled [commandbuffer waituntilscheduled]; // signal to the compositor that it can submit work to the GPU vr.vrcompositor()->postpresenthandoff();

68 vr.vrcompositor()->waitgetposes(vrtrackeddeviceposes, vr::k_unmaxtrackeddevicecount, nullptr, 0); id<mtlcommandbuffer> commandbuffer = [scenecommandqueue commandbuffer]; // render left and right eye images into eyetextures [self _drawscenewithcommandbuffer:commandbuffer]; [commandbuffer commit]; vr.vrcompositor()->submit(vr::eye_left, &eyetextures[0], &vreyebounds[0]); vr.vrcompositor()->submit(vr::eye_right, &eyetextures[1], &vreyebounds[1]); // wait until the GPU work is scheduled [commandbuffer waituntilscheduled]; // signal to the compositor that it can submit work to the GPU vr.vrcompositor()->postpresenthandoff();

69 vr.vrcompositor()->waitgetposes(vrtrackeddeviceposes, vr::k_unmaxtrackeddevicecount, nullptr, 0); id<mtlcommandbuffer> commandbuffer = [scenecommandqueue commandbuffer]; // render left and right eye images into eyetextures [self _drawscenewithcommandbuffer:commandbuffer]; [commandbuffer commit]; vr.vrcompositor()->submit(vr::eye_left, &eyetextures[0], &vreyebounds[0]); vr.vrcompositor()->submit(vr::eye_right, &eyetextures[1], &vreyebounds[1]); // wait until the GPU work is scheduled [commandbuffer waituntilscheduled]; // signal to the compositor that it can submit work to the GPU vr.vrcompositor()->postpresenthandoff();

70 vr.vrcompositor()->waitgetposes(vrtrackeddeviceposes, vr::k_unmaxtrackeddevicecount, nullptr, 0); id<mtlcommandbuffer> commandbuffer = [scenecommandqueue commandbuffer]; // render left and right eye images into eyetextures [self _drawscenewithcommandbuffer:commandbuffer]; [commandbuffer commit]; vr.vrcompositor()->submit(vr::eye_left, &eyetextures[0], &vreyebounds[0]); vr.vrcompositor()->submit(vr::eye_right, &eyetextures[1], &vreyebounds[1]); // wait until the GPU work is scheduled [commandbuffer waituntilscheduled]; // signal to the compositor that it can submit work to the GPU vr.vrcompositor()->postpresenthandoff();

71 vr.vrcompositor()->waitgetposes(vrtrackeddeviceposes, vr::k_unmaxtrackeddevicecount, nullptr, 0); id<mtlcommandbuffer> commandbuffer = [scenecommandqueue commandbuffer]; // render left and right eye images into eyetextures [self _drawscenewithcommandbuffer:commandbuffer]; [commandbuffer commit]; vr.vrcompositor()->submit(vr::eye_left, &eyetextures[0], &vreyebounds[0]); vr.vrcompositor()->submit(vr::eye_right, &eyetextures[1], &vreyebounds[1]); // wait until the GPU work is scheduled [commandbuffer waituntilscheduled]; // signal to the compositor that it can submit work to the GPU vr.vrcompositor()->postpresenthandoff();

72 vr.vrcompositor()->waitgetposes(vrtrackeddeviceposes, vr::k_unmaxtrackeddevicecount, nullptr, 0); id<mtlcommandbuffer> commandbuffer = [scenecommandqueue commandbuffer]; // render left and right eye images into eyetextures [self _drawscenewithcommandbuffer:commandbuffer]; [commandbuffer commit]; vr.vrcompositor()->submit(vr::eye_left, &eyetextures[0], &vreyebounds[0]); vr.vrcompositor()->submit(vr::eye_right, &eyetextures[1], &vreyebounds[1]); // wait until the GPU work is scheduled [commandbuffer waituntilscheduled]; // signal to the compositor that it can submit work to the GPU vr.vrcompositor()->postpresenthandoff();

73 VR Compositor Synchronization Frame cadence WGP() VBL WGP() VBL App Encode Encode VR Compositor Encode CPU Metal GPU App Render Warp 11.1 ms

74 VR Compositor Synchronization Frame cadence WGP() VBL WGP() VBL App Encode Encode VR Compositor Encode CPU Metal GPU App Render Warp 11.1 ms 11.1 ms

75 VR App Building 101 Overview of VR development macos platform specifics Anatomy of a VR frame VR best practices

76 Start Early VBL VBL CPU App Vc GPU Idle App Vc No Running Start

77 Start Early CPU work at frame start can introduce GPU bubbles VBL VBL CPU App Vc GPU Idle App Vc No Running Start

78 Start Early CPU work at frame start can introduce GPU bubbles SteamVR provides a mechanism to start early VBL VBL VBL VBL CPU App Vc App0 Vc App1 Vc GPU Idle App Vc App0 Vc App1 No Running Start With Running Start

79 Start Early CPU work at frame start can introduce GPU bubbles SteamVR provides a mechanism to start early Encode your frame after WaitGetPoses returns VBL VBL WGP VBL WGP VBL CPU App Vc App0 Vc App1 Vc GPU Idle App Vc App0 Vc App1 No Running Start With Running Start

80 Split Your Command Buffers VBL VBL CPU App Vc GPU Idle App Vc Single CB

81 Split Your Command Buffers Avoid submitting monolithic command buffers VBL VBL CPU App Vc GPU Idle App Vc Single CB

82 Split Your Command Buffers Avoid submitting monolithic command buffers Split and submit as you go to maximize GPU utilization VBL VBL VBL VBL CPU App Vc Vc GPU Idle App Vc Vc Single CB Split CB

83 Coalesce Left and Right Eye Draws

84 Coalesce Left and Right Eye Draws NEW Use the Metal 2 Viewport Array feature Per-primitive viewport selection in the vertex shader

85 Coalesce Left and Right Eye Draws NEW Use the Metal 2 Viewport Array feature Per-primitive viewport selection in the vertex shader Render to left and right eye with a single draw call

86 // Each viewport will cover half the texture size MTLViewport vrviewports[2]; vrviewports[0].originx = 0; vrviewports[0].originy = 0; vrviewports[0].width = width; vrviewports[0].height = height; vrviewports[0].znear = 0.0; vrviewports[0].zfar = 1.0; Height vrviewports[1].originx = width; vrviewports[1].originy = 0; vrviewports[1].width = width; vrviewports[1].height = height; vrviewports[1].znear = 0.0; vrviewports[1].zfar = 1.0; 2 x Width

87 // Each viewport will cover half the texture size MTLViewport vrviewports[2]; vrviewports[0].originx = 0; vrviewports[0].originy = 0; vrviewports[0].width = width; vrviewports[0].height = height; vrviewports[0].znear = 0.0; vrviewports[0].zfar = 1.0; Height (0, 0) vrviewports[1].originx = width; vrviewports[1].originy = 0; vrviewports[1].width = width; vrviewports[1].height = height; vrviewports[1].znear = 0.0; vrviewports[1].zfar = 1.0; (width, height) 2 x Width

88 // Each viewport will cover half the texture size MTLViewport vrviewports[2]; vrviewports[0].originx = 0; vrviewports[0].originy = 0; vrviewports[0].width = width; vrviewports[0].height = height; vrviewports[0].znear = 0.0; vrviewports[0].zfar = 1.0; Height (width, 0) vrviewports[1].originx = width; vrviewports[1].originy = 0; vrviewports[1].width = width; vrviewports[1].height = height; vrviewports[1].znear = 0.0; vrviewports[1].zfar = 1.0; 2 x Width (width, height)

89 // Set the viewports on your render command encoder for that render pass [renderencoder setviewports:vrviewports count:2]; // The instance_id will be used as the "eye" index [renderencoder drawindexedprimitives:primitivetype indexcount:count indextype:type indexbuffer:buffer indexbufferoffset:offset instancecount:2];

90 // Set the viewports on your render command encoder for that render pass [renderencoder setviewports:vrviewports count:2]; // The instance_id will be used as the "eye" index [renderencoder drawindexedprimitives:primitivetype indexcount:count indextype:type indexbuffer:buffer indexbufferoffset:offset instancecount:2];

91 // Set the viewports on your render command encoder for that render pass [renderencoder setviewports:vrviewports count:2]; // The instance_id will be used as the "eye" index [renderencoder drawindexedprimitives:primitivetype indexcount:count indextype:type indexbuffer:buffer indexbufferoffset:offset instancecount:2];

92 typedef struct { float4 position [[ position ]]; ushort viewport [[ viewport_array_index ]]; } ColorInOut; // Vertex Shader treats instance_id as an eye index vertex ColorInOut VS(Vertex in [[ stage_in ]], constant AAPLUniforms& uniforms [[ buffer(kbufferindexuniforms) ]], ushort iid [[ instance_id ]]) { ColorInOut out; out.position = uniforms.modelviewprojectionmatrix[iid] * float4(in.position, 1.0); out.viewport = iid; return out; }

93 typedef struct { float4 position [[ position ]]; ushort viewport [[ viewport_array_index ]]; } ColorInOut; // Vertex Shader treats instance_id as an eye index vertex ColorInOut VS(Vertex in [[ stage_in ]], constant AAPLUniforms& uniforms [[ buffer(kbufferindexuniforms) ]], ushort iid [[ instance_id ]]) { ColorInOut out; out.position = uniforms.modelviewprojectionmatrix[iid] * float4(in.position, 1.0); out.viewport = iid; return out; }

94 typedef struct { float4 position [[ position ]]; ushort viewport [[ viewport_array_index ]]; } ColorInOut; // Vertex Shader treats instance_id as an eye index vertex ColorInOut VS(Vertex in [[ stage_in ]], constant AAPLUniforms& uniforms [[ buffer(kbufferindexuniforms) ]], ushort iid [[ instance_id ]]) { ColorInOut out; out.position = uniforms.modelviewprojectionmatrix[iid] * float4(in.position, 1.0); out.viewport = iid; return out; }

95 typedef struct { float4 position [[ position ]]; ushort viewport [[ viewport_array_index ]]; } ColorInOut; // Vertex Shader treats instance_id as an eye index vertex ColorInOut VS(Vertex in [[ stage_in ]], constant AAPLUniforms& uniforms [[ buffer(kbufferindexuniforms) ]], ushort iid [[ instance_id ]]) { ColorInOut out; out.position = uniforms.modelviewprojectionmatrix[iid] * float4(in.position, 1.0); out.viewport = iid; return out; }

96 typedef struct { float4 position [[ position ]]; ushort viewport [[ viewport_array_index ]]; } ColorInOut; // Vertex Shader treats instance_id as an eye index vertex ColorInOut VS(Vertex in [[ stage_in ]], constant AAPLUniforms& uniforms [[ buffer(kbufferindexuniforms) ]], ushort iid [[ instance_id ]]) { ColorInOut out; out.position = uniforms.modelviewprojectionmatrix[iid] * float4(in.position, 1.0); out.viewport = iid; return out; }

97 Render Fewer Pixels

98 Render Fewer Pixels 15% of rendered pixels not displayed

99 Render Fewer Pixels 15% of rendered pixels not displayed Use SteamVR stencil mask to clip these pixels

100 VR App Building 101 Overview of VR development macos platform specifics Anatomy of a VR frame VR best practices

101 Nat Brown, Valve Software VR on macos

102 VR Motivation VR is a long-term investment for Valve room scale + input = magical Valve licenses VR tech non-exclusively

103 SteamVR Architecture VR application OpenVR APIs vrclient shared memory state and shared textures + high-frequency poses and input events vrmonitor vrserver vrcompositor vrdashboard device drivers Metal HMD and Controllers

104 SteamVR Architecture VR application OpenVR APIs vrclient shared memory state and shared textures + high-frequency poses and input events vrmonitor vrserver vrcompositor vrdashboard device drivers Metal HMD and Controllers

105 SteamVR Architecture VR application OpenVR APIs vrclient shared memory state and shared textures + high-frequency poses and input events vrmonitor vrserver vrcompositor vrdashboard device drivers Metal HMD and Controllers

106 SteamVR Architecture VR application OpenVR APIs vrclient shared memory state and shared textures + high-frequency poses and input events vrmonitor vrserver vrcompositor vrdashboard device drivers Metal HMD and Controllers

107 SteamVR Architecture VR application OpenVR APIs vrclient shared memory state and shared textures + high-frequency poses and input events vrmonitor vrserver vrcompositor vrdashboard device drivers Metal HMD and Controllers

108 SteamVR Architecture VR application OpenVR APIs vrclient shared memory state and shared textures + high-frequency poses and input events vrmonitor vrserver vrcompositor vrdashboard device drivers Metal HMD and Controllers

109 SteamVR on macos Closer engagement with Apple started Summer 2016 Bringing the compositor to Metal only took a few weeks Adopted the Metal 2 Direct to Display APIs

110 SteamVR on macos 90Hz ms VBL VBL VBL VBL WaitGetPoses() returns Running Start CPU Simulation / Prep 5 Simulation / Prep 6 Simulation / Prep 7 Submit 4 Submit 5 Submit 6 Submit 7 GPU Render 4 Render 5 Render 6 Render vrcompositor 5 6 Panel photons photons photons photons Rendering Prediction to Photons returned by WaitGetPoses() Simulation Prediction to Photons ~36 ms returned by GetDeviceToAbsoluteTrackingPose()

111 Where to get SteamVR Install Steam and create a free account Under Library / Tools, install SteamVR Opt into the SteamVR Beta Right-Click SteamVR, choose Properties Choose beta - SteamVR Beta Update Download OpenVR headers & framework

112 Using OpenVR in Your App Include OpenVR.framework inside your app bundle Conveys version to the SteamVR runtime Add OpenVR.Framework to Embedded Binaries under General settings Xcode will automatically set the Runtime Search Path appropriately for your bundle

113 Feedback and Support

114 Nat Brown, Valve Software

115 External GPU Support

116 Background External chassis with a desktop class GPU Connected to host via Thunderbolt

117 Goals Enable VR development

118 Goals Enable VR development Performance improvement in other GPU bound cases

119 External Graphics Developer Kit Sonnet 350W external GPU Chassis AMD Radeon RX 580 GPU Optimized for Thunderbolt 3 capable Macs Available for purchase today

120 // Identifying the External GPU id<mtldevice> externalgpu = nil; NSArray<id<MTLDevice>> * availabledevices = MTLCopyAllDevices(); for (id <MTLDevice> device in availabledevices) { if (device.removable) { externalgpu = device; return; } }

121 // Identifying the External GPU id<mtldevice> externalgpu = nil; NSArray<id<MTLDevice>> * availabledevices = MTLCopyAllDevices(); for (id <MTLDevice> device in availabledevices) { if (device.removable) { externalgpu = device; return; } }

122 // Identifying the External GPU id<mtldevice> externalgpu = nil; NSArray<id<MTLDevice>> * availabledevices = MTLCopyAllDevices(); for (id <MTLDevice> device in availabledevices) { if (device.removable) { externalgpu = device; return; } }

123 Thunderbolt 3 Bandwidth Relative Bandwidth Thunderbolt 2 1x 2x

124 Thunderbolt 3 Bandwidth Twice the bandwidth capability of Thunderbolt 2 Relative Bandwidth Thunderbolt 2 Thunderbolt 3 1x 2x

125 Thunderbolt 3 Bandwidth Twice the bandwidth capability of Thunderbolt 2 25% the bandwidth of PCIe x16 Relative Bandwidth Thunderbolt 2 Thunderbolt 3 PCIe x16 1x 2x 3x 4x 5x 6x 7x 8x

126 GPU Power vs. Bandwidth MacBook Pro System RAM Intel Chipset

127 GPU Power vs. Bandwidth MacBook Pro Internal GPU VRAM System RAM Intel Chipset

128 GPU Power vs. Bandwidth MacBook Pro Internal GPU VRAM System RAM Intel Chipset Thunderbolt 3 External GPU VRAM

129 GPU Power vs. Bandwidth Treat the GPU and link as a pair Optimal combination will depend on workload MacBook Pro Internal GPU VRAM System RAM Intel Chipset Thunderbolt 3 External GPU VRAM

130 Complex Display Topology

131 Complex Display Topology Displays connected to different GPUs

132 Complex Display Topology Displays connected to different GPUs Performance impact to render on one GPU and display on another

133 Complex Display Topology Displays connected to different GPUs Performance impact to render on one GPU and display on another Where your content is displayed matters

134 Golden Rule for GPU Selection Render on the same GPU your app displays on MacBook Pro Internal GPU VRAM Display System RAM Intel Chipset TBT3 External GPU VRAM Display

135 GPU Selection Decision Tree

136 GPU Selection Decision Tree Presenting to Display

137 GPU Selection Decision Tree Presenting to Display Yes Select GPU attached to Display

138 GPU Selection Decision Tree No (eg. Compute) Presenting to Display Yes Prefer low power Select GPU attached to Display

139 GPU Selection Decision Tree No (eg. Compute) Presenting to Display Yes Yes Prefer low power Select GPU attached to Display Select LowPower GPU

140 GPU Selection Decision Tree No (eg. Compute) Presenting to Display Yes Yes Prefer low power No Select GPU attached to Display Select LowPower GPU Select external GPU

141 GPU Selection Decision Tree No (eg. Compute) Presenting to Display Yes Yes Prefer low power No Select GPU attached to Display Select LowPower GPU Select external GPU

142 Metal Device from Display Use existing CoreGraphics API // Get the CGDirectDisplayID for the display your app window is on NSNumber* num = view.window.screen.devicedescription[@"nsscreennumber"]; CGDirectDisplayID viewdisplayid = [num unsignedintegervalue]; // query CG for the metal device id<mtldevice> newpreferreddevice = CGDirectDisplayCopyCurrentMetalDevice(viewDisplayID);

143 Metal Device from Display Use existing CoreGraphics API // Get the CGDirectDisplayID for the display your app window is on NSNumber* num = view.window.screen.devicedescription[@"nsscreennumber"]; CGDirectDisplayID viewdisplayid = [num unsignedintegervalue]; // query CG for the metal device id<mtldevice> newpreferreddevice = CGDirectDisplayCopyCurrentMetalDevice(viewDisplayID);

144 Metal Device from Display Use existing CoreGraphics API // Get the CGDirectDisplayID for the display your app window is on NSNumber* num = view.window.screen.devicedescription[@"nsscreennumber"]; CGDirectDisplayID viewdisplayid = [num unsignedintegervalue]; // query CG for the metal device id<mtldevice> newpreferreddevice = CGDirectDisplayCopyCurrentMetalDevice(viewDisplayID);

145 GPU Migration

146 GPU Migration Display migration may require GPU migration

147 GPU Migration Display migration may require GPU migration Register for NSWindowDidChangeScreenNotification Triggered when windows move across displays [[NSNotificationCenter defaultcenter] addobserver:self name:nswindowdidchangescreennotification object:nil];

148 // Get the Metal device for the GPU driving the display your app is now on NSNumber* num = view.window.screen.devicedescription[@"nsscreennumber"]; CGDirectDisplayID viewdisplayid = [num unsignedintegervalue]; id<mtldevice> newpreferreddevice = CGDirectDisplayCopyCurrentMetalDevice(viewDisplayID); // Early out if this display is being driven by the same GPU if (currentdevice == newpreferreddevice) return; // switch view to new device view.device = newpreferreddevice; // handle App migration to the new GPU // Call draw on this new device [MetalKitRenderer draw:view];

149 // Get the Metal device for the GPU driving the display your app is now on NSNumber* num = view.window.screen.devicedescription[@"nsscreennumber"]; CGDirectDisplayID viewdisplayid = [num unsignedintegervalue]; id<mtldevice> newpreferreddevice = CGDirectDisplayCopyCurrentMetalDevice(viewDisplayID); // Early out if this display is being driven by the same GPU if (currentdevice == newpreferreddevice) return; // switch view to new device view.device = newpreferreddevice; // handle App migration to the new GPU // Call draw on this new device [MetalKitRenderer draw:view];

150 // Get the Metal device for the GPU driving the display your app is now on NSNumber* num = view.window.screen.devicedescription[@"nsscreennumber"]; CGDirectDisplayID viewdisplayid = [num unsignedintegervalue]; id<mtldevice> newpreferreddevice = CGDirectDisplayCopyCurrentMetalDevice(viewDisplayID); // Early out if this display is being driven by the same GPU if (currentdevice == newpreferreddevice) return; // switch view to new device view.device = newpreferreddevice; // handle App migration to the new GPU // Call draw on this new device [MetalKitRenderer draw:view];

151 // Get the Metal device for the GPU driving the display your app is now on NSNumber* num = view.window.screen.devicedescription[@"nsscreennumber"]; CGDirectDisplayID viewdisplayid = [num unsignedintegervalue]; id<mtldevice> newpreferreddevice = CGDirectDisplayCopyCurrentMetalDevice(viewDisplayID); // Early out if this display is being driven by the same GPU if (currentdevice == newpreferreddevice) return; // switch view to new device view.device = newpreferreddevice; // handle App migration to the new GPU // Call draw on this new device [MetalKitRenderer draw:view];

152 Attach and Removal Notifications New with Metal 2 NEW MTLDeviceWasAddedNotification MTLDeviceWasRemovedNotification MTLDeviceRemovalRequestedNotification

153 NSArray< id<mtldevice> >* devices = nil; id<nsobject> metaldeviceobserver = nil; devices = MTLCopyAllDevicesWithObserver(&metalDeviceObserver, ^(id<mtldevice> device, MTLNotificationName name) { [self handlegpuhotplug:device notifier:name]; }); NEW

154 NSArray< id<mtldevice> >* devices = nil; id<nsobject> metaldeviceobserver = nil; devices = MTLCopyAllDevicesWithObserver(&metalDeviceObserver, ^(id<mtldevice> device, MTLNotificationName name) { [self handlegpuhotplug:device notifier:name]; }); NEW

155 NSArray< id<mtldevice> >* devices = nil; id<nsobject> metaldeviceobserver = nil; devices = MTLCopyAllDevicesWithObserver(&metalDeviceObserver, ^(id<mtldevice> device, MTLNotificationName name) { [self handlegpuhotplug:device notifier:name]; }); NEW

156 - (void)handlegpuhotplug:(id<mtldevice>)device notifier:(mtlnotificationname)notifier NEW { } if (notifier == MTLDeviceWasAddedNotification) // Device plugged in else if (notifier == MTLDeviceRemovalRequestedNotification) // Device Removal Requested. Cleanup and switch to preferred device else if (notifier == MTLDeviceWasRemovedNotification) // additional handling of surprise removal

157 - (void)handlegpuhotplug:(id<mtldevice>)device notifier:(mtlnotificationname)notifier NEW { } if (notifier == MTLDeviceWasAddedNotification) // Device plugged in else if (notifier == MTLDeviceRemovalRequestedNotification) // Device Removal Requested. Cleanup and switch to preferred device else if (notifier == MTLDeviceWasRemovedNotification) // additional handling of surprise removal

158 - (void)handlegpuhotplug:(id<mtldevice>)device notifier:(mtlnotificationname)notifier NEW { } if (notifier == MTLDeviceWasAddedNotification) // Device plugged in else if (notifier == MTLDeviceRemovalRequestedNotification) // Device Removal Requested. Cleanup and switch to preferred device else if (notifier == MTLDeviceWasRemovedNotification) // additional handling of surprise removal

159 Unexpected GPU Removal

160 Unexpected GPU Removal Metal API will return errors on GPU removal Survive until a GPU migration notification Regenerate data in GPU local memory

161 Best Practices

162 Best Practices Retain devices after migration

163 Best Practices Retain devices after migration Manage GPU migration for each window

164 Best Practices Retain devices after migration Manage GPU migration for each window Avoid transferring data between GPUs

165 VR Best Practices

166 VR Best Practices Attach the VR headset to the external GPU

167 VR Best Practices Attach the VR headset to the external GPU Present to a display driven by the external GPU

168 VR Best Practices Attach the VR headset to the external GPU Present to a display driven by the external GPU Cache resources on the external GPU

169 Summary VR development enabled with Metal 2 Support for HTC Vive and SteamVR VR enabled game engines or build a native VR app External GPU support

170 Related Sessions Introducing Metal 2 Executive Ballroom Tuesday 1:50PM Metal 2 Optimization and Debugging Executive Ballroom Thursday 3:15PM Using Metal 2 for Compute Grand Ballroom A Thursday 4:10PM

171 Metal Labs Metal 2 Lab Technology Lab A Tues 3:10PM 6:00PM VR with Metal 2 Lab Technology Lab A Wed 3:10PM 6:00PM Metal 2 Lab Technology Lab F Fri 9:00AM 12:00PM

172