Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Mark Kilgard, July 27, 2016
Virtual Reality Features of
NVIDIA GPUs
2
Mark Kilgard
Principal System Software Engineer
OpenGL driver implementation & API evolution
Other interests
Programmabl...
3
GPU Challenges for Virtual Reality Rendering
1. Must render unique view for each eye
Two scene renders per frame
2. Wide...
4
Magnification Effect of HMD Lens
Graph paper viewed & magnified through HMD lens
5
Head Mounted Display Lens Effect
Barrel Distorted Images
on HMD Screen
Magnified Overlapping Wide Field-of-view
Binocula...
6
Focus on Virtual Reality Features of Pascal GPUs
Virtual Reality rendering features in Pascal GPUs
GeForce 1080/1070/106...
7
Background: NVIDA GPU Architecture Road Map
Our interest NVIDIA GPU architectures of interest: Pascal
What is Pascal men...
8
Conventional 3D Single-eye Scene
9
Conventional 3D Single-eye Scene
10
Wide Stereo Views of Same Scene
Left Right
11
Left Right
12
Right Left
13
Left & Right Views Are
Similar but Different
Left and Right eye views have different occlusion
While left & right eye v...
14
Image Difference of Two Views
− + 0.5 =
Left eye view Right eye view
Clamped difference image
15
Left & Right Views Rendered Efficiently in One Pass
16
Lens Matched Shading
Left Right
17
Visualizing Lens Matched Shading
18
Lens Matched Shading
Left Right
19
Visualization of Lens Matched Shading Rendering
20
Still Needs Non-linear Warp
Lens Matched Shading
rendering
Warped image suitable
for HMD display
Non-linear
image
21
Still Needs Non-linear Warp
Lens Matched Shading
rendering
Warped image suitable
for HMD display
Non-linear
image
BUT s...
22
Example HMD Post-rendering Warp
23
Lens Matched Shading
with Window Rectangle Testing
24
Lens Matched Shading
with Window Rectangle Testing
Nothing in black corners is shaded or even rasterized
Yellow lines s...
25
Lens Matched Shading
with Window Rectangle Testing
26
What Didn’t Have to be Rendered
17% of left view pixels 17% of right view pixels
27
Window Rectangle Testing
During Rasterization
New multi-vendor OpenGL extension
EXT_window_rectangles
Provides a set of...
28
Warped Lens Matched Scene
Warped version of Lens Matched Shading to match HMD lens
29
Warped Lens Matched Shading
30
Visualizing Warp Window Rectangles
31
What Didn’t Have to be Warped
17% of left view pixels 17% of right view pixels
32
VR Rendering Pipeline in Images
LMS Right Eye View Warped Right Eye View
LMS Left Eye View Warped Left Eye View
Scene
L...
33
VR Rendering Pipeline in Images
LMS Right Eye View Warped Right Eye View
LMS Left Eye View Warped Left Eye View
Scene
D...
34
Traditional 3D Rendering Pipeline
Transform
3D View
Downsample
Display
Frame
App
Processing
if multisampled
uses pinhol...
35
Virtual Reality Rendering Pipeline
Rendering Left & Right Eye Views
Transform
Left View
Downsample
Display
Frame to HMD...
36
Virtual Reality Rendering Pipeline
With Lens Matched Shading
Transform
Left View
Downsample
Display
Frame to HMD
App
Pr...
37
Virtual Reality Rendering Pipeline
With Single Pass Stereo
Transform
Left & Right
View
Downsample
Display
Frame to HMD
...
38
Virtual Reality Rendering Pipeline
Single Pass Stereo + Lens Matched Shading
Downsample
Display
Frame to HMD
Rasterize
...
39
Pascal GPU Features for VR
Unique to NVIDIA
Pascal renders left & right eye
In single rendering pass
Halves CPU load fo...
40
Other NVIDIA Virtual Reality Initiatives
41
More Information
VRWORKS 2.0 Software Development Kit
Available NOW
Pascal OpenGL extensions for Virtual Reality
Specif...
Virtual Reality Features of NVIDIA GPUs
Upcoming SlideShare
Loading in …5
×

Virtual Reality Features of NVIDIA GPUs

570 views

Published on

Presented by Mark Kilgard on July 27, 2016 at NVIDIA's Booth at SIGGRAPH.in Anaheim

Published in: Technology
  • Be the first to comment

Virtual Reality Features of NVIDIA GPUs

  1. 1. Mark Kilgard, July 27, 2016 Virtual Reality Features of NVIDIA GPUs
  2. 2. 2 Mark Kilgard Principal System Software Engineer OpenGL driver implementation & API evolution Other interests Programmable shading languages, original Cg GPU shading language Web, vector graphics & path rendering, GPU rendering for 2D graphics Also known for creating OpenGL Utility Toolkit (GLUT) My Background
  3. 3. 3 GPU Challenges for Virtual Reality Rendering 1. Must render unique view for each eye Two scene renders per frame 2. Wide field-of-view rendered More pixels, distributed towards periphery 3. HMD lens magnifies rendered scene Non-linear inverse lens warp Allows the HMD lens magnification to provide convincing sense of presence
  4. 4. 4 Magnification Effect of HMD Lens Graph paper viewed & magnified through HMD lens
  5. 5. 5 Head Mounted Display Lens Effect Barrel Distorted Images on HMD Screen Magnified Overlapping Wide Field-of-view Binocular View
  6. 6. 6 Focus on Virtual Reality Features of Pascal GPUs Virtual Reality rendering features in Pascal GPUs GeForce 1080/1070/1060 NEW Quadro P6000 & P5000 As exposed in standard 3D APIs
  7. 7. 7 Background: NVIDA GPU Architecture Road Map Our interest NVIDIA GPU architectures of interest: Pascal What is Pascal mentioned on last slide?
  8. 8. 8 Conventional 3D Single-eye Scene
  9. 9. 9 Conventional 3D Single-eye Scene
  10. 10. 10 Wide Stereo Views of Same Scene Left Right
  11. 11. 11 Left Right
  12. 12. 12 Right Left
  13. 13. 13 Left & Right Views Are Similar but Different Left and Right eye views have different occlusion While left & right eye views look similar, must be draw independently Really only different in a small translation in X NVIDIA Single Pass Stereo supports generating left & right eye positions GPU automatically makes left & right eye versions of each primitive Then renders each view to a different framebuffer slice Exposed in OpenGL by NV_stereo_view_rendering extension Exposed for Direct3D 11 by NVAPI SinglePassStereoMode feature
  14. 14. 14 Image Difference of Two Views − + 0.5 = Left eye view Right eye view Clamped difference image
  15. 15. 15 Left & Right Views Rendered Efficiently in One Pass
  16. 16. 16 Lens Matched Shading Left Right
  17. 17. 17 Visualizing Lens Matched Shading
  18. 18. 18 Lens Matched Shading Left Right
  19. 19. 19 Visualization of Lens Matched Shading Rendering
  20. 20. 20 Still Needs Non-linear Warp Lens Matched Shading rendering Warped image suitable for HMD display Non-linear image
  21. 21. 21 Still Needs Non-linear Warp Lens Matched Shading rendering Warped image suitable for HMD display Non-linear image BUT substantially less pixel motion and distortion than conventional VR re-warping
  22. 22. 22 Example HMD Post-rendering Warp
  23. 23. 23 Lens Matched Shading with Window Rectangle Testing
  24. 24. 24 Lens Matched Shading with Window Rectangle Testing Nothing in black corners is shaded or even rasterized Yellow lines show overlaid 8 inclusive window rectangles Same 8 window rectangles “shared” by each view’s texture array layer
  25. 25. 25 Lens Matched Shading with Window Rectangle Testing
  26. 26. 26 What Didn’t Have to be Rendered 17% of left view pixels 17% of right view pixels
  27. 27. 27 Window Rectangle Testing During Rasterization New multi-vendor OpenGL extension EXT_window_rectangles Provides a set of window-space rectangles, 8 for NVIDIA GPUs Rasterizer either includes or excludes rasterization from rectangles Discards at speed-of-light As if free to rasterize to irregular rectangular regions Window Rectangles Test NEW stage OpenGL’s fragment processing pipeline
  28. 28. 28 Warped Lens Matched Scene Warped version of Lens Matched Shading to match HMD lens
  29. 29. 29 Warped Lens Matched Shading
  30. 30. 30 Visualizing Warp Window Rectangles
  31. 31. 31 What Didn’t Have to be Warped 17% of left view pixels 17% of right view pixels
  32. 32. 32 VR Rendering Pipeline in Images LMS Right Eye View Warped Right Eye View LMS Left Eye View Warped Left Eye View Scene LMS = Lens Matched Shading
  33. 33. 33 VR Rendering Pipeline in Images LMS Right Eye View Warped Right Eye View LMS Left Eye View Warped Left Eye View Scene Displayed within HMD Single Rendering Pass Single Pass Stereo + Lens Matched Shading + Window Rectangle Testing Drawn with Single Triangle Fragment Shader Warping Window Rectangle Testing Perception to user is linear rendering HMD lens “undoes” warping to provide a perceived wide field-of-view Pascal does all this efficiently in a single rendering pass! 8 viewports, 1 pass
  34. 34. 34 Traditional 3D Rendering Pipeline Transform 3D View Downsample Display Frame App Processing if multisampled uses pinhole camera projection, rasterizing to uniform rectilinear pixel grid monitor directly displays uniform rectilinear pixel grid Rasterize 3D View
  35. 35. 35 Virtual Reality Rendering Pipeline Rendering Left & Right Eye Views Transform Left View Downsample Display Frame to HMD App Processing Rasterize Left View Transform Right View Downsample Rasterize Right View HMD Warp HMD Warp warps uniform rectilinear grid based on HMD’s device profile Transform Right View Rasterize Right View Transform Right View Downsample Rasterize Right View Transform Right View HMD Warp Downsample Rasterize Right View Transform Right View second view rendering
  36. 36. 36 Virtual Reality Rendering Pipeline With Lens Matched Shading Transform Left View Downsample Display Frame to HMD App Processing Rasterize Left View Transform Right View Downsample Rasterize Right View HMD Warp HMD Warp Lens Matched Shading Lens Matched Shading replicates primitive to 4 projected quadrants less pixel resampling effort & error operating on LMS rasterized images
  37. 37. 37 Virtual Reality Rendering Pipeline With Single Pass Stereo Transform Left & Right View Downsample Display Frame to HMD App Processing Rasterize Left View Downsample Rasterize Right View HMD Warp HMD Warp one transform instance, outputs left & right position
  38. 38. 38 Virtual Reality Rendering Pipeline Single Pass Stereo + Lens Matched Shading Downsample Display Frame to HMD Rasterize Left View Downsample Rasterize Right View HMD Warp HMD Warp Lens Matched Shading Lens Matched Shading Transform Left & Right View App Processing + Advantages 1. Single transform for both views 2. Closer match between rendering & lens images 3. Fewer overall pixels for similar quality use window rectangle testing to bound lens region, during both rasterization & warp
  39. 39. 39 Pascal GPU Features for VR Unique to NVIDIA Pascal renders left & right eye In single rendering pass Halves CPU load for rendering Better concurrent texture locality OpenGL: NV_stereo_view_rendering Direct3D: NVAPI SinglePassStereoMode Better matches rendering to HMD lens Single pass Four separately projected quadrants Less rendering, less resampling error OpenGL: NV_clip_space_w_scaling Direct3D: NVAPI ModifiedWSupport
  40. 40. 40 Other NVIDIA Virtual Reality Initiatives
  41. 41. 41 More Information VRWORKS 2.0 Software Development Kit Available NOW Pascal OpenGL extensions for Virtual Reality Specification now public In official OpenGL.org extension registry OpenGL Extension Wrangler 2.0 (GLEW) support Example code in VRWORKS 2.0 SDK VRWORKS also documents NVAPI usage For Direct3D 11 Includes Examples code

×