Use Variable Rate Shading (VRS) to Improve the User Experience in Real-Time Game Engines | SIGGRAPH 2019 Technical Sessions

1. Variable Rate Shading Adam Lake, Filip Strugar, Kelly Gawne, Trapper Mcferron

2. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at www.intel.com Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. Optimization Notice: Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. You may not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Intel products described herein. You agree to grant Intel a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject matter disclosed herein. Intel, Core and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others © Intel Corporation. Legal Notices And Disclaimers

3. Agenda 3 • What is Variable Rate Shading (VRS)? • Breaking News for VRS • Case Studies in the lab • Objects in Motion • Particle Systems • Amazon Bistro Scene • Amazon Bistro + Depth of Field • Unreal Engine Integration • Questions

4. VRS Overview • Display resolution growing faster than pixel throughput • 1080p->4k->8k on the way • HDR formats (8bit->10bit->fp16 formats) • Intel Gen11 display controller supports fp16 render targets, no tone mapping req’d • Optional shading rates supported on Intel • [2x4], [4x2], and [4x4] • Shade at high frequency (1x1) where it has highest impact 4 Pixel shader invoked Pixel receives calculated result Maximizing pixel shader throughput is even more important in the 10W-15W market Intel (Gen11) AMD Nvidia (Turing) VRS Support Tier 1 Not yet Tier 2+Tier 1 Additional Shading Rates Supported Yes Not yet Yes

5. VRS not a panacea! • Helps • Selectively shade fewer pixels to reduce compute and memory bandwidth • Reduction in render target write pressure relative to VRS rate • Doesn’t help • Vertex shading, geometry • Small triangles - penalty for thread scheduling, rasterization, fixed function overhead • Blending • Can make things worse • Visual blockiness artifacts if not careful 5 We discuss lessons learned to improve performance with VRS while not reducing the perceived visual quality

6. DX12 VRS API Easy to use API: Microsoft* Windows 10 May Update+Windows* SDK 1. Query for support of VRS • Tier 1: Set shading rate via command list RSSetShadingRate(..) • All subsequent draws use this shading rate • Tier 2: image based and in pre-PS stage via SV_ShadingRate • Combining shading rates handled with host side combiner API • Min/max/combine/passthrough 2. Query for optional shading rates: options6.AdditionalShadingRatesSupported 3. Set shading rate, ID3D12GraphicsCommandList5 call RSSetShadingRate() 4. Draw Objects 6 Gen11 is VRS Tier1 with additional shading rates supported

7. Recent VRS Results • Architecture Day ‘18 and Computex ‘19: Unreal* Engine + Ren’s Demo: ~30% • Microsoft, Intel and Firaxis* at GDC 2019 for VRS DX12 API • Firaxis: Civilization VI, Tier 1 ~20% and Tier 2 ~14% • Spec: VRS Spec • Blog: Variable Rate Shading: a scalpel in a world of sledgehammers • Microsoft: Martin Fuller, focus on Tier 2: • Deriving Screen Space Shading Rate, Implementation Tips and Sparse Lighting • Reduction in pixel shader invocations of ~50% • Nvidia*/Bethesda*: Wolfenstein II • Reduction of ‘bad’ frames, > 16.7ms for example • GDC ’19 talk on adaptive shading, Lei Yang et al • I3D paper on lossless content and motion adaptive shading, Lei Yang et al 7

9. Case Study: Objects under Motion SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST

10. Case Study – Objects under motion • Idea: Use VRS when objects are moving fast enough that the reduced shading rate is not noticeable • Increase FPS or quality via <favorite additional effect> 10

11. VRS 1x1 vs. 2x2 animated 11 • Indistinguishable in foreground • Slight degradation in background observed • Fewer texture samples across same amount of surface (in object space) • Geometry boundary preserved-full per pixel coverage even though shading rate reduced 1x1 Shading Rate 2x2 Shading Rate

12. VRS 2x2 vs. 4x4 animated 12 • Again, indistinguishable in foreground • Observable degradation in 4x4 background L • Atmospheric blur or fog can help, hides high frequency texture aliasing in background 4x4 Shading Rate2x2 Shading Rate

13. Let’s examine this up close… • Almost no observable degradation on 2x2, however 4x4 easy to see aliasing • All high frequency texture content significantly blurred, less noticeable under motion • Consider increasing roughness to help useful lifetime of coarser VRS rates • Edges perfectly preserved in all cases J 13

14. Fog and other atmospheric effects 14 Consider shading rate reduction for objects in the distance when atmospherics are being used, not in general

15. 3DMark* VRS Feature Test • LOD integration • Pre-release Gen11 20-40% speedup between VRS on/off • 3DMark VRS Feature Test Available for download Q3 2019 Used with Permission of 3DMark* and UL*

16. Informal user study • Constraints: • 1080p, ~3 ft away from 4k display, image not resized, shown 1080p back to back • 8 graphics engineers asked to compare displayed 1x1 vs. 2x2 and 2x2 vs. 4x4 16 Description Result Distinguish between 1x1 and 2x2 0% of 8 graphics engineers Distinguish between 2x2 and 4x4 100% of 8 graphics engineers 1x1 2x2 4x4

17. • Object moving closer to view plane cover more pixels in same unit of time as objects in background assuming world space size and velocity are constant • Objects undergoing motion blur in foreground will be more blurred than same object in background at same velocity • See Unreal use case for more details One more thing… 17 V1 V2

18. Takeaways • Visual artifacts most noticeable in high frequency detail • Examples: normal maps, specularity • In these cases, consider INCREASING shading frequency in background, not decreasing, unless we have fog, atmospherics, or blurs • GOOD cases: ambient occlusion, diffuse shading, albedo as these are slowly varying • Edges remain perfectly preserved because we have full pixel rate coverage J • If supporting Tier2, review Lei Yang’s work quantitative work on adaptive techniques, consider adoption into your engine 18

20. Case Study: Particles SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST

21. Why VRS in the lab? • We try out things so you don’t have to! • Two different workloads: Particles, Bistro scene with forward renderer • Performance under varying conditions (resolutions, shader complexities, etc.) • Quality considerations, integration ‘gotchas’ 21

22. VRS Particles • Simple particles POC, 10k quads, 50-200 alpha blends per pixel • Compare traditional low res offscreen vs VRS 22

23. VRS Particles • Can we do https://developer.nvidia.com/gpugems/GPUGems3/gpugems3_ch23.html (High- Speed, Off-Screen Particles) better & easier with VRS? • We try different resolutions, shader complexities, etc. low: ~20 medium: ~100 high: ~300 (shader complexity, dxbc instructions) 23

24. •VRS performance is excellent for more complex pixel shaders, ok even on simple •Bound on the output and changing resolution makes almost no difference! Performance of VRS Particles 24 no point in VRS4x4 (maybe 2x1/1x2?) perfect scaling even for VRS4x4 Measured on prototype ICL Gen 11 HW but very similar scaling observed on RTX 2060 (absolutenumbers) (relativenumbers)

25. Quality of VRS Particles • VRS blocks are render target aligned so alpha-blended stuff accumulates along the same boundaries L (we have some filtering ideas in the works!) • On the good side, no need for compositing, depth testing just works 25

26. Case Study: Amazon Bistro Scene

27. VRS w/ Amazon Bistro Scene • The opposite of particles POC: Amazon Lumberyard Bistro dataset (https://developer.nvidia.com/orca/amazon-lumberyard-bistro) • Around 3.7m triangles (mix of big & small), alpha tested foliage, some transparencies • Simple forward renderer, point lights with shadows • Different resolutions, and shader complexities • VRS applied on opaque & transparencies only • Flythrough benchmark 27

28. • VRS 2x1/1x2 and 2x2 bring most savings in a general case (but 4x2/2x4 & 4x4 are still good for specific cases: particles, high res.) *combined average of 2400 frames, various shading complexities, 3 resolutions (1920x1080, 2560x1600, 3840x2160), on ICL (Gen11) and RTX 2060 VRS Bistro perf vs. shade rate 28 30% 16% 7% 4%

29. • Higher resolutions make VRS more effective, especially at higher block sizes! (Balance is changed: same geometry but more shading, etc.) *combined average of 2400 frames, various shading complexities VRS Bistro Perf vs. Resolution 29 0.43 0.26 0.58 0.37

30. VRS Quality: Blockiness • Blockiness is bad… • … but less noticeable at high resolutions, partially obscured or in motion (exploring filtering options, watch this space) 30

31. Quantifying VRS Quality 31 VRS 2x4 at 4k better than VRS 2x2 at 1080p!* *will depend on the content, ymmv Idea: choose horizontal/vertical per object?

32. VRS Gotchas and Ideas • Careful with pixel size and location: • ddx/ddy scaled accordingly (VRS 2x2 means they are 2x – what we usually want) • SV_Position no longer n+0.5 • Can affect tiled/clustered lighting or anything relying on full screen textures: depth/color for soft particles, SSR, heat-haze/refraction effects. Sampling between texels can introduce artifacts and VRS might not help with perf when no MIPs. Create MIPs for each VRS level? • Best to make shaders as VRS-agnostic (pixel-size/location agnostic) as possible. SV_ShadingRate doesn’t work on Tier1 (or at and below SM_5.x); alternative: shadingRate = float2( ddx( pos.x ), ddy( pos.y ) ), with pos : SV_Position J • Automatic classification of materials for best performance gain / least visual impact? 32

33. VRS takeaways • Best case measured, VRS 4x4 on particles: 90+% reduction in draw time! • Roughly speaking, with forward renderer we get 30% savings with VRS 1x2/2x1 and 46% at VRS 2x2 rates (but YMMV, VRS exposes unseen bottlenecks: small triangles, fill-rate, memory b/w, etc.) • Blockiness is a quality problem. See CIV6 GDC 2019 talk, https://youtu.be/f- SklVb2MDI?t=2072 – custom filter helps. Active field of research. • The higher the resolution the more VRS helps perf* and the less it hurts quality *unless you’re at the blend/fillrate or framebuffer memory bandwidth wall 33

34. Case Study: Depth of Field SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST

35. 35 Depth of field: case study This whole area is blurry! So why shade in high detail? (these bits here too)

36. 36 Depth of field: depth of field areas near out of focus area near transition in focus area far out of focus area in focus to in focus from far transition

37. Depth of field: adding VRS • VRS Tier1 lets us only do per-draw call, focusing on that for now • We don’t want direct transition from 1x1 to 4x4, switch looks bad in motion • Use progressive VRS shading rate steps: • 1x1 (in focus) • 2x1 (transition) • 2x2 (transition) • 4x2 (transition) • 4x4 (completely out of focus) (pick suitable max rate – could be 4x2 or 2x2 or even 2x1)

38. 38 Depth of field: per-object VRS rate nearest distance to camera plane near out of focus area near transition in focus area far out of focus area in focus to in focus from far transition farthest distance to camera plane

39. 39 Depth of field: per-object VRS rate near out of focus area near transition in focus area far out of focus area in focus to in focus from far transition 4x4 2x2 1x1 2x1 4x2

40. Depth of field: choosing the vrs rate • Select rate conservatively to avoid affecting any in-focus pixels 1. For far out of focus use object’s nearest distance 2. For near out of focus use object’s farthest distance 3. Compute rates independently, combine conservatively • Same logic for Tier2 LOD mask: use macroblock’s nearest/farthest depth

41. Depth of field: first issue • Geometry not fine grained enough? • Solution (if possible): split up only the meshes that cover a lot of pixels

42. Depth of field: aliasing in motion • DoF-blurred areas with VRS look great on still images but bad in motion • Reason? In this case, mainly specular reflections! • Solution: dampen the offender based on VRS rate NO VRS VRS 4x2 VRS 4x2 + spec dampening

43. Depth of field: aliasing in motion, continued • For our specific case we use a hack that increases material roughness and reduces specular reflections based on VRS rate • This approach can be refined and/or used in other sources of VRS- amplified aliasing • This code works in non-VRS as well as VRS Tier 1 and Tier 2 cases (although might be best to compile-out in non-VRS case)

44. Depth of field: No VRS

45. Depth of field: VRS-optimized up to VRS 4x2

46. Depth of field: takeaways • Performance: • On ICL, 2560x1600, we get approx. 16%* savings (1.0x -> 0.84x) • On RTX 2060, 4K, we get approx. 14%* savings (1.0x -> 0.86x) • In our case this translates to approx. 10% overall FPS increase (depending on shading complexity choice) • DoF is just one use case; would work similarly for fog/atmospherics • Tier2: control block size of 16x16 is temporally unstable, but maybe use both per-object and image based with a combiner? *Automated benchmark, measured across 2400 frame animation with varying DoF settings, Tier 1 (per-object) implementation, max VRS rate set to 4x2, forward opaque and transparent passes

48. Case Study: Unreal Integration SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST

49. Spelunking with VRS Goal: Integrate Tier One VRS techniques with Unreal Engine and create a demo

50. Step One: Detect VRS Support Fold Microsoft* Windows 10 May Update SDK into build system Unreal Tier One Integration: Detect 50

51. Step Two: Implement RSSetShadingRate Exposed on the RHI layer to future proof cross-API functionality—stubbed out for unsupported APIs Remember: Commandlist 5 Unreal Tier One Integration: Define 51

52. Step Three: Apply shading rate to select pass in Render() REMEMBER: Global State! Must “unset” back to 1x1 after Started with mesh drawing and then separate per pass settings Unreal Tier One Integration: Apply 52

53. Woo! VRS in Engine!

54. Rens Demo 54

55. VRS Disabled 36.9ms/f VRS Enabled 28.1ms/f Performance 55 30% Speedup!

56. GOOD • Expensive Pixel Shaders • Definition from depth (foliage outlines) • Obfuscation: depth of field, motion blur, distance fog, etc. “CHALLENGING” • Geometry bottlenecks • Low resolution textures • For distance based techniques: Large meshes spanning foreground and background What makes a workload? 56

57. Fly Through GBuffer: 2x2 Shadow: 1x1 Reflections: 2x2 DOF: Enabled Interactive GBuffer: 2x1 Shadow: 1x1 Reflections: 2x2 DOF: Disabled VRS Implementation Simple per-pass integration

58. • Specific to Workload • Largely panning laterally • Integration with motion detection 1x1 à 2x1: 2ms 2x1 à 2x2: 3ms Asymmetric VRS rates

59. Depth of field paid for by VRS Could even reduce VRS rate Simple implementation, no popping Could tag focal points in cinematic Gen 11 Pre-Release HW at 1080p DOF Cost Recovery DOF Off DOF On VRS Off 36.9ms 40.9ms VRS 2x2 On 28.1ms 31.7ms

60. Reflection Environment pixel shader • Minimal reflection probes • Skybox rim lighting • SetVRSRate(RefEnvShadingRate) • RenderReflectionEnvironment() • SetVRSRate(1x1) 2ms at 2x2 4ms at 4x4 Cheating 60

61. Step Zero: Strip It • Strip the renderer to what you need • Accelerates artifact debugging Minimum viable project: Triangle with high variance texture To start, we disabled translucency, reflections, temporal AA, etc. to expose most primitive artifacts Tier One Integration: Learnings 61

62. Try: • Different shading rates per-pass, e.g. Mesh Drawing, Shadows, Reflections • Expose as console variables and experiment live • Mesh Tagging • Deeper integrations with obfuscation techniques: DOF, motion blur, distance fog, etc Tier One Integration: Experiment & Explore 62

63. Check for Accidental Inclusions: Make sure you’re only affecting what you think you are: SET and UNSET • Reflections • Decal Buffer • Screen Space Techniques • Up/Down Samples Debugging Artifacts 63 Tool Support • Intel GPA Version 19.2+ • Pix Version 1903.26+ Artifact appears with VRS 1x1, but not with VRS disabled? Could be driver issues, give us a call.

64. What Worked: • VRS paid for depth of field • Asymmetric Shading Rates • Exposing shading rates as console vars for experimenting • Tight engineer/artist feedback loop Wish we’d had: • Ability to tag hero meshes for high/low shading rates • Fully tested render feature pipeline to eliminate late breaking artifacts • More stable workflow The Good and The Bad 64

65. Open Work • Tier One and Tier Two cointegration • Per-Draw techniques in Unreal’s new mesh drawing pipeline • Mesh Tagging system: give artists option to never/always use coarse rate • Proper motion detection • How will it interact with dynamic res? 65

66. Results Getting started with VRS is relatively simple The performance boost from VRS can be extremely impactful while maintaining visual quality, depending on workload and implementation Even simple VRS implementations can deliver meaningful performance improvements. 66

67. Questions? This Photo by Unknown Author is licensed under CC BY-SA

68. • Subtitle Copy Goes Here

69. backup

70. References • [we already have links provided previously, will enumerate here time permitting] • Lei Yang, Dmitry Zhdan, Emmett Kilgariff, Eric B. Lum, Yubo Zhang, Matthew Johnson, and Henrik Rydgård. 2019. Visually Lossless Content and Motion Adaptive Shading in Games. Proc. ACM Comput. Graph. Interact. Tech. 2, 1, Article 6 (June 2019), 19 pages. DOI: https://doi.org/10.1145/3320287 • Lei Yang et al’s GDC talk: https://www.gdcvault.com/browse/gdc-19/play/1026205 • Martin Fuller presentation: • Our previous work: • VRMark demo: • Rens demo:

71. Hotoffthepresses • Intel announced support for VRS at Architecture Day 2018 for Gen11 • Gen11 support for VRS • Public reveal with MSFT and Firaxis at GDC 2019 for VRS DX12 API • Blog: Variable Rate Shading: a scalpel in a world of sledgehammers • Spec: VRS Spec • VRS on Youtube • IHVs public with DX12 VRS support 71 Intel (Gen11) AMD Nvidia (Turing) VRS Support Tier 1 Not yet Tier 2 (includes Tier 1) Additional Shading Rates Supported Yes Not yet Yes

72. RecentWorkcontinued • +30% from VRS on Gen11 hardware • Details later! 72 [ArtbyRensDeBoer,UEcodingKellyGawne@Intel] https://www.youtube.com/watch?v=GLfFUQaBXgQ&feature=youtu.be

73. Intel Confidential VisualTechnologiesTeam DrivingBest-in-classineverythingwedo 73

77. Intel Confidential VisualTechnologiesTeam DrivingBest-in-classineverythingwedo 77 1x1 big

78. Intel Confidential VisualTechnologiesTeam DrivingBest-in-classineverythingwedo 78 2x2big

79. Intel Confidential VisualTechnologiesTeam DrivingBest-in-classineverythingwedo 79 4x4 big

80. Intel Confidential VisualTechnologiesTeam DrivingBest-in-classineverythingwedo 80 4x4 smaller in big scene2x2 smaller in big scene 1x1 smaller in big scene

84. Nvidia/Bethesda 84

85. MSFT 85

86. Firaxis 86

87. VRMarkDemo 87

88. § TIMING: 50 Min. for Presentation & 10 Min. for Q & A § Instructions: § Use this template for presentation. § Key concepts, not full descriptions. Expand on points verbally. § Be brief. Text-heavy slides are uninteresting to view. § Use graphics, charts, visually-appealing pictures to support concepts. § No fonts smaller than 16pt, use contrasting colors § DO NOT grab pictures off Google to use in your presentation. Purchase pictures off Shutterstock.com or other royalty-free image site. § Recommended slides: Agenda, Call to Action, Reference Presentationsliderecommendations[DELETEWHENDONE] Work Week Milestone 22 Outline Due 24 50% drafts complete 26 80% drafts complete 27-28 Dry Runs 29 100% decks provided to Michaelle for legal review 32 Final Deck with corrections from legal review provided to Michaelle 33 SIGGRAPH DEADLINES:

89. Adam Lake, Filip Strugar, Trapper McFerron

90. 90 Agenda • Intro / Previous Work - Adam • The API - Adam • VRS in Practice - Filip • Case Study: Objects in Motion - Adam • Case Study: VRS Depth of Field - Trapper • Summary – Adam • Questions • Backup? • Case Study: LOD – Adam [Low Priority] • Particle POC – Filip

91. 91 Agenda • What is VRS • Enabling VRS • When to use VRS • Case Study: Particles • Case Study: VRS Depth of Field • Summary • Questions

92. 92 Whatisvrs • Explain VRS modes (2x2, 4x4, etc.) • Explain it helps perf when pixel shader complexity is the bottleneck • Show Pictures • API support and Tiers • HW support

93. 93 Enablingvrs • Show code which tests for VRS • Show code which enables VRS for Tier 1 (per draw)

94. 94 Whentousevrs • Talk about potential use cases (both explored and unexplored) • Shading obscured by fog?, water?, blurs (e.g. Depth of Field)? • Depth? Bring up potential issues w.r.t. over-blurring

95. 95 Casestudy:particles • Particles – show and talk about Filip’s R&D

96. 96 Casestudy:vrsdepthoffield • General overview of real time Depth of Field techniques • Talk about our Depth of Field specifics and how we leveraged VRS • Show before and after visuals and performance

97. 97 Summary • Using Tier 1 VRS on specific draws will improve performance on Gen11 with little extra work required from the ISV • Especially consider it for any objects obscure by post process or screen space effects • Be aware of stair stepping when used with alpha • Check out our papers and code: • Adam’s white paper link • Github link vrs_dof • Github link particles

98. Abstract In this presentation we will discuss a new feature available in the latest release of DirectX12 and supported on Intel Gen11 Graphics Hardware called Variable Rate Shading (VRS). We will first present an overview of the DX12 VRS API and best practices of the use of the feature on Intel hardware. Then we will talk about our experience modifying traditional 3D effects including depth of field and particle systems to leverage Variable Rate Shading. We will share our results and insights gained from these experiments and provide recommendations of how to use VRS in your rendering pipeline. Finally, we will have demos and/or videos to showcase these techniques that show performance and visual quality results .

99. 10 0 Depthoffield:splitblurs Near Blur Buffer Far Blur Buffer

100. 10 1 Depthoffield:nearandfarbuffers • Pass 3 – Gaussian Blur

101. 10 2 Depthoffield:composite float3 color = g_color[ d_pixel ].rgb; float far_coc = g_coc[ d_pixel ]; if ( far_coc > 0 ) { float3 far_color = g_dof_far.SampleLevel( … ).rgb; color = lerp( color, far_color, far_coc ); } float4 dof_near = g_dof_near.SampleLevel( … ); float alpha = dof_near.a * (1.0 - far_coc); g_color[ d_pixel ].rgb = lerp( color, dof_near.rgb, alpha );

102. 10 3 Depthoffield:bokeh Bokeh Blurs for Foreground, Background static const float2 kernel[max_bokeh_samples] = { float2(0.25,0), float2(0.07429166058208397,0.2933711331132184), float2(-0.31244462213914737,0.16908657384211828), float2(-0.2762595884262893,-0.3000979109324614), float2(0.2518840201879596,-0.3855371939366909), … } for ( i = 0; i < max_bokeh_samples; i++ ) { float2 sample_uv = uv + (texel_size * kernel[i] * dist); bokeh_samples[i] = g_farblur_color.SampleLevel( g_samplerLinearClamp, sample_uv, 0 ); } Near Bokeh Far Bokeh

103. 10 4 Depthoffield:farbokeh Pass 2 – Bokeh Blur for Background if ( coc == 0 ) return color; for ( i = 0; i < max_bokeh_samples; i++ ) { blur_color += (bokeh_samples[i] * bokeh_samples[i].a); valid_count += bokeh_samples[i].a; } blur_color /= valid_count; Prevents samples which are in the focal range from leaking into the blur Far Blur CoCFar Blur Buffer If no CoC for this pixel, it should not be blurred

104. Prevents samples which are in the focal range from leaking into the blur If no CoC for this pixel, it should not be blurredif ( coc == 0 ) return color; 10 5 Depthoffield:farbokeh Pass 2 – Bokeh Blur for Background for ( i = 0; i < max_bokeh_samples; i++ ) { blur_color += (bokeh_samples[i] * bokeh_samples[i].a); valid_count += bokeh_samples[i].a; } blur_color /= valid_count; Prevents samples which are in the focal range from leaking into the blur Far Blur CoCFar Blur Buffer

105. 10 6 Depthoffield:nearbokeh Pass 2 – Bokeh Blur for Foreground for ( i = 1; i < max_bokeh_samples; i++ ) { if ( bokeh_samples[ i ].a > 0 ) { blur_color += bokeh_samples[i]; valid_count++; } } blur_color /= valid_count; Prevents samples which are in the focal range from leaking into the blur Don’t do this, every pixel should blur. The compositing step will fade in based off of CoC if ( coc == 0 ) return color;

106. 10 7 Depthoffield:nearbokeh Pass 2 – Bokeh Blur for Foreground for ( i = 1; i < max_bokeh_samples; i++ ) { if ( bokeh_samples[ i ].a > 0 ) { blur_color += bokeh_samples[i]; valid_count++; } } blur_color /= valid_count; Prevents samples which are in the focal range from leaking into the blur Don’t do this, every pixel should blur. The compositing step will fade in based off of CoC if ( coc == 0 ) return color;

107. 10 8 Depthoffield:gaussianblurs Pass 3 – Gaussian Blur float3 avg_color = 0; int i; for ( i = 0; i < total_colors; i++ ) avg_color += color[ i ].a > 0 ? color[ i ].rgb : color[ 0 ].rgb; avg_color = avg_color / total_colors; float4 blur_color = 0; for ( i = 0; i < total_colors; i++ ) blur_color += (color[ i ].a > 0 ? color[ i ] : float4(avg_color, color[i].a)) * weights[ i ]; Compute average color of all pixels which should be blurred If a gaussian weight corresponds with a pixel which should not be included in the blur, use the average color instead.

108. 10 9 Depthoffield:nearandfarbuffers • Pass 3 – Gaussian Blur

109. 11 0 Depthoffield:composite float3 color = g_color[ d_pixel ].rgb;

110. 11 1 Depthoffield:composite float3 color = g_color[ d_pixel ].rgb; float far_coc = g_coc[ d_pixel ]; if ( far_coc > 0 ) { float3 far_color = g_dof_far.SampleLevel( … ).rgb; color = lerp( color, far_color, far_coc ); }

111. 11 2 Depthoffield:composite float3 color = g_color[ d_pixel ].rgb; float far_coc = g_coc[ d_pixel ]; if ( far_coc > 0 ) { float3 far_color = g_dof_far.SampleLevel( … ).rgb; color = lerp( color, far_color, far_coc ); } float4 dof_near = g_dof_near.SampleLevel( … ); float alpha = dof_near.a * (1.0 - far_coc); g_color[ d_pixel ].rgb = lerp( color, dof_near.rgb, alpha );

112. References 113

113. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST

114. backup

115. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST

116. Intel Exhibitor Session Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis. Ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST Bringing Intelligent Motion Using Reinforcement Learning Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis. Ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum.

117. Intel Exhibitor Session Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis. Ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST Bringing Intelligent Motion Using Reinforcement Learning

118. Graph and Charts Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis. Ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST Advanced SIMD Programming Title 1st Qtr 2nd Qtr

119. Graph and Charts Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis. Ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST Advanced SIMD Programming 4th Qtr 3rd Qtr 2nd Qtr 1st Qtr 0 1 2 3 4 5 6 7 8 9 V 1 V 2

120. Graphs and Charts Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST MPEG-I Immersive Video Sales 0 1 2 3 4 5 6 Sales Series 1 Series 2 Series 3

121. Graphs and Charts SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST MPEG-I Immersive Video Haribus con peria voluptinvel et accuscienia cullupi squiae. Haribus con peria voluptinvel et accuscienia cullupi squiae. Haribus con peria voluptinvel et accuscienia cullupi squiae. Haribus con peria voluptinvel et accuscienia cullupi squiae.

122. Graphs and Charts Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis. Ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST Advanced SIMD Programming 0 1 2 3 4 5 6

123. Intel Exhibitor Session “Advanced SIMD Programming with ISPC quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi et ut as poreprae”. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST Bringing Intelligent Motion Using Reinforcement Learning

124. Intel Exhibitor Session Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis. Ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST Bringing Intelligent Motion Using Reinforcement Learning

125. The Future of Entertainment to Your Living Room SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST Arts & Design Gaming & Interactive New Technologies Production & Animation

126. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST 12 7

127. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. Legal Notices And Disclaimers

128. SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. Haribus con peria voluptinvel et accuscienia cullupi squiae sersped quidiasit lam volectur solupta denimpo rporesecae sit offic tectur si officiu ntiatem rendam, saped que officillibus alibusa dolupta nimilis et, quibea nimi, ut ut as re rae audi dus, od untis cuptatio molorenit odignihitate aute delit modia sitium aut ea aut molores torestiis ad que alitationse venim essuntia con conesti que poribus dis expelic itecaeperit qui qui destis disquatae volor sin ped quis molenet, ut voluptae officia aliti quiatiam quamet ulparum que net exerum accabor aute num ipienihilit volorec epudio venimus atur as et et aut inis et ut as poreprae in estenditium lit lat veliberum id exped modit, niet ut quatur as elenis quis earum ipsum. Legal Notices And Disclaimers

129. CHAPTER HEADER 1 SIGGRAPH 2019 | LOS ANGLES | 28 JULY - 1 AUGUST Subtitle Copy Goes Here

136. • Subtitle Copy Goes Here