Successfully reported this slideshow.

Deferred shading


Published on

  • Be the first to comment

Deferred shading

  1. 1. Deferred Shading Chun-Fu “Frank” Chao Image from source 1
  2. 2. Outline Why(Why not) deferred shading History of deferred shading Basic of deferred shading Shadow in deferred shading Extension of deferred shading ◦ Light Pre-Pass deferred shading ◦ Anti-aliasing in deferred shading ◦ Inferred shading ◦ Tile-based deferred shading
  3. 3. What is deferred shading? Instead of intermediately calculate the color and write to frame buffer, we divide the information into smaller parts and write them into intermediate buffer (i.e. G(eometric)-buffer). Then combine and calculate color later. Video from source 15
  4. 4. Killzone 2 Behind the Bullet forPlayStation 3 Video from source 2
  5. 5. Why deferred shading? Image from source 26
  6. 6. Why deferred shading? Decouple the scene geometry complexity and light complexity ◦ Going from O( #Light * #Triangle ) to O( #Light ) + O( #Triangle ) Only do the BRDF calculation on the visible surfaces Simplify the rendering process ◦ Few passes on G-buffer V.S. Switching shaders G-buffer is useful for post-processing
  7. 7. Comparison with ForwardRendering Single pass lighting: For each object Render object Apply all the lights in shader Why this is not good? ◦ Boated shader if we do code generation for all the light sources ◦ Wasteful ( We are throwing away fully shaded fragment in depth test !) From source 4, slide 3
  8. 8. Comparison with ForwardRendering Multi-pass lighting: For each light For each object affected by light Framebuffer += BRDF( object, light ) Why this is not good? ◦ Repetitive vertex transformation for the same object ◦ Wasteful ( We are “still” throwing away fully shaded fragment in depth test !) From source 4, slide 4
  9. 9. Deferred Shading For each object Render the lighting(material) property into G-buffer For each light Framebuffer += BRDF( G-buffer, light ) Simplify the lighting process O(1) depth complexity for lighting From source 4, slide 5
  10. 10. Why “not” deferred shading? Memory intensive ◦ Multiple buffer ( Multiple render target ) ◦ Buffer read-back ◦ Limited material type Cannot apply hardware assisted anti-aliasing ◦ MSAA(Multi-sample anti-aliasing) is for pixel already shaded Cannot handle translucent object ◦ We only have the information of the object with smallest depth value
  11. 11. History Deering, M., et al. “The Triangle Processor and Normal Vector Shader: A VLSI System for High Performance Graphics”, Proceedings of SIGGRAPH ‘88 ◦ Only shading once after depth resolution T. Saito and T. Takahashi. “Comprehensible rendering of 3d shapes”, Proceedings of SIGGRAPH ’90 ◦ It is designed for contour rendering ◦ The first time the term of G-Buffer appears Image from source 6, 7
  12. 12. Multiple Render target A feature that allows programmable rendering pipeline to render images to multiple render target texture at the same time. Introduced by OpenGL 2.0 and DirectX 9.0 Supported by Xbox360 and Playstation 3
  13. 13. Multiple Render target We can do deferred shading without Multiple render target. ◦ But the performance is going to be a problem Image from source 6, 8
  15. 15. What kind of G-buffer do we need? Going from the SSAO assignment we did
  16. 16. Say we want something like this From source 9, slide 9
  17. 17. What kind of G-buffer do we need? From source 9, slide 10
  18. 18. What kind of G-buffer do we need? From source 9, slide 11
  19. 19. What kind of G-buffer do we need? From source 9, slide 12
  20. 20. What kind of G-buffer do we need? From source 9, slide 13
  21. 21. What kind of G-buffer do we need? From source 9, slide 14
  22. 22. What kind of G-buffer do we need? From source 9, slide 15
  23. 23. What kind of G-buffer do we need? From source 9, slide 16
  24. 24. What kind of G-buffer do we need? From source 9, slide 17
  25. 25. The G-Buffer of Killzone 2 4 X R8G8B8 + 24Depth + 8Stencil ~= 36 MB ◦ We only got 256 MB RAM and 256 MB VRAM on PS3  No Normal.z term ◦ Normal.z = sqrt( 1.0 – Normal.x^2 – Normal.y^2 ) ◦ Trade memory space with computation time From source 9, slide 19
  26. 26. The G-buffer of Battlefield 3  Material ID is a common way to manage material R8 G8 B8 A8GB0 Normal .xyz Spec. SmoothnessGB1 Diffuse albedo .rgb Specular albedoGB2 Sky visibility Custom envmap ID Material Param. Material IDGB3 Irradiance (dynamic radiosity) Image from source 9, slide 12 and source 10
  27. 27. Lighting Result Image from source 10, slide 31
  28. 28. SHADOW
  29. 29. That’s cool. But where is my shadow? 1000 lights without shadow is easy, but 100 lights with shadow is impossible now  Shadow map ◦ A good fit with deferred shading ◦ Need more memory  Shadow volume ◦ Render light as geometry volume ◦ Depth test to determine whether the object is inside the volume or not
  30. 30. Shadow map Create a depth map in the perspective of light source. Compare the position of the geometry surface with the depth map of light source. Position = Screen space position + depth Image from source 14
  31. 31. Ouch! Image from source 16, Page 16
  32. 32. Shadow Volume Only apply the light to the pixel enclose by the shadow volume Depth test needs to be handle carefully Image from source 4, slide 12 Image from source 17
  33. 33. Xbox Gladiator 2k4 demo Video from source 5
  34. 34. Wait! We still have plenty ofproblems Memory usage ◦ Light Pre-Pass deferred shading Anti-aliasing ◦ MLAA ( Morphological Anti-aliasing ) ◦ SRAA ( Sub-pixel Reconstruction Anti-Aliasing ) Alpha blending ◦ Inferred shading
  36. 36. Light Pre-pass / Deferred Lighting Two pass deferred shading 1st pass ◦ Only render depth and normal ◦ Do not need MRT! Image from source 17
  37. 37. Light Pre-pass / Deferred Lighting Write LightColor * N dot L * Attenuation in RGB, specular in A channel Image from source 17
  38. 38. Light Pre-pass / Deferred Lighting 2nd geometry pass ◦ Fetch the material property ◦ Combine with light buffer Image from source 17
  39. 39. Light Pre-pass / Deferred Lighting Pro ◦ Less memory required ◦ Doesn’t not require MRT feature = Enable programmer to turn on MSAA in DirectX9 ◦ Only one material fetch regardless number of light Con ◦ “Two” geometry passes
  41. 41. Anti-aliasing Super-sampling one framebuffer is painful, super-sampling 5 MRT is 5X painful  Hardware supported MSAA is incompatible with deferred shading ◦ MSAA cannot apply to MRT in DirectX 9 ◦ Super-sampling and averaging normal or depth might cause wired result Imaged based approach of anti-aliasing We have depth and normal information 
  42. 42. Basic Edge-detection Anti-aliasing Do the 2D edge detection algorithm on frame buffer Apply low-pass filter across the edge The discontinuities of normal and depth are better features to locate edges
  43. 43. Basic Edge-detection Anti-aliasing Detect depth or normal discontinuity is better than the color discontinuity Image from source 17
  44. 44. Morphological Anti-aliasing Detect the pattern of color discontinuity Draw triangular area based on the pattern Image from source 18
  45. 45. Morphological Anti-aliasing Connect the mid-points of the L, Z, and U shape segments. The blending weight is based on the triangle area enclosed by mid points Image from source 18
  46. 46. Sub-pixel Reconstruction Anti-Aliasing MLAA will alter the shape of the original image Image from source 20
  47. 47. Sub-pixel Reconstruction Anti-Aliasing Determine the blending pattern based on the super-sampled depth and normal information Image from source 20
  48. 48. Anti-aliasing comparison Image from source 21, slide 44
  49. 49. Anti-aliasing comparison Image from source 21, slide 45
  50. 50. Anti-aliasing comparison Image from source 21, slide 46
  51. 51. Anti-aliasing comparison Image from source 21, slide 47
  52. 52. Anti-aliasing comparison Image from source 21, slide 48
  53. 53. Anti-aliasing comparison Image from source 21, slide 49
  55. 55. Alpha blending How do we get transparent/transculant effect in deferred shading ◦ One more forward rendering pass  ◦ Depth peeling  So expensive   Limited layer of transculant material
  56. 56. Depth Peeling Multiple pass of depth buffer generation from front to back Peel the fragments away after depth buffer generated Render image form back to front Image from source 22
  57. 57. Depth Peeling Image from source 22
  58. 58. Inferred Shading 3-pass algorithm based on the 2-pass Light Pre-pass deferred shading Can create transparent effect in deferred shading scheme ◦ Even multiple layer of transparency! Image from source 23
  59. 59. Inferred Shading Geometry Pass ◦ RT1 : Normal.X / Normal.Y ◦ RT2 : Depth / Object ID ◦ The resolution is lower than target Light Pass ◦ Apply lighting to the G-buffer to create L-buffer ◦ The resolution is also lower that target Material Pass ◦ Material fetch ◦ Combine material with L-buffer information
  60. 60. Inferred Shading Discontinuity Sensitive Filtering ◦ Sample lighting information based on the object ID information stored in G-buffer ◦ If the sample is not from the same object, then ignore the sample Image from source 23
  61. 61. Lighting Alpha Polygons Render the transparent objects last in the geometry pass and render them into stipple pattern Special sampling rule parse over the stipple pattern in the material pass Image from source 23
  62. 62. Stipple Pattern Image from source 23
  63. 63. Material Pass : Render OpaqueObjects Image from source 23
  64. 64. Material Pass : Render TransparentObjects Image from source 23
  65. 65. Multiple Layer of TransparencyTransparent Opaque 1 Opaque Opaque Transparent Transparent 1 3 Transparent Opaque 2 Image from source 23
  66. 66. Resolution Decrease Image from source 23
  68. 68. Tile-based Deferred Shading Introduced into Frostbite 2 engine Divide the screen in screen-space tiles Cull analytical lights (point, cone, line), per tile Compute lighting for all contributing lights, per tile Slide from source 25
  69. 69. Tile-based Deferred Shading Image from source 10, slide 42
  70. 70. Tile-based Deferred Shading Image from source 10, slide 43
  71. 71. Tile-based Deferred Shading Image from source 10, slide 43
  72. 72. Tile-based Deferred Shading• The screen is divided in 920 tiles of 32x32 pixels• Downsample and classify the scene from 720p to 40x23 (1 pixel == 1 tile) • Find each tile’s Min/Max depth • Find each tile’s material permutations • Downsampling is done in multi-pass and via MRTs Slide from source 25
  73. 73. Tile-based Deferred Shading Image from source 25, slide 72
  74. 74. Tile-based Deferred Shading Image from source 25, slide 73
  75. 75. Tile-based Deferred Shading Image from source 25, slide 74
  76. 76. Tile-based Deferred Shading Build mini-frustas for each tile Cull lights against sky-free tiles in a shader Store the culling results in a texture: ◦ Column == Light ID ◦ Row == Tile ID Actually, 4 lights can be processed at once (A-R-G-B) Read back the contribution results on the CPU and prepare for lighting! Slide from source 25
  77. 77. Tile-based Deferred Shading Parse the culling results texture on CPU For each light type For each tile For each material permutation, Regroup & set the light parameters for the pixel shader constants Setup the shader loop counter (Limited number of light passed) Additively render lights with a single draw call (to the final HDR lighting buffer) Slide from source 25
  78. 78. Tile-based Deferred Shading Image from source 25, slide 66
  79. 79. Tile-based Deferred Shading Image from source 25, slide 67
  80. 80. Tile-based Deferred Shading Image from source 25, slide 68
  81. 81. Tile-based Deferred Shading Image from source 25, slide 69
  82. 82. Thank you Questions? Image from source 25
  83. 83. BONUS
  85. 85. Demonstration Image from source 1
  86. 86. Approximating Translucency Image from source 1
  87. 87. Approximating Translucency Image from source 1
  88. 88. SOURCE
  89. 89. Source1. John Chapman, “Deferred Rendering, Transparency & Alpha Blending Tutorial”, Available at: Guerrilla Games, “Killzone 2 Behind the Bullet for PlayStation 3”, Available at: Shawn Hargreaves, “Deferred Shading”, Available at: Shawn Hargreaves, “Deferred Shading : 6800 Leagues Under The Sea”, Available at: 6800_Leagues_Deferred_Shading.pdf5. Rich Geldreich, “Deferred Lighting and Shading”, Available at: Deering, M., et al. “The Triangle Processor and Normal Vector Shader: A VLSI System for High Performance Graphics”, Proceedings of SIGGRAPH ‘88 , Available at:
  90. 90. Source7. T. Saito and T. Takahashi. “Comprehensible rendering of 3d shapes”, Proceedings of SIGGRAPH ’90, Available at: Filion, D. and McNaughton, R. (Blizzard Entertainment), “ StarCraft II: Effects & Techniques”, slide available at: Course note available at: Filion-StarCraftII.pdf9. Michal Valient (Guerrilla), “Deferred Rendering in Killzone 2 “, Available at: Christina Coffin (DICE), “SPU-based Deferred Shading for Battlefield 3 on Playstation 3”, Available at: y=Rendering11. Colin Barré-Brisebois (DICE), “Approximating Translucency for a Fast, Cheap and Convincing Subsurface Scattering Look”, Available at:
  91. 91. Source12. Michal Valient (Guerrilla), “The Rendering Technology of Killzone 2”, Available at: 213. Michiel van der Leeuw (Guerrilla), “The PlayStation®3’s SPUs in the Real World: A KILLZONE 2 Case Study “, Available at: world-a-killzone-2-case-study-988622414. Wikipedia : Shadow Mapping Wikipedia : Deferred Shading Johan Andersson (DICE), “5 Major Challenges in Interactive Rendering”, Available at: y=Rendering17. Nicolas Thibieroz, “Deferred Shading Optimizations” , Available at: ps
  92. 92. Source18. Alexander Reshetov (Intel Labs ), “Morphological Antialiasing” , Available at: Alexandre De Pereyra (Intel), “MLAA: Efficiently Moving Antialiasing from the GPU to the CPU”, Available at: from-the-gpu-to-the-cpu/20. Chajdas, McGuire, and Luebke, Subpixel Reconstruction Antialiasing, ACM Symposium on Interactive 3D Graphics and Games (I3D 2011 proceedings), February 2011, Available at: Johan Andersson (DICE), “DirectX 11 Rendering in Battlefield 3”, Available at: y=Rendering22. Cass Everitt (Nvidia), “Interactive Order-Independent Transparency”, Available at: transparency
  93. 93. Source23. Scott Kircher and Alan Lawrance. 2009. Inferred lighting: fast dynamic lighting and shadows for opaque and translucent objects. In Proceedings of the 2009 ACM SIGGRAPH Symposium on Video Games (Sandbox 09), Stephen N. Spencer (Ed.). ACM, New York, NY, USA, 39-45. Available at: ENGEL, W., 2008. Diary of a graphics programmer: Light pre-pass renderer. Online, accessed Jan. 27th, 2009. Available at: renderer.html.25. John White (Black Box), Colin Barré-Brisebois (DICE)“More Performance! Five Rendering Ideas from Battlefield 3 and Need For Speed The Run”, Available at: Kenny Magnusson (DICE),“Lighting you up in Battlefield 3”
  94. 94. Indirect Reference Source1. Oles Shishkovtsov (GSC Game World), "GPU Gems 2 Chapter 9. Deferred Shading in S.T.A.L.K.E.R.", Available at: Rusty Koonce (NCsoft Corporation), "GPU Gems 3 chapter 19. Deferred Shading in Tabula Rasa", Available at: Tiago Sousa(Crytek), "GPU Gems 3 chapter 16. Vegetation Procedural Animation and Shading in Crysis", Available at: Wolfgang Engel, “ShaderX2 : introductions and tutorials with DirectX 9”, Available at: X_9.pdf5. Wolfgang Engel, “Light Pre-Pass -Deferred Lighting: Latest Development”, Available at: ppt
  95. 95. Indirect Reference Source6. Mark Lee(Insomniac games), “Pre-lighting, Available at: Mark Lee(Insomniac games), “Pre-lighting in Resistance 2”, Available at: Rouslan Dimitrov(Nvidia), “Cascaded Shadow Maps”, Available at: maps/doc/cascaded_shadow_maps.pdf9. Damian Trebilco, “Light Indexed Deferred Lighting”, Available at: Andrew Lauritzen(Intel), "Deferred Rendering for Current and Future Rendering Pipelines", Available at: B. Segovia, J. C. Iehl, R. Mitanchey, and B. Péroche. 2006. Non-interleaved deferred shading of interleaved sample patterns. In Proceedings of the 21st ACM SIGGRAPH/EUROGRAPHICS symposium on Graphics hardware (GH 06). ACM, New York, NY, USA, 53-60.
  96. 96. Indirect Reference Source12. John Tsiombikas, “Volume Shadows Tutorial”, Available at: Stefan Brabec , Thomas Annen , Hans-peter Seidel, ” Shadow Mapping for Hemispherical and Omnidirectional Light Sources”, Available at: Emil Persson (AMD), “Depth In-depth “, Available at: Sam Martin, Per Einarsson(Geomerics), “A Real Time Radiosity Architecture” Martin Mittring(Crytek), “A bit more deferred – CryEngine3”, Available at: Anton Kaplanyan(Crytek), “CryENGINE 3: reaching the speed of light” speed-of-light18. Nickolay Kasyan, Nicolas Schulz, Tiago Sousa(Crytek), ”Secrets of CryENGINE 3 Graphics Technology”, Available at: graphics-technology
  97. 97. Indirect Reference Source19. Martin Mittring(Epic games), Bryan Dudash(Nvidia), “The Technology Behind the DirectX 11 Unreal Engine "Samaritan" Demo”, Available at: Dean Calver , “Photo-realistic Deferred Lighting” Anton Kaplanyan (Crytek) ,”Light Propagation Volumes in CryEngine 3” , Available at: volumes-in-cryengine-322. Aths, “Multisampling Anti-Aliasing: A Closeup View “, Available at: Fabio Policarpo, Francisco Fonseca(CheckMate Games), “ Deferred Shading Tutorial”, Available at: http://www710.univ- pdf24. Josh Klint (CEO, Leadwerks Corporation ), “Deferred Rendering in Leadwerks Engine “, Available at:
  98. 98. Previous Presentation [Spring 2011] Sean Thomas [Spring 2010] Ian Perera TURE2010/Deferred%20RenderingMacro. pptm