Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Rendering Techniques in Deus Ex: Mankind Divided

2,736 views

Published on

This talk provides a cohesive overview of the advanced rendering techniques developed for Deus EX: Mankind Divided. It covers a collection of diverse features, the challenges they presented, where current approaches succeed and fail, and solutions and implementation details.

Published in: Software
  • Be the first to comment

Rendering Techniques in Deus Ex: Mankind Divided

  1. 1. DEUS EX MANKIND DIVIDED Labs R&D: The Rendering Techniques of Deus EX: Mankind Divided and Rise of the Tomb Raider SIGGRAPH 2015 www.eidosmontreal.com/www.square-enix-montreal.com
  2. 2. DEUS EX MANKIND DIVIDED VFX Effect for Adam Jensen Titan 2 v Nicolas Longhamps Senior Technical Artist at Labs, Eidos Montreal Peter Sikachev 3D Programmer at Labs, Eidos Montreal
  3. 3. DEUS EX MANKIND DIVIDED Titan 3 VFX Description  Trademark effect for DXMD  A protective shell covering character  Should look solid but feel “alive”  Should look similar to CG version v
  4. 4. DEUS EX MANKIND DIVIDED Titan 4 Visual Features  Triangular surface  Animation  Hit reaction  First-person vignette v
  5. 5. DEUS EX MANKIND DIVIDED Titan 5 Tessellation  Hardware tessellation  Dynamic tessellation factor based on distance  Small triangles used for geometry animations and displacement  Tessellation barycentric coordinates used for edge bevel effect v
  6. 6. DEUS EX MANKIND DIVIDED www.eidosmontreal.com www.square-enix-montreal.com Titan 6 Geometry Animation  Masks control specific areas  “Living skin”: periodic surface disturbance  Full-body noise on hit  Sequential vertex/normal offset v
  7. 7. DEUS EX MANKIND DIVIDED www.eidosmontreal.com www.square-enix-montreal.com Titan 7 Hit Reaction  Weapon hits  Parameters set per ammunition type  Hits found on CPU and passed in CB  Titan mesh animated near hit areas  Underlying character mesh is shown in holes only  (De)construction  Reusing proxy weapon hits  Placed at joints selected by artists v
  8. 8. DEUS EX MANKIND DIVIDED www.eidosmontreal.com www.square-enix-montreal.com Titan 8 Texture and Normal Map  Normal bevel (helps smooth edges)  Normal softening  Detail normal/data maps for normal animation  Surface noise map to simulate animated micro detail  Custom back reflections (aka ‘fake specular’) v
  9. 9. DEUS EX MANKIND DIVIDED Normal Bevel Off
  10. 10. DEUS EX MANKIND DIVIDED Normal Bevel On
  11. 11. DEUS EX MANKIND DIVIDED Normal Softening Off
  12. 12. DEUS EX MANKIND DIVIDED Normal Softening On
  13. 13. DEUS EX MANKIND DIVIDED Detail Normal Map Off
  14. 14. DEUS EX MANKIND DIVIDED Detail Normal Map On
  15. 15. DEUS EX MANKIND DIVIDED www.eidosmontreal.com www.square-enix-montreal.com Titan 15 First Person Vignette  Post Process effect in same visual style as Titan  Should convey the presence of armor but not obstruct view  Artist created mesh rendered in screen space  Shader supports Titan-like effects :  Sequential normal and vertex position offset animation  Edge bevel  (De)construction  Other shader effects :  Refraction  Chromatic aberration v
  16. 16. DEUS EX MANKIND DIVIDED www.eidosmontreal.com www.square-enix-montreal.com Titan 16 v Implementation Details  Not a performance-critical feature (~0.1 ms per character with Titan)  Flat normal using pos gradients  Tessellation was retro-fitted to the engine  Especially painful on PS4  Number of simultaneous hits is limited (32)  Limit displacement based on camera distance and artist mask
  17. 17. DEUS EX MANKIND DIVIDED 17 vParallax Occlusion Mapping Peter Sikachev 3D Programmer at Labs, Eidos Montreal
  18. 18. DEUS EX MANKIND DIVIDED Parallax Occlusion Mapping 18 Relaxed Cone-Step Tracing Refresher  Introduced in GPU Gems 3 [Policarpo07]  Pre-generate a cone-step map (CSM) acceleration structure  Use CSM instead linear steps  Improve intersection with binary steps v surface
  19. 19. DEUS EX MANKIND DIVIDED Parallax Occlusion Mapping 19 Relaxed Cone-Step Tracing Refresher  Introduced in GPU Gems 3 [Policarpo07]  Pre-generate a cone-step map (CSM) acceleration structure  Use CSM instead linear steps  Improve intersection with binary steps view raysurface cone
  20. 20. DEUS EX MANKIND DIVIDED Parallax Occlusion Mapping 20 Relaxed Cone-Step Tracing Refresher  Introduced in GPU Gems 3 [Policarpo07]  Pre-generate a cone-step map (CSM) acceleration structure  Use CSM instead linear steps  Improve intersection with binary steps view raysurface cone
  21. 21. DEUS EX MANKIND DIVIDED Parallax Occlusion Mapping 21 Blending POM surfaces with puddles  Water should look like it is between reliefs  Use height as a threshold between two materials fMaterialMask *= 1.0f - saturate( (fPOMHeight - u_fPOMThreshold) * u_fPOMMultiplier )
  22. 22. DEUS EX MANKIND DIVIDED Visuals Examples All Rights Reserved SquareEnix Eidos Montreal 2015
  23. 23. DEUS EX MANKIND DIVIDED www.eidosmontreal.com www.square-enix-montreal.com
  24. 24. DEUS EX MANKIND DIVIDED www.eidosmontreal.com www.square-enix-montreal.com Height blend threshold = 1 (no blend)
  25. 25. DEUS EX MANKIND DIVIDED www.eidosmontreal.com www.square-enix-montreal.com Height blend threshold = 0 (no blend)
  26. 26. DEUS EX MANKIND DIVIDED www.eidosmontreal.com www.square-enix-montreal.com Height blend threshold = 0.4
  27. 27. DEUS EX MANKIND DIVIDED www.eidosmontreal.com www.square-enix-montreal.com Height blend threshold = 0.8
  28. 28. DEUS EX MANKIND DIVIDED Parallax Occlusion Mapping 28 Depth Center  An artist-inspired idea  Feature from Marmoset Toolbag 2  Allows details to stick out  Used on charactersv EyeDirTangentSpace.xy *= POMRange; EyeDirTangentSpace /= max(EyeDirTangentSpace.z, NUMERIC_EPSILON); float Distance = length(EyeDirTangentSpace.xy); float2 BaseOffset = EyeDirTangentSpace.xy * POMDepthCenter / (Distance + 1.0f); view ray POM surface mesh surface
  29. 29. DEUS EX MANKIND DIVIDED Parallax Occlusion Mapping 29 Depth Center  An artist-inspired idea  Feature from Marmoset Toolbag 2  Allows details to stick out  Used on charactersv EyeDirTangentSpace.xy *= POMRange; EyeDirTangentSpace /= max(EyeDirTangentSpace.z, NUMERIC_EPSILON); float Distance = length(EyeDirTangentSpace.xy); float2 BaseOffset = EyeDirTangentSpace.xy * POMDepthCenter / (Distance + 1.0f); view ray POM surface mesh surface
  30. 30. DEUS EX MANKIND DIVIDED Visuals Examples All Rights Reserved SquareEnix Eidos Montreal 2015
  31. 31. DEUS EX MANKIND DIVIDED Without POM 31
  32. 32. DEUS EX MANKIND DIVIDED With POM 32
  33. 33. DEUS EX MANKIND DIVIDED Without POM 33
  34. 34. DEUS EX MANKIND DIVIDED With POM 34
  35. 35. DEUS EX MANKIND DIVIDED Parallax Occlusion Mapping 35 Optimizations and Cost  Reduce number of cone/binary steps based on distance  Texture mip is approximated as there were no POM  Incremental cost: ~1 ms full-screen 1080p up-close at PS4 v
  36. 36. DEUS EX MANKIND DIVIDED 36 Pure Hair v Presented by Anton Kai Michels R&D Programmer for Labs at Eidos Montreal Uriel Doyon Technical Director for Labs at Eidos Montreal
  37. 37. DEUS EX MANKIND DIVIDED Pure Hair Data Model 37 Asset Types  Textured head  Hair mesh  Hair strands Textured Head v
  38. 38. DEUS EX MANKIND DIVIDED Pure Hair Data Model 38 Asset Types  Textured head  Hair mesh  Hair strands Textured Head + Hair Strands vv
  39. 39. DEUS EX MANKIND DIVIDED Pure Hair Data Model 39 Asset Types  Textured head  Hair mesh  Hair strands Hair Mesh v
  40. 40. DEUS EX MANKIND DIVIDED Pure Hair Data Model 40 Asset Types  Textured head  Hair mesh  Hair strands Hair Strands v
  41. 41. DEUS EX MANKIND DIVIDED Pure Hair Data Model 41 Asset Types  Textured head  Hair mesh  Hair strands Hair MeshHair Mesh + Hair Strands v
  42. 42. DEUS EX MANKIND DIVIDED Pure Hair Hair Strands 42 Wisp Data  Required at runtime  Exported into Dawn Engine  Keep DCC wisps if available  Else grouping splines manually  Generate master strand per wisp Master strands (one per wisp) vv
  43. 43. DEUS EX MANKIND DIVIDED Pure Hair Hair Strands 43 Master to slave skinning  Less strands to simulate  Shape preserving LODs  Retains haircut shape despite simulation Slave strands (many per wisp) vv
  44. 44. DEUS EX MANKIND DIVIDED Pure Hair Hair Strands 44 Shape preserving LODs  Reduce strand count while increasing strand width  LOD is applied per wisp, so shape is preserved v LOD 0 LOD 1 LOD 2 LOD 3
  45. 45. DEUS EX MANKIND DIVIDED Pure Hair Simulation Variable number of vertices  Between 4 to 32 vertices per strand  Master strands within groups have same vertex count  Slave strands can have any number of vertices  Fixed length enforcing (a la Muller) vv
  46. 46. DEUS EX MANKIND DIVIDED Pure Hair Simulation vv World space simulation  Issues on fast moving objects  Calculate and scale root delta position  Apply it to previous frame positions
  47. 47. DEUS EX MANKIND DIVIDED Pure Hair Simulation vv Wind  Build a cone from wind direction  Gives each hair slightly different vector  Makes hairs spread nicely  Creates desirable variation between frames
  48. 48. DEUS EX MANKIND DIVIDED Pure Hair Simulation 48 vv Wisp Interactions  Slave strands affected by surrounding masters if close enough  Decided on stateless approach  Simplifies LOD handling  View frustum changes  Slaves only affected by closest master strand in reference position
  49. 49. DEUS EX MANKIND DIVIDED v Lighting probes  Evaluate lighting over hair bounds  Store directional intensities using cubemaps or spherical harmonics  Reduces lighting cost significantly  Head occlusion approximated by sphere in shader Tangent based lighting  Use tangent based lighting instead of normal  Especially visible in diffuse because strands are billboarded Pure Hair Lighting Model
  50. 50. DEUS EX MANKIND DIVIDED 50 Dual specular highlights  Most significant hair lighting feature  Adding tangent noise breaks "audio tape" look. Fresnel factor  More important than back scattering.  Extra term in the lighting equation. v Pure Hair Lighting Model 1 specular highlight Dual specular highlights Dual specular highlights + tangent noise
  51. 51. DEUS EX MANKIND DIVIDED Pure Hair Translucency 51 Per Pixel Linked List (PPLL)  Node buffer = 200Mb in 1080p  With 12 bytes per node  Koverdraw = 8  More overdraw makes resolve more expensive vv Fill PPLL Resolve Color Resolve Depth
  52. 52. DEUS EX MANKIND DIVIDED Pure Hair Translucency 52 PPLL with Hair Depth Pre-Pass  Optimization for PPLL, greatly benefits LODs  Allows smaller Koverdraw (~ 3) for the same quality  Artists must limit translucency to strand tips and mesh edges  ~100Mb total vv Fill PPLL Resolve Color Resolve Depth Hair Depth Pre-pass
  53. 53. DEUS EX MANKIND DIVIDED 53 Depth Peeling  Use atomics to compute closest depths 0 and 1  Fill nodes only if they are entry 0 or 1  Similar performance to PPLL with depth pre-pass  Half memory cost with no overflow  ~50Mb total Resolve Depth (D0 or D1) Fill Nodes Resolve Color DepthPeeling,AlphaAcc Pure Hair Translucency
  54. 54. DEUS EX MANKIND DIVIDED 54 Depth Peeling vs PPLL with Depth Pre-Pass  Performance comparison on a Radeon HD 7970, using Adam Jensen, cinematic LOD Fullscreen Near Far* PPLL & Depth Pre-pass 1.774 1.004 0.852 Resolve PPLL 2.059 0.422 0.146 Depth Peeling 1.452 0.892 0.833 Resolve DP 1.907 0.338 0.098 Note: Depth Peeling is faster at close range in part because the PPLL depth pre-pass does not have early depth, and also because data access have better locality. Still this is very hardware specific. Pure Hair Translucency
  55. 55. DEUS EX MANKIND DIVIDED Presented by Anton Kai Michels R&D Programmer for Labs at Eidos Montreal Motion Blur 55 v Samuel Delmont Senior 3D Programmer for Labs at Eidos Montreal
  56. 56. DEUS EX MANKIND DIVIDED Motion Blur 56  Real cameras produce motion blur by accumulating exposure of moving objects while shutter is open  Mimicking this in video games adds realism and a cinematic look  Helps reduce the perception of frame rate v
  57. 57. DEUS EX MANKIND DIVIDED Motion Blur Inspiration  Crysis, [Sousa08]  Iterative sampling approach : multiple blur passes  [McGuire13]  Tile neighborhood max velocity removes hard edges v
  58. 58. DEUS EX MANKIND DIVIDED Motion Blur v Overview  Motion Blur done in 2 steps:  Velocity Buffer Generation Velocity buffer is RG_FLOAT16, contains motion vectors in screen space coordinates (delta UV of a pixel’s previous position and current position)  Blur Post process, uses velocity buffer to compute blur direction and samples offsets.  Velocity buffer is full resolution: used for temporal reprojection
  59. 59. DEUS EX MANKIND DIVIDED Motion Blur v Velocity Buffer Generation  Dynamic objects  Velocity caused by camera movement + object movement + object skinned deformation  Applies to opaque dynamic/skinned primitives  Static objects  Velocity caused by camera movement only  Applies to opaque static primitives
  60. 60. DEUS EX MANKIND DIVIDED Motion Blur 60 v v Dynamic Object Velocity Pass  Render dynamic objects using:  Previous world matrix  Previous skinning matrix if applicable  Any previous vertex deformation  Write in stencil buffer to reduce static object velocity pass overhead Velocity buffer RG_Float16 visualization
  61. 61. DEUS EX MANKIND DIVIDED Motion Blur 61 v Static Object Velocity Pass  Screen space pass, skip dynamic object pixels from previous pass using stencil buffer  Calculate pixel’s last frame screen position using depth buffer and previous frame’s view-projection matrix
  62. 62. DEUS EX MANKIND DIVIDED Motion Blur 62 Blur  Half or full resolution  Artist velocity exponent, smoothens velocity curve and diminishes motion blur on low velocity targets  To deal with hard edge artifacts, use tile max velocity  Compute max in 20x20 pixel tiles  Compute max in 3x3 tile neighborhood
  63. 63. DEUS EX MANKIND DIVIDED Motion Blur 63 Blur  Blur done in two passes  Low Frequency: blur 4 samples along tile max velocity Tile neighborhood max velocity Base Samples
  64. 64. DEUS EX MANKIND DIVIDED Motion Blur 64 Blur  Blur done in two passes  Low Frequency: blur 4 samples along tile max velocity  High Frequency: blur between samples from first pass Tile neighborhood max velocity Base Samples
  65. 65. DEUS EX MANKIND DIVIDED Motion Blur 65 Blur  1st Pass blur weighted with pixel velocity  2nd Pass can’t use original pixel velocity  1st Pass outputs a comparison value to alpha channel  2nd Pass reads these values and uses them for weighting First pass velocity comparison average First pass blur 4 samples Second pass blur 4 samples
  66. 66. DEUS EX MANKIND DIVIDED Motion Blur 66 Blur Recap  Separate iterative passes give high quality at lower cost.  Total of 8 samples for virtually 16 samples.  Tile max deals with hard edge artifact. Hard edge artifact without Tile neighbor max No hard edge with Tile neighbor max
  67. 67. DEUS EX MANKIND DIVIDED Motion Blur 67 Performance  0.8ms on PS4, includes:  Color buffer downsample  Velocity buffer downsample  Tile max and tile neighborhood max passes  Both separate blur passes  Bilateral upsampling v
  68. 68. DEUS EX MANKIND DIVIDED 68 Heart and Soul v Presented by Anton Kai Michels R&D Programmer for Labs at Eidos Montreal Dominique Demers Director of Photography at Eidos Montreal Philippe Desrosiers Lead Interactive Objects at Eidos Montreal
  69. 69. DEUS EX MANKIND DIVIDED 69 Heart and Soul v Heart of the machine  Deus Ex very interconnected  Living, breathing world  Applies to environments  Interaction increases immersion
  70. 70. DEUS EX MANKIND DIVIDED 70 Heart and Soul v Creating Interaction  Dawn Engine  Artist & game designer created behaviors  Modular System  Quick Iteration  Fast Prototyping  Coherent and standardized
  71. 71. DEUS EX MANKIND DIVIDED 71 Heart and Soul Graph View  Scripting UI  Node based  Display and edit everything  Lego block approach  Connections visualized as lines  Shows active links  Easy to Debug
  72. 72. DEUS EX MANKIND DIVIDED 72 Heart and Soul v Gameplay Objects 3 Geometry Types  Static objects (level geometry)  Physics objects (rigid bodies)  Kinematic objects (bones + animation) All gameplay objects are a mix of geometry and scripting entities.
  73. 73. DEUS EX MANKIND DIVIDED 73 Heart and Soul Pins  Input pins receive trigger signals  Output pins send trigger signals  Property pins expose values  Accessor pins fetch values  Event scripting: left / right  Data scripting: top / bottom
  74. 74. DEUS EX MANKIND DIVIDED 74 Heart and Soul v Signal Types  Various signal types  Void, Bool, Int, Float, Vector, String, Color, etc.  Types are color coded  Makes compatibility explicit Custom Signals  Dawn Engine allows user created signals  Expose signals from child to parent
  75. 75. DEUS EX MANKIND DIVIDED 75 Heart and Soul Composite Objects
  76. 76. DEUS EX MANKIND DIVIDED Heart and Soul Images with Soul  Ambitious artistic direction  Push graphic quality  Push image tangibility  Volumetric lighting + air density  Apply photography to videogames  Tell stories with color v
  77. 77. DEUS EX MANKIND DIVIDED Heart and Soul Photographic Postfilters  Depth of Field with Bokeh  Motion Blur  Volumetric Lighting  Parallax Occlusion Mapping  Screen Space Reflection  Glare + Glow  Lens Effects (Flare, Dust, Scratch) v
  78. 78. DEUS EX MANKIND DIVIDED Heart and Soul Color Language  Color = emotional cues  Meanings vary by culture  Devised own color language based on:  Factions  Metaphors  Story hints  Subtle but noticeable  Coherent + Consistent
  79. 79. DEUS EX MANKIND DIVIDED Heart and Soul Colorscripts  Colors should tell a story  Use color language as base coat  Fluctuate saturation and contrast  Affects brain stimulation and stress  Climatic scenes more saturated and contrasted  Colorscript is a map of these fluctuations “Colorscripts provide a roadmap for shifts in color, lighting, emotion and mood throughout a film.” – Amid Amidi, The Art of Pixar
  80. 80. DEUS EX MANKIND DIVIDED Heart and Soul Color Correction Tool  Adjust multiple aspects of color  3 color zones based on luma curve  2 depth zones: near and far Tool split into 3 parts:  Luma  Color 1  Color 2 109
  81. 81. DEUS EX MANKIND DIVIDED Heart and Soul Two luma curves define the color zone distribution: Shadows, Midtones and Highlights Luma 110
  82. 82. DEUS EX MANKIND DIVIDED Heart and Soul Parameters for each color zone defined by luma curves Adjusts Hue, Saturation, Contrast and Global Tint Color 1 111
  83. 83. DEUS EX MANKIND DIVIDED Heart and Soul More color zone parameters. These can affect RGB individually Adjusts Level, Brightness, Contrast and Color Temperature Color 2 112
  84. 84. DEUS EX MANKIND DIVIDED Heart and Soul Foreground & Background  Each can be affected separately  Full color correction  Adjustable for photography needs  Distances set by lighting artists v

×