1.
The Rendering Technology of
Lords of the Fallen
Philip Hammer

2.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Intro
● Deck13 Interactive is one of the leading game developers in Germany
○ In-house multiplatform technology FLEDGE
(PS4, XB1, PC, PS3, XB360, iOS)
○ Previously: Venetica, Jack Keane, Blood Knights, Tiger&Chicken, etc.

3.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Intro
● Currently working on “Lords of the Fallen”
○ 3rd-person challenging Action-RPG
○ Platforms PS4, XB1, PC
○ Development together with CI Games
○ Release End of 2014

4.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Development
● LotF started as a PS3/XB360 title
● Gradually shifted to (back then) “Next-Gen”
○ Lot of changes in content
● Tech challenges
○ Switch from Light-Prepass to Deferred Rendering
○ Drop of DX9-support unlocked a lot of freedom
○ Feature a lot more multithreading

5.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Overview
● Our Deferred Rendering Approach
● Deferred Material Parameter Buffer
● Deferred Particle Lighting
● Short Teaser
○ Transparent Shadows
○ Volumetric Lighting
○ GPU Particles

6.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Our Deferred Rendering Approach
● Switch from Light-Prepass Rendering to more traditional Deferred
Rendering
○ Getting rid of the second geometry-pass
○ Overhead for Geometry and API was horrible on consoles
○ Memory amount and bandwidth is good nowadays
■ can afford a lot more rendertarget memory
○ Had to move a lot effects from object-shaders to deferred stages
(reflections, fog, etc.)

7.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Our Deferred Rendering Approach
● Want have more freedom in lighting
○ IBL / localized cubemaps for ambient specular
○ Translucency should *not* be a hacky corner case anymore
○ Transparent objects should receive lighting and cast transparent shadows
○ Dynamic Global Illumination
■ We use dynamic directional irradiance lightmaps generated by Geomerics
Enlighten© Middleware

8.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Our Deferred Rendering Approach
● Render minimal G-Buffer
○ 2x RGBA8_UNORM +
R11G11B10_FLOAT + D24S8 = 128 bpp
● Render L-Buffer
○ 2x R11G11B10_FLOAT
○ separate diffuse + specular lighting
● Render deferred reflections
○ Several Environment-Probes
Cubemaps + Screenspace Reflections
● Render combine pass
○ Composite Albedo, Spec-Color, Lighting, Reflections
● Render transparent objects
● Render deferred fog
● Render postprocessing

9.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
G-Buffer Layout
● Should be still as small as possible
● Parameter encoding
○ 2-channel normals
○ Compact YCoCg for albedo + specular-color [Mavridis12]
■ Interleave chroma component
○ Parameter packing
■ Pack single bits and use lower precision for some attributes
○ Share channels for mutual exclusive parameters
● Material-ID/-Index for referencing per-material parameters
○ more on this later
● 128 bpp (31,64 MB @ 1080p) even fits into XB1 ESRAM

10.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
G-Buffer Layout
RED GREEN BLUE ALPHA
MRT0
RGBA8_UNORM
Encoded Normal Material-
ID / Index
Specular
Exponent
MRT1
RGBA8_UNORM
Albedo
Luma
Albedo
Chroma
Spec.
Luma
Spec. Chroma /
Translucency
L-Buffer (Diffuse)
R11G11B10_F
Irradiance Lightmap -
Depth/Stencil
D24S8
Depth Stencil
● Colored Specular and Translucency are mutual
exclusive!
○ Determined by Material-ID

11.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
G-Buffer Layout
RED GREEN BLUE ALPHA
MRT0
RGBA8_UNORM
Encoded Normal Material-
ID / Index
Specular
Exponent
MRT1
RGBA8_UNORM
Albedo
Luma
Albedo
Chroma
Spec.
Luma
Spec. Chroma /
Translucency
L-Buffer (Diffuse)
R11G11B10_F
Irradiance Lightmap -
Depth/Stencil
D24S8
Depth Stencil
● 1 Bit Material-ID determines whether MRT1.a
contains Translucency or Spec. Chroma.
● 7 Bit Material-Index used to reference Material
Parameter Buffers

12.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
G-Buffer Layout
RED GREEN BLUE ALPHA
MRT0
RGBA8_UNORM
Encoded Normal Material-
ID / Index
Specular
Exponent
MRT1
RGBA8_UNORM
Albedo
Luma
Albedo
Chroma
Spec.
Luma
Spec. Chroma /
Translucency
L-Buffer (Diffuse)
R11G11B10_F
Irradiance Lightmap -
Depth/Stencil
D24S8
Depth Stencil
● 1 Bit Material-ID determines whether MRT1.a
contains Translucency or Spec. Chroma.
● 7 Bit Material-Index used to reference Material
Parameter Buffers
// extract material-id from g-buffer
float unpackMaterialID ( float packedIndexID )
{
return floor( ( packedIndexID * 255.0 ) / 128.0 ) ;
}
// extract material-index from g-buffer
uint unpackMaterialIndex ( float packedIndexID )
{
const uint bufferElements = 4 ;
uint index = uint( packedIndexID * 255.0 ) & 0x7f ;
return index * bufferElements ;
}

13.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
G-Buffer Layout
RED GREEN BLUE ALPHA
MRT0
RGBA8_UNORM
Encoded Normal Material-
ID / Index
Specular
Exponent
MRT1
RGBA8_UNORM
Albedo
Luma
Albedo
Chroma
Spec.
Luma
Spec. Chroma /
Translucency
L-Buffer (Diffuse)
R11G11B10_F
Irradiance Lightmap -
Depth/Stencil
D24S8
Depth Stencil
● Stencil used for masking
○ 3 Bits for Decals, Reflections, Skin/SSSSS
● 2 Bits Light-Volume Stenciling

14.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
G-Buffer Layout
● Other Attributes affecting only a few objects will be rendered in a
separate pass
○ Tangents for smooth alpha-test
○ Per-object rimlights, emissive/glow & other FX
○ Motion-Vectors of moving objects for motionblur
■ vectors for static geometry are computed by reprojecting the depthbuffer
(“camera-motionblur”)
○ Distortion-Vectors for Post-Distort

15.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Material Parameter Buffer (MP-Buffers)
● What to do with additional material parameters when G-Buffer is full?
○ Increasing G-Buffer size not always an option
○ Information is sometimes only needed per-material, not necessarily per-
pixel frequency
● Examples
○ Reflection-Parameters (Strength, Fresnel)
○ Subsurface-Scattering Parameters (Distortion, Tint)
○ etc.

16.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Material Parameter Buffer (MP-Buffers)
● Our solution: Material Parameter Buffers
● Generate buffer every frame on CPU and bind to lighting-/reflection-
shader
○ Lookup Table for material parameters
■ DX9: multiple 1D RGBA8 textures
■ DX11: structured buffer
○ Stores parameters from material
○ Indexed by Material-Index (G-Buffer)
■ can index up to 128 different materials per frame
● 7 Bit, because 1 Bit reserved for Material-ID
■ determine for each drawcall a set of parameters
● Smart value quantization and interleaving
● Allows a lot more material variety for deferred rendering

17.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
MP-Buffers Example
● Subsurface-scattering / backscattering inspired by [Brisebois12]
● Some problems in deferred rendering
○ Too many per-surface parameters
○ Parametrization per light can work, but inflexible (e.g. white light
filters through differently colored objects)
● We store a single scalar translucency value per-pixel
● Other parameters (e.g. fresnel parameters and tint) are stored in MP-
Buffers and can therefore only vary per material

18.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
● Subsurface-scattering / backscattering inspired by [Brisebois12]
MP-Buffers Example
G-Buffer MRT0.a = Material-ID/-Index
Translates to LUT / structured buffer
Material-
Index
sssDistortion sssPower sssMaterialColor …
0 4.5 2.4 0.43, 0.39, 0.84 …
1 8.0 3.5 0.22, 0.51, 0.52 …
2 5.6 4.1 0.41, 0.43, 0.11 …
... ... ... ... ...

19.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
● Subsurface-scattering / backscattering inspired by [Brisebois12]
MP-Buffers Example
// deferred light-volume fragment shader
Buffer<float4> parameterBuffer ;
uint materialIndex = unpackMaterialIndex(gbuffer0Value . a);
float4 matParamBuffer0 = parameterBuffer[materialIndex];
float4 matParamBuffer1 = parameterBuffer[materialIndex + 1];
float sssDistortion = matParamBuffer0 . x ;
// backscattering according to [Brisebois12]
float3 sssLightVector = L + N * sssDistortion ;
float sssDot = exp2 ( saturate ( dot ( V, -sssLightVector ) ) * sssPower - sssPower ) * sssScale ;
float3 sssColor = ( sssDot * sssMaterialColor * sssLightColor ) * backscattering_mask ;
Parameters per Light (cbuffer)
Parameters per Material (mp-buffer)
Parameters per Pixel (g-bufer)
Material-
Index
sssDistortion sssPower sssMaterialColor …
0 4.5 2.4 0.43, 0.39, 0.84 …
1 8.0 3.5 0.22, 0.51, 0.52 …
2 5.6 4.1 0.41, 0.43, 0.11 …
... ... ... ... ...

20.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Lighting
● 4 different light-types: directional, point, spot, box (local directional)
○ currently around dynamic 500 lights per level
○ most of them featuring animated gobo-textures & shadow-casting
● Classic light-volume accumulation into L-Buffer
○ With optional double-sided stencil masking for large lights
○ Tiled Deferred is currently under development

21.
Digital Dragons 2014
Lighting
● Diffuse L-Buffer
○ Indirect lighting
○ Filled in G-Buffer stage
○ Directional irradiance lightmaps for static
geometry
○ SH lightprobes for dynamic objects
○ We use Geomerics Enlighten

22.
Digital Dragons 2014
Lighting
● Diffuse L-Buffer
○ Accumulate direct diffuse +
translucent light

23.
Digital Dragons 2014
Lighting
● Specular L-Buffer
○ Accumulate specular light

24.
Digital Dragons 2014
Lighting
● Reflection Buffer
○ Shared with specular L-Buffer
○ Static reflections (indirect specular lighting)
○ IBL with localized, parallax corrected cubemaps [Largarde12]

25.
Digital Dragons 2014
Lighting
● Reflection Buffer
○ Shared with specular L-Buffer
○ Static reflections (indirect specular lighting)
○ IBL with localized, parallax corrected cubemaps [Largarde12]
● Environment-Probes
○ Create 6 “screenshots” (raw RGBA16_F) with 90° Fov and 1:1
aspect. Skip postprocessing, gui etc.
○ Make cubemap out of 6 screenshots (256x256x256 / BC6)
○ Use ATI CubemapGen API to filter mipmap-chain + Edge-Fixup
○ Doesn’t match BRDF, but looks good with some artist tweaks
● All reflections done in a single fullscreen pass [Drobot12]
○ Gather 8 most influential env-probes and check if inside one or
more env-probe AABB

26.
Digital Dragons 2014
Lighting
● Final Frame
○ Composite
○ Volumetric Light Scattering
○ Transparent Objects
○ Postprocessing

27.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Deferred Particle Lighting
● We wanted to have lit particles
○ Better integration into scene
○ Reuse particle effects under different lighting conditions or in shadow

28.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Deferred Particle Lighting
● Common type of particles: atmospheric, smoke, dust & fog effects
○ Usually transparent, dense ... and cause a lot of overdraw
○ Cost explosion when lighting every fragment
○ Non-directional, low-frequency information
○ Perfect for vertex-lighting
● Deferred vertex-lighting
○ No compromise in number of affecting lights and shadows
○ Fits perfectly into deferred pipeline

29.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Deferred Particle Lighting: G-Buffer
● Step 1: render a non-screenspace particle “G-Buffer”
○ Basically a sequential list
○ Render particle vertex-buffer as point-list
(D3D11_PRIMITIVE_TOPOLOGY_POINTLIST)
○ Rasterize viewspace-positions of particles subsequently into a buffer
■ Currently a 1024x512 RGBA16F texture for all visible lit particles
○ G-Buffer currently contains only vertex-positions
■ Could be extended with more surface attributes like translucency or
normals (which doesn’t make too much sense with billboard-particles)

30.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Deferred Particle Lighting: G-Buffer
● Step 1: render a non-screenspace particle “G-Buffer”
○ Compute the next list index in vertex shader and rasterize
■ Based on current SV_VertexID and the previously rendered particles count
as shader constant vertexCountPerInstance
■ When using instanced particles, also factor in SV_InstanceID
// vertex-shader
// pos = SV_Position
float4 pos = 1.0 ;
// vertexId = SV_VertexID, instanceId = SV_InstanceID
// param_vertexcount = shader constant holding the currently rendered number of vertices
float vertexIndex = vertexId + ( instanceId * vertexCountPerInstance) + param_vertexcount ;
float rasterPosY = trunc ( vertexIndex / textureWidth ) ;
float rasterPosX = trunc ( vertexIndex - rasterPosY * textureWidth ) ;
pos . x = ( ( rasterPosX * 2.0 ) - textureWidth ) / textureWidth ;
pos . y = ( ( rasterPosY * 2.0 ) - textureHeight ) / textureHeight ;
pos . zw = 1.0 ;

31.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Deferred Particle Lighting: Lighting
● Step 2: Apply lighting similar to normal deferred lighting
○ Use fullscreen-quads instead of light-volumes, because the fragments
have no spatial relationship (Non-Screenspace Fragment-List)
○ Different light-types, shadows, projected textures, etc. “just works”
○ Very simple “lighting model”
○ Shadowing: PCF with large kernel width reduces flickering in shadow/light
transitions
○ Optimization: Stencil-Mask each fragment in Per-Vertex G-Buffer
// pixel shader
// trivial implementation (different for each light type):
// - sample g-buffer containing the viewspace positions of particle vertices
// - compute light attenuation base on position and light parameters (...)
float3 positionVS = SAMPLE ( perVertexGBufferSampler, screenUV . xy ) . rgb ;
col0 . rgb = light_color . rgb * getLightAttenuation ( positionVS ) * getShadowing ( positionVS ) ;

32.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Deferred Particle Lighting: Material
● Step 3: Render particles with lighting applied
○ Take care of exact rendering order (same as G-Buffer stage)
○ Sample particle L-Buffer in vertexshader
○ Interpolate lighting to fragment shader and modulate with particle texture
// vertex-shader
// vertexId = SV_VertexID, instanceId = SV_InstanceID
// param_vertexcount = shader constant holding the currently rendered number of vertices
float vertexIndex = vertexId + ( instanceId * vertexCountPerInstance ) + param_vertexcount ;
float3 perVertexLightBufferUV = 0;
perVertexLightBufferUV . y =trunc ( vertexIndex / textureWidth ) ;
perVertexLightBufferUV . x =trunc ( vertexIndex - perVertexLightBufferUV . y * textureWidth ) ;
perVertexLightBufferUV . y = ( textureHeight - 1.0 ) - perVertexLightBufferUV . y ;
float4 lbufferValue = perVertexLightBufferTexture .Load ( (int3)perVertexLightBufferUV . xyz ) ;
// pixel-shader
col0 . rgb = particleTextureColor . rgb * f_in . lbufferValue . rgb ;

33.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Deferred Particle Lighting - Conclusion
● Great for many types of particles
○ Cheap and effective
● Inherits problems of all per-vertex techniques
○ Low spatial resolution
● Works best with small particles
● Very small and bright lights can lead to flickering if only one corner of
particle gets suddenly lit
● Be wary of buffer overflow with many lit particles

34.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Transparent Shadows
● Shadowmask for PSSM
○ Accumulate cascades in screenspace
○ Depth-preserving gauss-blur
○ Can accumulate up to 4 different monochromatic shadow-sources
● Each light can also cast colored transparent shadows
○ Simple approach:
■ Render certain objects alpha-blended into shadowmap-sized RGBA8 buffer
■ Use shadowmap as depth buffer
■ Sample RGBA8 buffer with shadowmap uv and multiply with lighting
■ Works seamless with volumetric lights and particle lighting
○ Problems:
■ Shadowmask reduced to 1 shadow-source
■ Self-shadowing not correct, but normally not an issue

35.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
● All lights can cast depth-tested transparent shadows
Transparent Shadows

36.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Transparent Shadows
● Works also with volumetric lights, transparent particles, etc.

37.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Volumetric Lighting
● For details, see Benjamin Glatzels talk yesterday
● Raymarching in light’s view space while evaluating the shadowmap
○ Also evaluate projector-texture, transparent shadows, IES light-profiles,
noise volume-texture, etc
● Based on the technique presented by [Toth09]

38.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Volumetric Lighting

39.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
GPU Particles
● Yeah, we have GPU particles as well
● Compute Shader-based simulations of large amounts of particles
○ Screenspace collisions (test against depthbuffer)
○ Lots of different emitter types
○ Driven by prebaked 3D vector fields and other force-field types
○ Topic would fill another 45 minutes, so just a teaser

40.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
GPU Particles

41.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
Things for the future
● Physical Based Rendering
○ We already have a PBR renderpath, but a lot of content was already
done, so it didn’t make it into LotF
● Re-introduce a pure z-prepass
○ Z-Layout done before G-Buffer
○ Reduces Overdraw and can use implicit alpha-test per depth-test=equal
○ Helps with decals
● Tools, tools tools
○ Optimized workflows, better tools and therefore faster iteration times are
more important than more funky graphics effects

42.
Digital Dragons 2014
@philiphammer0
phammer@deck13.com
http://www.deck13.com
Thanks for listening!
...and special thanks to the amazing Deck13 tech-team

43.
Philip Hammer, The Rendering Technology of ‘Lords of the Fallen’
References
[Mavridis12] "The Compact YCoCg Frame Buffer", Journal of Computer Graphics Techniques, Vol. 1, No. 1, 2012
[Lagarde12] “Local Image-based Lighting With Parallax-corrected Cubemap”, Siggraph, 2012
[Brisebois12] “Approximating Translucency for a Fast, Cheap and Convincing Subsurface Scattering Look”, GDC 2011
http://dice.se/wp-content/uploads/Colin_BarreBrisebois_Programming_ApproximatingTranslucency.pdf
[Drobot12] "Lighting of Killzone: Shadow Fall"
http://de.slideshare.net/guerrillagames/lighting-of-killzone-shadow-fall
[Toth09] “Real-time Volumetric Lighting in Participating Media”, Eurographics 2009
http://sirkan.iit.bme.hu/~szirmay/lightshaft.pdf