The document presents a comprehensive guide on post-processing techniques in rendering, covering topics such as gamma control, high-dynamic range rendering (HDR), color filters, and various visual effects like motion blur and light streaks. It delves into the mathematics and implementations of these effects, highlighting the importance of techniques like tone mapping and depth of field to achieve desired visual outcomes. Additionally, it discusses the challenges in rendering pipelines and optimization methods to enhance image quality while managing computational resources.

1. A Post-Processing Pipeline
Wolfgang Engel
Confetti Special Effects Inc., Carlsbad
Paris Master Class

2. Agenda
• PostFX
– Color Filters
– High-Dynamic Range Rendering
– Light Streaks
– Depth of Field
– Motion Blur
– Miscellaneous Effects Snippets
• References

3. Basics
• PostFX:
– Image space is cheaper: do 3D stuff in 2D
– Control color, color quality and color range
• Lots of render-target to render-target blits

4. Gamma control
• Problem: most color manipulation methods
applied in a renderer become physically non-
linear
• RGB color operations do not look right (adds up)
[Brown]
• dynamic lighting
• several light sources
• shadowing
• texture filtering
• alpha blending
• advanced color filters

5. Gamma control
• Gamma is compression method because 8-bit
per channel is not much precision to
represent color
– Non-linear transfer function to compress color
– Compresses into roughly perceptually uniform
space
-> called sRGB space [Stokes]
• Used everywhere by default for render targets
like the framebuffer and textures

6. Gamma Control
• We want: renderer without gamma
correction == gamma 1.0
• Art Pipeline is most of the time running
gamma 2.2 everywhere
• ->convert from gamma 2.2 to 1.0 while
fetching textures and color values and back
to gamma 2.2 at the end of the renderer

7. Gamma control
• From sRGB to linear gamma
float3 Color; Color = ((Color <= 0.03928) ? Color / 12.92 : pow((Color + 0.055) / 1.055, 2.4))

8. Gamma control
• From linear gamma to sRGB
float3 Color; Color = (Color <= 0.00304) ? Color * 12.92 : (1.055 * pow(Color, 1.0/2.4) - 0.055);

9. Gamma Control
• Converting to gamma 1.0 [Stokes]
Color = ((Color <= 0.03928) ? Color / 12.92 : pow((Color + 0.055) / 1.055, 2.4))
• Converting to gamma 2.2
Color = (Color <= 0.00304) ? Color * 12.92 : (1.055 * pow(Color, 1.0/2.4) - 0.055);
• Hardware can convert textures and the end
result… but some hardware uses linear
approximations here
• Vertex colors still need to be converted “by
hand”

10. Color Filters
• Oldest post effects in games -> Call of Duty /
Medal of Honor
• For example:
– Winter scene -> grey-ish tint
– Jungle scene -> greenish / blueish tint
• Color changes based on the area where the
player is

11. Color Filters: Saturation
• Remove color from image ~= convert to
luminance [ITU1990]
• Y of XYZ color space represents luminace
float Lum = dot(Color,float3(0.2126, 0.7152, 0.0722));
float3 Color = lerp(Lum.xxx, Color, Sat)

12. Color Filters: Contrast
• Brain determines color of objects with color of
the surrounding of the object
• Cubic Polynomial

13. Color Filters: Color Correction
• Simplest color filters just add or multiply RGB
values
float3 Color = Color * ColorCorrect * 2.0 + ColorAdd;

14. HD RR
• High-Dynamic Range Rendering
• Why?
– Real world:
• Starlight -> sun-lit snow up to 1012
:1 is viewable
• Shadows to hightlights in a normal scene up to 104
:1
– Human visual system can cover around 103
:1 at a
particular exposure
-> in order to see a wider range it adjusts exposure to
light automatically (light adaptation)
– Monitor can only cover a dynamic range of 102
:1

15. HD RR: Requirements
• High-Dynamic Range Data -> larger than 0..1 and
more deltas than 8:8:8:8
– Textures: how to compress / gamma correct HDR
textures
– Render Targets: how to keep high-dynamic range data
• Tone Mapping -> compresses back to low-dynamic
range for monitor
• Light Adaptation -> similar to human visual system
• Glare == Bloom -> overload optic nerve of eye
• Tone Mapping for the Night -> human visual system
under low lighting conditions

16. HD RR
• Ansel Adam’s Zone System [Reinhard]

17. HD RR
• Data with higher range than 0..1
• Storing High-Dynamic Range Data in Textures
– RGBE - 32-bit per pixel
– DXGI_FORMAT_R9G9B9E5_SHAREDEXP - 32-bit per pixel
– DXT1 + quarter L16 – 8-bit per pixel
– DXT1: storing common scale + exponent for each of the color channels in
a texture by utilizing unused space in the DXT header – 4-bit per-pixel
– -> Challenge: gamma control -> calc. exp. without gamma
– DX11 has a HDR texture but without gamma control
• Keeping High-Dynamic Range Data in Render Targets
– 10:10:10:2 (DX9: MS, blending, no filtering)
– 7e3 format XBOX 360: configure value range & precision with color exp.
Bias [Tchou]
– 16:16:16:16 (DX9: some cards: MS+blend others filter+blend)
– DX10: 11:11:10 (MS, source blending, filtering)

18. HD RR
• HDR data in 8:8:8:8 Render Targets
• *based on Greg Wards LogLuv model
• RGB12A2 for primary render target:
• Two 8:8:8:8 render targets
• 1-8 bits in render target 0 / 4 – 12 bits in render target 1
• Overlap of 4 bits for alpha blending
High-Dynamic Range Rendering
Color
Space
# of cycles
(encoding)
Bilinear
Filtering
Blur Filter Alpha
Blending
RGB - Yes Yes Yes
HSV ~34 Yes No No
CIE Yxy ~19 Yes Yes No
L16uv* ~19 Yes Yes No
RGBE ~13 No No No

19. HD RR
• Tone mapping operator to compress HDR to LDR
– Luminance Transform
– Range Mapping

20. HD RR
• Convert whole screen to an average
luminance
• Logarithmic average not arithmetic average ->
non-linear response of the eye to a linear
increase in luminance

21. HD RR
• To convert RGB to Luminance [ITU1990]
• RGB->CIE XYZ->CIE Yxy
• CIE Yxy->CIE XYZ->RGB

22. HD RR
• Simple Tone Mapping Operator [Reinhard]
• Scaling with MiddleGrey
• Map range from 0..1

23. HD RR
• Advanced Tone Mapping Operator
• Artistically desirable to burn out bright areas
• Source art not always HDR
• Leaves 0..1

24. HD RR
• Light Adaptation
• Re-use luminance data to mimic light adaptation of the
eye -> cheap
• Temporal changes in lighting conditions
» Day -> Night: Rods ~30 minutes
» Outdoor <-> Indoor: Cones ~few seconds
• Game Scenarios:
» Outdoor <-> Indoor
» Weather Changes
» Tunnel drive

25. HD RR
• Exponential decay function [Pattanaik]
• Frame-rate independent
• Adapted luminance chases average luminance
– Stable lighting conditions -> the same
• tau interpolates between adaptation rates of
cones and rods
• 0.2 for rods / 0.4 for cones

26. HD RR
• Luminance History function [Tchou]
• Even out fast luminance changes (flashes etc.)
• Keeps track of the luminance of the last 16 frames
• If the last 16 values >= || < current adapted luminance
-> run light adaptation
• If some of the 16 values are going in different
directions
-> no light adaptation
• Runs only once per frame -> cheap

27. HD RR
• Glaring

28. HD RR
• Glaring
– Intense lighting -> optic nerve of the eye
overloads
• Bright pass filter
• Gaussian convolution filter to bloom

29. HD RR
• Bright pass filter
• Compresses dark pixels leave bright pixels
Threshold 0.5 Offset = 1.0
• Same tone mapping operator as in tone mapping
-> consistent

30. HD RR
• Gauss filter
• σ - standard deviation
• x, y coordinates relativ
to center of filter kernel

31. HD RR
• Scotopic View
• Contrast is lower
• Visual acuity is lower
• Blue shift
• Convert RGB to CIE XYZ
• Scotopic Tone Mapping Operator [Shirley]
• Multiply with a grey-bluish color

32. Depth of Field

33. Depth of Field

34. Depth of Field
• Range of acceptable sharpness == Depth of
Field see [Scheuermann] and [Gillham]
• Define a near and far blur plane
• Everything in front of the near blur plane and
everything behind the far blur plane is blurred

35. Depth of Field
• Convert depth buffer values into camera
space
• Multiply vector with third column of proj.
matrix
• x.2 shows how to factor in / w here w = z
• x.3 result

36. Depth of Field
• Applying Depth of Field
• Convert to Camera Z == pixel distance from
camera
float PixelCameraZ = (-NearClip * Q) / (Depth - Q);
• Focus + Depth of Field Range [DOFRM]
lerp(OriginalImage, BlurredImage, saturate(Range * abs(Focus - PixelCameraZ)));
-> Auto-Focus effect possible
• Color leaking: change draw order or ignore it

37. Light Streaks

38. Light Streaks
• Light scattering through a lens, eye lens or
aperture
-> cracks, scratches, grooves, dust and lens
imperfections cause light to scatter and diffract
• Painting light streaks would be done with a
brush by pressing a bit harder on the common
point and releasing the pressure while moving
the brush away from this point
->Light streak filter “smears” a bright spot into
several directions [Kawase][Oat]

39. Light Streaks
• Sample Pattern
• Masaki Kawase [Kawase] -> 4-tap == XBOX

40. Light Streaks
• Results
Left the scene image | intermediate result | right result from two streaks

41. Light Streaks
• Sample Pattern
• Next-Gen (Developed by Alex Ehrath)-> 7 tap
filter kernel

42. Light Streaks
• Do 4 – 6 passes to achieve cross like streak
pattern
• To adjust the strength of the streaks the
samples are weighted differently with the
following equation

43. Light Streaks
• Direction for the streak is coming from
orientation vector [Oat] like this
• b holds the power value of four or seven – shown on previous slide
• TxSize holds the texel size (1.0f / SizeOfTexture)

44. Light Streaks
float4 Color = 0.0f;
// get the center pixel
// Weights represents four w values. It is calculated on the application level and send to the pixel shader
Color = saturate(Weights.x) * tex2D(RenderMapSampler, TexCoord.xy);
// setup the SC variable
// the streak vector and b is precomputed and send down from the application level to the pixel shader
float2 SC = LightStreakDirection.xy * b * TexelSize.xy;
// take sample 2 and 4
Color += saturate(Weights.y) * tex2D(RenderMapSampler, TexCoord.xy + SC);
Color += saturate(Weights.y) * tex2D(RenderMapSampler, TexCoord.xy - SC);
// take sample 1 and 5
SC += SC;
Color += saturate(Weights.z) * tex2D(RenderMapSampler, TexCoord.xy + SC);
Color += saturate(Weights.z) * tex2D(RenderMapSampler, TexCoord.xy - SC);
// take sample 0 and 6
SC += SC;
Color += saturate(Weights.w) * tex2D(RenderMapSampler, TexCoord.xy + SC);
Color += saturate(Weights.w) * tex2D(RenderMapSampler, TexCoord.xy - SC);
return saturate(Color);

45. Light Streaks
• Picking antipodal texture coordinates is for
optimization and coolness 
• A light streak filter probably runs on the same
render target that is used to bloom the scene

46. Motion Blur

47. Motion Blur
• Mainly two ways to motion blur (… there are much more )
– Geometry Stretching
– Velocity Vector Field

48. Motion Blur
• Geometry Stretching [Wloka]
• Create a motion vector in view or world-space Vview
• Compare this vector to the normal in view or world-
space
– If similar in direction -> the current position value is
chosen to transform the vertex
– Otherwise the previous position value

49. Motion Blur
// transform previous and current pos
float4 P = mul(WorldView, in.Pos);
float4 Pprev = mul(prevWorldView, in.prevPos);
// transform normal
float3 N = mul(WorldView, in.Normal);
// calculate eye space motion vector
float3 motionVector = P.xyz - Pprev.xyz;
// calculate clip space motion vector
P = mul(WorldViewProj, in.Pos); // current position
Pprev = mul(prevWorldViewProj, in.prevPos); // previous position
// choose previous or current position based
// on dot product between motion vector and normal
float flag = dot(motionVector, N) > 0;
float4 Pstretch = flag ? P : Pprev;
out.hpos = Pstretch;

50. Motion Blur
• Velocity Vector Field
• Store a vector in homogenous post-projection space
(HPPS) in a render target like this
• For this the view and world matrices for all objects
from the previous frame need to be stored
// do divide by W -> NDC coordinates
P.xyz = P.xyz / P.w;
Pprev.xyz = Pprev.xyz / Pprev.w;
Pstretch.xyz = Pstretch.xyz / Pstretch.w;
// calculate window space velocity
out.velocity = (P.xyz - Pprev.xyz);

51. Motion Blur
• Alternatively re-construct the previous position from
the depth buffer
• Shortcoming pixel velocity is calculated per-vertex
and interpolated across primitives -> not very
precise

52. Motion Blur
float2 velocity = tex2D(VelocityMapSampler, in.tex.xy);
static const int numSamples = 8;
float2 velocityStep = velocity / numSamples;
float speed = length(velocity);
float3 FullscreenSample = tex2D(FullMapSampler, IN.tex.xy);
// Look up into our jitter texture using this 'magic' formula
static const half2 JitterOffset = { 58.164f, 47.13f };
const float2 jitterLookup = IN.tex.xy * JitterOffset + (FullscreenSample.rg * 8.0f);
JitterScalar = tex2D(JitterMapSampler, jitterLookup) - 0.5f;
IN.tex.xy += velocityStep * JitterScalar;
for (int i = 0; i < numSamples / 2; ++i)
{
float2 currentVelocityStep = velocityStep * (i + 1);
sampleAverage += float4(tex2D(QuarterMapSampler, IN.tex.xy + currentVelocityStep), 1.0f);
sampleAverage += float4(tex2D(QuarterMapSampler, IN.tex.xy - currentVelocityStep), 1.0f);
}
float3 outColor = sampleAverage.rgb / sampleAverage.w;
float2 normScreenPos = IN.tex.xy;
float2 vecToCenter = normScreenPos - 0.5f;
float distToCenter = pow(length(vecToCenter) * 2.25f, 0.8f);
float3 desat = dot(outColor.rgb,float3(0.3f,0.59f,0.11f));
outColor = lerp(outColor, desat, distToCenter * blurEffectDesatScale);
float3 tinted = outColor.rgb * blurEffectTintColor;
outColor.rgb = lerp(outColor.rgb, tinted.rgb, distToCenter * blurEffectTintFade);
return float4(outColor.rgb, velMag);

53. Sepia
• Different than saturation control
• Marwan Ansari [Ansari] developed an efficient way
to convert to Sepia
• Basis is the YIQ color space
• Converting from RGB to YIQ with the matrix M
• Converting from YIQ to RGB is M’s inverse

54. Sepia

55. Sepia
• To optimize the conversion from RGB to YIQ and vice versa ->
reduce to the luminance value Y
-> dot product between the RGB value and the first row of
the RGB-To-YIQ matrix
...
float3 IntensityConverter={0.299, 0.587, 0.114};
Y = dot(IntensityConverter, Color);
...
...
float4 sepiaConvert ={0.191, -0.054,-0.221,0.0};
return Y + sepiaConvert;
...

56. Film Grain

57. Film Grain
• Use a 3D volume texture of 64x64x64 or 32x32x32 that holds
random noise by running through its z slices based on a time
counter (check out RenderMonkey example ScreenSpace
Effects.rfx -> old TV)
...
// pos has a value range of -1..1
// the texture address mode is wrapped
float rand = tex3D(RandomSampler, float3(pos.x, pos.y, counter));
float4 image = tex2D(ImageSampler, Screenpos.xy);
return rand + image;
...

58. Frame Border

59. Frame Border
• Applying a power function to the extrema of the
screen + clamp to 0..1 (check out RenderMonkey
example ScreenSpace Effects.rfx -> old TV)
...
float f = (1-pos.x*pos.y) * (1-pos.y*pos.y);
float frame = saturate(frameSharpness * (pow(f, frameShape) - frameLimit));
return frame * (rand + image);
...
• frameSharpness makes the transition between the frame and
the visible scene softer
• frameshape makes the shape of the frame rounder or more
edgy
• framelimit makes the frame bigger or smaller

60. Median Filter
• Used to remove “salt and pepper noise” from a picture
[Mitchell]
• Can be used to reduce color quality -> cheap camera with
MPEG compression
• Picks the middle value

61. Median Filter
float FindMedian(float a, float b, float c)
{
float Median;
if(a < b)
{
if(b<c)
Median = b;
else Median = max(a, c);
}
else
{
if(a < c) Median = a;
else Median = max(b, c);
}
return Median;
}
Vectorized:
float3 FindMedian(float3 RGB,float3 RGB2, float3 RGB3)
{
return (RGB < RGB2) ? ((RGB2 < RGB3) ? RGB2 : max(RGB,RGB3) ) : ((RGB < RGB3) ? RGB :
max(RGB2,RGB3));
}

62. Interlace Effect
• Figure out every second line with frac()
float fraction = frac(UnnormCoord.y * 0.5);
if (fraction)
… // do something with this line
else
… // do something different with every second line

63. Implementation
Depth of Field
Tone Mapping
Color Filters
Gamma Control
Frame-Buffer
16x16 RT
Luminance
4x4 RT
Luminance
1x1 RT
Luminance
64x64 RT
Luminance
1x1 RT
Adapt Lum.
Measure Luminance
Adapt Luminance
Primary Render Target
Bright
Pass Filter
Z-Buffer
Downsample
½ Size
Downsample
¼ Size
Gauss Filter
Gauss Filter II

64. References
[Ansari] Marwan Y. Ansari, "Fast Sepia Tone Conversion", Game Programming Gems 4, Charles River Media, pp 461 - 465, ISBN 1-58450-295-9
[Baker] Steve Baker, "Learning to Love your Z-Buffer", http://sjbaker.org/steve/omniv/love_your_z_buffer.html
[Berger] R.W. Berger, “Why Do Images Appear Darker on Some Displays?”, http://www.bberger.net/rwb/gamma.html
[Brown] Simon Brown, “Gamma-Correct Rendering”, http://www.sjbrown.co.uk/?article=gamma
[Caruzzi] Francesco Carucci, "Simulating blending operations on floating point render targets", ShaderX2
- Shader Programming Tips and Tricks with DirectX 9, Wordware Inc., pp 172 - 177, ISBN 1-
55622-988-7
[DOFRM] Depth of Field Effect in RenderMonkey 1.62
[Gilham] David Gilham, "Real-Time Depth-of-Field Implemented with a Post-Processing only Technique", ShaderX5
: Advanced Rendering, Charles River Media / Thomson, pp 163 - 175, ISBN 1-
58450-499-4
[Green] Simon Green, "Stupid OpenGL Shader Tricks", http://developer.nvidia.com/docs/IO/8230/GDC2003_OpenGLShaderTricks.pdf"
[ITU1990] ITU (International Tlecommunication Union), Geneva. ITU-R Recommendation BT.709, Basic Parameter Values for the HDTV Standard for the Studio and for International Programme
Exchange, 1990 (Formerly CCIR Rec. 709).
[Kawase] Masaki Kawase, "Frame Buffer Postprocessing Effects in DOUBLE-S.T.E.A.L (Wreckless)", http://www.daionet.gr.jp/~masa/
[Kawase2] Masumi Nagaya, Masaki Kawase, Interview on WRECKLESS: The Yakuza Missions (Japanese Version Title: "DOUBLE-S.T.E.A.L"), http://spin.s2c.ne.jp/dsteal/wreckless.html
[Krawczyk] Grzegorz Krawczyk, Karol Myszkowski, Hans-Peter Seidel, "Perceptual Effects in Real-time Tone Mapping", Proc. of Spring Conference on Computer Graphics, 2005
[Mitchell] Jason L. Mitchell, Marwan Y. Ansari, Evan Hart, "Advanced Image Processing with DirectX 9 Pixel Shaders", ShaderX2 - Shader Programming Tips and Tricks with DirectX 9, Wordware
Inc., pp 439 - 464, ISBN 1-55622-988-7
[Oat] Chris Oat, "A Steerable Streak Filter", , ShaderX3
: Advanced Rendering With DirectX And OpenGL, Charles River Media, pp 341 - 348, ISBN 1-58450-357-2
[Pattanaik] Sumanta N. Pattanaik, Jack Tumblin, Hector Yee, Donald P. Greenberg, "Time Dependent Visual Adaptation For Fast Realistic Image Display", SIGGRAPH 2000, pp 47-54
[Persson] Emil Persson, "HDR Texturing" in the ATI March 2006 SDK.
[Poynton] Charles Poynton, "The rehabilitation of gamma", http://www.poynton.com/PDFs/Rehabilitation_of_gamma.pdf
[Reinhard] Erik Reinhard, Michael Stark, Peter Shirley, James Ferwerda, "Photographic Tone Reproduction for Digital Images", http://www.cs.utah.edu/~reinhard/cdrom/
[Scheuermann] Thorsten Scheuermann and Natalya Tatarchuk, ShaderX3
: Advanced Rendering With DirectX And OpenGL, Charles River Media, pp 363 - 377, ISBN 1-58450-357-2
[Seetzen] Helge Seetzen, Wolfgang Heidrich, Wolfgang Stuerzlinger, Greg Ward, Lorne Whitehead, Matthew Trentacoste, Abhijeet Ghosh, Andrejs Vorozcovs, "High Dynamic Range Display
Systems",SIGGRAPH 2004, pp 760 - 768
[Shimizu] Clement Shimizu, Amit Shesh, Baoquan Chen, "Hardware Accelerated Motion Blur Generation", EUROGRAPHICS 2003 22:3
[Shirley] Peter Shirley et all., "Fundamentals of Computer Graphics", Second Edition, A.K. Peters pp. 543 - 544, 2005, ISBN 1-56881-269-8
[Spenzer] Greg Spenzer, Peter Shirley, Kurt Zimmerman, Donald P. Greenberg, "Physically-Based Glare Effects for Digital Images", SIGGRAPH 1995, pp. 325 - 334.
[Stokes] Michael Stokes (Hewlett-Packard), Matthew Anderson (Microsoft), Srinivasan Chandrasekar (Microsoft), Ricardo Motta (Hewlett-Packard) “A Standard Default Color Space for the
Internet - sRGB“ http://www.w3.org/Graphics/Color/sRGB.html
[Tchou] Chris Tchou, HDR the Bungie Way, http://msdn2.microsoft.com/en-us/directx/aa937787.aspx
[W3CContrast] W3C Working Draft, 26 April 2000, Technique, Checkpoint 2.2 - Ensure that foreground and background color combinations provide sufficient contrast when viewed by someone
having color deficits or when viewed on a black and white screen, htp://www.w3.org/TR/AERT#color-contrast
[Waliszewski] Arkadiusz Waliszewski, "Floating-point Cube maps", ShaderX2 - Shader Programming Tips and Tricks with DirectX 9, Wordware Inc., pp 319 - 323, ISBN 1-55622-988-7