HDR meets Black&White 2 Francesco Carucci Graphics Programmer/Lionhead Studios [email_address]
What’s HDR rendering? <ul><li>We all know…   </li></ul><ul><li>…  rendering the scene to a format which can represent col...
Why HDR? <ul><li>An HDR image can represent bright and dark areas in the scene ‘equally’ well </li></ul><ul><li>Different ...
Black&White 2 by night (1) <ul><li>Day/night in BW2 cycle makes it difficult to find the right lighting set up for all cas...
Black&White 2 by night (2) More than 95% of the pixels have luminance smaller than 0.5
Color luminance <ul><li>Quick and simple: luminance is a color property defined in the HSL space and intuitively represent...
Tonemapping (1) <ul><li>Here’s the simplest tonemapping operator: </li></ul><ul><ul><li>if (luminance > 1.0) color = white...
Tone mapping (2) <ul><li>For a better operator give a look at: </li></ul><ul><li>“ Photographic Tone Reproduction for Digi...
HDR in Games  <ul><li>Some games are now supporting HDR lighting or HDR rendering: </li></ul><ul><ul><li>Half Life 2 Lost ...
HDR “a la” Half Life 2 (1) <ul><li>Compute the light equation using high dynamic range values </li></ul><ul><li>Tone map a...
HDR “a la” Half Life 2 (2) <ul><li>Pros: </li></ul><ul><ul><li>It doesn’t require HDR hardware support </li></ul></ul><ul>...
HDR “a la” Half Life 2 (3) <ul><li>Cons: </li></ul><ul><ul><li>Need to touch each and every single shader to include tone ...
HDR: The “floating” way (1) <ul><li>Compute lighting at high dynamic range, write the result to a floating point (16, 32 b...
HDR: The “floating” way (2) <ul><li>Pros: </li></ul><ul><ul><li>Fixed and predictable cost per pixel </li></ul></ul><ul><u...
HDR: The “floating” way (3) <ul><li>Cons: </li></ul><ul><ul><li>It’s expensive! Lots of bandwidth required, lots of memory...
Other ways to do HDR <ul><li>Use RGBE to represent HDR colors </li></ul><ul><ul><li>Blending must be done manually </li></...
HDR: Fran’s way   <ul><li>But first, a word from the sponsor: </li></ul><ul><li><insert animation of a silly Black&White ...
Average luminance and the GPU <ul><li>Computing the average luminance of a scene is a typical ‘gathering’ operation </li><...
Average luminance on the CPU <ul><li>The HDR image data has to be transferred to the main memory </li></ul><ul><ul><li>SLO...
Image Histogram <ul><li>Find maximum and minimum absolute luminance (Lmax, Lmin) </li></ul><ul><li>Divide the luminance ra...
Histogram Equalisation (1) <ul><li>Once we got the image histogram we can equalise to cover the whole available dynamic ra...
Histogram Equalisation (2) <ul><li>From “Digital Image Processing”, page 91-93: </li></ul><ul><ul><li>map each slot with l...
Autoexposure (1) <ul><li>From the equalized histogram compute Lnew_min, Lnew_max, Lnew_avg: </li></ul><ul><ul><li>Lnew_min...
Autoexposure (2) <ul><li>Minimum and maximum percentage can be chosen to ignore very small bright and dark areas and have ...
Autoexposure (3) <ul><li>Auto exposure is implemented by tracking the actual minimum, maximum and average luminance to rea...
Luminance mapping (1) <ul><li>Create a 1024x1 linear texture to store the luminance mapping: </li></ul><ul><ul><li>Luminan...
Luminance mapping (2) <ul><li>The bias value is adjusted to increase or decrease contrast in the scene </li></ul><ul><ul><...
Luminance mapping (3) <ul><li>To reduce banding artefacts, the luminance map should be D3DFMT_R16F </li></ul><ul><li>The p...
Histogram based HDR Pros <ul><li>Trade bandwidth from GPU to CPU (usually spare but scarce) for GPU power </li></ul><ul><l...
Histogram based HDR Cons <ul><li>Needs a lot of GPU to CPU bandwidth  </li></ul><ul><li>  but free on console </li></ul><u...
HDR in Black&White 2 (1) <ul><li>Plugging HDR into Black&White 2 required only localized changes to the post processing fr...
HDR in Black&White 2 (2) <ul><li>A fp16 render target is used to render the main rendering to have good color precision in...
Performance analysis <ul><li>A scaled down version of a 1024x768 fp16 render target requires approximately 11mb/s of bandw...
Before
After
Before
After
BW2 HDR Patch <ul><li>The HDR Patch of Black&White 2 will be released soon (April) </li></ul><ul><li>It will support MSAA ...
References <ul><li>“ Photographic Tone Reproduction for Digital Images ” </li></ul><ul><li>Game Programming Gems 4 </li></...
Black&White 2 Engine Team <ul><li>Thanks to the Black&White 2 engine team who have made an amazingly looking game:  Ben, D...
? [email_address]
Upcoming SlideShare
Loading in …5
×

Hdr Meets Black And White 2

2,314 views
2,177 views

Published on

Game Developers Conferences 2006.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,314
On SlideShare
0
From Embeds
0
Number of Embeds
60
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Hdr Meets Black And White 2

  1. 1. HDR meets Black&White 2 Francesco Carucci Graphics Programmer/Lionhead Studios [email_address]
  2. 2. What’s HDR rendering? <ul><li>We all know…  </li></ul><ul><li>… rendering the scene to a format which can represent color luminance greater than 1.0 (usually floating point) </li></ul><ul><li>Display devices are limited to LDR (8/10 bit): the HDR image must be converted to LDR (tonemapped) before it can be displayed </li></ul>
  3. 3. Why HDR? <ul><li>An HDR image can represent bright and dark areas in the scene ‘equally’ well </li></ul><ul><li>Different scenes with drastically different lighting conditions can be handled the same way without saturating </li></ul><ul><li>Does your game need it? </li></ul><ul><ul><li>No: Go To End. Pub. </li></ul></ul><ul><ul><li>Yes: Go To Next Slide. </li></ul></ul>
  4. 4. Black&White 2 by night (1) <ul><li>Day/night in BW2 cycle makes it difficult to find the right lighting set up for all cases </li></ul><ul><li>Night scenes in BW2 are suffering from severe low dynamic range </li></ul><ul><li>HDR rendering and auto exposure as a solution to improve visual quality in night scenes </li></ul>
  5. 5. Black&White 2 by night (2) More than 95% of the pixels have luminance smaller than 0.5
  6. 6. Color luminance <ul><li>Quick and simple: luminance is a color property defined in the HSL space and intuitively represents color brightness </li></ul><ul><li>To compute luminance from RGB: luminance = dot(color,LUMINANCE_VECTOR) </li></ul><ul><li>A typical LUMINANCE_VECTOR: (0.2125 0.7154 0.0721) </li></ul>
  7. 7. Tonemapping (1) <ul><li>Here’s the simplest tonemapping operator: </li></ul><ul><ul><li>if (luminance > 1.0) color = white </li></ul></ul><ul><li>Simple but not very interesting, let’s try another one: </li></ul><ul><ul><li>new_luminance = luminance / max_luminance </li></ul></ul><ul><ul><li>new_color = color * new_luminance / luminance </li></ul></ul>
  8. 8. Tone mapping (2) <ul><li>For a better operator give a look at: </li></ul><ul><li>“ Photographic Tone Reproduction for Digital Images ” </li></ul><ul><li>… but it doesn’t matter how hard you try: Tone mapping is an artistic process, and it does require input from an artist . </li></ul>
  9. 9. HDR in Games <ul><li>Some games are now supporting HDR lighting or HDR rendering: </li></ul><ul><ul><li>Half Life 2 Lost Coast (HDRL) </li></ul></ul><ul><ul><li>Far Cry and Splinter Cell (HDRR) </li></ul></ul><ul><ul><li>Some XBOX 360 games </li></ul></ul>
  10. 10. HDR “a la” Half Life 2 (1) <ul><li>Compute the light equation using high dynamic range values </li></ul><ul><li>Tone map at the end of the lighting computation before writing the result to the framebuffer </li></ul><ul><li>Render to a standard i8 (D3DFMT_A8R8G8B8) render target </li></ul>
  11. 11. HDR “a la” Half Life 2 (2) <ul><li>Pros: </li></ul><ul><ul><li>It doesn’t require HDR hardware support </li></ul></ul><ul><ul><li>Compatible with MSAA on a broad range of hardware </li></ul></ul><ul><ul><li>Relatively cheap </li></ul></ul>
  12. 12. HDR “a la” Half Life 2 (3) <ul><li>Cons: </li></ul><ul><ul><li>Need to touch each and every single shader to include tone mapping </li></ul></ul><ul><ul><li>Not easy to understand and control the overall dynamic range of the scene </li></ul></ul><ul><ul><li>Can become expensive it there’s lots of overdraw (but a depth pass can save the day here!) </li></ul></ul>
  13. 13. HDR: The “floating” way (1) <ul><li>Compute lighting at high dynamic range, write the result to a floating point (16, 32 bit per component) render target (D3DFMT_A16FG16FB16F) </li></ul><ul><li>Compute average and maximum luminance from the HDR image </li></ul><ul><li>Tone map the HDR image to a LDR render target </li></ul>
  14. 14. HDR: The “floating” way (2) <ul><li>Pros: </li></ul><ul><ul><li>Fixed and predictable cost per pixel </li></ul></ul><ul><ul><li>High precision in representing the color (16bit per component) </li></ul></ul><ul><ul><li>Easy to control exposure using average and maximum luminance directly computed from the scene </li></ul></ul>
  15. 15. HDR: The “floating” way (3) <ul><li>Cons: </li></ul><ul><ul><li>It’s expensive! Lots of bandwidth required, lots of memory to store the fp16 render target </li></ul></ul><ul><ul><li>It requires special hardware support available on latest hardware </li></ul></ul><ul><ul><li>It doesn’t play well with MSAA (needs hardware support to resolve MSAA of a floating point render target) </li></ul></ul>
  16. 16. Other ways to do HDR <ul><li>Use RGBE to represent HDR colors </li></ul><ul><ul><li>Blending must be done manually </li></ul></ul><ul><ul><li>No MSAA </li></ul></ul><ul><li>Convert to HSL color space before writing to the frame buffer, allocate more bits for L for HDR </li></ul><ul><ul><li>expensive, hard to implement “right” </li></ul></ul>
  17. 17. HDR: Fran’s way  <ul><li>But first, a word from the sponsor: </li></ul><ul><li><insert animation of a silly Black&White 2 creature here > </li></ul>
  18. 18. Average luminance and the GPU <ul><li>Computing the average luminance of a scene is a typical ‘gathering’ operation </li></ul><ul><li>The GPU is not good at gathering data </li></ul><ul><li>Where the CPU is very good at it </li></ul>
  19. 19. Average luminance on the CPU <ul><li>The HDR image data has to be transferred to the main memory </li></ul><ul><ul><li>SLOW! (not so slow on PCI-E) </li></ul></ul><ul><ul><li>not a problem on next-gen consoles </li></ul></ul><ul><li>Why only average luminance? Compute the entire image histogram! </li></ul>
  20. 20. Image Histogram <ul><li>Find maximum and minimum absolute luminance (Lmax, Lmin) </li></ul><ul><li>Divide the luminance range in a certain number of slots (e.g. 1024) </li></ul><ul><li>For each slot, find the number of pixels with that particular luminance </li></ul><ul><li>Divide each slot by the number of pixels to obtain its frequency </li></ul>
  21. 21. Histogram Equalisation (1) <ul><li>Once we got the image histogram we can equalise to cover the whole available dynamic range </li></ul>
  22. 22. Histogram Equalisation (2) <ul><li>From “Digital Image Processing”, page 91-93: </li></ul><ul><ul><li>map each slot with luminance level R in the input image, to a slot with level S where S is the sum of all frequencies less or equal than R </li></ul></ul><ul><li>The result a good approximation of an equalized histogram </li></ul>
  23. 23. Autoexposure (1) <ul><li>From the equalized histogram compute Lnew_min, Lnew_max, Lnew_avg: </li></ul><ul><ul><li>Lnew_min is the first slot where S is greater than Pmin </li></ul></ul><ul><ul><li>Lnew_max is the first slot where S is greater than Pmax </li></ul></ul><ul><ul><li>Lnew_avg is the first slot where S is greater than Pavg </li></ul></ul><ul><li>Typical Pmin, Pmax and Pavg are 0.01 (1%), 0.99 (50%), and 0.50 (50%) </li></ul>
  24. 24. Autoexposure (2) <ul><li>Minimum and maximum percentage can be chosen to ignore very small bright and dark areas and have a more uniform behaviour </li></ul>
  25. 25. Autoexposure (3) <ul><li>Auto exposure is implemented by tracking the actual minimum, maximum and average luminance to reach the values computed earlier: </li></ul><ul><ul><li>act_value += new_value * (1.0f - pow(1.0f - speed, 30.0f * elapsed_time)); </li></ul></ul>
  26. 26. Luminance mapping (1) <ul><li>Create a 1024x1 linear texture to store the luminance mapping: </li></ul><ul><ul><li>Luminance less than Lact_min is mapped to 0.0 </li></ul></ul><ul><ul><li>Luminance greater than Lact_max is mapped to 1.0 (and then bloomed) </li></ul></ul><ul><ul><li>The rest of the range could be mapped with a simple gamma ramp: </li></ul></ul><ul><ul><li>float xp = 1.0f + (Lact_avg - Lact_min) / (Lact_max - Lact_min); </li></ul></ul><ul><ul><li>gamma_ramp = powf(x, xp + bias); </li></ul></ul>
  27. 27. Luminance mapping (2) <ul><li>The bias value is adjusted to increase or decrease contrast in the scene </li></ul><ul><ul><li>Higher contrast during the day </li></ul></ul><ul><ul><li>Lower contrast at night </li></ul></ul>
  28. 28. Luminance mapping (3) <ul><li>To reduce banding artefacts, the luminance map should be D3DFMT_R16F </li></ul><ul><li>The pixel shader fragment in the tone mapper: </li></ul><ul><ul><li>float Li = dot(colorIn, LUMINANCE_VECTOR); </li></ul></ul><ul><ul><li>float Ld = tex1D(LuminanceMapSampler, Li); </li></ul></ul><ul><ul><li>colorOut.rgb = LuminanceScale * (colorIn.rgb / Li) * Ld; </li></ul></ul>
  29. 29. Histogram based HDR Pros <ul><li>Trade bandwidth from GPU to CPU (usually spare but scarce) for GPU power </li></ul><ul><li>More control over image characteristics </li></ul><ul><li>Very flexible and simple pixel shader tone mapper </li></ul><ul><li>Easy to plug into color correction techniques based on 3d textures </li></ul>
  30. 30. Histogram based HDR Cons <ul><li>Needs a lot of GPU to CPU bandwidth </li></ul><ul><li> but free on console </li></ul><ul><li>Needs a lot of memory to double buffer the HDR image </li></ul><ul><ul><li>but downloading a scaled version will help </li></ul></ul><ul><li>Needs CPU cycles to analyze the HDR image </li></ul><ul><ul><li>what about that idle core there? </li></ul></ul>
  31. 31. HDR in Black&White 2 (1) <ul><li>Plugging HDR into Black&White 2 required only localized changes to the post processing framework </li></ul><ul><ul><li>NO shader needed to be touched </li></ul></ul><ul><ul><li>NO change to the art asset </li></ul></ul><ul><li>But to take full advantage of HDR the artists should take part in the process! </li></ul>
  32. 32. HDR in Black&White 2 (2) <ul><li>A fp16 render target is used to render the main rendering to have good color precision in input for the tone mapper </li></ul><ul><li>A scaled down (4x) version of the HDR image is downloaded to system memory and analyzed every frame (at 30fps) </li></ul>
  33. 33. Performance analysis <ul><li>A scaled down version of a 1024x768 fp16 render target requires approximately 11mb/s of bandwidth </li></ul><ul><li>It takes about 10ms to download the image and about 1ms to analyze it on a AMD64 3500mhz </li></ul><ul><ul><li>It’s bandwidth bound </li></ul></ul>
  34. 34. Before
  35. 35. After
  36. 36. Before
  37. 37. After
  38. 38. BW2 HDR Patch <ul><li>The HDR Patch of Black&White 2 will be released soon (April) </li></ul><ul><li>It will support MSAA where available </li></ul>
  39. 39. References <ul><li>“ Photographic Tone Reproduction for Digital Images ” </li></ul><ul><li>Game Programming Gems 4 </li></ul><ul><li>“ Digital Image Processing” – Gonzales, Woods </li></ul><ul><li>“ High Dynamic Range Rendering on the GeForce 6800” – Simon Green, Cem Cebenoyahn </li></ul>
  40. 40. Black&White 2 Engine Team <ul><li>Thanks to the Black&White 2 engine team who have made an amazingly looking game: Ben, Dave and Mark </li></ul>
  41. 41. ? [email_address]

×