Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hdr Meets Black And White 2


Published on

Game Developers Conferences 2006.

  • Be the first to comment

  • Be the first to like this

Hdr Meets Black And White 2

  1. 1. HDR meets Black&White 2 Francesco Carucci Graphics Programmer/Lionhead Studios [email_address]
  2. 2. What’s HDR rendering? <ul><li>We all know…  </li></ul><ul><li>… rendering the scene to a format which can represent color luminance greater than 1.0 (usually floating point) </li></ul><ul><li>Display devices are limited to LDR (8/10 bit): the HDR image must be converted to LDR (tonemapped) before it can be displayed </li></ul>
  3. 3. Why HDR? <ul><li>An HDR image can represent bright and dark areas in the scene ‘equally’ well </li></ul><ul><li>Different scenes with drastically different lighting conditions can be handled the same way without saturating </li></ul><ul><li>Does your game need it? </li></ul><ul><ul><li>No: Go To End. Pub. </li></ul></ul><ul><ul><li>Yes: Go To Next Slide. </li></ul></ul>
  4. 4. Black&White 2 by night (1) <ul><li>Day/night in BW2 cycle makes it difficult to find the right lighting set up for all cases </li></ul><ul><li>Night scenes in BW2 are suffering from severe low dynamic range </li></ul><ul><li>HDR rendering and auto exposure as a solution to improve visual quality in night scenes </li></ul>
  5. 5. Black&White 2 by night (2) More than 95% of the pixels have luminance smaller than 0.5
  6. 6. Color luminance <ul><li>Quick and simple: luminance is a color property defined in the HSL space and intuitively represents color brightness </li></ul><ul><li>To compute luminance from RGB: luminance = dot(color,LUMINANCE_VECTOR) </li></ul><ul><li>A typical LUMINANCE_VECTOR: (0.2125 0.7154 0.0721) </li></ul>
  7. 7. Tonemapping (1) <ul><li>Here’s the simplest tonemapping operator: </li></ul><ul><ul><li>if (luminance > 1.0) color = white </li></ul></ul><ul><li>Simple but not very interesting, let’s try another one: </li></ul><ul><ul><li>new_luminance = luminance / max_luminance </li></ul></ul><ul><ul><li>new_color = color * new_luminance / luminance </li></ul></ul>
  8. 8. Tone mapping (2) <ul><li>For a better operator give a look at: </li></ul><ul><li>“ Photographic Tone Reproduction for Digital Images ” </li></ul><ul><li>… but it doesn’t matter how hard you try: Tone mapping is an artistic process, and it does require input from an artist . </li></ul>
  9. 9. HDR in Games <ul><li>Some games are now supporting HDR lighting or HDR rendering: </li></ul><ul><ul><li>Half Life 2 Lost Coast (HDRL) </li></ul></ul><ul><ul><li>Far Cry and Splinter Cell (HDRR) </li></ul></ul><ul><ul><li>Some XBOX 360 games </li></ul></ul>
  10. 10. HDR “a la” Half Life 2 (1) <ul><li>Compute the light equation using high dynamic range values </li></ul><ul><li>Tone map at the end of the lighting computation before writing the result to the framebuffer </li></ul><ul><li>Render to a standard i8 (D3DFMT_A8R8G8B8) render target </li></ul>
  11. 11. HDR “a la” Half Life 2 (2) <ul><li>Pros: </li></ul><ul><ul><li>It doesn’t require HDR hardware support </li></ul></ul><ul><ul><li>Compatible with MSAA on a broad range of hardware </li></ul></ul><ul><ul><li>Relatively cheap </li></ul></ul>
  12. 12. HDR “a la” Half Life 2 (3) <ul><li>Cons: </li></ul><ul><ul><li>Need to touch each and every single shader to include tone mapping </li></ul></ul><ul><ul><li>Not easy to understand and control the overall dynamic range of the scene </li></ul></ul><ul><ul><li>Can become expensive it there’s lots of overdraw (but a depth pass can save the day here!) </li></ul></ul>
  13. 13. HDR: The “floating” way (1) <ul><li>Compute lighting at high dynamic range, write the result to a floating point (16, 32 bit per component) render target (D3DFMT_A16FG16FB16F) </li></ul><ul><li>Compute average and maximum luminance from the HDR image </li></ul><ul><li>Tone map the HDR image to a LDR render target </li></ul>
  14. 14. HDR: The “floating” way (2) <ul><li>Pros: </li></ul><ul><ul><li>Fixed and predictable cost per pixel </li></ul></ul><ul><ul><li>High precision in representing the color (16bit per component) </li></ul></ul><ul><ul><li>Easy to control exposure using average and maximum luminance directly computed from the scene </li></ul></ul>
  15. 15. HDR: The “floating” way (3) <ul><li>Cons: </li></ul><ul><ul><li>It’s expensive! Lots of bandwidth required, lots of memory to store the fp16 render target </li></ul></ul><ul><ul><li>It requires special hardware support available on latest hardware </li></ul></ul><ul><ul><li>It doesn’t play well with MSAA (needs hardware support to resolve MSAA of a floating point render target) </li></ul></ul>
  16. 16. Other ways to do HDR <ul><li>Use RGBE to represent HDR colors </li></ul><ul><ul><li>Blending must be done manually </li></ul></ul><ul><ul><li>No MSAA </li></ul></ul><ul><li>Convert to HSL color space before writing to the frame buffer, allocate more bits for L for HDR </li></ul><ul><ul><li>expensive, hard to implement “right” </li></ul></ul>
  17. 17. HDR: Fran’s way  <ul><li>But first, a word from the sponsor: </li></ul><ul><li><insert animation of a silly Black&White 2 creature here > </li></ul>
  18. 18. Average luminance and the GPU <ul><li>Computing the average luminance of a scene is a typical ‘gathering’ operation </li></ul><ul><li>The GPU is not good at gathering data </li></ul><ul><li>Where the CPU is very good at it </li></ul>
  19. 19. Average luminance on the CPU <ul><li>The HDR image data has to be transferred to the main memory </li></ul><ul><ul><li>SLOW! (not so slow on PCI-E) </li></ul></ul><ul><ul><li>not a problem on next-gen consoles </li></ul></ul><ul><li>Why only average luminance? Compute the entire image histogram! </li></ul>
  20. 20. Image Histogram <ul><li>Find maximum and minimum absolute luminance (Lmax, Lmin) </li></ul><ul><li>Divide the luminance range in a certain number of slots (e.g. 1024) </li></ul><ul><li>For each slot, find the number of pixels with that particular luminance </li></ul><ul><li>Divide each slot by the number of pixels to obtain its frequency </li></ul>
  21. 21. Histogram Equalisation (1) <ul><li>Once we got the image histogram we can equalise to cover the whole available dynamic range </li></ul>
  22. 22. Histogram Equalisation (2) <ul><li>From “Digital Image Processing”, page 91-93: </li></ul><ul><ul><li>map each slot with luminance level R in the input image, to a slot with level S where S is the sum of all frequencies less or equal than R </li></ul></ul><ul><li>The result a good approximation of an equalized histogram </li></ul>
  23. 23. Autoexposure (1) <ul><li>From the equalized histogram compute Lnew_min, Lnew_max, Lnew_avg: </li></ul><ul><ul><li>Lnew_min is the first slot where S is greater than Pmin </li></ul></ul><ul><ul><li>Lnew_max is the first slot where S is greater than Pmax </li></ul></ul><ul><ul><li>Lnew_avg is the first slot where S is greater than Pavg </li></ul></ul><ul><li>Typical Pmin, Pmax and Pavg are 0.01 (1%), 0.99 (50%), and 0.50 (50%) </li></ul>
  24. 24. Autoexposure (2) <ul><li>Minimum and maximum percentage can be chosen to ignore very small bright and dark areas and have a more uniform behaviour </li></ul>
  25. 25. Autoexposure (3) <ul><li>Auto exposure is implemented by tracking the actual minimum, maximum and average luminance to reach the values computed earlier: </li></ul><ul><ul><li>act_value += new_value * (1.0f - pow(1.0f - speed, 30.0f * elapsed_time)); </li></ul></ul>
  26. 26. Luminance mapping (1) <ul><li>Create a 1024x1 linear texture to store the luminance mapping: </li></ul><ul><ul><li>Luminance less than Lact_min is mapped to 0.0 </li></ul></ul><ul><ul><li>Luminance greater than Lact_max is mapped to 1.0 (and then bloomed) </li></ul></ul><ul><ul><li>The rest of the range could be mapped with a simple gamma ramp: </li></ul></ul><ul><ul><li>float xp = 1.0f + (Lact_avg - Lact_min) / (Lact_max - Lact_min); </li></ul></ul><ul><ul><li>gamma_ramp = powf(x, xp + bias); </li></ul></ul>
  27. 27. Luminance mapping (2) <ul><li>The bias value is adjusted to increase or decrease contrast in the scene </li></ul><ul><ul><li>Higher contrast during the day </li></ul></ul><ul><ul><li>Lower contrast at night </li></ul></ul>
  28. 28. Luminance mapping (3) <ul><li>To reduce banding artefacts, the luminance map should be D3DFMT_R16F </li></ul><ul><li>The pixel shader fragment in the tone mapper: </li></ul><ul><ul><li>float Li = dot(colorIn, LUMINANCE_VECTOR); </li></ul></ul><ul><ul><li>float Ld = tex1D(LuminanceMapSampler, Li); </li></ul></ul><ul><ul><li>colorOut.rgb = LuminanceScale * (colorIn.rgb / Li) * Ld; </li></ul></ul>
  29. 29. Histogram based HDR Pros <ul><li>Trade bandwidth from GPU to CPU (usually spare but scarce) for GPU power </li></ul><ul><li>More control over image characteristics </li></ul><ul><li>Very flexible and simple pixel shader tone mapper </li></ul><ul><li>Easy to plug into color correction techniques based on 3d textures </li></ul>
  30. 30. Histogram based HDR Cons <ul><li>Needs a lot of GPU to CPU bandwidth </li></ul><ul><li> but free on console </li></ul><ul><li>Needs a lot of memory to double buffer the HDR image </li></ul><ul><ul><li>but downloading a scaled version will help </li></ul></ul><ul><li>Needs CPU cycles to analyze the HDR image </li></ul><ul><ul><li>what about that idle core there? </li></ul></ul>
  31. 31. HDR in Black&White 2 (1) <ul><li>Plugging HDR into Black&White 2 required only localized changes to the post processing framework </li></ul><ul><ul><li>NO shader needed to be touched </li></ul></ul><ul><ul><li>NO change to the art asset </li></ul></ul><ul><li>But to take full advantage of HDR the artists should take part in the process! </li></ul>
  32. 32. HDR in Black&White 2 (2) <ul><li>A fp16 render target is used to render the main rendering to have good color precision in input for the tone mapper </li></ul><ul><li>A scaled down (4x) version of the HDR image is downloaded to system memory and analyzed every frame (at 30fps) </li></ul>
  33. 33. Performance analysis <ul><li>A scaled down version of a 1024x768 fp16 render target requires approximately 11mb/s of bandwidth </li></ul><ul><li>It takes about 10ms to download the image and about 1ms to analyze it on a AMD64 3500mhz </li></ul><ul><ul><li>It’s bandwidth bound </li></ul></ul>
  34. 34. Before
  35. 35. After
  36. 36. Before
  37. 37. After
  38. 38. BW2 HDR Patch <ul><li>The HDR Patch of Black&White 2 will be released soon (April) </li></ul><ul><li>It will support MSAA where available </li></ul>
  39. 39. References <ul><li>“ Photographic Tone Reproduction for Digital Images ” </li></ul><ul><li>Game Programming Gems 4 </li></ul><ul><li>“ Digital Image Processing” – Gonzales, Woods </li></ul><ul><li>“ High Dynamic Range Rendering on the GeForce 6800” – Simon Green, Cem Cebenoyahn </li></ul>
  40. 40. Black&White 2 Engine Team <ul><li>Thanks to the Black&White 2 engine team who have made an amazingly looking game: Ben, Dave and Mark </li></ul>
  41. 41. ? [email_address]