A Practical and Robust Bump-mapping Technique for Today’s GPUs Stanford Shading Group presentation May 8, 2000
Mark J. Kilgard Graphics Software Engineer NVIDIA Corporation
“Hardware Bump Mapping” mostly hype so far <ul><li>Previous techniques </li></ul><ul><ul><li>Prone to aliasing artifacts <...
Example
Overview of the Technique <ul><li>Uses new NVIDIA GPU features </li></ul><ul><ul><li>Cube maps - for per-pixel normalizati...
Features of the technique <ul><li>Correctly handles </li></ul><ul><ul><li>Surface self-shadowing for both diffuse and spec...
Anatomy of the technique <ul><li>Three passes (or just two if no specular) </li></ul> + diffuse decal specular
High-level view <ul><li>Tangent space per-pixel lighting [Peercy 97] </li></ul><ul><ul><li>CPU supplies tangent space ligh...
Per-pixel lighting building blocks <ul><li>Cube maps enable per-pixel vector normalization </li></ul><ul><ul><li>Cube map ...
Per-pixel lighting building blocks <ul><li>Register combiners perform per-pixel math </li></ul><ul><ul><li>Signed math for...
Surface self-shadowing <ul><li>Two kinds of self-shadowing </li></ul><ul><ul><li>max(0, L dot N’) based on the perturbed n...
Surface self-shadowing <ul><li>Self-shadowing computation </li></ul><ul><ul><li>Technique supports bumped self-shadowing b...
Complete lighting model <ul><li>More per-pixel math than you might think! </li></ul><ul><ul><li>Ambient = constant </li></...
Self-shadowing example Light leaks onto torus “back side” due to bad geometric self-shadowing Proper geometric self-shadow...
Some CPU work required <ul><li>Per-vertex work </li></ul><ul><ul><li>Model requires per-vertex tangent space basis </li></...
Normal Map Filtering Issues <ul><li>Normal map filtering is important </li></ul><ul><ul><li>Consider two objects in the di...
Sub-fragment diffuse lighting <ul><li>If expense was inconsequential </li></ul><ul><ul><li>Light every normal in fragment’...
Examining the Approximation <ul><li>Works in important cases </li></ul><ul><ul><li>true if no L  N terms are clamped </li...
Diffuse Normal Map Filtering <ul><li>Simpler than you think </li></ul><ul><ul><li>Simply average normals in the normal map...
Sub-fragment Specular Lighting <ul><li>Again, if expense was inconsequential </li></ul><ul><ul><li>Light every normal in f...
Attempting a Specular Approximation <ul><li>Why specular approximation is hard </li></ul><ul><ul><li>specular exponent can...
Specular Filtering Stand-in <ul><li>Same as diffuse filtering, but re-normalize </li></ul><ul><ul><li>About the best we ca...
Single Filtered Normal Map Texture <ul><li>Combines diffuse and specular filtering </li></ul><ul><ul><li>Keep normalized f...
Issues <ul><li>Truth in advertising </li></ul><ul><ul><li>Specular exponent only 8 so not too shiny </li></ul></ul><ul><ul...
Claim: practical <ul><li>Justification for this claim </li></ul><ul><ul><li>Real-time frame rates on current generation GP...
Claim: robust <ul><li>Justification for this claim </li></ul><ul><ul><li>Animates with minimal temporal aliasing </li></ul...
Source available on the web already <ul><li>Bumpdemo source code </li></ul><ul><ul><li>Go to www.nvidia.com Developer Sect...
Other novel uses for cube mapping <ul><li>Stable specular highlights </li></ul><ul><ul><li>Encode specular & diffuse light...
Example for stable specular highlights Bright, stable specular highlight, even at low tessellation. Poor per-vertex sampli...
Another example of stable specular highlights High tessellation does not completely fix per-vertex artifacts Still bright ...
<ul><li>Per-pixel specular normal mapping </li></ul><ul><ul><li>Assumes fancy per-pixel dot product operations! </li></ul>...
One pass per-pixel diffuse & specular example! OpenGL per-vertex lighting with mesh of thousands of vertices (note displac...
Example in wireframe Dense, dense mesh. One polygon!
Object-space per-pixel lighting example
Object-space per-pixel example in wire-frame The detail is per-pixel!
Register combiners overview <ul><ul><li>overrides texture stages/environment, color sum, and fog in current APIs </li></ul...
Register combiners operational overview general combiner stages final combiner RGB Portion Alpha Portion initialize regist...
General combiners  RGB operation zero primary color secondary color constant color 0 constant color 1 fog spare 1 spare 0 ...
General combiners Alpha operation zero primary color secondary color constant color 0 constant color 1 fog spare 1 spare 0...
Final combiner operation zero primary color secondary color constant color 0 constant color 1 fog spare 1 spare 0 texture ...
Upcoming SlideShare
Loading in...5
×

A Practical and Robust Bump-mapping Technique for Today’s GPUs (slides)

1,174

Published on

I presented this on May 8, 2000 to the Stanford Shading Group in Palo Alto, California. The presentation explains how to use the, then state-of-the-art, NVIDIA register combiners of the GeForce 256 to implement per-pixel bump mapping, a technique that is now ubiquitous in most 3D computer games.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,174
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
37
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

A Practical and Robust Bump-mapping Technique for Today’s GPUs (slides)

  1. 1. A Practical and Robust Bump-mapping Technique for Today’s GPUs Stanford Shading Group presentation May 8, 2000
  2. 2. Mark J. Kilgard Graphics Software Engineer NVIDIA Corporation
  3. 3. “Hardware Bump Mapping” mostly hype so far <ul><li>Previous techniques </li></ul><ul><ul><li>Prone to aliasing artifacts </li></ul></ul><ul><ul><li>Do not correctly handle surface self-shadowing </li></ul></ul><ul><ul><li>Cut too many corners, i.e. fail to renormalize, etc. </li></ul></ul><ul><ul><li>Too expensive: many passes (OIM), needs accumulation buffer or glCopyPixels </li></ul></ul><ul><ul><li>Only handle infinite, non-attenuated, non-spotlight lights </li></ul></ul><ul><ul><li>Assume local viewer </li></ul></ul><ul><ul><li>Correct only for essentially flat surfaces </li></ul></ul><ul><ul><li>Difficult to implement and author </li></ul></ul>
  4. 4. Example
  5. 5. Overview of the Technique <ul><li>Uses new NVIDIA GPU features </li></ul><ul><ul><li>Cube maps - for per-pixel normalization </li></ul></ul><ul><ul><ul><li>EXT_texture_cube_map extension </li></ul></ul></ul><ul><ul><li>Register combiners - for per-pixel dot products and additional math </li></ul></ul><ul><ul><ul><li>NV_register_combiners </li></ul></ul></ul><ul><ul><li>And dual-textured - for normalization cube map and 2D normal map </li></ul></ul><ul><ul><ul><li>ARB_multitexture </li></ul></ul></ul>
  6. 6. Features of the technique <ul><li>Correctly handles </li></ul><ul><ul><li>Surface self-shadowing for both diffuse and specular contributions </li></ul></ul><ul><ul><li>Diffuse minification so that bumped objects in the distance look dimmer than equivalent smooth distant objects </li></ul></ul><ul><ul><li>Mipmap filtering of normal maps minimizes aliasing </li></ul></ul><ul><ul><li>Local, attenuated, and/or spot-light light sources with local viewer </li></ul></ul><ul><ul><li>Ambient and surface decals </li></ul></ul>
  7. 7. Anatomy of the technique <ul><li>Three passes (or just two if no specular) </li></ul> + diffuse decal specular
  8. 8. High-level view <ul><li>Tangent space per-pixel lighting [Peercy 97] </li></ul><ul><ul><li>CPU supplies tangent space light vector or half-angle vector as (s,t,r) texture coordinates </li></ul></ul><ul><ul><ul><li>linearly interpolated </li></ul></ul></ul><ul><ul><ul><li>normalized by an RGB8 cube map texture </li></ul></ul></ul><ul><ul><li>Normal map is RGBA8 2D texture </li></ul></ul><ul><ul><ul><li>RGB contains “compressed” normalized perturbed normal </li></ul></ul></ul><ul><ul><ul><li>Alpha contains averaged normal shortening </li></ul></ul></ul><ul><ul><li>But no hard-wired per-pixel “lighting engine”! </li></ul></ul>
  9. 9. Per-pixel lighting building blocks <ul><li>Cube maps enable per-pixel vector normalization </li></ul><ul><ul><li>Cube map normalizes interpolated (s,t,r) into RGB </li></ul></ul><ul><ul><ul><li>(s,t,r) is interpolated per-vertex light or half-angle vector </li></ul></ul></ul><ul><ul><li>Combiners expand RGB from [0,1] range to [-1,1] range </li></ul></ul><ul><ul><ul><li>expansion done by register combiners </li></ul></ul></ul><ul><ul><li>Optimization : If infinite viewer or infinite light used, half-angle can be computed within the cube map! </li></ul></ul><ul><ul><li>32x32x6 RGB8 cube map texture is sufficient </li></ul></ul>
  10. 10. Per-pixel lighting building blocks <ul><li>Register combiners perform per-pixel math </li></ul><ul><ul><li>Signed math for dot products </li></ul></ul><ul><ul><ul><li>L dot N’ for diffuse, H dot N’ for Blinn specular </li></ul></ul></ul><ul><ul><ul><li>L and H properly normalized by cube map </li></ul></ul></ul><ul><ul><ul><li>Pre-normalized N’ supplied by normal map </li></ul></ul></ul><ul><ul><li>Uses tangent space L z for geometric surface self-shadowing </li></ul></ul><ul><ul><li>Constant color for adding in ambient contribution </li></ul></ul><ul><ul><li>Successive squaring for specular exponent </li></ul></ul>
  11. 11. Surface self-shadowing <ul><li>Two kinds of self-shadowing </li></ul><ul><ul><li>max(0, L dot N’) based on the perturbed normal </li></ul></ul><ul><ul><li>Also should clamp when L dot N goes negative! </li></ul></ul>L N N’ L N’ N Surface should self-shadow due to perturbed normal, I.e., L dot N’<0 Surface should self-shadow due to unperturbed normal, I.e., L dot N<0
  12. 12. Surface self-shadowing <ul><li>Self-shadowing computation </li></ul><ul><ul><li>Technique supports bumped self-shadowing by computing min(8*max(0,Lz), 1) * max(0, L dot N’) </li></ul></ul><ul><ul><li>Also stashes this self-shadowing term in destination alpha min(8*max(0,Lz), 1) </li></ul></ul><ul><ul><li>Specular pass blends with DST_ALPHA,ONE computing min(8*max(0,Lz), 1) * max(0, H dot N’)^8 </li></ul></ul><ul><ul><li>Illumination does not appear on geometric “back side” </li></ul></ul><ul><ul><ul><li>Steep ramp avoids winking and popping at self-shadow boundary </li></ul></ul></ul>
  13. 13. Complete lighting model <ul><li>More per-pixel math than you might think! </li></ul><ul><ul><li>Ambient = constant </li></ul></ul><ul><ul><li>Diffuse = min(8*max(0,Lz), 1) * max(0, L dot N’) </li></ul></ul><ul><ul><li>Specular = min(8*max(0,Lz), 1) * max(0, H dot N’)^8 </li></ul></ul><ul><ul><li>Final = (Diffuse + Ambient) * Decal + Specular </li></ul></ul><ul><li>Attenuation and/or spotlight </li></ul><ul><ul><li>Final = (Attenuate*Diffuse + Ambient) * Decal + Attenuate*Specular </li></ul></ul>
  14. 14. Self-shadowing example Light leaks onto torus “back side” due to bad geometric self-shadowing Proper geometric self-shadowing!
  15. 15. Some CPU work required <ul><li>Per-vertex work </li></ul><ul><ul><li>Model requires per-vertex tangent space basis </li></ul></ul><ul><ul><ul><li>any bump mapping scheme needs surface normals (N), tangents (T), and binormals (B) </li></ul></ul></ul><ul><ul><li>If light or viewer (in the specular case) changes relative to the object, CPU re-compute tangent space light and half-angle </li></ul></ul><ul><ul><ul><li>Transform object-space vector by [ T B N ] 3x3 matrix </li></ul></ul></ul><ul><ul><li>Nice split: CPU does per-vertex work, GPU does transform and per-pixel work </li></ul></ul>
  16. 16. Normal Map Filtering Issues <ul><li>Normal map filtering is important </li></ul><ul><ul><li>Consider two objects in the distance </li></ul></ul><ul><ul><ul><li>chalk golf ball (diffuse, bumpy) </li></ul></ul></ul><ul><ul><ul><li>smooth chalk ball (diffuse, but not bumpy) </li></ul></ul></ul><ul><ul><ul><li>bumpy ball should look dimmer </li></ul></ul></ul><ul><ul><li>How to properly account for this? </li></ul></ul><ul><ul><li>Corresponds to heavy minification of the normal map </li></ul></ul><ul><ul><ul><li>would imply sub-fragment lighting </li></ul></ul></ul>
  17. 17. Sub-fragment diffuse lighting <ul><li>If expense was inconsequential </li></ul><ul><ul><li>Light every normal in fragment’s normal map footprint </li></ul></ul><ul><ul><ul><li>filter the result </li></ul></ul></ul><ul><ul><ul><li>too expensive, so make an approximation </li></ul></ul></ul>
  18. 18. Examining the Approximation <ul><li>Works in important cases </li></ul><ul><ul><li>true if no L  N terms are clamped </li></ul></ul><ul><ul><li>true if all L  N terms are clamped </li></ul></ul><ul><ul><li>inaccurate when mixture of self-shadowing and not </li></ul></ul><ul><ul><ul><li>worst case: highly distributed normals case, often rare </li></ul></ul></ul>
  19. 19. Diffuse Normal Map Filtering <ul><li>Simpler than you think </li></ul><ul><ul><li>Simply average normals in the normal map </li></ul></ul><ul><ul><ul><li>minified normals are often denormalized </li></ul></ul></ul><ul><ul><ul><li>do NOT renormalize </li></ul></ul></ul><ul><ul><li>Better than hyper-tessellation and per-vertex lighting </li></ul></ul><ul><ul><ul><li>hyper-tessellation cannot simulate sub-pixel lighting without sub-pixel rendering </li></ul></ul></ul><ul><ul><li>Issue: mipmap bleeding </li></ul></ul><ul><ul><ul><li>possible solution: borders and LOD clamping </li></ul></ul></ul>
  20. 20. Sub-fragment Specular Lighting <ul><li>Again, if expense was inconsequential </li></ul><ul><ul><li>Light every normal in fragment’s normal map footprint </li></ul></ul><ul><ul><ul><li>filter the result </li></ul></ul></ul><ul><ul><ul><li>too expensive, but what approximation? </li></ul></ul></ul>
  21. 21. Attempting a Specular Approximation <ul><li>Why specular approximation is hard </li></ul><ul><ul><li>specular exponent cannot be factored out </li></ul></ul><ul><ul><li>specular exponent needs H  N to be 1.0 when H and N are coincident </li></ul></ul><ul><ul><ul><li>an issue if N is de-normalized </li></ul></ul></ul><ul><ul><ul><li>if length of N is too short, specular term never “glints” </li></ul></ul></ul><ul><ul><li>Known issue, see [Fournier 92] and [Schilling 97] </li></ul></ul><ul><ul><ul><li>current solutions too expensive for hardware </li></ul></ul></ul>
  22. 22. Specular Filtering Stand-in <ul><li>Same as diffuse filtering, but re-normalize </li></ul><ul><ul><li>About the best we can do </li></ul></ul><ul><ul><li>Like [Fournier 92] approximation, just one “surface” </li></ul></ul>
  23. 23. Single Filtered Normal Map Texture <ul><li>Combines diffuse and specular filtering </li></ul><ul><ul><li>Keep normalized filtered normal in (R,G,B) </li></ul></ul><ul><ul><ul><li>Use (R,G,B) directly for specular </li></ul></ul></ul><ul><ul><li>Keep de-normalization factor in A </li></ul></ul><ul><ul><ul><li>Scale (R,G,B) by A for diffuse </li></ul></ul></ul><ul><ul><li>Efficient since RGB 24-bit texel format is often padded out to 32-bit RGBA texel format for nice word accesses </li></ul></ul>
  24. 24. Issues <ul><li>Truth in advertising </li></ul><ul><ul><li>Specular exponent only 8 so not too shiny </li></ul></ul><ul><ul><ul><li>banding evident in specular highlights, though easy to hide artifacts in surface decal and diffuse contribution </li></ul></ul></ul><ul><ul><li>Perturbed normal slightly de-normalized by filtering </li></ul></ul><ul><ul><ul><li>causes very slight dimming, alternative is aliasing </li></ul></ul></ul><ul><ul><ul><li>most objectionable when magnifying, “grated” look </li></ul></ul></ul><ul><ul><li>Not “bumped environment mapping” </li></ul></ul><ul><ul><ul><li>but DX6 bump-env support is a joke </li></ul></ul></ul><ul><ul><li>Requires GeForce 256, Quadro, or any future NVIDIA GPU </li></ul></ul>
  25. 25. Claim: practical <ul><li>Justification for this claim </li></ul><ul><ul><li>Real-time frame rates on current generation GPUs </li></ul></ul><ul><ul><ul><li>tweakable: eliminate specular or decal pass, etc. </li></ul></ul></ul><ul><ul><li>Mixable with other multi-pass rendering techniques </li></ul></ul><ul><ul><ul><li>stenciled shadows, multiple lights, fog, etc. </li></ul></ul></ul><ul><ul><li>Bump map easy to author as an 8-bit gray-scale image </li></ul></ul><ul><ul><ul><li>examples: </li></ul></ul></ul>
  26. 26. Claim: robust <ul><li>Justification for this claim </li></ul><ul><ul><li>Animates with minimal temporal aliasing </li></ul></ul><ul><ul><li>Models both self-shadowing terms, including for specular </li></ul></ul><ul><ul><li>Diffuse illumination filtered reasonably </li></ul></ul><ul><ul><li>Light and half-angle vectors renormalized per-pixel </li></ul></ul><ul><ul><li>Local, attenuated, and spot-light light sources </li></ul></ul><ul><ul><li>Local viewer </li></ul></ul><ul><ul><li>Overall, very reasonable fidelity to Blinn’s original formulation </li></ul></ul>
  27. 27. Source available on the web already <ul><li>Bumpdemo source code </li></ul><ul><ul><li>Go to www.nvidia.com Developer Section </li></ul></ul><ul><ul><ul><li>http://www.nvidia.com/developer.nsf/htmlmedia/devMainFR_top.html </li></ul></ul></ul><ul><ul><li>Find and download bumpdemo.zip </li></ul></ul><ul><ul><ul><li>requires GeForce 256 or Quadro to run </li></ul></ul></ul><ul><ul><li>Also download “NVIDIA OpenGL Extension Specifications” </li></ul></ul><ul><ul><ul><li>nvOpenGLspecs.pdf </li></ul></ul></ul><ul><ul><ul><li>Adobe Acrobat, 197 pages </li></ul></ul></ul><ul><ul><ul><li>documents OpenGL extensions used by bumpdemo </li></ul></ul></ul>
  28. 28. Other novel uses for cube mapping <ul><li>Stable specular highlights </li></ul><ul><ul><li>Encode specular & diffuse lighting solution </li></ul></ul><ul><ul><ul><li>Diffuse cube map using normal map texgen </li></ul></ul></ul><ul><ul><ul><li>Specular cube map using reflection map texgen </li></ul></ul></ul><ul><ul><ul><li>Encode unlimited number of directional lights </li></ul></ul></ul><ul><ul><li>Result is stable specular highlights </li></ul></ul><ul><ul><li>Less significant for diffuse lighting </li></ul></ul><ul><ul><ul><li>Average of dot products  dot product of averages </li></ul></ul></ul><ul><ul><ul><li>Excepting clamping </li></ul></ul></ul>
  29. 29. Example for stable specular highlights Bright, stable specular highlight, even at low tessellation. Poor per-vertex sampling of the highlight. Wobbles during animation. Cube map lighting Standard per-vertex lighting
  30. 30. Another example of stable specular highlights High tessellation does not completely fix per-vertex artifacts Still bright and stable. Better sampled, but still has streaky artifacts.
  31. 31. <ul><li>Per-pixel specular normal mapping </li></ul><ul><ul><li>Assumes fancy per-pixel dot product operations! </li></ul></ul><ul><ul><li>Cube map encodes normalized half-angle </li></ul></ul><ul><ul><ul><li>Texgen supplies view vector </li></ul></ul></ul><ul><ul><ul><li>Cube map generates normalize(V+L) vector </li></ul></ul></ul><ul><ul><li>Another 2D texture supplies per-pixel surface normals </li></ul></ul><ul><ul><li>Per-pixel specular lighting </li></ul></ul><ul><ul><li>Per-pixel “normalized(V+L) dot N” ! </li></ul></ul>Still more novel uses for cube maps & combiners
  32. 32. One pass per-pixel diffuse & specular example! OpenGL per-vertex lighting with mesh of thousands of vertices (note displacement). Per-pixel specular normal mapping with cube maps (flat, but lighting all there).
  33. 33. Example in wireframe Dense, dense mesh. One polygon!
  34. 34. Object-space per-pixel lighting example
  35. 35. Object-space per-pixel example in wire-frame The detail is per-pixel!
  36. 36. Register combiners overview <ul><ul><li>overrides texture stages/environment, color sum, and fog in current APIs </li></ul></ul><ul><ul><li>signed math (negative one to positive range) </li></ul></ul><ul><ul><ul><li>extended range through scaling </li></ul></ul></ul><ul><ul><li>dot products for lighting and image processing applications </li></ul></ul><ul><ul><ul><li>designed for specular, diffuse, and ambient per-pixel lighting </li></ul></ul></ul><ul><ul><ul><li>object space bump map lighting </li></ul></ul></ul><ul><ul><ul><li>tangent space bump map lighting </li></ul></ul></ul><ul><ul><ul><li>post-filtering 3x3 color matrix for color space conversions </li></ul></ul></ul><ul><ul><li>register model supports non-linear data flows </li></ul></ul><ul><ul><ul><li>superior to linear chain in current APIs </li></ul></ul></ul><ul><ul><li>effectively, a VLIW instruction set for fragment coloring </li></ul></ul><ul><ul><li>very efficient hardware implementation </li></ul></ul>
  37. 37. Register combiners operational overview general combiner stages final combiner RGB Portion Alpha Portion initialize registers stage #0 stage #1 (optional) AB+CD, AB, CD AB+CD, AB, CD AB+CD, AB, CD AB+CD, AB, CD AB+(1-A)C+D, EF, G RGBA fragment
  38. 38. General combiners RGB operation zero primary color secondary color constant color 0 constant color 1 fog spare 1 spare 0 texture 0 texture 1 A B C D input map input map input map not writeable RGB A RGB A input registers computations output registers scale and bias input map not readable zero primary color secondary color constant color 0 constant color 1 fog spare 1 spare 0 texture 0 texture 1 A B + C D A B mux C D -or- A B A  B -or- C  D C D -or-
  39. 39. General combiners Alpha operation zero primary color secondary color constant color 0 constant color 1 fog spare 1 spare 0 texture 0 texture 1 A B C D A B C D input map input map input map not writeable RGB A RGB A input registers computations output registers scale and bias input map not readable zero primary color secondary color constant color 0 constant color 1 fog spare 1 spare 0 texture 0 texture 1 A B + C D A B mux C D -or-
  40. 40. Final combiner operation zero primary color secondary color constant color 0 constant color 1 fog spare 1 spare 0 texture 0 texture 1 A B C D RGB A input registers computations A B + ( 1 - A) C + D E F E F G spare 0 + secondary color input map input map input map input map input map input map input map fragment RGB out fragment Alpha out G
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×