Geometry Shader-based Bump Mapping Setup

4,615 views

Published on

Geometry shader based bump mapping setup using Cg 3.1.

Published in: Technology, Art & Photos
  • Be the first to comment

Geometry Shader-based Bump Mapping Setup

  1. 1. Geometry shader-based bump mapping setup Mark Kilgard NVIDIA Corporation February 5, 2007 Updated October 18, 2007
  2. 2. Overview <ul><li>GPU bump-mapping traditionally involves complicated “hard to get right” per-primitive setup math to compute how object-space quantities map into texture space </li></ul><ul><ul><li>Existing art requires CPU work </li></ul></ul><ul><li>This invention uses a geometry shader to accomplish this more efficiently </li></ul><ul><ul><li>Artist authors traditional textured 3D model </li></ul></ul><ul><ul><li>Authors height field with same texture parameterization as the model’s decal texture </li></ul></ul><ul><ul><li>Vertex shader can animate & skin model </li></ul></ul><ul><ul><li>Geometry shader constructs object-to-texture space mapping </li></ul></ul><ul><ul><ul><li>Object-space light and view vectors can now be transformed to texture space </li></ul></ul></ul><ul><ul><li>Pixel shader does efficient texture-space bump mapping </li></ul></ul>
  3. 3. Geometry Shader in Cg TRIANGLE void md2bump_geometry( AttribArray < float4 > position : POSITION , AttribArray < float2 > texCoord : TEXCOORD0 , AttribArray < float3 > objPosition : TEXCOORD1 , AttribArray < float3 > objNormal : TEXCOORD2 , AttribArray < float3 > objView : TEXCOORD3 , AttribArray < float3 > objLight : TEXCOORD4 ) { float3 dXYZdU = objPosition[1] - objPosition[0]; float dSdU = texCoord[1].s - texCoord[0].s; float3 dXYZdV = objPosition[2] - objPosition[0]; float dSdV = texCoord[2].s - texCoord[0].s; float3 tangent = normalize (dSdV * dXYZdU - dSdU * dXYZdV); for ( int i=0; i<3; i++) { float3 normal = objNormal[i], binormal = cross (tangent,normal); float3x3 basis = float3x3 ( tangent , binormal, normal); float3 surfaceLightVector : TEXCOORD1 = mul (basis, objLight[i]); float3 surfaceViewVector : TEXCOORD2 = mul (basis, objView[i]); emitVertex (position[i], texCoord[i], surfaceLightVector, surfaceViewVector); } } warning: not tolerant of mirroring (see next version)
  4. 4. Canonical Triangle A B C A = ( u=0, v=0, x0, y0, z0, s0, t0 ) B = ( u=1, v=0, x1, y1, z1, s1, t1 ) C = ( u=0, v=1, x2, y2, z2, s2, t2 ) ∂ xyz / ∂u = (x1,y1,z1) - (x0,y0,z0) ∂ xyz / ∂v = (x2,y2,z2) - (x0,y0,z0) ∂ s / ∂u = s1 – s0 ∂ t / ∂v = t2 – t0 v u quotient rule for partial derivatives:
  5. 5. Forming an Object-to-Texture Space Basis assumes left-handed texture space provided by vertex shader
  6. 6. Transform Object-Space Vectors for Lighting to Texture-Space object-space texture-space light vector view vector
  7. 7. Issue: Texture Mirroring <ul><li>Texture space is ad-hoc </li></ul><ul><ul><li>Mapping defined by artists during model construction </li></ul></ul><ul><ul><li>Artists often “re-use” regions of texture space </li></ul></ul><ul><ul><li>Includes mirroring regions of the texture </li></ul></ul><ul><ul><ul><li>Example: bilateral symmetry of face or humanoid </li></ul></ul></ul><ul><li>Mathematical consequence of mirroring </li></ul><ul><ul><li>Signed area of triangle in texture space is negative </li></ul></ul><ul><ul><li>Requires reversal of the sense of gradients </li></ul></ul><ul><ul><li>Geometry shader can compensate </li></ul></ul>
  8. 8. Compensating for Texture Mirroing If signed area in texture space “less than” zero, compute basis with negated tangent: 2x2 determinant
  9. 9. Example of Mirroring artist’s original decal derived height field RGB normal map generated from height field Visualization of 3D model’s signed area in texture space green =front-facing red =back-facing only half of face appears in decal
  10. 10. Consequence on Lighting from Accounting for Mirroring Bad: mirrored skull buckle and belt lighting wrong on left side (indents in) Correct lighting
  11. 11. Mirroring-tolerant Geometry Shader in Cg TRIANGLE void md2bump_geometry( AttribArray < float4 > position : POSITION , AttribArray < float2 > texCoord : TEXCOORD0 , AttribArray < float3 > objPosition : TEXCOORD1 , AttribArray < float3 > objNormal : TEXCOORD2 , AttribArray < float3 > objView : TEXCOORD3 , AttribArray < float3 > objLight : TEXCOORD4 ) { float3 dXYZdU = objPosition[1] - objPosition[0]; float dSdU = texCoord[1].s - texCoord[0].s; float3 dXYZdV = objPosition[2] - objPosition[0]; float dSdV = texCoord[2].s - texCoord[0].s; float3 tangent = normalize (dSdV * dXYZdU - dSdU * dXYZdV); float area = determinant ( float2x2 (dSTdV, dSTdU)); float3 orientedTangent = area >= 0 ? tangent : -tangent; for ( int i=0; i<3; i++) { float3 normal = objNormal[i], binormal = cross (tangent,normal); float3x3 basis = float3x3 (orientedTangent, binormal, normal); float3 surfaceLightVector : TEXCOORD1 = mul (basis, objLight[i]); float3 surfaceViewVector : TEXCOORD2 = mul (basis, objView[i]); emitVertex (position[i], texCoord[i], surfaceLightVector, surfaceViewVector); } } additional & changed code for mirroring
  12. 12. Discarding Triangles Significantly Back Facing w.r.t. the Light TRIANGLE void md2bump_geometry( AttribArray < float4 > position : POSITION , AttribArray < float2 > texCoord : TEXCOORD0 , AttribArray < float3 > objPosition : TEXCOORD1 , AttribArray < float3 > objNormal : TEXCOORD2 , AttribArray < float3 > objView : TEXCOORD3 , AttribArray < float3 > objLight : TEXCOORD4 ) { float3 dXYZdU = objPosition[1] - objPosition[0]; float dSdU = texCoord[1].s - texCoord[0].s; float3 dXYZdV = objPosition[2] - objPosition[0]; float dSdV = texCoord[2].s - texCoord[0].s; float3 tangent = normalize (dSdV * dXYZdU - dSdU * dXYZdV); float maxLightZ, maxLightThreshold = -0.3; for ( int i=0; i<3; i++) { float3 normal = objNormal[i], binormal = cross (tangent,normal); float3x3 basis = float3x3 ( tangent , binormal, normal); float3 surfaceLightVector : TEXCOORD1 = mul (basis, objLight[i]); maxLightZ = i==0 ? normalize(surfaceLightVector).z : max(maxLightZ, normalize(surfaceLightVector).z); float3 surfaceViewVector : TEXCOORD2 = mul (basis, objView[i]); if (i < 2 || maxLightZ > maxLightThreshold) emitVertex (position[i], texCoord[i], surfaceLightVector, surfaceViewVector); } }
  13. 13. Graphics pipeline dataflow with geometry shader-based setup application Vertex shader Primitive assembly Geometry shader Rasterizer Fragment shader Raster operations framebuffer <ul><li>Animation and skinning of object-space vertices </li></ul><ul><li>Output texture coordinates </li></ul><ul><li>Output object-space light & view vectors </li></ul><ul><li>Output clip-space position </li></ul><ul><li>Compute object-to-texture space basis for triangle </li></ul><ul><li>Transform object-space vectors to texture space </li></ul><ul><li>Output triangle </li></ul><ul><li>Access perturbed normal from normal map texture </li></ul><ul><li>Compute lighting in texture space </li></ul><ul><li>Output resulting color </li></ul>
  14. 14. Sans Per-vertex Normals
  15. 15. Geometry Shader in Cg Sans Normals TRIANGLE void md2bump_geometry_sans_normal( AttribArray < float4 > position : POSITION , AttribArray < float2 > texCoord : TEXCOORD0 , AttribArray < float3 > objPosition : TEXCOORD1 , // IGNORE per-vertex normal! AttribArray < float3 > objView : TEXCOORD3 , AttribArray < float3 > objLight : TEXCOORD4 ) { float3 dXYZdU = objPosition[1] - objPosition[0]; float2 dSTdU = texCoord[1] - texCoord[0]; float3 dXYZdV = objPosition[2] - objPosition[0]; float2 dSTdV = texCoord[2] - texCoord[0]; float3 tangent = normalize (dSTdV.s * dXYZdU - dSTdU.s * dXYZdV); float3 tangent2 = normalize (dSTdV.t * dXYZdU - dSTdU.t * dXYZdV); float area = determinant (float2x2(dSTdV, dSTdU)); tangent = area >= 0 ? tangent : -tangent; float3 normal = cross(tangent2,tangent); tangent2 = area >= 0 ? tangent2 : -tangent2; for ( int i=0; i<3; i++) { float3x3 basis = float3x3 (tangent, tangent2, normal); float3 surfaceLightVector : TEXCOORD1 = mul(basis, objLight[i]); float3 surfaceViewVector : TEXCOORD2 = mul(basis, objView[i]); emitVertex (position[i], texCoord[i], surfaceLightVector, surfaceViewVector); } }
  16. 16. Geometry shader setup with and without per-vertex normals Setup without per-vertex normals, relying on normalized gradients only; faceted look Setup with per-vertex normals; smoother lighting appearance
  17. 17. Visualizing Tangent, Normal, and Bi-normals for Discontinuities <ul><li>Using per-vertex normals </li></ul>Visualize normals Reasonably smooth Visualize tangents Flat per-triangle Visualize tangents Semi-smooth, semi-flat
  18. 18. Issue: Lighting Discontinuities <ul><li>Problem: Geometry shader computes single tangent vector for entire triangle </li></ul><ul><ul><li>See tangent visualization (prior slide) </li></ul></ul><ul><ul><li>Still each vertex of triangle has its own unique normal so basis has some variation of triangle </li></ul></ul><ul><li>This per-triangle “tangent flatness” can lead to lighting discontinuities </li></ul>
  19. 19. Lighting Discontinuities <ul><li>Example of unsightly discontinuities when applying geometry shader bump mapping setup to a sphere: </li></ul>
  20. 20. Dealing with Lighting Discontinuities <ul><li>Final shading when bump mapping can hide these discontinuities somewhat </li></ul><ul><ul><li>Imperfect solution </li></ul></ul><ul><li>Perturb the normals within the normal map to compensate for the discontinuities </li></ul><ul><ul><li>How? </li></ul></ul><ul><ul><li>I have some (unproven, uninvestigated) ideas </li></ul></ul>
  21. 21. More Information and Examples <ul><li>Get the Cg Toolkit </li></ul><ul><ul><li>Cg 3.1 is available now </li></ul></ul><ul><ul><li>http://developer.nvidia.com/cg-toolkit-download </li></ul></ul><ul><ul><ul><li>Or http://http.developer.nvidia.com/Cg/cg_3_1_0010.html </li></ul></ul></ul><ul><ul><ul><ul><li>See the “Download” section of this page </li></ul></ul></ul></ul><ul><li>Find source code once installed in </li></ul><ul><ul><li>examples/OpenGL/advanced/gs_md2render </li></ul></ul>gs_md2bump Geometry shader bump map example gs_md2shadow Geometry shader bump map + shadow volumes example
  22. 22. Supplemental Slides
  23. 23. Authored 3D Model Inputs Key frame blended 3D model Decal skin Bump skin Gloss skin GPU Rendering
  24. 24. Animation via Key Frames or Vertex Skinning GPU Rendering Frame A Frame B Other possible key frames

×