The document describes a geometry shader-based approach to bump mapping that has several advantages over traditional CPU-based approaches. The geometry shader constructs an object-to-texture space mapping for each triangle, allowing lighting computations to be done efficiently in texture space in the pixel shader. It addresses issues like texture mirroring and lighting discontinuities. Examples and Cg source code are provided to illustrate the technique.

1. Geometry shader-based bump mapping setup Mark Kilgard NVIDIA Corporation February 5, 2007 Updated October 18, 2007

2. Overview GPU bump-mapping traditionally involves complicated “hard to get right” per-primitive setup math to compute how object-space quantities map into texture space Existing art requires CPU work This invention uses a geometry shader to accomplish this more efficiently Artist authors traditional textured 3D model Authors height field with same texture parameterization as the model’s decal texture Vertex shader can animate & skin model Geometry shader constructs object-to-texture space mapping Object-space light and view vectors can now be transformed to texture space Pixel shader does efficient texture-space bump mapping

3. Geometry Shader in Cg TRIANGLE void md2bump_geometry( AttribArray < float4 > position : POSITION , AttribArray < float2 > texCoord : TEXCOORD0 , AttribArray < float3 > objPosition : TEXCOORD1 , AttribArray < float3 > objNormal : TEXCOORD2 , AttribArray < float3 > objView : TEXCOORD3 , AttribArray < float3 > objLight : TEXCOORD4 ) { float3 dXYZdU = objPosition[1] - objPosition[0]; float dSdU = texCoord[1].s - texCoord[0].s; float3 dXYZdV = objPosition[2] - objPosition[0]; float dSdV = texCoord[2].s - texCoord[0].s; float3 tangent = normalize (dSdV * dXYZdU - dSdU * dXYZdV); for ( int i=0; i<3; i++) { float3 normal = objNormal[i], binormal = cross (tangent,normal); float3x3 basis = float3x3 ( tangent , binormal, normal); float3 surfaceLightVector : TEXCOORD1 = mul (basis, objLight[i]); float3 surfaceViewVector : TEXCOORD2 = mul (basis, objView[i]); emitVertex (position[i], texCoord[i], surfaceLightVector, surfaceViewVector); } } warning: not tolerant of mirroring (see next version)

4. Canonical Triangle A B C A = ( u=0, v=0, x0, y0, z0, s0, t0 ) B = ( u=1, v=0, x1, y1, z1, s1, t1 ) C = ( u=0, v=1, x2, y2, z2, s2, t2 ) ∂ xyz / ∂u = (x1,y1,z1) - (x0,y0,z0) ∂ xyz / ∂v = (x2,y2,z2) - (x0,y0,z0) ∂ s / ∂u = s1 – s0 ∂ t / ∂v = t2 – t0 v u quotient rule for partial derivatives:

5. Forming an Object-to-Texture Space Basis assumes left-handed texture space provided by vertex shader

6. Transform Object-Space Vectors for Lighting to Texture-Space object-space texture-space light vector view vector

7. Issue: Texture Mirroring Texture space is ad-hoc Mapping defined by artists during model construction Artists often “re-use” regions of texture space Includes mirroring regions of the texture Example: bilateral symmetry of face or humanoid Mathematical consequence of mirroring Signed area of triangle in texture space is negative Requires reversal of the sense of gradients Geometry shader can compensate

8. Compensating for Texture Mirroing If signed area in texture space “less than” zero, compute basis with negated tangent: 2x2 determinant

9. Example of Mirroring artist’s original decal derived height field RGB normal map generated from height field Visualization of 3D model’s signed area in texture space green =front-facing red =back-facing only half of face appears in decal

10. Consequence on Lighting from Accounting for Mirroring Bad: mirrored skull buckle and belt lighting wrong on left side (indents in) Correct lighting

11. Mirroring-tolerant Geometry Shader in Cg TRIANGLE void md2bump_geometry( AttribArray < float4 > position : POSITION , AttribArray < float2 > texCoord : TEXCOORD0 , AttribArray < float3 > objPosition : TEXCOORD1 , AttribArray < float3 > objNormal : TEXCOORD2 , AttribArray < float3 > objView : TEXCOORD3 , AttribArray < float3 > objLight : TEXCOORD4 ) { float3 dXYZdU = objPosition[1] - objPosition[0]; float dSdU = texCoord[1].s - texCoord[0].s; float3 dXYZdV = objPosition[2] - objPosition[0]; float dSdV = texCoord[2].s - texCoord[0].s; float3 tangent = normalize (dSdV * dXYZdU - dSdU * dXYZdV); float area = determinant ( float2x2 (dSTdV, dSTdU)); float3 orientedTangent = area >= 0 ? tangent : -tangent; for ( int i=0; i<3; i++) { float3 normal = objNormal[i], binormal = cross (tangent,normal); float3x3 basis = float3x3 (orientedTangent, binormal, normal); float3 surfaceLightVector : TEXCOORD1 = mul (basis, objLight[i]); float3 surfaceViewVector : TEXCOORD2 = mul (basis, objView[i]); emitVertex (position[i], texCoord[i], surfaceLightVector, surfaceViewVector); } } additional & changed code for mirroring

12. Discarding Triangles Significantly Back Facing w.r.t. the Light TRIANGLE void md2bump_geometry( AttribArray < float4 > position : POSITION , AttribArray < float2 > texCoord : TEXCOORD0 , AttribArray < float3 > objPosition : TEXCOORD1 , AttribArray < float3 > objNormal : TEXCOORD2 , AttribArray < float3 > objView : TEXCOORD3 , AttribArray < float3 > objLight : TEXCOORD4 ) { float3 dXYZdU = objPosition[1] - objPosition[0]; float dSdU = texCoord[1].s - texCoord[0].s; float3 dXYZdV = objPosition[2] - objPosition[0]; float dSdV = texCoord[2].s - texCoord[0].s; float3 tangent = normalize (dSdV * dXYZdU - dSdU * dXYZdV); float maxLightZ, maxLightThreshold = -0.3; for ( int i=0; i<3; i++) { float3 normal = objNormal[i], binormal = cross (tangent,normal); float3x3 basis = float3x3 ( tangent , binormal, normal); float3 surfaceLightVector : TEXCOORD1 = mul (basis, objLight[i]); maxLightZ = i==0 ? normalize(surfaceLightVector).z : max(maxLightZ, normalize(surfaceLightVector).z); float3 surfaceViewVector : TEXCOORD2 = mul (basis, objView[i]); if (i < 2 || maxLightZ > maxLightThreshold) emitVertex (position[i], texCoord[i], surfaceLightVector, surfaceViewVector); } }

13. Graphics pipeline dataflow with geometry shader-based setup application Vertex shader Primitive assembly Geometry shader Rasterizer Fragment shader Raster operations framebuffer Animation and skinning of object-space vertices Output texture coordinates Output object-space light & view vectors Output clip-space position Compute object-to-texture space basis for triangle Transform object-space vectors to texture space Output triangle Access perturbed normal from normal map texture Compute lighting in texture space Output resulting color

14. Sans Per-vertex Normals

15. Geometry Shader in Cg Sans Normals TRIANGLE void md2bump_geometry_sans_normal( AttribArray < float4 > position : POSITION , AttribArray < float2 > texCoord : TEXCOORD0 , AttribArray < float3 > objPosition : TEXCOORD1 , // IGNORE per-vertex normal! AttribArray < float3 > objView : TEXCOORD3 , AttribArray < float3 > objLight : TEXCOORD4 ) { float3 dXYZdU = objPosition[1] - objPosition[0]; float2 dSTdU = texCoord[1] - texCoord[0]; float3 dXYZdV = objPosition[2] - objPosition[0]; float2 dSTdV = texCoord[2] - texCoord[0]; float3 tangent = normalize (dSTdV.s * dXYZdU - dSTdU.s * dXYZdV); float3 tangent2 = normalize (dSTdV.t * dXYZdU - dSTdU.t * dXYZdV); float area = determinant (float2x2(dSTdV, dSTdU)); tangent = area >= 0 ? tangent : -tangent; float3 normal = cross(tangent2,tangent); tangent2 = area >= 0 ? tangent2 : -tangent2; for ( int i=0; i<3; i++) { float3x3 basis = float3x3 (tangent, tangent2, normal); float3 surfaceLightVector : TEXCOORD1 = mul(basis, objLight[i]); float3 surfaceViewVector : TEXCOORD2 = mul(basis, objView[i]); emitVertex (position[i], texCoord[i], surfaceLightVector, surfaceViewVector); } }

16. Geometry shader setup with and without per-vertex normals Setup without per-vertex normals, relying on normalized gradients only; faceted look Setup with per-vertex normals; smoother lighting appearance

17. Visualizing Tangent, Normal, and Bi-normals for Discontinuities Using per-vertex normals Visualize normals Reasonably smooth Visualize tangents Flat per-triangle Visualize tangents Semi-smooth, semi-flat

18. Issue: Lighting Discontinuities Problem: Geometry shader computes single tangent vector for entire triangle See tangent visualization (prior slide) Still each vertex of triangle has its own unique normal so basis has some variation of triangle This per-triangle “tangent flatness” can lead to lighting discontinuities

19. Lighting Discontinuities Example of unsightly discontinuities when applying geometry shader bump mapping setup to a sphere:

20. Dealing with Lighting Discontinuities Final shading when bump mapping can hide these discontinuities somewhat Imperfect solution Perturb the normals within the normal map to compensate for the discontinuities How? I have some (unproven, uninvestigated) ideas

21. More Information and Examples Get the Cg Toolkit Cg 3.1 is available now http://developer.nvidia.com/cg-toolkit-download Or http://http.developer.nvidia.com/Cg/cg_3_1_0010.html See the “Download” section of this page Find source code once installed in examples/OpenGL/advanced/gs_md2render gs_md2bump Geometry shader bump map example gs_md2shadow Geometry shader bump map + shadow volumes example

22. Supplemental Slides

23. Authored 3D Model Inputs Key frame blended 3D model Decal skin Bump skin Gloss skin GPU Rendering

24. Animation via Key Frames or Vertex Skinning GPU Rendering Frame A Frame B Other possible key frames