• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

SIGGRAPH Asia 2008 Modern OpenGL

on

  • 21,095 views

A long-time implementer of OpenGL (Mark Kilgard, NVIDIA) and the system's original architect (Kurt Akeley, Microsoft) explain OpenGL's design and evolution. OpenGL's state machine is now a complex ...

A long-time implementer of OpenGL (Mark Kilgard, NVIDIA) and the system's original architect (Kurt Akeley, Microsoft) explain OpenGL's design and evolution. OpenGL's state machine is now a complex data-flow with multiple programmable stages. OpenGL practitioners can expect candid design explanations, advice for programming modern GPUs, and insight into OpenGL's future.

These slides were presented at SIGGRAPH Asia 2008 for the "Modern OpenGL: Its Design and Evolution" course.

Course abstract: OpenGL was conceived in 1991 to provide an industry standard for programming the hardware graphics pipeline. The original design has evolved considerably over the last 17 years. Whereas capabilities mandated by OpenGL such as texture mapping and a stencil buffer were present only on the world's most expensive graphics hardware back in 1991, now these features are completely pervasive in PCs and now even available in several hand-held devices. Over that time, OpenGL's original fixed-function state machine has evolved into a complex data-flow including several application-programmable stages. And the performance of OpenGL has increased from 100x to over 1,000x in many important raw graphics operations.

In this course, a long-time implementer of OpenGL and the system's original architect explain OpenGL's design and evolution.

You will learn how the modern (post-2006) graphics hardware pipeline is exposed through OpenGL. You will hear Kurt Akeley's personal retrospective on OpenGL's development. You will learn nine ways to write better OpenGL programs. You will learn how modern OpenGL implementations operate. Finally we discuss OpenGL's future evolution.

Whether you program with OpenGL or program with another API such as Direct3D, this course will give you new insights into graphics hardware architecture, programmable shading, and how to best take advantage of modern GPUs.

Statistics

Views

Total Views
21,095
Views on SlideShare
20,910
Embed Views
185

Actions

Likes
15
Downloads
719
Comments
2

18 Embeds 185

http://www.shader.jp 95
http://knitinr.blogspot.com 43
http://www.slideshare.net 19
http://knitinr.blogspot.in 4
http://knitinr.blogspot.co.uk 4
http://knitinr.blogspot.ca 3
http://www.mefeedia.com 3
http://209.85.175.132 2
http://74.125.153.132 2
http://knitinr.blogspot.de 2
http://knitinr.blogspot.it 1
http://knitinr.blogspot.ru 1
http://knitinr.blogspot.cz 1
http://knitinr.blogspot.kr 1
http://knitinr.blogspot.com.au 1
http://192.168.10.100 1
http://static.slideshare.net 1
http://knitinr.blogspot.ae 1
More...

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

12 of 2 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • thanks
    Are you sure you want to
    Your message goes here
    Processing…
  • thks.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

SIGGRAPH Asia 2008 Modern OpenGL SIGGRAPH Asia 2008 Modern OpenGL Presentation Transcript

  •  
  • Mark J. Kilgard, NVIDIA Kurt Akeley, Microsoft Research 13 December 2008 Singapore Modern OpenGL: Its Design and Evolution
  • Introductions
  • Kurt Akeley
    • Led development of OpenGL at Silicon Graphics (SGI)
      • Co-founded SGI
      • Lead development of SGI’s high-end graphics hardware
      • Co-author of OpenGL specification
    • Returned to Stanford University to complete Ph.D.
    • Co-developed Cg “C for graphics” language at NVIDIA
    • Principal Researcher, Microsoft Research Silicon Valley
      • Spent time at Microsoft Research Asia in Beijing
    • Member of US National Academy of Engineering
  • Mark Kilgard
    • Principal System Software Engineer, NVIDIA, Austin, Texas
      • Developed original OpenGL driver for 1 st GeForce GPU
      • Specified many key OpenGL extensions
      • Works on Cg for portable programmable shading
      • NVIDIA Distinguished Inventor
    • Before NVIDIA, worked at Silicon Graphics
      • Worked on X Window System integration for OpenGL
      • Developed popular OpenGL Utility Toolkit (GLUT)
    • Wrote book on OpenGL and X, co-authored Cg Tutorial
  • Marc Levoy
    • Moderator for our facilitated discussion
    • Professor of Computer Science and Electrical Engineering
      • Stanford University
    • SIGGRAPH Computer Graphics Achievement Award
    • ACM Fellow
  • Course Schedule
    • Modern OpenGL (Kilgard)
    • OpenGL’s evolution: a personal retrospective (Akeley)
    • Writing better OpenGL (Kilgard)
    • Implementing OpenGL (Kilgard)
    • OpenGL’s future evolution (Kilgard)
    • OpenGL in Context (Akeley, Kilgard, Levoy)
      • Facilitated conversation
    – Mid-session break –
  • Check Out the Course Notes (1)
    • Look to www.opengl.org web site for our final slides
    • New Material
      • “An Incomplete History of OpenGL” (Kilgard)
        • How the OpenGL graphics system developed
      • “Using Vertex Buffer Objects Well” (Kilgard)
        • Learn how to use Vertex Buffers objects for high vertex processing rates
  • Check Out the Course Notes (2)
    • Paper Reprints
      • OpenGL design rationale from its specification co-authors (Segal, Akeley)
      • Realizing OpenGL: two implementations of one architecture (Kilgard)
      • Graphics hardware: GTX, RealityEngine, InfiniteReality, GeForce 6800
        • Key developments in graphics hardware design over last 20 years
      • GPU Programmability: “User-Programmable Vertex Engine” and “Cg” SIGGAPH papers
      • “How GPUs Work” (Luebke, Humpherys)
  • Modern OpenGL Mark Kilgard Principal System Software Engineer NVIDIA
  • Modern OpenGL
    • History
      • How did OpenGL get where it is now?
    • Present
      • Version 3.0
      • Functionality beyond 3.0
  • An Overview History of OpenGL
    • Pre-history 1991
      • IRIS GL, a proprietary Graphics Library by SGI
    • OpenGL, an open standard for 3D
      • Focus: procedural hardware-accelerated 3D graphics
      • Governed by Architectural Review Board (ARB)
      • Extensibility planned into design
    • Competition
      • Proprietary APIs (1991-1995)
      • PHIGS & PEX for X Window System (1992-1997)
      • Microsoft’s Direct3D (1998-)
  • OpenGL’s Pre-history IRIS GL 1 Window system: MEX IRIS GL 2 Window system: MEX Operating system: UNIX IRIS GL 3 Window system: NeWS/X11 Operating system: IRIX 3. x IRIS GL 4 Window system: Native X11 Operating system: IRIX 4.3 OpenGL 1.0 Window system: Native X11 with GLX Operating system: IRIX 5.1 1991 1993 1988 1986 1983 First work on GL 5.0 proposal 1989 Dates are for shipping commercial SGI implementation 1983-2008 = 25 years
  • OpenGL’s Design Philosophy
    • High-performance
      • Assumes hardware acceleration
    • Defined by a specification
      • Rather than a de-facto implementation
    • Rendering state machine
      • Procedural
      • Not a window system, not a scene graph
    • No initial sub-setting
    • Extensible
    • Data type rich
    • Cross-platform
      • Window system-independent core
      • X Window System, Microsoft Windows, OS/2, OS X, etc.
    • Multi-language bindings
      • C, FORTRAN, etc.
      • Not merely an API, rather a system
  • Timeline of OpenGL’s Development 1992 1994 1996 1998 2000 2002 2004 2006 2008 OpenGL 1.0 approved OpenGL 1.1 OpenGL 1.2 Multitexture added (1.2.1) OpenGL 1.3 OpenGL 1.4 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 SGI Infinite- Reality OpenGL Utility Toolkit (GLUT) released Mesa 3D open source Khronos controls OpenGL 1 st GPU for PCs with single-chip transform & lighting for OpenGL (GeForce) NT 3.51 bring OpenGL to PCs OpenGL ES for embedded devices 1 st commercial OpenGL implementation (DEC)
  • Competitive 3D APIs
    • OpenGL has always existed in competition with other APIs
      • Strengthened OpenGL by driving feature parity
    • OpenGL’s competitive strengths:
      • Cross platform, open process
      • API stability, extensibility
      • Clean initial design & specification
    1992 1994 1996 1998 2000 2002 2004 2006 2008 Proprietary Unix workstation 3D APIs XGL Dor é Starbase IRIS GL X Consortium 3D standard PEX Microsoft Direct3D DirectX 3 DirectX 5 DirectX 6 DirectX 7 DirectX 8 DirectX 9 DirectX 10
  • OpenGL 1.0 1992 1994 1996 1998 2000 2002 2004 2006 2008 OpenGL 1.0 approved OpenGL 1.1 OpenGL 1.2 Multitexture added (1.2.1) OpenGL 1.3 OpenGL 1.4 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 SGI Infinite- Reality OpenGL Utility Toolkit (GLUT) released Mesa 3D open source Khronos controls OpenGL 1 st GPU for PCs with single-chip transform & lighting for OpenGL (GeForce) NT 3.51 bring OpenGL to PCs OpenGL ES for embedded devices 1 st commercial OpenGL implementation (DEC)
    • Immediate mode
    • Vertex transformation and lighting
    • Points, lines, polygons
    • Stippling, wide points and lines
    • Bitmaps, image rectangles, and pixel reads
    • Pixel store and transfer
    • 1D and 2D textures, fog, and scissor
    • Display lists and evaluators
    • RGBA and color index color models
    • Color, depth, stencil, and accumulation buffers
    • Selection and feedback modes
    • Queries
  • OpenGL State Machine
    • From OpenGL 3.0 specification, unchanged since 1.0
  • SGI “Classic” Hardware View of OpenGL 3D Application or Game
    • Entirely fixed-function, no programmability
    • High-end SGI hardware manifested functionality in distinct chips
    OpenGL API Front End Vertex Assembly Vertex Transform & Lighting Primitive Assembly, Clipping, Setup, and Rasterization Texture & Fog Texture Fetch Raster Operations Framebuffer Access Memory Interface Graphics Hardware Boundary 1992 Graphics data flow Memory operations Fixed-function unit Programmable unit
  • OpenGL 1.1 1992 1994 1996 1998 2000 2002 2004 2006 2008 OpenGL 1.0 approved OpenGL 1.1 OpenGL 1.2 Multitexture added (1.2.1) OpenGL 1.3 OpenGL 1.4 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 SGI Infinite- Reality OpenGL Utility Toolkit (GLUT) released Mesa 3D open source Khronos controls OpenGL 1 st GPU for PCs with single-chip transform & lighting for OpenGL (GeForce) NT 3.51 bring OpenGL to PCs OpenGL ES for embedded devices 1 st commercial OpenGL implementation (DEC)
    • Vertex arrays
    • Texture objects
    • Texture internal formats
    • Texture sub-image updates
    • Texture proxies
    • Copy framebuffer-to-texture
    • Polygon offset
    • RGBA logical operations
  • The Look of OpenGL 1.1 SGI skyfly demo Stenciled shadow volumes Ideas in Motion
  • OpenGL 1.2 1992 1994 1996 1998 2000 2002 2004 2006 2008 OpenGL 1.0 approved OpenGL 1.1 OpenGL 1.2 Multitexture added (1.2.1) OpenGL 1.3 OpenGL 1.4 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 SGI Infinite- Reality OpenGL Utility Toolkit (GLUT) released Mesa 3D open source Khronos controls OpenGL 1 st GPU for PCs with single-chip transform & lighting for OpenGL (GeForce) NT 3.51 bring OpenGL to PCs OpenGL ES for embedded devices 1 st commercial OpenGL implementation (DEC)
    • 3D textures
    • Texture edge clamp wrap mode
    • Texture level-of-detail clamping
    • BGRA component order
    • Packed pixel formats
    • Imaging subset ( optional )
    • Normal rescaling
    • Separate specular
    • Vertex array draw elements range
  • Akeley’s (Modernized) OpenGL Data Flow vertex shading rasterization & fragment shading texture raster operations framebuffer pixel unpack pixel pack vertex puller client memory pixel transfer glReadPixels / glCopyPixels / glCopyTex{Sub}Image glDrawPixels glBitmap glCopyPixels glTex{Sub}Image glCopyTex{Sub}Image glDrawElements glDrawArrays selection / feedback / transform feedback glVertex* glColor* glTexCoord* etc. blending depth testing stencil testing accumulation storage operations
  • OpenGL 1.2 Imaging Subset Color Table Convolution (separable or general) Post-convolve Scale & Bias Post-convolve Color Table Color Matrix Post-color matrix Scale & Bias Post-color matrix Color Table Histogram Min-max Look-up Table (RGBA-to-RGBA) Look-up Table (Index-to-RGBA) Scale & Bias Shift & Add Index pixels RGBA pixels Pixel Rectangle Rasterization core functionality ARB_imaging subset discard discard
  • OpenGL 1.2.1 1992 1994 1996 1998 2000 2002 2004 2006 2008 OpenGL 1.0 approved OpenGL 1.1 OpenGL 1.2 Multitexture added (1.2.1) OpenGL 1.3 OpenGL 1.4 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 SGI Infinite- Reality OpenGL Utility Toolkit (GLUT) released Mesa 3D open source Khronos controls OpenGL 1 st GPU for PCs with single-chip transform & lighting for OpenGL (GeForce) NT 3.51 bring OpenGL to PCs OpenGL ES for embedded devices 1 st commercial OpenGL implementation (DEC)
    • Multi-texture ( optional )
  • Multi-texture Poster Child: Quake 2 Light Maps  (modulate) = lightmaps only decal only combined scene
  • GeForce 256 (NV10) View of OpenGL 3D Application or Game
    • Vertex pulling (vertex buffer objects) via DMA
    • Dual-texture, cube maps, and register combiners
    OpenGL API GPU Front End Vertex Assembly Vertex Transform & Lighting Primitive Assembly, Clipping, Setup, and Rasterization Texture & Fog Texture Fetch Raster Operations Framebuffer Access Memory Interface CPU – GPU Boundary 1999 Attribute Fetch
  • Hardware Cube Maps Rendered scene Dynamically created cube map image Image credit: “Guts” GeForce 2 GTS demo, Thant Thessman
  • OpenGL 1.3 1992 1994 1996 1998 2000 2002 2004 2006 2008 OpenGL 1.0 approved OpenGL 1.1 OpenGL 1.2 Multitexture added (1.2.1) OpenGL 1.3 OpenGL 1.4 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 SGI Infinite- Reality OpenGL Utility Toolkit (GLUT) released Mesa 3D open source Khronos controls OpenGL 1 st GPU for PCs with single-chip transform & lighting for OpenGL (GeForce) NT 3.51 bring OpenGL to PCs OpenGL ES for embedded devices 1 st commercial OpenGL implementation (DEC)
    • Multi-texture ( required now )
    • Cube map texturing
    • Compressed texture formats
    • Texture border clamp
    • Texture environment functions
      • Add, combine, dot product
    • Multisample anti-aliasing
    • Transpose matrix
  • GeForce 3 & 4 Ti (NV2x) View of OpenGL 3D Application or Game
    • Programmable vertex processing
    • Highly configurable fragment processing
    OpenGL API GPU Front End Vertex Assembly Vertex Program Primitive Assembly, Clipping, Setup, and Rasterization Multi-texture shaders & Combiners Texture Fetch Raster Operations Framebuffer Access Memory Interface CPU – GPU Boundary 2001 Attribute Fetch
  • Vertex Programmability Paletted matrix skinning Twister vertex program Per-vertex cartoon shading
  • Configurable Fragment Processing Bumpy shiny environment mapping Chromatic aberration Offset 2D bump mapping Depth sprites
  • OpenGL 1.4 1992 1994 1996 1998 2000 2002 2004 2006 2008 OpenGL 1.0 approved OpenGL 1.1 OpenGL 1.2 Multitexture added (1.2.1) OpenGL 1.3 OpenGL 1.4 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 SGI Infinite- Reality OpenGL Utility Toolkit (GLUT) released Mesa 3D open source Khronos controls OpenGL 1 st GPU for PCs with single-chip transform & lighting for OpenGL (GeForce) NT 3.51 bring OpenGL to PCs OpenGL ES for embedded devices 1 st commercial OpenGL implementation (DEC)
    • Automatic mipmap generation
    • Shadow-mapping
      • Depth textures and shadow comparisons
    • Texture level-of-detail bias
    • Texture mirrored repeat wrap mode
    • Multi-texture combination
    • Fog coordinate
    • Secondary color
    • Configurable point size attenuation
    • Color blending improvements
    • Stencil wrap operations
    • Window-space raster position specification
  • Hardware Shadow Mapping Without shadow mapping With shadow mapping Depth map from light source’s view Darker is closer light position Projective Texturing (1.0) & Polygon Offset (1.1) key enablers
  • Shadow Mapping Explained Planar distance from light Depth map projected onto scene ≤ = less than True “un-shadowed” region shown green equals
  • OpenGL 1.5 1992 1994 1996 1998 2000 2002 2004 2006 2008 OpenGL 1.0 approved OpenGL 1.1 OpenGL 1.2 Multitexture added (1.2.1) OpenGL 1.3 OpenGL 1.4 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 SGI Infinite- Reality OpenGL Utility Toolkit (GLUT) released Mesa 3D open source Khronos controls OpenGL 1 st GPU for PCs with single-chip transform & lighting for OpenGL (GeForce) NT 3.51 bring OpenGL to PCs OpenGL ES for embedded devices 1 st commercial OpenGL implementation (DEC)
    • Vertex buffer objects (VBOs)
    • Occlusion queries
    • Generalized shadow mapping functions
  • GeForce FX (NV3x) View of OpenGL 3D Application or Game
    • Programmable fragment processing
      • 16 texture units, IEEE 754 32-bit floating-point
    • Vertex program branching
    OpenGL API GPU Front End Vertex Assembly Vertex Program Primitive Assembly, Clipping, Setup, and Rasterization Fragment Program Texture Fetch Raster Operations Framebuffer Access Memory Interface CPU – GPU Boundary 2003 Attribute Fetch
  • Floating-point Fragment Programmability
  • OpenGL Fragment Program Flowchart More Instructions? Read Interpolants and/or Registers Map Input values: Swizzle, Negate, etc. Perform Instruction Math / Operation Write Output Register with Masking Begin Fragment Fetch & Decode Next Instruction Temporary Registers initialized to 0,0,0,0 Output Depth & Color Registers initialized to 0,0,0,1 Initialize Parameters Emit Output Registers as Transformed Vertex End Fragment Fragment Program Instruction Loop Fragment Program Instruction Memory Texture Fetch Instruction? yes no no Compute Texture Address & Level-of-detail & Fetch Texels Filter Texels yes Texture Images Primitive Interpolants
  • Key Trend: Configurability becomes Programmability Fixed-function Programmable Simple Configurability Complex Configurability
  • Core OpenGL fragment texturing & coloring Line Rasterization Polygon Rasterization Pixel Rectangle Rasterization Bitmap Rasterization From Primitive Assembly DrawPixels Bitmap Conventional Texture Fetching Texture Environment Application Color Sum Fog To raster operations Coverage Application Texture Unit 0 Texture Unit 1 Texture Unit 0 Texture Unit 1 Point Rasterization
  • NV1x OpenGL fragment texturing & coloring Line Rasterization Polygon Rasterization Pixel Rectangle Rasterization Bitmap Rasterization From Primitive Assembly DrawPixels Bitmap Conventional Texture Fetching Texture Environment Application Color Sum Fog To raster operations Coverage Application Register Combiners Texture Unit 0 General Stage 1 Final Stage Texture Unit 1 General Stage 0 Texture Unit 0 Texture Unit 1 GL_REGISTER_COMBINERS_NV enable Point Rasterization
  • Register Combiners NV2x OpenGL fragment texturing & coloring Line Rasterization Polygon Rasterization Pixel Rectangle Rasterization Bitmap Rasterization From Primitive Assembly DrawPixels Bitmap Conventional Texture Fetching Texture Environment Application Color Sum Fog To raster operations Coverage Application Texture Shaders Final Combiner General Stage 7 … GLTEXTURE_SHADER_NV enable GL_REGISTER_COMBINERS_NV enable Texture Shader 3 … Texture Shader 1 Texture Shader 0 Point Rasterization General Stage 1 General Stage 0 Texture Unit 3 … Texture Unit 1 Texture Unit 0 Texture Unit 3 … Texture Unit 1 Texture Unit 0
  • Fragment Program Instruction 0 NV3x OpenGL fragment texturing & coloring Line Rasterization Polygon Rasterization Pixel Rectangle Rasterization Bitmap Rasterization From Primitive Assembly DrawPixels Bitmap Conventional Texture Fetching Texture Environment Application Color Sum Fog To raster operations Coverage Application Texture Shaders Final Combiner General Stage 7 … … Fragment Program Fragment Program Instruction 1023 GL_REGISTER_COMBINERS_NV enable GLTEXTURE_SHADER_NV enable GL_FRAGMENT_PROGRAM_NV enable !!FP1.0 or !!ARBfp1.0 programs Texture Shader 3 … Texture Shader 1 Texture Shader 0 Point Rasterization General Stage 1 General Stage 0 Texture Unit 3 … Texture Unit 1 Texture Unit 0 Texture Unit 3 … Texture Unit 1 Texture Unit 0
  • OpenGL 2.0 1992 1994 1996 1998 2000 2002 2004 2006 2008 OpenGL 1.0 approved OpenGL 1.1 OpenGL 1.2 Multitexture added (1.2.1) OpenGL 1.3 OpenGL 1.4 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 SGI Infinite- Reality OpenGL Utility Toolkit (GLUT) released Mesa 3D open source Khronos controls OpenGL 1 st GPU for PCs with single-chip transform & lighting for OpenGL (GeForce) NT 3.51 bring OpenGL to PCs OpenGL ES for embedded devices 1 st commercial OpenGL implementation (DEC)
    • Programmable shading
      • OpenGL Shading Language (GLSL)
    • Multiple color buffer rendering targets
    • Non-power-of-two texture dimensions
    • Point sprites
    • Separate blend equation
    • Two-sided stencil testing
  • GeForce 6 & 7 (NV4x/G7x) View of OpenGL 3D Application or Game
    • Limited vertex texturing
    • Fragment branching
    • Multiple render targets & floating-point blending
    OpenGL API GPU Front End Vertex Assembly Vertex Program Primitive Assembly, Clipping, Setup, and Rasterization Fragment Program Texture Fetch Raster Operations Framebuffer Access Memory Interface CPU – GPU Boundary 2004 Attribute Fetch
  • GeForce 8 & 9 (G8x/G9x) View of OpenGL Primitive Program 3D Application or Game
    • Primitive (geometry) programs
    • Parameter reads from buffer objects
    • Transform feedback (stream out)
    OpenGL API GPU Front End Vertex Assembly Vertex Program , Clipping, Setup, and Rasterization Fragment Program Texture Fetch Raster Operations Framebuffer Access Memory Interface CPU – GPU Boundary 2006 Attribute Fetch Primitive Assembly Parameter Buffer Read
  • OpenGL Pipeline Fixed-function Steps Primitive Program
    • Much of functional pipeline remains fixed-function
      • Vital to maintaining performance and data flow
      • Hard to compete with hard-wired rasterization, Zcull, and pixel compression
    GPU Front End Vertex Assembly Vertex Program , Clipping, Setup, and Rasterization Fragment Program Texture Fetch Raster Operations Framebuffer Access Memory Interface 2006 Attribute Fetch Primitive Assembly Parameter Buffer Read
  • OpenGL Pipeline Programmable Domains Primitive Program
    • New geometry shader domain for per-primitive programmable processing
    • Unified Streaming Processor Array (SPA) architecture means same capabilities for all domains
    GPU Front End Vertex Assembly Vertex Program , Clipping, Setup, and Rasterization Fragment Program Texture Fetch Raster Operations Framebuffer Access Memory Interface 2006 Attribute Fetch Primitive Assembly Parameter Buffer Read Can be unified hardware!
  • OpenGL 2.1 1992 1994 1996 1998 2000 2002 2004 2006 2008 OpenGL 1.0 approved OpenGL 1.1 OpenGL 1.2 Multitexture added (1.2.1) OpenGL 1.3 OpenGL 1.4 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 SGI Infinite- Reality OpenGL Utility Toolkit (GLUT) released Mesa 3D open source Khronos controls OpenGL 1 st GPU for PCs with single-chip transform & lighting for OpenGL (GeForce) NT 3.51 bring OpenGL to PCs OpenGL ES for embedded devices 1 st commercial OpenGL implementation (DEC)
    • OpenGL Shading Language (GLSL) improvements
      • Non-square matrices
    • Pixel buffer objects (PBOs)
    • sRGB color space texture formats
  • OpenGL 3.0 1992 1994 1996 1998 2000 2002 2004 2006 2008 OpenGL 1.0 approved OpenGL 1.1 OpenGL 1.2 Multitexture added (1.2.1) OpenGL 1.3 OpenGL 1.4 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 SGI Infinite- Reality OpenGL Utility Toolkit (GLUT) released Mesa 3D open source Khronos controls OpenGL 1 st GPU for PCs with single-chip transform & lighting for OpenGL (GeForce) NT 3.51 bring OpenGL to PCs OpenGL ES for embedded devices 1 st commercial OpenGL implementation (DEC)
    • OpenGL Shading Language (GLSL) improvements
      • New texture fetches
      • True integer data types and operators
      • switch/case/default flow control statements
    • Conditional rendering based on occlusion query results
    • Transform feedback
    • Vertex array objects
    • Floating-point textures, color buffers, and depth buffers
    • Half-precision vertex arrays
    • Texture arrays
    • Integer textures
    • Red and red-green texture formats
      • Compressed red and red-green formats
    • Framebuffer objects (FBOs)
    • Packed depth-stencil pixel formats
    • Per-color buffer clearing, blending, and masking
    • sRGB color space color buffers
    • Fine-grain buffer mapping and flushing
  • Areas of 3.0 Functionality Improvement
    • Programmability
      • Shader Model 4.0 features
      • OpenGL Shading Language (GLSL) 1.30
    • Texturing
      • New texture representations and formats
    • Framebuffer operations
      • Framebuffer objects
      • New formats
      • New copy (blit), clear, blend, and masking operations
    • Buffer management
      • Non-blocking and fine-grain update of buffer object data stores
    • Vertex processing
      • Vertex array configuration objects
      • Conditional rendering for occlusion culling
      • New half-precision vertex attribute formats
    • Pixel processing
      • New half-precision external pixel formats
    All Brand New Core Features
  • OpenGL 3.0 Programmability
    • Shader Model 4.0 additions
      • True signed & unsigned integer values
        • True integer operators: ^ , & , | , << . >> , % , ~
      • Texture additions
        • Texture arrays
        • Base texture size queries
        • Texel offsets to fetches
        • Explicit LOD and derivative control
      • Integer samplers
      • Interpolation modifiers: centroid , noperspective , and flat
      • Vertex array element number: gl_VertexID
    • OpenGL Shading Language (GLSL) improvements
      • ## concatenation in pre-processor for macros
      • switch / case / default statements
  • OpenGL 3.0 Texturing Functionality
    • Texture representation
      • Texture arrays: indexed access to a set of 1D or 2D texture images
    • Texture formats
      • Floating-point texture formats
        • Single-precision (32-bit, IEEE s23e8)
        • Half-precision (16-bit, s10e5)
      • Red & red/green texture formats
        • Intended as FBO framebuffer formats too
      • Compressed red & red/green texture formats
      • Shared exponent texture formats
      • Packed floating-point texture formats
  • Texture Arrays
    • Conventional texture = One logical pre-filtered image
    • Texture array = index-able plurality of pre-filtered images
      • Rationale is fewer texture object binds when drawing different objects
      • No filtering between mipmap sets in a texture array
      • All mipmap sets in array share same format/border & base dimensions
      • Both 1D and 2D texture arrays
      • Require shaders, no fixed-function support
    • Texture image specification
      • Use glTexImage3D, glTexSubImage3D, etc. to load 2D texture arrays
        • No new OpenGL commands for texture arrays
      • 3 rd dimension specifies integer array index
        • No halving in 3 rd dimension for mipmaps
        • So 64 ×128x17 reduces to 32×64×17 all the way to 1×1×17
  • Texture Arrays Example
    • Multiple skins packed in texture array
      • Motivation : binding to one multi-skin texture array avoids texture bind per object
    Texture array index 0 1 2 3 4 0 1 2 3 4 Mipmap level index
  • Compact Floating-point Textures
    • Shared exponent & packed float representations are ideal of High Dynamic Range (HDR) applications
  • Compact Floating-point Texture Formats
    • Packed float format
      • No sign bit, independent exponents
    • Shared exponent format
      • No sign bit, shared exponent, no implied leading 1
    5-bit mantissa 5-bit exponent 6-bit mantissa 5-bit exponent 6-bit mantissa 5-bit exponent bit 31 bit 0 9-bit mantissa 5-bit shared exponent 9-bit mantissa 9-bit mantissa bit 31 bit 0
  • 1- and 2-component Block Compression Scheme
    • Basic 1-component block compression format
      • Borrowed from alpha compression scheme of S3TC 5
    8-bit B 8-bit A 2 min/max values 64 bits total per block + 4x4 Pixel Decoded Block Encoded Block 16 pixels x 8-bit/componet = 128 bits decoded so effectively 2:1 compression 16 bits
  • Framebuffer Operations
    • Framebuffer objects
      • Standardized framebuffer objects (FBOs) for rendering to textures and renderbuffers
        • Render-to-texture
      • Multisample renderbuffers for FBOs
    • Framebuffer operations
      • Copies from one FBO to another, including multisample data
      • Per-color attachment color clears, blending, and write masking
    • Framebuffer formats
      • Floating-point color buffers
      • Floating-point depth buffers
      • Rendering into framebuffer format with 3 small unsigned floating-point values packed in a 32-bit value
      • Rendering into sRGB color space framebuffers
  • Framebuffer Object Example
    • Depth peeling for correctly ordered transparency
      • Great render-to-texture application for FBOs
  • Depth Peeling Behind the Scenes
    • Depth buffer has closest fragment at all pixels
      • Save depth buffer
    • Render again, but use depth buffer as shadow map
      • Discard fragment in front of shadow map’s depth value
      • Effectively peels one layer of depth!
    • Resulting color buffer is 2 nd closest fragment
      • And depth buffer for 2 nd closest fragments’ depth
    • Now repeat peeling more layers
      • Use ping-pong depth buffer scheme
      • Use occlusion query to detect when no more fragments to peel
    • Composite color layers front-to-back (or back-to-front)
      • Front-to-back peeling can be done during the peeling process
  • Delicate Color Fidelity with sRGB
    • Problem : PC display devices have non-linear (sRGB) display gamut—delicate color shading looks wrong
    Conventional rendering (uncorrected color) Gamma correct (sRGB rendered) Softer and more natural Unnaturally deep facial shadows NVIDIA’s Adriana GeForce 8 Launch Demo
  • What is sRGB?
    • A standard color space
      • Intended for monitors, printers, and the Internet
      • Created cooperatively by HP and Microsoft
      • Non-linear, roughly gamma of 2.2
      • Intuitively “encodes more dark values”
    • OpenGL 2.1 already added sRGB texture formats
      • Texture fetch converts sRGB to linear RGB, then filters
      • Result takes more than 8-bit fixed-point to represent in shader
    • 3.0 adds complementary sRGB framebuffer support
      • “ sRGB correct blending” converts framebuffer sRGB to linear, blend with linear color from shader, then convert back to sRGB
      • Works with FrameBuffer Objects (FBOs)
    sRGB chromaticity
  • So why sRGB? Standard Windows Display is Not Gamma Corrected
    • 25+ years of PC graphics, icons, and images depend on not gamma correcting displays
      • sRGB textures and color buffers compensates for this
    “ Expected” appearance of Windows desktop & icons but 3D lighting too dark Wash-ed out desktop appearance if color response was linear but 3D lighting is correct Gamma 1.0 Gamma 2.2 linear color response
  • Vertex Processing
    • Vertex array configuration
      • Objects to manage vertex array configuration client state
      • Half-precision floating-point vertex array formats
    • Vertex output streaming
      • Stream transformed vertex results into buffer object data stores
    • Occlusion culling
      • Skip rendering based on occlusion query result
  • Miscellaneous
    • Pixel Processing
      • Half-precision floating-point pixel external formats
    • Buffer Management
      • Non-blocking and fine-grain update of buffer object data stores
  • ARB Extensions to OpenGL 3.0
    • OpenGL 3.0 standard provides new ARB extensions
      • Extensions go beyond OpenGL 3.0
        • Standardized at same time as OpenGL 3.0
      • Support features in hardware today
    • Specifically
      • ARB_geometry_shader4 —provides per-primitive programmable processing
      • ARB_draw_instanced —gives shader access to instance ID
      • ARB_texture_buffer_object —allows buffer object to be sampled as a huge 1D unfiltered texture
    • Shipping today
      • NVIDIA driver provides all three
  • Transform Feedback for Terrain Generation by Recursive Subdivision
    • Geometry shaders + transform feedback
    • Render quads (use 4-vertex line adjacency primitive) from vertex buffer object
    • Fetch height field
    • Stream subdivided positions and normals to transform feedback “other” buffer object
    • Use buffer object as vertex buffer
    • Repeat, ping-pong buffer objects
    • Computation and data all stays on the GPU!
  • Skin Deformation
    • Capture & re-use geometric deformations
    Transform feedback allows the GPU to calculate the interactive, deforming elastic skin of the frog
  • Silhouette Edge Rendering
    • Uses geometry shader
    silhouette edge detection geometry shader Complete mesh Silhouette edges Useful for non-photorealistic rendering Looks like human sketching
  • More Geometry Shader Examples Shimmering point sprites Generate fins for lines Generate shells for fur rendering
  • Improved Interpolation Techniques
    • Using geometry shader functionality
    Quadratic normal interpolation True quadrilateral rendering with mean value coordinate interpolation
  • “Fair” Quadrilateral Interpolation
    • glBegin (GL_QUADS);
    • glColor3fv (red); glVertex3fv (lowerLeft);
    • glColor3fv (green); glVertex3fv (lowerRight);
    • glColor3fv (red); glVertex3fv (upperRight);
    • glColor3fv (blue); glVertex3fv (upperLeft);
    • glEnd ();
    • Geometry shader actually operates on 4-vertex GL_LINE_ADJACENCY primitives instead of quads
    Wrong , slash triangle split Wrong , backslash triangle split Better : Mean value coordinates
  • OpenGL 2.x ARB Extensions
    • Many OpenGL 3.0 extensions have corresponding ARB extensions for OpenGL 2.1 implementations to advertise
      • Helps get 3.0 functionality out sooner, rather than later
    • New ARB extensions for 3.0 functionality
      • ARB_framebuffer_object —framebuffer objects (FBOs) for render-to-texture
      • ARB_texture_rg —red and red/green texture formats
      • ARB_map_buffer_region —non-blocking and fine-grain update of buffer object data stores
      • ARB_instanced_arrays —instance ID available to shaders
      • ARB_half_float_vertex — half-precision floating-point vertex array formats
      • ARB_framebuffer_sRGB — rendering into sRGB color space framebuffers
      • ARB_texture_compression_rgtc —compressed red and red/green texture formats
      • ARB_depth_buffer_float —floating-point depth buffers
      • ARB_vertex_array_object —objects to manage vertex array configuration client state
  • Beyond OpenGL 3.0
    • OpenGL 3.0
    • EXT_gpu_shader4
    • NV_conditional_render
    • ARB_color_buffer_float
    • NV_depth_buffer_float
    • ARB_texture_float
    • EXT_packed_float
    • EXT_texture_shared_exponent
    • NV_half_float
    • ARB_half_float_pixel
    • EXT_framebuffer_object
    • EXT_framebuffer_multisample
    • EXT_framebuffer_blit
    • EXT_texture_integer
    • EXT_texture_array
    • EXT_packed_depth_stencil
    • EXT_draw_buffers2
    • EXT_texture_compression_rgtc
    • EXT_transform_feedback
    • APPLE_vertex_array_object
    • EXT_framebuffer_sRGB
    • APPLE_flush_buffer_range (modified)
    • In GeForce 8, 9, & 2xx Series but not yet core
    • EXT_geometry_shader4 (now ARB)
    • EXT_bindable_uniform
    • NV_gpu_program4
    • NV_parameter_buffer_object
    • EXT_texture_compression_latc
    • EXT_texture_buffer_object (now ARB)
    • NV_framebuffer_multisample_coverage
    • NV_transform_feedback2
    • NV_explicit_multisample
    • NV_multisample_coverage
    • EXT_draw_instanced (now ARB)
    • EXT_direct_state_access
    • EXT_vertex_array_bgra
    • EXT_texture_swizzle
    Plenty of proven OpenGL extensions for OpenGL Working Group to draw upon for OpenGL 3.1
  • OpenGL Version Evolution
    • Now OpenGL is part of Khronos Group
      • Previously OpenGL’s evolution was governed by the OpenGL Architectural Review Board (ARB)
      • Now officially a Khronos working group
        • Khronos also standardizes OpenCL, OpenVG, etc.
    • How OpenGL version updates happen
      • OpenGL participants proposing extensions
      • Successful extensions are polished and incorporated into core
      • OpenGL 3.0 is great example of this process
        • Roughly 20 extensions folded into “core”
        • Just 3 of those previously unimplemented
  • OpenGL Extensions by Source
    • 44% of extensions are “core” or multi-vendor
    • Lots of vendors have initiated extensions
    • Extending OpenGL is industry-wide collaboration
    EXT SGI SGIS SGIX ARB NV Others Others ATI APPLE MESA Source: http://www.opengl.org/registry (Dec 2008)
  • What’s Driving OpenGL Modernization? Human desire for Visual Intuition and Entertainment Embarrassing Parallelism of Graphics Increasing Semiconductor Density Particularly the hardware-amenable, latency tolerant nature of rasterization Particularly interactive video games
  • Kurt Akeley Principal Researcher Microsoft Research Silicon Valley OpenGL’s Evolution: A Personal Retrospective
  • A personal retrospective
    • My background:
      • Silicon Graphics, 1982-2001
      • OpenGL, 1990-2004
    • Today’s topics:
      • Computer architecture
      • Culture and process
    • For a more complete coverage see:
      • https://graphics.stanford.edu/wikis/cs448-07-spring/
      • Mark Kilgard’s excellent course notes
  • Jim Clark and the Geometry Engine
    • This text is 24 points
      • Sub bullets look like this
    The Geometry Engine: A VLSI Geometry System for Graphics Computer Graphics, Volume 16, Number 3 (Proceedings of SIGGRAPH 1982) p127-133, 1982
  • Jim’s helpers: the Stanford gang IRIS GL Geometry Engine IRIS GL Hardware back-end Hardware front-end
  • Success! (in 1995)
  • Computer Architecture
  • What is computer architecture?
    • Architecture: “the minimal set of properties that determine what programs will run and what results they will produce”
    • Implementation: “the logical organization of the [computer’s] dataflow and controls”
    • Realization: “the physical structure embodying the implementation”
  • Example: the analog clock
    • Architecture
      • Circular dial divided into twelfths
      • Hour hand (short) and minute hand (long)
    Example from Computer Architecture, Concepts and Evolution , Gerrit A. Blaauw and Frederick P. Brooks, Jr., Addison-Wesley, 1997
    • Implementation
      • A weight, driving a pendulum, or
      • A spring, driving a balance wheel, or
      • A battery, driving an oscillator, or ….
    • Realization
      • Gear ratios, pendulum lengths, battery sizes, ...
    12 11 10 6 8 9 7 5 4 2 1 3
  • A useful distinction
    • NVIDIA 8800
      • SIMD, or
      • SPMD ?
    Thread Processor Vertex Thread Issue Setup / Rasterization / ZCull Primitive Thread Issue Fragment Thread Issue Data Assembler Application
    • Architecture:
      • SPMD
    • Implementation:
      • SIMD
    • Realization:
      • ASIC
    SIMD = Single Instruction, Multiple Data SPMD = Single Program, Multiple Data ASIC = Application Specific Integrated Circuit L2 FB SP SP L1 TF SP SP L1 TF SP SP L1 TF SP SP L1 TF SP SP L1 TF SP SP L1 TF SP SP L1 TF SP SP L1 TF L2 FB L2 FB L2 FB L2 FB L2 FB
  • The mainstream view
    • Table of Contents:
      • Fundamentals
      • Instruction Sets
      • Pipelining
      • Advanced Pipelining and ILP
      • Memory-Hierarchy Design
      • Storage Systems
      • Interconnection Networks
      • Multiprocessors
  • OpenGL is an architecture All errors specified No side effects Little undefined operation No undefined operation Validity of inputs Specification rules! When implementation errors are found, they are fixed. Enforcement No performance queries Not a formal aspect of architecture Speed No feature sub-setting Configuration attributes (e.g., framebuffer) Can vary amount of resource (e.g., memory) Configuration Carefully planned, though mistakes were made It’s an architecture, whether it was planned or not  . Intentional design Top-level goal Conformance tests, … Code runs equivalently on all implementations Compatibility SGI Indy/Indigo/InfiniteReality NVIDIA GeForce, ATI Radeon, … IBM 360 30/40/50/65/75 Amdahl Different implementations OpenGL Blaauw/Brooks
  • But OpenGL is an API (Application Programming Interface)
    • Yes, Blaauw and Brooks talk about (computer) architecture as though it is always expressed as ISA (Instruction-Set Architecture)
    • But …
      • API is just a higher-level programming interface
      • “ Instruction-Set” Architecture implies other types of computer architectures (such as “API” Architecture)
      • OpenGL has evolved to include ISA-like interfaces (e.g., the interface below GLSL)
  • We didn’t know …
    • No mention in spec (even 3.0)
      • “ We view OpenGL as a state …”
    • First use in “ARB”
      • Architecture Review Board
      • Coined by Bill Glazier from “Palo Alto Architecture Review Board”
    • First formal usage (I know of)
      • Mark J. Kilgard, Realizing OpenGL: two implementations of one
      • architecture, Proceedings of the ACM SIGGRAPH/EUROGRAPHICS
      • workshop on Graphics hardware, p.45-55, August 03-04, 1997, Los Angeles, California, United States.
  • Fred is magnanimous
  • What is implied by “programmable”?
    • What does it mean to teach programming?
      • Does running a microwave oven count?
      • Does defining the geometry of a game “level” count?
      • Does specifying OpenGL modes count?
    • This seems to be a somewhat open question
      • Butler Lampson couldn’t tell me  .
      • Microsoft developers of teaching tools couldn’t tell me.
      • An online search wasn’t very helpful.
    • Do we just “know it when we see it”?
      • Justice Potter Stewart’s definition of pornography
  • My try at some formalization
    • Key ideas:
      • Composition  choice of placement, sequence
      • Non-obvious  s emantics are interesting and novel
      • Imperative  maybe there are other kinds of programming
    “ Composition, the organization of elemental operations into a non-obvious whole, is the essence of imperative programming.” -- Kurt Akeley (Foreword to GPU Gems 3)
  • OpenGL has always been programmable
    • Follows directly from being an “architecture”
      • OpenGL commands are instructions (API as an ISA)
      • They can be “composed” to create programs
      • Multi-pass rendering is the prototypical example
      • But Peercy et al. implemented a RenderMan shader compiler
      • Invariance was specified from the start (e.g., same fragments)
    • We set out to enable “usage that we didn’t anticipate”
      • Obvious for a traditional ISA (e.g., IA32)
      • Not so obvious for a graphics API
      • Example: texture applies to all primitives, not just triangles
  • Example multi-pass OpenGL “program” glEnable(GL_DEPTH_TEST); glDisable(GL_LIGHTING); glColorMask( false , false , false , false ); glEnable(GL_POLYGON_OFFSET_FILL); glPolygonOffset( maxwidth /2, 1); draw solid objects glDepthMask(GL_FALSE); glColorMask( true , true , true , true ); glColor3f( linecolor ); glDisable(GL_POLYGON_OFFSET_FILL); glPolygonMode(GL_FRONT_AND_BACK, GL_LINE); draw solid objects again glDisable(GL_DEPTH_TEST); glPolygonMode(GL_FRONT_AND_BACK, GL_FILL); glDepthMask(GL_TRUE); Hidden-line rendering
  • Example multi-pass OpenGL “program” glEnable(GL_DEPTH_TEST); glDisable(GL_LIGHTING); glColorMask( false , false , false , false ); glEnable(GL_POLYGON_OFFSET_FILL); glPolygonOffset( maxwidth /2, 1); draw solid objects glDepthMask(GL_FALSE); glColorMask( true , true , true , true ); glColor3f(1, 1, 1); glDisable(GL_POLYGON_OFFSET_FILL); glPolygonMode(GL_FRONT_AND_BACK, GL_LINE); glEnable(GL_CULL_FACE); glCullFace(GL_FRONT); draw solid objects again draw true edges // for a complete hidden-line drawing glDisable(GL_DEPTH_TEST); glPolygonMode(GL_FRONT_AND_BACK, GL_FILL); glDepthMask(GL_TRUE); glDisable(GL_CULL_FACE); Additions to the hidden-line algorithm (previous slide) highlighted in red Silhouette rendering
  • Invariance Corollary 1 Fragment generation is invariant with respect to the state values marked with in Rule 2.
    • Intended to capture complete sequence of operations
    • Also inspired design changes
  • Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations Framebuffer Texture memory Pixel assembly (unpack) Pixel operations Pixel pack Vertex pipeline Pixel pipeline Application All primitives (including pixels) are rasterized All vertexes are treated equally (e.g., lighted) All fragments are treated equally (e.g., texture mapped and depth-buffered) Not a required implementation, but “abstraction distance” matters
  • Culture and Process
  • Suppose … http://www.opengl.org/registry/ Name ARB_texture_cube_map Name Strings GL_ARB_texture_cube_map Notice Copyright OpenGL Architectural Review Board, 1999. Contact Michael Gold, NVIDIA (gold 'at' nvidia.com) Status Complete. Approved by ARB on 12/8/1999 Version Last Modified Date: December 14, 1999 Number ARB Extension #7 Dependencies None. Written based on the wording of the OpenGL 1.2.1 specification but not dependent on it. Overview This extension provides a new texture generation scheme for cube map textures. Instead of the current texture providing a 1D, 2D, or 3D lookup into a 1D, 2D, or 3D texture image, the texture is a set of six 2D images representing the faces of a cube. The (s,t,r) texture coordinates …
  • Complete specification Name Name Strings Notice Contact Status Version Number Dependencies Overview Issues New Procedures and Functions New Tokens Additions to Chapter 2 of the OpenGL Specification Additions to Chapter 3 of the OpenGL Specification Additions to Chapter 4 of the OpenGL Specification Additions to Chapter 5 of the OpenGL Specification Additions to Chapter 6 of the OpenGL Specification Additions to the GLX Specification Errors New State (type, query mechanism, initial value, attribute set, specification section) Usage Examples
  • 19 issues The spec just linearly interpolates the reflection vectors computed per-vertex across polygons. Is there a problem interpolating reflection vectors in this way? Probably. The better approach would be to interpolate the eye vector and normal vector over the polygon and perform the reflection vector computation on a per-fragment basis. Not doing so is likely to lead to artifacts because angular changes in the normal vector result in twice as large a change in the reflection vector as normal vector changes. The effect is likely to be reflections that become glancing reflections too fast over the surface of the polygon. Note that this is an issue for REFLECTION_MAP_ARB, but not NORMAL_MAP_ARB.
  • 19 issues … What happens if an (s,t,q) is passed to cube map generation that is close to (0,0,0), ie. a degenerate direction vector? RESOLUTION: Leave undefined what happens in this case (but may not lead to GL interruption or termination). Note that a vector close to (0,0,0) may be generated as a result of the per-fragment interpolation of (s,t,r) between vertices.
  • Trust and integrity
    • Lots of collaboration during the initial design
      • But final decisions made by a small group
    • SGI played fair
      • OpenGL 1.0 didn’t favor SGI equipment (our ports were late  )
      • SGI obeyed all conformance rules
      • SGI didn’t adjust the spec to match our equipment
    • The ARB avoided marketing tasks such as benchmarks
      • We stuck with technical design issues
    • We documented rigorously
      • Specification, man pages, …
  • Five Kinkos in Austin Texas The OpenGL Graphics System: A Specification (Version 1.1) Mark Segal Kurt Akeley Editor: Chris Frazier Copyright © 1992-1997 Silicon Graphics, Inc. This document contains unpublished information of Silicon Graphics, Inc.
  • Extension facts
    • 442 Vendor and “EXT” extension specifications
      • Vendor: specific to a single vendor
      • EXT: shared by two or more vendors
    • 56 “ARB” extensions
      • Standardized , likely to be in the next spec revision
    • Lots of text …
    Source: OpenGL extension registry, December 2008
  • “ Specification” sizes 9,079,063 1,076,008 209,426 All 442 Extensions 3,998,303 632,515 86,783 Old Testament 1,197,812 188,430 27,319 New Testament 5,214,085 823,647 114,535 King James Bible 2,221,347 263,908 48,674 56 ARB Extensions Chars Words Lines
  • Beyond the specification
    • The ARB (now replaced with Khronos)
      • Rules of order, secretary, IP, …
    • The extension process
      • Categories, token syntax, spec templates, enums, registry, …
    • Licensing
    • Conformance
  • Summary
    • Many mistakes made (see other presentations for lists)
    • Created a sustainable culture that values quality and rigorous documentation
    • Defined and evolved the architecture for interactive 3-D computer graphics
  • Writing better OpenGL Mark Kilgard Principal System Software Engineer NVIDIA
  • Motivation
    • Complex APIs and systems have pitfalls
      • After 17 years of designed evolution, OpenGL certainly has its share
    • Normal documentation focus:
      • What can you do?
      • Rather than: What should you do?
  • Communicating Vertex Data
    • The way you learn OpenGL:
      • Immediate mode
        • glBegin , glColor3f , glVertex3f , glEnd
      • Straightforward—no ambiguity about vertex data is
        • All vertex components are function parameters
    • The problem—too function call intensive
      • And all vertex data must flow through CPU
  • Example Scenario
    • An OpenGL application has to render a set of rectangles
    • Rectangle with its parameters
      • x, y, height, width, left color, right color, depth
    ( x,y ) depth order 0.0 1.0 left side color right side color height width
  • Scene Representation
    • Each rectangle specified by following RectInfo structure:
    • Array of RectInfo structures describes “scene”
      • Simplistic scene for sake of teaching
    typedef struct { GLfloat x, y, width, height; GLfloat depth_order; GLfloat left_side_color[3]; // red, green, then blue GLfloat right_side_color[3]; // red, green, then blue } RectInfo;
  • Example Scene and Rendering Result
    • Scene of 4 rectangles:
    • RectInfo rect_list[4] = {
    • { 10, 20, 180, 140, 0.5,
    • { 1, 1, 1 }, { 1, 0, 1 } },
    • { 30, 40, 100, 60, 0.5,
    • { 1, 0, 0 }, { 0, 0, 1 } },
    • { 140, 60, 100, 80, 0.5,
    • { 0, 0, 1 }, { 0, 1, 0 } },
    • { 70, 120, 80, 60, 0.7,
    • { 1, 1, 0 }, { 0, 1, 1 } },
    • };
    • OpenGL-rendered result
  • Immediate Mode Rectangle Rendering
    • Given sized RectInfo array, render vertices of quads
    1 st vertex 2 nd vertex 3 rd vertex 4 th vertex void drawRectangles( int count, const RectInfo *list) { glBegin ( GL_QUADS ); for ( int i=0; i<count; i++) { const RectInfo *r = &list[i]; glColor3fv (r->left_side_color); glVertex3f (r->x, r->y, r->depth_order); glColor3fv (r->right_side_color); glVertex3f (r->x+r->width, r->y, r->depth_order); // right_side_color “sticks” glVertex3f (r->x+r->width, r->y+r->height, r->depth_order); glColor3fv (r->left_side_color); glVertex3f (r->x, r->y+r->height, r->depth_order); } glEnd (); } For each rectangle
  • Critique of Immediate Mode
    • Advantages
      • Straightforward to code and debug
      • Easy-to-understand conceptual model
        • Building stream of vertices with OpenGL commands
      • Avoids driver & application copies of vertex data
      • Flexible, allowing totally dynamic vertex generation
    • Disadvantages
      • Rendering continuously streams attributes through CPU
        • Pollutes CPU cache with vertex data
      • Function call intensive
      • Unable to saturate fast graphics hardware
        • CPUs just too slow
    • Contrast with vertex array approach…
  • Vertex Array Approach
    • Step 1 : Copy vertex attributes into vertex arrays
      • From: RectInfo array ( CPU memory )
      • To: interleaved arrays of vertex attributes ( CPU memory )
    • Step 2 : To render
      • Configure OpenGL vertex array client state
        • Use glEnableClientState , glVertexPointer , glColorPointer
      • Render quads based on indices into vertex arrays
        • Use glDrawArrays
  • Vertex Array Format
    • Interleave vertex attributes in color & position arrays
    24 bytes per vertex color position float = 4 bytes vertex 0 vertex 1 red green blue x y z red green blue x y z color position
  • Step 1: Copy Rectangle Attributes to Vertex Arrays
    • void *initVarrayRectangles( int count, const RectInfo *list)
    • {
    • void *varray = ( char *) malloc ( sizeof ( GLfloat )*6*4*count);
    • GLfloat *p = varray;
    • for ( int i=0; i<count; i++, p+=24) {
    • const RectInfo *r = &list[i];
    • // quad vertex #1
    • memcpy (&p[0], r->left_side_color, sizeof( GLfloat )*3);
    • p[3] = r->x; p[4] = r->y; p[5] = r->depth_order;
    • // quad vertex #2
    • memcpy (&p[6], r->right_side_color, sizeof( GLfloat )*3);
    • p[9] = r->x+r->width; p[10] = r->y; p[11] = r->depth_order;
    • // quad vertex #3
    • memcpy (&p[12], r->right_side_color, sizeof( GLfloat )*3);
    • p[15] = r->x+r->width; p[16] = r->y+r->height; p[17] = r->depth_order;
    • // quad vertex #4
    • memcpy (&p[18], r-> left_side_color, sizeof( GLfloat )*3);
    • p[21] = r->x; p[22] = r->y+r->height; p[23] = r->depth_order;
    • }
    • return varray;
    • }
  • Step 2: Configure & Render from Vertex Arrays
    • void drawVarrayRectangles( int count, const RectInfo *list)
    • {
    • char *varray = initVarrayRectangles(count, list);
    • const GLfloat *p = ( const GLfloat *) varray;
    • const GLsizei stride = sizeof ( GLfloat )*6 ; //3 RGB floats,3 XYZ floats
    • glColorPointer (/*rgb*/3, GL_FLOAT , stride, p+0);
    • glVertexPointer (/*xyz*/3, GL_FLOAT , stride, p+3);
    • glEnableClientState ( GL_COLOR_ARRAY );
    • glEnableClientState ( GL_VERTEX_ARRAY );
    • glDrawArrays ( GL_QUADS , /*firstIndex*/0, /*indexCount*/count*4);
    • free (varray);
    • }
  • Critique of Simplistic Vertex Array Rendering
    • Advantages
      • Far fewer OpenGL commands issued
    • Disadvantages
      • Every render with drawVarrayRectangles calls initVarrayRectangles
        • Allocates, initializes, & frees vertex array memory every render
    • Improve by separating vertex array construction from rendering
  • Initialize Once, Render Many Approach
    • This routine expects base pointer returned by initVarrayRectangles
    • void drawInitializedVarrayRectangles( int count, const void *varray)
    • {
    • const GLfloat *p = ( const GLfloat *) varray;
    • const GLsizei stride = sizeof ( GLfloat )*6; // 3 RGB floats, 3 XYZ floats
    • glColorPointer (/*rgb*/3, GL_FLOAT , stride, p+0);
    • glVertexPointer (/*xyz*/3, GL_FLOAT , stride, p+3);
    • // Assume GL_COLOR_ARRAY and GL_VERTEX_ARRAY are already enabled!
    • glDrawArrays ( GL_QUADS , /*firstIndex*/0, /*indexCount*/count*4);
    • }
  • Client Memory Vertex Attribute Transfer GPU Processor command processor vertex puller hardware rendering pipeline CPU CPU writes of command + vertex data GPU DMA transfer of command + vertex data vertex data travels through CPU memory reads CPU command queue application (client) memory vertex array
  • Vertex Buffer Object Vertex Attribute Pulling OpenGL (vertex) buffer object GPU command processor vertex puller hardware rendering pipeline CPU CPU writes of command + vertex indices vertex array GPU DMA transfer of command data application (client) memory memory reads CPU GPU DMA transfer of vertex data—CPU never reads data command queue
  • Initializing Vertex Buffer Objects (VBOs)
    • Once using vertex arrays, easy to switch to VBOs
      • Make the vertex array as before
      • Then bind to buffer object and copy data to the buffer
    • void initVarrayRectanglesInVBO( GLuint bufferName,
    • int count, const RectInfo *list)
    • {
    • char *varray = initVarrayRectangles(count, list);
    • const GLsizei stride = sizeof ( GLfloat )*6; // 3 RGB floats, 3 XYZ floats
    • const GLint numVertices = 4*count;
    • const GLsizeiptr bufferSize = stride*numVertices;
    • glBindBuffer ( GL_ARRAY_BUFFER , bufferName);
    • glBufferData ( GL_ARRAY_BUFFER , bufferSize, varray, GL_STATIC_DRAW );
    • free (varray);
    • }
  • Rendering from Vertex Buffer Objects
    • Once initialized, glBindBuffer to bind to buffer ahead of vertex array configuration
      • Send offsets instead of points
    • void drawVarrayRectanglesFromVBO(GLuint bufferName,
    • int count)
    • {
    • const char *base = NULL ;
    • const GLsizei stride = sizeof (GLfloat)*6; // 3 RGB floats, 3 XYZ floats
    • glBindBuffer (GL_ARRAY_BUFFER, bufferName);
    • glColorPointer ( /*rgb*/ 3, GL_FLOAT , stride, base+0* sizeof (GLfloat));
    • glVertexPointer ( /*xyz*/ 3, GL_FLOAT , stride, base+3* sizeof (GLfloat));
    • // Assume GL_COLOR_ARRAY and GL_VERTEX_ARRAY are already enabled!
    • glDrawArrays ( GL_QUADS , /*firstIndex*/0, /*indexCount*/count*4);
    • }
  • Understanding glBindBuffer
    • Buffer object bindings are frequent point of confusion for programmers
      • What does glBindBuffer do really?
    • Lots of buffer binding targets:
      • GL_ARRAY_BUFFER target—for vertex attribute arrays
        • Query with GL_ARRAY_BUFFER_BINDING
      • GL_ARRAY_ELEMENT_BUFFER target—for vertex indices, effectively topology
        • Query with GL_ELEMENT_ARRAY_BUFFER_BINDING
      • Each vertex array has its own buffer, query with
        • GL_VERTEX_ARRAY_BUFFER_BINDING
        • GL_COLOR_ARRAY_BUFFER_BINDING
        • GL_TEXCOORD_ARRAY_BUFFER_BINDING , etc.
  • Bind and Query Buffer Targets
    • Buffer Bind Tokens
    • GL_ARRAY_BUFFER
    • GL_ELEMENT_ARRAY_BUFFER
    • Buffer Query Tokens
    • GL_ARRAY_BUFFER_BINDING
    • GL_ELEMENT_ARRAY_BUFFER_BINDING
    • GL_COLOR_ARRAY_BUFFER_BINDING
    • GL_VERTEX_ARRAY_BUFFER_BINDING
    • GL_FOGCOORD_ARRAY_BUFFER_BINDING
    • GL_TEXCOORD_ARRAY_BUFFER_BINDING
    • GL_VERTEX_ATTRIB_ARRRAY_BUFFER_BINDING
    Target tokens for glBindBuffer Query tokens to glGetIntegerv Query tokens to glGetVertexAttribiv
  • Latched Vertex Array Buffer Bindings
    • Here’s the confusing part:
      • glBindBuffer ( GL_ARRAY_BUFFER , 34);
      • glColorPointer (3, GL_FLOAT , color_stride, (void*)color_offset);
    • The glBindBuffer doesn’t change any vertex array binding
      • The GL_ARRAY_BUFFER_BINDING state that glBindBuffer sets does not itself affect rendering
    • It is the glColorPointer call that latches the array buffer binding to change the color array’s buffer binding!
      • Same with all vertex array buffer bindings
  • Binding Buffer Zero is Special
    • By default, vertex arrays don’t access buffer objects
      • Instead client memory is accessed
    • This is because
      • The initial buffer binding for a context is zero
      • And zero is special
        • Zero means access client memory
    • You can always resume client memory vertex array access for a given array like this
      • glBindBuffer ( GL_ARRAY_BUFFER , 0); // use client memory
      • glColorPointer (3, GL_FLOAT , color_stride, color_pointer);
    • Different treatment of the “pointer” parameter to vertex array specification commands
      • When the current array buffer binding is zero, the pointer value is a client memory pointer
      • When the current array buffer binding is non-zero (meaning it names a buffer object), the pointer value is “recast” as an offset from the beginning of the buffer
    • Once again
      • The glBindBuffer ( GL_ARRAY_BUFFER ,0) call alone doesn’t change any vertex array buffer bindings
      • It takes a vertex array specification command such as glColorPointer to latch the zero
    ensures compatibility with pre-VBO OpenGL
  • Texture Coordinate Set Selector
    • A selector in OpenGL is
      • A state variable that controls what state a subsequent command updates
      • Examples of commands that modify selectors
        • glMatrixMode , glActiveTexture , glClientActiveTexture
    • A selector is different from latched state
      • Latched state is a specified value that is set (or “latched”) when a subsequent command is called
    • Pitfall warning: glTexCoordPointer both
      • Relies on the glClientActiveTexture command’s selector
      • And latches the current array buffer binding for the selected texture coordinate vertex array
    • Example
      • glBindBuffer ( GL_ARRAY_BUFFER , 34);
      • glClientActiveTexture ( GL_TEXTURE3 );
      • glTexCoordPointer (2, GL_FLOAT , uv_stride, (void*)buffer_offset);
    buffer value glTexCoordPointer latches selector glTexCoordPointer uses
  • OpenGL’s Modern Buffer-centric Processing Model Vertex Array Buffer Object (VaBO) Transform Feedback Buffer (XBO) Parameter Buffer (PaBO) Pixel Unpack Buffer (PuBO) Pixel Pack Buffer (PpBO) Bindable Uniform Buffer (BUB) Texture Buffer Object (TexBO) Vertex Puller Vertex Shading Geometry Shading Fragment Shading Texturing Array Element Buffer Object (VeBO) Pixel Pipeline vertex data texel data pixel data parameter data ( not ARB functionality yet ) glBegin, glDrawElements, etc. glDrawPixels, glTexImage2D, etc. glReadPixels, etc. Framebuffer
  • Usages of OpenGL Buffers Objects
    • Vertex uses (VBOs)
      • Input to GL : Vertex attribute buffer objects
        • Color, position, texture coordinate sets, etc.
      • Input to GL : Vertex element buffer objects
        • Indices
      • Output from GL : Transform feedback
        • Streaming vertex attributes out
    • Texture uses (TexBOs)
      • Texturing from : Texture buffer objects
    • Pixel uses (PBOs)
      • Output from GL : Pixel pack buffer objects
        • glReadPixels
      • Input from GL : Pixel unpack buffer objects
        • glDrawPixels, glBitmap, glTexImage2D, etc.
    • Shader uses (PaBOs, UBOs)
      • Input to assembly program : Parameter buffer objects
      • Input to GLSL program : Bind-able uniform buffer objects
    Key point: OpenGL buffers are containers for bytes; a buffer is not tied to any particular usage
  • Continuum of OpenGL Usage Tweak-able Performance Immediate mode Client vertex arrays Vertex buffer objects (VBOs) Display lists
  • Mid-session break 15 minutes
  • Implementing OpenGL Mark Kilgard Principal System Software Engineer NVIDIA
  • Topics in OpenGL Implementation
    • Dual-core OpenGL driver operation
    • What goes into a texture fetch?
      • You give me some texture coordinates
      • I give you back a color
      • Could it be any simpler?
  • OpenGL Drivers for Multi-core CPUs
    • Today dual-core processors in PCs is nearly ubiquitous
      • 4, 6, 8, and more cores are clearly coming
    • How does OpenGL implementation exploit this trend?
    • Answer : develop dual-core OpenGL driver
  • Dual-core OpenGL Driver Architecture Application thread … Application thread D Context 1 Application thread A Application rendering thread App ICD ICD’s app thread (tokenize thread) Worker thread 1 (server thread) Application thread C Application audio thread (no OpenGL) Context 2 Application thread B Application rendering thread ICD’s app thread (tokenize thread) Worker thread 2 (server thread) Circular command FIFO Circular command FIFO
  • Dual-core Performance Results
    • A well-behaved OpenGL application benefiting from a dual-core mode of OpenGL driver operations
    Frames per second Mode of OpenGL driver operation
  • Good Dual-core Driver Practices
    • General advice
      • Display lists execute on the driver’s worker thread!
      • You want to avoid situations where the application thread must “sync” with the driver thread
    • Specific advice
      • Avoid OpenGL state queries
        • More on this later
      • Avoid querying OpenGL errors in production code
    • Bad behavior is detected automatically and leads to exit from the dual-core mode
      • Back to the standard single-core driver mode of operation
      • “ Do no harm”
  • Consider an OpenGL texture fetch
    • Seems very simple
      • Input: texture coordinates (s,t,r,q)
      • Output: some color (r,g,b,a)
      • Just a simple function, written in Cg/HLSL:
        • uniform sampler2D decal : TEXUNIT2 ; float4 texcoord : TEXCOORD3 ; float4 rgba = tex2D (decal, texcoordset.st);
      • Compiles to single instruction:
        • TEX o[COLR], f[TEX3], TEX2, 2D;
    • Implementation is much more involved!
  • Anatomy of a Texture Fetch Filtered texel vector Texel Selection Texel Combination Texel offsets Texel data Texture images Combination parameters Texture coordinate vector Texture parameters
  • Texture Fetch Functionality (1)
    • Texture coordinate processing
      • Projective texturing ( OpenGL 1.0 )
      • Cube map face selection ( OpenGL 1.3 )
      • Texture array indexing ( OpenGL 2.1 )
      • Coordinate scale: normalization ( ARB_texture_rectangle )
    • Level-of-detail (LOD) computation
      • Log of maximum texture coordinate partial derivative ( OpenGL 1.0 )
      • LOD clamping ( OpenGL 1.2 )
      • LOD bias ( OpenGL 1.3 )
      • Anisotropic scaling of partial derivatives ( SGIX_texture_lod_bias )
    • Wrap modes
      • Repeat, clamp ( OpenGL 1.0 )
      • Clamp to edge ( OpenGL 1.2 ), Clamp to border ( OpenGL 1.3 )
      • Mirrored repeat ( OpenGL 1.4 )
      • Fully generalized clamped mirror repeat ( EXT_texture_mirror_clamp )
      • Wrap to adjacent cube map face
      • Region clamp & mirror ( PlayStation 2 )
  • Texture Fetch Functionality (2)
    • Filter modes
      • Minification / magnification transition ( OpenGL 1.0 )
      • Nearest, linear, mipmap ( OpenGL 1.0 )
      • 1D & 2D ( OpenGL 1.0 ), 3D ( OpenGL 1.2 ), 4D ( SGIS_texture4D )
      • Anisotropic ( EXT_texture_filter_anisotropic )
      • Fixed-weights: Quincunx, 3x3 Gaussian
        • Used for multi-sample resolves
      • Detail texture magnification ( SGIS_detail_texture )
      • Sharpen texture magnification ( SGIS_sharpen_texture )
      • 4x4 filter ( SGIS_texture_filter4 )
      • Sharp-edge texture magnification ( E&S Harmony )
      • Floating-point texture filtering ( ARB_texture_float , OpenGL 3.0 )
  • Texture Fetch Functionality (3)
    • Texture formats
      • Uncompressed
        • Packing: RGBA8, RGB5A1, etc. ( OpenGL 1.1 )
        • Type: unsigned, signed ( NV_texture_shader )
        • Normalized: fixed-point vs. integer ( OpenGL 3.0 )
      • Compressed
        • DXT compression formats ( EXT_texture_compression_s3tc )
        • 4:2:2 video compression ( various extensions )
        • 1- and 2-component compression ( EXT_texture_compression_latc, OpenGL 3.0 )
        • Other approaches: IDCT, VQ, differential encoding, normal maps, separable decompositions
      • Alternate encodings
        • RGB9 with 5-bit shared exponent ( EXT_texture_shared_exponent )
        • Spherical harmonics
        • Sum of product decompositions
  • Texture Fetch Functionality (4)
    • Pre-filtering operations
      • Gamma correction ( OpenGL 2.1 )
        • Table: sRGB / arbitrary
      • Shadow map comparison ( OpenGL 1.4 )
        • Compare functions: LEQUAL, GREATER, etc. ( OpenGL 1.5 )
        • Needs “R” depth value per texel
      • Palette lookup ( EXT_paletted_texture )
      • Thresh-holding
        • Color key
        • Generalized thresh-holding
  • Texture Fetch Functionality (5)
    • Optimizations
      • Level-of-detail weighting adjustments
      • Mid-maps (extra pre-filtered levels in-between existing levels)
    • Unconventional uses
      • Bitmap textures for fonts with large filters ( Direct3D 10 )
      • Rip-mapping
      • Non-uniform texture border color
      • Clip-mapping ( SGIX_clipmap )
      • Multi-texel borders
      • Silhouette maps (Pardeep Sen’s work)
        • Shadow mapping
        • Sharp piecewise linear magnification
  • Phased Data Flow
    • Must hide long memory read latency between Selection and Combination phases
    Memory reads for samples FIFOing of combination parameters Texel Selection Texel Combination Texel offsets Texel data Texture images Combination parameters Texture coordinate vector Texture parameters
  • What really happens?
    • Let’s consider a simple tri-linear mip-mapped 2D projective texture fetch
    • Logically just one instruction
      • TXP o[COLR], f[TEX3], TEX2, 2D;
    • Logically
      • Texel selection
      • Texel combination
    • How many operations are involved?
  • Medium-Level Dissection of a Texture Fetch Convert texel coords to texel offsets integer / fixed-point texel combination texel offsets texel data texture images combination parameters interpolated texture coords vector texture parameters Convert texture coords to texel coords filtered texel vector texel coords floor / frac integer coords & fractional weights floating-point scaling and combination integer / fixed-point texel intermediates
  • Interpolation
    • First we need to interpolate (s,t,r,q)
    • This is the f[TEX3] part of the TXP instruction
    • Projective texturing means we want (s/q, t/q)
      • And possible r/q if shadow mapping
    • In order to correct for perspective, hardware actually interpolates
      • (s/w, t/w, r/w, q/w)
    • If not projective texturing, could linearly interpolate inverse w (or 1/w)
      • Then compute its reciprocal to get w
        • Since 1/(1/w) equals w
      • Then multiply (s/w,t/w,r/w,q/w) times w
        • To get (s,t,r,q)
    • If projective texturing, we can instead
      • Compute reciprocal of q/w to get w/q
      • Then multiple (s/w,t/w,r/w) by w/q to get (s/q, t/q, r/q)
    Observe projective texturing is same cost as perspective correction
  • Interpolation Operations
    • Ax + By + C per scalar linear interpolation
      • 2 MADs
    • One reciprocal to invert q/w for projective texturing
      • Or one reciprocal to invert 1/w for perspective texturing
    • Then 1 MUL per component for s/w * w/q
      • Or s/w * w
    • For (s,t) means
      • 4 MADs, 2 MULs, & 1 RCP
      • (s,t,r) requires 6 MADs, 3 MULs, & 1 RCP
    • All floating-point operations
  • Texture Space Mapping
    • Have interpolated & projected coordinates
    • Now need to determine what texels to fetch
    • Multiple (s,t) by (width,height) of texture base level
      • Could convert (s,t) to fixed-point first
        • Or do math in floating-point
      • Say based texture is 256x256 so
        • So compute (s*256, t*256)=(u,v)
  • Mipmap Level-of-detail Selection
    • Tri-linear mip-mapping means compute appropriate mipmap level
    • Hardware rasterizes in 2x2 pixel entities
      • Typically called quad-pixels or just quad
      • Finite difference with neighbors to get change in u and v with respect to window space
        • Approximation to ∂u/∂x, ∂u/∂y, ∂v/∂x, ∂v/∂y
        • Means 4 subtractions per quad (1 per pixel)
    • Now compute approximation to gradient length
      • p = max(sqrt((∂u/∂x) 2 +(∂u/∂y) 2 ), sqrt((∂v/∂x) 2 +(∂v/∂y) 2 ))
    one-pixel separation
  • Level-of-detail Bias and Clamping
    • Convert p length to power-of-two level-of-detail and apply LOD bias
      • λ = log2(p) + lodBias
    • Now clamp λ to valid LOD range
      • λ ’ = max(minLOD, min(maxLOD, λ ))
  • Determine Mipmap Levels and Level Filtering Weight
    • Determine lower and upper mipmap levels
      • b = floor( λ ’)) is bottom mipmap level
      • t = floor( λ ’+1) is top mipmap level
    • Determine filter weight between levels
      • w = frac( λ ’) is filter weight
  • Determine Texture Sample Point
    • Get (u,v) for selected top and bottom mipmap levels
      • Consider a level l which could be either level t or b
        • With (u,v) locations (ul,vl)
    • Perform GL_CLAMP_TO_EDGE wrap modes
      • u w = max(1/2*widthOfLevel(l), min(1-1/2*widthOfLevel(l), u))
      • v w = max(1/2*heightOfLevel(l), min(1-1/2*heightOfLevel(l), v))
    • Get integer location (i,j) within each level
      • (i,j) = ( floor(u w * widthOfLevel(l)), floor(v w * ) )
    border edge s t
  • Determine Texel Locations
    • Bilinear sample needs 4 texel locations
      • (i0,j0), (i0,j1), (i1,j0), (i1,j1)
    • With integer texel coordinates
      • i0 = floor(i-1/2)
      • i1 = floor(i+1/2)
      • j0 = floor(j-1/2)
      • j1 = floor(j+1/2)
    • Also compute fractional weights for bilinear filtering
      • a = frac(i-1/2)
      • b = frac(j-1/2)
  • Determine Texel Addresses
    • Assuming a texture level image’s base pointer, compute a texel address of each texel to fetch
      • Assume bytesPerTexel = 4 bytes for RGBA8 texture
    • Example
      • addr00 = baseOfLevel(l) + bytesPerTexel*(i0+j0*widthOfLevel(l))
      • addr01 = baseOfLevel(l) + bytesPerTexel*(i0+j1*widthOfLevel(l))
      • addr10 = baseOfLevel(l) + bytesPerTexel*(i1+j0*widthOfLevel(l))
      • addr11 = baseOfLevel(l) + bytesPerTexel*(i1+j1*widthOfLevel(l))
    • More complicated address schemes are needed for good texture locality!
  • Initiate Texture Reads
    • Initiate texture memory reads at the 8 texel addresses
      • addr00, addr01, addr10, addr11 for the upper level
      • addr00, addr01, addr10, addr11 for the lower level
    • Queue the weights a, b, and w
      • Latency FIFO in hardware makes these weights available when texture reads complete
  • Phased Data Flow
    • Must hide long memory read latency between Selection and Combination phases
    Memory reads for samples FIFOing of combination parameters Texel Selection Texel Combination Texel offsets Texel data Texture images Combination parameters Texture coordinate vector Texture parameters
  • Texel Combination
    • When texels reads are returned, begin filtering
      • Assume results are
        • Top texels: t00, t01, t10, t11
        • Bottom texels: b00, b01, b10, b11
    • Per-component filtering math is tri-linear filter
      • RGBA8 is four components
    • result = (1-a)*(1-b)*(1-w)*b00 + (1-a)*b*(1-w)*b*b01 + a*(1-b)*(1-w)*b10 + a*b*(1-w)*b11 + (1-a)*(1-b)*w*t00 + (1-a)*b*w*t01 + a*(1-b)*w*t10 + a*b*w*t11;
    • 24 MADs per component, or 96 for RGBA
      • Lerp-tree could do 14 MADs per component, or 56 for RGBA
  • Total Texture Fetch Operations
    • Interpolation
      • 6 MADs, 3 MULs, & 1 RCP (floating-point)
    • Texel selection
      • Texture space mapping
        • 2 MULs (fixed-point)
      • LOD determination (floating-point)
        • 1 pixel difference, 2 SQRTs, 4 MULs, 1 LOG2
      • LOD bias and clamping (fixed-point)
        • 1 ADD, 1 MIN, 1 MAX
      • Level determination and level weighting (fixed-point)
        • 1 FLOOR, 1 ADD, 1 FRAC
      • Texture sample point
        • 4 MAXs, 4 MINs, 2 FLOORs (fixed-point)
      • Texel locations and bi-linear weights
        • 8 FLOORs, 4 FRACs, 8 ADDs (fixed-point)
      • Addressing
        • 16 integer MADs (integer)
    • Texel combination
      • 56 fixed-point MADs (fixed-point)
  • Observations about the Texture Fetch
    • Lots of ways to implement the math
      • Lots of clever ways to be efficient
      • Lots more texture operations not considered in this analysis
        • Compression
        • Anisotropic filtering
        • sRGB
        • Shadow mapping
    • Arguably TEX instructions are “world’s most CISC instructions”
      • Texture fetches are incredibly complex instructions
    • Good deal of GPU’s superiority at graphics operations over CPUs is attributable to TEX instruction efficiency
      • Good for compute too
  • OpenGL’s Future Evolution Mark Kilgard Principal System Software Engineer NVIDIA
  • What drives OpenGL’s future?
    • GPU graphics functionality
      • Tessellation & geometry amplification
    • Ratio of GPU to single-core CPU performance
    • Compatibility
      • Direct3Disms
      • OpenGLisms
      • Deprecation
    • Compute support
      • OpenCL, CUDA, Stream processing
    • Unconventional graphics devices
  • Better Graphics Functionality
    • Expect more graphics performance
      • Easy prediction
      • Rasterization nowhere near peaked
        • Ray tracing fans—GPUs make rays and triangles faster
          • Market still values triangles more than rays
    • Expect more generalized graphics functionality
      • Trend for texture enhancements likely to continue
  • Geometry Amplification
    • Tessellation
      • Programmable hardware support coming
    • True market demand probably not tessellation per se
      • Games want visual richness
        • Texture and shading have created much richness
          • Often “pixel richness” as substitute for geometry richness
        • Increasingly “ visual richness” means geometric complexity
      • Geometry Amplification may be better term
        • Tessellation is one way to improve tessellation
          • Recognize the limits of bi-variate patches for representing geometry
  • Programmable Tessellation
    • Stunning real-time geometric detail + animation possible
      • Programmable tessellation + vertex textured displacements
  • Continuous Level-of-detail for Tessellation
    • Same patch mesh for all 3 scenes
    Increasing tessellation level-of-detail
  • Adaptive Programmable Tessellation Programmable level-of-detail determination allows more tessellation along silhouette edges
  • Limits of Patch Tessellation
    • What games tend to want
      • Here’s 8 vertices (bounding box), go draw a fire truck
    • Here’s a few vertices, go draw a tree
  • Tessellation Not New to OpenGL
    • At least three different bi-variate patch tessellation schemes have been added to OpenGL
      • Evaluators (OpenGL 1.0)
      • NV_evaluators (GeForce 3)
        • water-tight
        • adaptive level-of-detail
        • forward differencing approach
      • ATI_pn_triangles Curved PN Triangles (Radeon)
        • tessellated triangle based on positions+normals
    • None succeeded
      • Hard to integrate into art pipelines
      • Didn’t offer enough performance advantage
    GLUT’s wire-frame teapot [Moreton 20001] [Vlachos 20001]
  • Ratio of CPU core-to-GPU Performance
    • Well known computer architecture trends now
      • Single-threaded CPU performance trends are stalled
        • Multi-core is CPU designer response
      • GPU performance continues on-trend
    • What does this mean for graphics API design?
      • CPUs must generate more visually rich API command streams to saturate GPUs
      • Can’t just send more commands faster
        • Single-threaded CPUs can only do so much
      • So must send more powerful commands
  • Déjà vu
    • We’ve been here before
      • Early 1980s : Graphics terminals used to be connected to minicomputers by slow speed interconnects
        • CPUs themselves far too slow for real-time rendering
      • Resulting rendering model
        • Download scene database to graphics terminal
        • Adjust viewing and modeling parameters
        • Send “redraw scene” command
  • What Happened
    • Such “scene processor” hardware not very flexible
      • Difficult to animate anything beyond rigid dynamics
    • Eventually SGI and others matched CPUs and interconnects to graphics performance
      • Result was IRIS GL’s immediate mode
      • CPU fast enough to send geometry every frame
    • OpenGL took this model
      • Over time added vertex arrays, vertex buffers, texturing, programmable shading, and more performance
      • CPU performance became limiter still
        • Better graphics driver tuning helped
        • Dual-core drivers help some more
  • OpenGL’s Most Powerful Command
    • Available since OpenGL 1.0
    • Can render essentially anything OpenGL can render!
    • Takes just one parameter
    • The command
      • glCallList ( GLuint displayListName);
    • Power of display lists comes from
      • Playing back arbitrary compiled commands
      • Allowing for hierarchical calling of display list
        • A display list can contain glCallList or glCallLists
      • Ability of application to re-define display lists
        • No editing, but can be re-defined
  • Enhanced Display Lists
    • OpenGL 1.0 display lists are too inflexible
      • Pixel & vertex data “compiled into” display lists
      • Binding objects always “by name”
        • Rather than “by reference
    • These problems can be fixed
      • Modern OpenGL supports buffers for transferring vertices and pixels
        • Compile commands into display lists that defer vertex and pixel transfers until execute-time
          • Rather than compile-time
      • Allow objects (textures, buffers, programs) to be bound “by reference” or “by name”
  • Other Display List Enhancements
    • Conditional display list execution
    • Relaxed vertex index and command order
    • Parallel construction of display lists by multiple threads
    • General insight : Easier for driver to optimize application’s graphics command stream if it gets to 1) see the repetition in the command stream clearly 2) take time to analyze and optimize usage
  • Conditional Display List Execution
    • Today’s occlusion query
      • Application must “query” to learn occlusion result
        • Latency too great to respond
      • Application can use OpenGL 3.0’s conditional render capability
        • But just skips vertex pulling, not state changes
    • Conditional display list execution
      • Allow a glCallList to depend on the occlusion result from an occlusion query object
        • Allows in-band occlusion querying
        • Skip both vertex pulling and state changes
  • Relaxed Vertex Index and Command Order
    • OpenGL today always executes commands “in order”
      • Sequentially requirement
    • Provide compile-time specification of re-ordering allowances
      • Allows GL implementation to re-order
        • Vertex indices within display list’s vertex batch
        • Commands within display list
      • Key rule : state vector rendering command executes in must match the state if command was rendered sequentially
      • Allow static or dynamic re-ordering
        • Static re-ordering needed for multi-pass invariances
    • Past practice
      • IRIS Performer would sort rendering by state changes for performance
      • [Sander 2007] show substantial benefit for vertex ordering
  • Parallel Display List Construction
    • Today’s model
      • Single thread makes all OpenGL rendering calls
        • Minimizes GPU context switch overhead
        • Ties command generation rate to single core’s CPU performance
    • Enhanced display list model
      • Multiple threads can build display lists in parallel
      • Single thread still executes display lists
      • Countable semaphore objects used to synchronize hand-off of display lists built by other threads with main rendering thread
  • Rethinking Display Lists
    • Display lists have been proposed for deprecation
      • Right as we really need them!
    • Much more interesting to enhance display lists
      • Dual-core driver already off-loads display list traversal to driver’s thread
      • Multi-core driver could scan frequently executed display lists to optimize their order and error processing
        • Includes adding pre-fetching to avoid stalling CPU on cache misses for object accesses
  • Direct3Disms
    • Developing a shader-rich game title costs $$$
      • For top titles, often US$ 5,000,000+
      • Investment typically amortized over multiple platforms
    • Consoles are primary target, then PCs
      • PC version typically developed for Direct3D
      • Reality: OpenGL is often 3 rd or worse priority
    • API differences = porting & performance pitfalls
      • Stops or slows Direct3D-developed 3D content from working easily on OpenGL platforms
  • Supporting Direct3D: Not New
    • OpenGL has always supported multiple formats well
      • OpenGL’s plethora of pixel and vertex formats
      • Very first OpenGL extension: EXT_bgra
        • Provides a pixel component ordering to match the color component ordering of Windows for 2D GDI rendering
        • Made core functionality by OpenGL 1.3
    • Many OpenGL extensions have embraced Direct3Disms
      • Secondary color
      • Fog coordinate
      • Point sprites
  • Direct3D vs. OpenGL Coordinate System Conventions
    • Window origin conventions
      • Direct3D = upper-left origin
      • OpenGL = lower-left origin
    • Pixel center conventions
      • Direct3D9 = pixel centers at integer locations
      • OpenGL (and Direct3D 10) = pixel centers at half-pixel locations
    • Clip space conventions
      • Direct3D = [-1,+1] for XY, [0,1] for Z
      • OpenGL = [-1,+1] range for XYZ
    • Affects
      • How projection matrix is loaded
      • Fragment shaders that access the window position
      • Point sprites have upper-left texture coordinate origin
        • OpenGL already lets application choose lower-left or upper-left
  • Direct3D vs. OpenGL Provoking Vertex Conventions
    • Direct3D uses “first” vertex of a triangle or line to determine which color is used for flat shading
    • OpenGL uses “last” vertex for lines, triangles, and quads
      • Except for polygons ( GL_POLYGON ) mode that use the first vertex
    Direct3D 9 pDev->SetRenderState( D3DRS_SHADEMODE, D3DSHADE_FLAT); OpenGL glShadeModel(GL_FLAT); Input triangle strip with per-vertex colors
  • BGRA Vertex Array Order
    • Direct3D 9’s most common usage for sending per-vertex colors is 32-bit D3DCOLOR data type:
      • Red in bits 16:23
      • Green in bits 8:15
      • Blue in bits 0:7
      • Alpha in bits 24:31
    • Laid in memory, looks like BGRA order
      • OpenGL assumes RGBA order for all vertex arrays
    • Direct3Dism EXT_vertex_array_bgra extension allows: glColorPointer( GL_BGRA , GL_UNSIGNED_BYTE, stride, pointer); glSecondaryColorPointer( GL_BGRA , GL_UNSIGNED_BYTE, stride, pointer); glVertexAttribPointer( GL_BGRA , GL_UNSIGNED_BYTE, stride, pointer);
    8-bit red 8-bit alpha 8-bit green 8-bit blue bit 31 bit 0
  • OpenGLisms
    • Things about OpenGL’s operation that make it hard for non-OpenGL applications to port to OpenGL
    • Examples
      • Selectors
      • Linked GLSL program objects
  • Eliminating Selectors from OpenGL
    • OpenGL has lots of selectors
      • Selectors set state that indicates what state subsequent commands will update
      • Already mentioned selectors: glClientActiveTexture
        • Other examples: glActiveTexture , glMatrixMode , glBindTexture , glBindBuffer , glUseProgram , glBindProgramARB
        • OpenGL is full of selectors
          • Partly OpenGL’s extensibility strategy
          • Partly because objects are bound into context
            • Bind-to-edit objects
            • Rather than edit-by-name
    • Direct State Access extension: EXT_direct_state_access
      • Provides complete selector-free additional API for OpenGL
      • Shipping in NVIDIA’s 180.43 drivers
  • Reasons to Eliminate Selectors
    • Direct3D has an “edit-by-name” model of operation
      • Means Direct3D has no selectors
      • Having to manage selectors when porting Direct3D or console code to OpenGL is awkward
        • Requires deferring updates to minimize selector and object bind changes
    • Layered libraries can’t count of selector state
      • To be safe when updating sate controlled by selectors, such libraries must use idiom
        • Save selector, Set selector, Update state, Restore selector
        • Bad for performance, particularly bad for dual-core drivers since queries are expensive
  • GLSL Program Object Linking
    • GLSL requires shader objects from different domains (vertex, geometry, fragment) to be linked into single GLSL program object
      • Means you can’t mix-and-match shaders easily
    • Other APIs don’t have this limitation
      • Direct3D
      • Prior OpenGL assembly language extensions
      • Consoles
    • Have a “separate shader objects” extension could fix this problem
  • Separate Shader Objects Example
    • Combining different GLSL shaders at once
    Specular brick bump mapping Red diffuse Wobbly torus Smooth torus Different GLSL vertex shaders Different GLSL fragment shaders
  • Deprecation
    • Part of OpenGL 3.0 is a marking of features for deprecation
      • LOTS of functionality is marked for deprecation
      • I contend no real application today uses the non-deprecated subset of OpenGL—all apps would have to change due to deprecation
    • Some vendors believe getting rid of features will make OpenGL better in some way
      • NVIDIA does not believe in abandoning API compatibility this way
    • OpenGL is part of a large ecosystem so removing features this way undermines the substantial investment partners have made in OpenGL over years
      • API compatibility and stability is one of OpenGL’s great strengths
  • Synergy between OpenGL and OpenCL
    • Complimentary capabilities
      • OpenGL 3.0 = state-of-the-art, cross-platform graphics
      • OpenCL 1.0 = state-of-the-art, cross-platform compute
    • Computation & Graphics should work together
      • Most natural way to intuit compute results is with graphics
      • When Compute is done on a GPU, there’s no need to “copy” the data to see it visualized
    • Appendix B of OpenCL specification
      • Details with sharing objects between OpenGL and OpenCL
        • Called “GL” and “CL” from here on…
  • Four Kinds of Shared Objects OpenCL 3D image object cl_mem OpenGL renderbuffer object GLuint renderbuffer OpenGL buffer object GLuint bufferobj OpenCL buffer object cl_mem OpenGL texture 2D object GLenum target GLuint texture GLint miplevel OpenGL texture 3D object GLenum target GLuint texture GLint OpenCL 2D image object cl_mem 2D image object cl_mem clCreateFromGLBuffer clCreateFromGLTexture2D clCreateFromGLTexture3D clCreateFromGLRenderbuffer OpenGL OpenCL
  • OpenGL / OpenCL Sharing
    • Requirements for GL object sharing with CL
      • CL context must be created with an OpenGL context
      • Each platform-specific API will provide its appropriate way to create an OpenGL-compatible CL context
        • For WGL (Windows), CGL (OS X), GLX (X11/Linux), EGL (OpenGL ES), etc.
    • Creating cl_mem for GL Objects does two things
      • Ensures CL has a reference to the GL objects
      • Provides cl_mem handle to acquire GL object for CL’s use
    • clRetainMemObject & clReleaseMemObject can create counted references to cl_mem objects
  • Acquiring GL Objects for Compute Access
    • Still must “enqueue acquire” GL objects for compute kernels to use them
      • Otherwise reading or writing GL objects with CL is undefined
      • Enqueue acquire and release provide sequential consistency with GL command processing
    • Enqueue commands for GL objects
      • clEnqueue Acquire GLObjects
        • Takes list of cl_mem objects for GL objects & list of cl_events that must complete before acquire
        • Returns a cl_event for this acquire operation
      • clEnqueue Release GLObjects
        • Takes list of cl_mem objects for GL objects & list of cl_events that must complete before release
        • Returns a cl_event for this release operation
  • Unconventional OpenGL Deployments
    • Workstation PCs—Quadro
    • Consumer PCs—GeForce
    • High-end Visualization—QuadroPlex Visual Computing Solution (VCS)
    • Embedded Applications
    • Handheld Devices
    • Game Consoles
    Conventional PC OpenGL Products Unconventional
  • OpenGL in Context A facilitated conversation with Dr. Marc Levoy, Stanford University
  • Questions?