• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
 

Advanced Graphics Workshop - GFX2011

on

  • 3,000 views

This was presented at GFX2011 - conducted by IEEE Consumer Electronics society, Bangalore chapter. This is the public version of the presentation.

This was presented at GFX2011 - conducted by IEEE Consumer Electronics society, Bangalore chapter. This is the public version of the presentation.

Statistics

Views

Total Views
3,000
Views on SlideShare
2,994
Embed Views
6

Actions

Likes
1
Downloads
72
Comments
0

4 Embeds 6

http://www.linkedin.com 3
http://www.docseek.net 1
http://www.slashdocs.com 1
https://www.linkedin.com 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Advanced Graphics Workshop - GFX2011 Advanced Graphics Workshop - GFX2011 Presentation Transcript

    • Advanced Graphics Workshop - Prabindh Sundareson, Texas Instruments GFX2011 Dec 3 rd 2011 Bangalore Note: This slide set is a public version of the actual slides presented at the workshop
    • GFX2011 8.30 AM [Registration and Introduction, Equipment setup] 9.00 AM Why Graphics ? Present and Future – Prof. Vijay Natarajan, Assistant Professor, Department of Computer Science and Automation, IISc, Bangalore 9.45 AM Introduction to the OpenGL/ES Rendering pipeline, and algorithms Detailed walkthrough of the OpenGL ES2.0 spec and APIs – Part 1 1.00 PM [Lunch] Detailed walkthrough of the OpenGL ES2.0 spec and APIs – Part 2 - Break - Framework and platform integration - EGL, Android (SurfaceFlinger) Tools for performance benchmarking, and Graphics Development Q&A, Certificate presentation to participants – Networking
    • Detailed Agenda
      • Inaugural Talk – Dr. Vijay Natarajan – “Graphics – Present and Future”
      • - Break
      • GPU HW Architectures, and the GL API
      • The CFF APIs
        • Lab 1
      • Texturing objects
        • Lab 94 – Rectangle wall
      • Vertices and Transformations
        • Lab 913 – Eyeing the eyes
      • Lunch Break
      • Real-life modeling and loading 3D models
        • Lab 9 – Review .obj file loading method
      • Shaders – Vertex and Fragment
        • Lab 96 – Squishing the slices
      • - Break
      • Rendering targets
        • Lab 910 – 3D in a 2D world
      • Creating special effects
      • EGL – and Platform Integration
        • Overview of GLES2.0 usage in Android/ iOS
      • *This PPT is to be used in conjunction with the labs at http://www.gpupowered.org
    • GPU HW Architectures
      • CPUs are programmed with sequential code
        • Typical C program – linear code
        • Well defined Pre-fetch architectures, cache mechanisms
        • Problem ?
          • Limited by how fast “a” processor can execute, read, write
      • GPUs are parallel
        • Small & same code, multiple data
        • Don’t care - control dependencies
          • If used, drastically reduces throughput
        • “ Output” is a result of a matrix operation (n x n)
          • Graphics output – color pixels
          • Computational output – matrix values
    • GPU integrated SOCs
      • The A5 chipset
      CPU size ~= GPU size
    • Integrated GPU architectures (Samsung SRP) From – Khronos presentation
    • Vivante GPU Architecture
      • GPUs vary in
      • Unified vs separate shader HW architecture
      • internal cache size
      • Bus size
      • rendering blocks
      • Separated 2D and 3D blocks
      From - http://www.socip.org/socip/speech/pdf/2-Vivante-SoCIP%202011%20Presentation.pdf Spec evolution
    • OpenGL specification evolution Reading the spec
    • A note on reading the GLES specifications
      • It must be clear that GLES specification is a derivative of the GL specification
        • It is recommended to read the OpenGL 2.0 specification first
        • Then read the OpenGL ES2.0 specification
      • Similarly, for shading language
        • It is recommended to read the OpenGL SL specification first
        • Then read the OpenGL ES SL specification
      Extensions
    • What are Extensions ?
      • Extension types
        • OES – Conformance tested by Khronos
        • EXT – Extension supported by >1 IP vendor
        • Proprietary (vendor_ prefix) – Extension from 1 IP vendor
      • How to check for extensions ?
        • getSupportedExtensions (WebGL), getExtension()
        • glGetString (openGL ES)
      • Number of extensions
        • OpenGL  400 +
        • OpenGL ES  100+
      Dependencies
    • OpenGL Dependencies
      • OpenGL depends on a number of external systems to run
        • A Windowing system (abstracted by EGL/ WGL/ XGL …)
        • External inputs – texture files, 3D modelling tools, shaders, sounds, …
      • OpenGL is directly used by
        • OS/ Driver developers (ofcourse!)
        • HW IP designers
        • Game studios (optimisation)
        • Researchers (Modelling, Realism, ) …
        • Tools developers
      • Application developers do not generally program on OpenGL, but rather do it on an Android API binding, or Java binding
      GL vs ES
    • OpenGL 2.0 vs OpenGL ES2
      • All Fixed function functionality is removed
        • Specific drawing calls like Fog etc
        • Matrix setup
      • Replaced with programmable entities, GLES SL is ~= GL SL
      • Compatibility issues GL  GLES
        • Shaders
          • GLSL does not enable fixed function state to be available
          • ex, gl_TexCoord
      • To enable compatibility, Architecture Review Board (ARB) extension – “ GL_ARB_ES2_compatibility”
        • http://www.opengl.org/registry/specs/ARB/ES2_compatibility.txt
      • Good reference
        • http://developer.amd.com/gpu_assets/GDC06-GLES_Tutorial_Day-Munshi-OpenGLES_Overview.pdf
      GLES API
    • The OpenGL ES API
      • From the Khronos OpenGL ES Reference Card
        • “ OpenGL ® ES is a software interface to graphics hardware.
        • The interface consists of a set of procedures and functions that allow a programmer to specify the objects and operations involved in producing high-quality graphical images, specifically color images of three-dimensional objects”
      • Keep this document handy for API reference
        • http://www.khronos.org/opengles/sdk/docs/reference_cards/OpenGL-ES-2_0-Reference-card.pdf
      Client server
    • The GL Client – Server Model
      • Client (application on host), server (OpenGL on GPU)
      • Server can have multiple rendering contexts, but has a global state
        • Client will connect to one of the contexts at any point of time
      • Client can set the states of the server by sending commands
        • Further API calls will thus be affected by the previous states set by the client
      • Server expects not-to-be interrupted by the client during operation
        • Inherent nature of the parallel processor
      GL,SL, EGL spec versions
    • OpenGL Specifications OPENGL Full version ES version Common Common-Lite GLSL companion GLSL-ES companion What we miss in ES compared to desktop version: Polygons, Display lists, Accumulation buffers,… Currently in 4.0+ Currently in 2.0 Currently in 1.0.16 Currently in 1.20 EGL Currently in 1.3 Core GL Spec Shader Spec Platform Integration EGL Currently in 1.3 Programming Flow
    • Programming in OpenGL / ES
      • Step1:
        • Initialise EGL for rendering – context, surface, window
      • Step2:
        • Describe the scene (VBOs, Texture coordinates) – objects with Triangles, lighting
      • Step3:
        • Load the textures needed for the objects in the scene
      • Step4:
        • Compile the Vertex and Fragment Shaders
      • Step 5:
        • Select the output target (FBO, Fb, Pixmap …)
      • Step5:
        • Draw the scene
      • Step 6
        • Run this in a loop
      Frames/vertex/basics
    • Preliminaries
      • Pixel Throughput
        • Memory bandwidth for a 1080P display @ 60 fps
        • Did we forget Overdraw ?
      • Vertex Throughput
        • Vertex throughput for a 100k triangle scene
      • Tearing
      Frame switch (Uniform) Driver frame draw (non-uniform) Real frame switch happens here Triangles
    • Why Triangles ?
      • Connectivity, Simplicity, Cost
      + + ??? pipeline
    • ES2.0 Pipeline What do the APIs look like ? …
    • The GLES API – Overall view Global Platform Management Vertex Operations Texture Operations Shader Operations Rendering Operations
      • VBOs
      • Attaching attributes
      • Attaching textures
      • Loading texture data
      • Mipmaps
      • Loading, compiling,
      • Linking to program
      • Binary shaders
      • Rendering to Framebuffer
      • Rendering to FBO
      • RTT
      State Management
      • CFF
      • Front/Back facing
      • Enable/Disable (culling, ..)
      • Get/ Set uniforms
      • egl, wgl, glx ..
      • Antialiasing,
      • Configuration
      Context Management
      • Surface - window
      • Threading models
      • Context sharing
      EGL GL
    • Starting on the GPU
    • Flush, and Finish
      • Several types of rendering methods adopted by GPUs
        • Immediate rendering
        • Deferred rendering
          • Tiled rendering (ex, QCOM_tiled_rendering)
      • Immediate rendering – everyone is happy
        • Except the memory bus!
      • Deferred – the GPU applies its “intelligence” to do/not do certain draw calls/ portions of draw calls
        • Used most commonly in embedded GPUs
      • Flush() – The call ensures pending operations are kicked off, returns
      • Finish() – The call ensures pending operations are kicked off, “waits for completion” , returns
    • A note on WebGL
      • In native code, need to handle Surface, context
        • For example, refer to this native code
        • eglGetDisplay
        • eglInitialize – for this display
        • eglBindAPI – bind to GLES/ VG context
        • eglChooseConfig – configure surface type, GLES1/2/VG, Depth/Stencil buffer size
        • eglCreateWindowSurface / eglCreatePixmapSurface
        • eglCreateContext
        • eglMakeCurrent
      • Platform initialisation in WebGL is handled by browser
        • Only configurations are – stencil, depth, AA, Alpha, preserve
        • No EGL calls in JS application code
        • No multi-threading issues (not yet, but “workers” are coming)
      • The hands on labs will focus on GL, not EGL
      • Note : - GL context gets lost, when user account is locked/screen saver mode etc – Restart browser as required.
    • Programming
      • ClearColor
      • Clear – clears with color specified earlier
      • Flush/ Finish
      • “ setup”, “vertex”, “fragment”, “animate” functions in class
      • Will be called by framework
      • Clear web cache first time
      • Login
      • Open “Default Lab #1” - Copy
      • Open “My labs” #1 – Paste code
      • Change code in setupFunc()
      • Save, Run again
      • Lab #1
      Below link contains introductory video for starting the labs: http://www.youtube.com/watch?v=TM6t2ev9MHk
    • Texturing
    • A note on Binding, Buffer Objects
      • What is “Binding” ?
        • Binding a server to a client – ex, VBO to a texture
        • All objects are associated with a context state
        • Binding an object is ~ copying the object state  context
        • Removes client  server movement everytime
        • “ Xfer-once-to-server, keep the token, Use-multipletimes-later”
        • Good practice to “unbind” after operations– set binding to 0/null to avoid rogue programs changing state of bound object
      • Buffer-Objects
        • Allows data to be stored on the “server” ie, the GPU memory, rather than client memory (via pointer)
          • GPU can decide where to place it for the fastest performance
    • Correct approach of data transfers
      • Generate Object (ex, glGenBuffers, glGenTextures )
      • Bind Object to an ID “xyz” ( glBindBuffer(xyz), .. )
      • Transfer data to Object ( glBufferData, glTexImage2D )
      • Unbind ( glBindBuffer(0) )
      •  After this point, the data remains bound to “xyz” and is managed by GPU.
      • Can be accessed later by referencing “xyz”
      •  Applies to VBOs, Textures, …
      • Note the implicit “no atomicity” – needs locking
    • Texturing basics
      • Texture Formats available
        • RGB* formats, Luminance only formats
        • Relevance of YUV
      • Texture Filtering
        • Maps texture coordinates to object coordinates – think of wrapping cloth over object
      • Mipmaps
        • Local optimisation – use pre-determined “reduced” size images, if object is far away from viewer – as compared to filtering full image
        • Objective is to reduce bandwidth, not necessarily higher quality
        • Application can generate and pass through TexImage2D()
        • GPU can generate using GenerateMipMap()
        • Occupies more memory
      Uv mapping
    • Texturing 3D objects
      • Mapping from a bitmap to a 3D object involves matching the texture coordinates to the object surface
      • Texture coordinates are calculated along with the vertex coordinates
      • 3D tools output Texture coordinates along with vertex information, for the scene
      • Lab 12 (Sphere), Lab 13
      compression
    • Texture Compression types
      • GLES spec supports RGBA textures, Luminance …
        • To reduce memory bandwidth, compression used
      • Texture Compression major types
        • PVRTC
        • ETC1
        • Others
      • Android primarily supports ETC1
      • iOS supports PVRTC (and no other)
      • Extension support queryable using GL API queries
      • How to store this information in an uniform manner ?
        • Texture file formats
        • PVRTC (using Textool converter from IMG) commonly used
        • KTX file format
      KTX
    • Khronos KTX file format
      • To render a texture, steps to be used today:
        • Application needs apriori knowledge of texture type, format, storage type, mipmap levels, and filename or the buffer data itself
        • Then load into server using TexImage2D()
      • Proprietary formats exist to separate this application+texture dependency –
        • ex, PVRT from IMG
      • KTX file format from Khronos is a standard way to store texture information, and the texture itself
        • See next slide for structure of the file
    • KTX format … Passing coords
    • Texture coordinates to GPU
      • Texture coordinates are passed to GPU as “Attributes” along with the vertices
      • Gen-bind-bufferdata, then bindAttrib
      WebGL/Textures
    • Note on WebGL and Textures
      • Because of the way file loads work on browsers (asynchronous), texture loading may not happen before the actual draw
        • Expect black screen for a very short-while till the Texture image loads from the website
      • On native applications, due to the synchronous nature of loading the texture this issue will not be present
      Programming
    • Programming with Textures
      • bindTexture
      • pixelStorei (webGL only)
        • UNPACK_FLIP_Y_WEBGL
      • texImage2D
      • texParameteri
        • TEXTURE_MAG_FILTER
        • TEXTURE_MIN_FILTER
      • Note: WebGL “null” binding instead of “0”
      • Lab #2 (why not square ?)
      • Lab #94 – The wall
    • Note on the lab hints
      • Each session ends with a lab. The lab sessions online intentionally have errors that the reader has to debug to show the rendered object on screen.
      • Keys are provided for each such lab at the end of the section in this PPT
    • Lab 94 – Texturing (Keys to remove errors)
      • var indexArray = new Uint16Array([0, 1, 2, 2 , 1, 3]);
      • var texCoordArray = new Float32Array([0,0, 10 ,0, 0, 10, 10,10]);
      • context.enableVertexAttribArray( 1 );
      • context.vertexAttribPointer(1, 2 , context.FLOAT, context.FALSE, 0, 0);
    • Vertices, Transformations
    • What are vertices ?
      • Vertices –
        • Points defined in a specific coordinate axes, to represent 3D geometry
        • Atleast 3 vertices are used to define a Triangle – one of the primitives supported by GLES
    • Vertex operations
      • Where do vertices come from ?
        • Output of Modelling tools
        • Mesh rendering / transforms – optimisations
      • For 2D operations (ex Window systems), just 2 triangles
      Attributes
    • Vertex Attributes
      • A vertex is characterised by its position {x,y,z}
        • {x,y,z} are floating point values
      • Additionally, normals are required for directional lighting calculations in shader
        • 3D Tools output the normal map also along with vertex information
      • Additionally, texture coordinates are required
        • Again, 3D tools output the texture coordinates
      • Each HW implementation must support a minimum number of vertex attributes
        • Maximum number can be queried using MAX_VERTEX_ATTRIBS
      CPU to GPU xfer
    • Vertices – CPU to GPU
      • Optimising Vertex operations
        • A 3D object will have a lot of “common” vertices
          • Ex – Cube has 6*2 triangles, (6*2)*3 vertices, but only 8 “points”
        • So rather than passing vertices, pass 8 vertices, and 36 indices to the vertices to reduce Bandwidth
          • Indices can be 16bit, so reduce BW by ~50%
        • GL_ELEMENT_ARRAY_BUFFER and GL_ARRAY_BUFFER
        • STATIC_DRAW, DYNAMIC_DRAW
        • Not uploading again and again but re-use
      • What are Vertex Buffer Objects ?
        • genBuffers (createBuffer in WebGL), binding, bufferData/offset and usage
        • Usage of Index Buffers (ELEMENT_ARRAY_BUFFER)
      Cartesian
    • Translation
      • matrix1.translate(X,0.0,0);
      X = 0 X = 0.4  Translation applied to all objects (effect is not dependent on depth of object)
    • Rotation x y z Rotation  Observe effect of x offset! Refresh M,V,P after every rotate -0 Lookat
    • Getting the eye to see the object
      • “ Model” Matrix made the object “look” right
      • Now make the object visible to the “eye” – The “View”
        • Eye is always at the origin {0,0,0}
        • So using matrices, move the current object to the eye
      • “ LookAt” is implemented in many standard toolkits
        • The LookAt transformation is defined by
          • Viewpoint - from where the view ray starts (eye)
          • A Reference point (where the view ray ends) – in middle of scene (center)
          • A look-”up” direction (up)
      • ex – gluLookAt Utility function
      • Significant contributor of grey-hair
      Viewport
    • Viewport Transformation
      • Convert from the rendering to the final screen size
        • ie physical screen
      • Define the viewport using glViewport()
        • Viewport can be an area anywhere within the physical screen
      • This takes care of aspect ratio
        • Ex, square becomes rectangle in laptop
      • After the transformation, successful triangles get to the rasterisation HW, and then to the Fragment shader
      HW optimisations
    • Summary - The Transformation Sequence Translation example mathematical step - w
    • HW Optimisations
      • Not all triangles are visible
        • HW can reject based on depth
        • coverage
        • Front-facing or back-facing (Culling)
      • Culling is disabled by default per specification
        • However, most HW do this optimisation by default to save on bandwidth/ later pixel processing
      Programming
    • Programming !
      • Recall the Bandwidth needs for the vertex transfers / frame
      • Passing Vertices
        • Create Buffer Object
        • bindBuffer
        • bufferData
        • Indices are passed as type ELEMENT_ARRAY
      • Passing Attributes
        • bindAttribLocation
        • enableVertexAttribArray
        • vertexAttribPointer
      • matrix.getAsArray()
      • Lab #913 – Eyeing the eyes
    • Lab 913 – Keys for “Eyeing the eyes”
      • var totalArcs = 36 ; //shadervertexsetup_tunnel
      • texCoords.push( 10 *numArcs / totalArcs);
      • texCoords.push( 10 *zslice / numZSlices);
      • matrix1.scale(1.0, 1.0, 1.0 ); //not 15.0
    • Real life 3D models
    • Real-life modelling of objects
      • 3D models are stored in a combination of
        • Vertices
        • Indices / Faces *
        • Normals
        • Texture coordinates
      • Ex, .OBJ, 3DS, STL, FBX …
        • f, v, v//norm, v/t, o
        • Export of vertices => scaling to 1.0-1.0
        • Vertex normals vs face normals
        • Materials (mtl), animations
        • Problem of multiple indices not allowed in openGL
      • Tools and Models
        • Blender, Maya, …
        • http://assimp.sourceforge.net/ - tool for importing multiple types
        • http://www.blendswap.com/ - Blender models
      • Tessellation of meshes can be aided by HW in GPUs
    • Programming
      • Loading 3D models is an application functionality
        • No new APIs from OpenGLES are needed
      • A parser is required to parse the model files, and extract the vertex, attribute, normal, texture coordinate information
      • Look through objdata.js in
        • Lab #9
    • Shaders
    • Vertices, Fragments - Revisited
      • Vertices –
        • Points defined in a specific coordinate axes, to represent 3D geometry
        • Atleast 3 vertices are used to define a Triangle – one of the primitives supported by GLES
      • Fragments
        • The primitives are “rasterised” to convert the “area” under the primitive to a set of color pixels that are then placed in the output buffer
      Shader characteristics
    • Shader characteristics
      • Uniforms – uniform for all shader passes
        • Can be updated at run time from application
      • Attributes – changes per shader pass
      • Varying – Passed between vertex and fragment shaders
        • Ex, written by Vertex shader, and used by Fragment shader
          • gl_Position
      • Programs
        • Why do we need multiple programs in an application
          • for offscreen animation, different effects
      • MAX VARYING VECTORS – enum
      Inputs to shader
    • Inputs to the Shaders
      • Vertex Shader
        • Vertices, attributes,
        • Uniforms
      • Fragment Shader
        • Rasterised fragments (ie, after rasteriser fixed function HW)
        • Varyings from vertex shader
        • Uniforms
      Shader types
    • Fragment Shaders
      • A fragment is – a pixel belonging to an area of the target render screen (on-screen or off-screen)
        • Primitives are rasterised, after clipping
      • Fragment shader is responsible for the output colour, just before the post-processing operations
      • A Fragment shader can operate on “1” fragment at a time
      • Minimum number of “TEXTURE UNITS” is 8
      • Calculation of colors
        • Colors are interpolated across vertices automatically (Ref Lab 6 in the hands-on session) – ie, “varyings” are interpolated in Fragment shaders during rendering
        • Colors can be generated from a texture “sampler”
        • Each HW has a specific number of “Texture Units” that need to be activated, and textures assigned to it for operation in the shader
        • Additional information from vertex shader through “varyings”
      • Outputs
        • gl_FragColor
      Sample Frag shader
    • Program
      • Each program consists of 1 fragment shader, and 1 vertex shader
      • Within a program, all uniforms share a single global space
      Precision
    • Advanced Shaders
      • Animation
      • Environment Mapping
      • Per-Pixel Lighting (As opposed to textured lighting)
      • Bump Mapping
      • Ray Tracers
      • Procedural Textures
      • CSS – shaders (HTML5 – coming up)
    • Programming with Shaders
      • Pass in shader strings
      • Compile, link, Use
      • Set uniforms
      • Do calculations
      • Lab #96
    • Lab 96 – Keys for “Squishing the slice”
      • uniform mediump float skyline;
      • vec4 tempPos;
      • tempPos = MVPMatrix * inVertex;
      • tempPos.y=min(skyline, tempPos.y); //or, try below – one of the 2
      • tempPos.y=min(sin(inVertex.x*5.0)+cos(inVertex.y*2.0), tempPos.y);
      • gl_Position = tempPos;
      • var skylineLoc = context.getUniformLocation(sprogram,"skyline");
      • context.uniform1f(skylineLoc, -0.1);
      • context.drawArrays(context.TRIANGLES, 0, vertexparams[1 ] /3 );
    • Rendering Targets
    • Rendering Targets
      • A rendering context is required before drawing a scene. And a correponding Framebuffer
        • Recall bindFramebuffer()
      • It can be
        • Window system Framebuffer (Fb)
        • Offscreen buffer (Implemented in a Frame Buffer Object)
          • FBO is not a memory area – it is information about the actual color buffer in memory, depth/ stencil buffers
      • By default, rendering happens to the Window system framebuffer (ID ‘0’)
      Need
    • Need for offscreen rendering
      • Special effects
        • Refer the fire effect specified earlier (Multiple passes)
      • Interfacing to “non-display” use-cases
        • Ex, passing video through GPU, perform 3D effects, then re-encode back to compressed format
        • Edge detection/ computation – output is sent to a memory buffer for use by other (non-GL) engines
      FBO
    • FrameBuffer Object
      • A Frame Buffer Object
        • Can be just a color buffer (ex, a buffer of size 1920x1080x 4)
        • Typically also has depth/ stencil buffer
        • By default – FBO – ID “0” is never assigned to new FBO
          • It is assigned to Window system provided Frame Buffer (onscreen)
        • Renderbuffers and Textures can be “attached” to FBO
          • For RB – application has to allocate storage
          • For FBO, the GL server will allocate the storage
      rtt
    • Render-To-Texture
      • By binding a Texture to a FBO, the FBO can be used as
        • Stage 1 – target of a rendering operation
        • Stage 2 – used as a texture to another draw
        • This is “Render-To-Texture” (RTT)
      • This allows the flexibility of “discreetly” using the server to do 3D operations (not visible onscreen), then use this output as texture input to a visible object
        • If not for RTT, we have to render to regular Framebuffer then do CopyTexImage2D() or readPixels() which are inefficient
      • Offscreen rendering is needed for dynamic-reflections
      APIs
    • Post-processing operations
      • Blending with Framebuffer - enables nice effects (Ref Lab #6)
      • Standard Alpha-Blending
        • glEnable ( GL_BLEND );
        • glBlendFunc ( GL_SRC_ALPHA, GL_ONE );
      • Is a “bad” way of creating effects
        • Reads back previous framebuffer contents, then blend
        • Makes application memory bound, specially at larger resolutions
        • Stalls parallel operations within the GPU
        • Recommended way is to perform Render-To-Texture, and blending where necessary in the shader
      • But needed for medical image viewing – ex Ultrasound images, > 128 slices blending
      programming
    • Programming FBO and back to Fb
      • glGenFramebuffers
      • glBindFramebuffer
        • Makes this FBO used
      • glFramebufferTexture2D(id)
        • Indicate ‘id’ is to be used for rendering to TEXTURE, so storage is different
      • glDeleteFramebuffers
      • Then, create separate object to texture with TEXTURE ‘id’
      • Then, use previous textureID id as input to texImage2D next
      • Switching to FB
        • Change binding to screen FB
        • Load different set of vertices as needed, different program as needed
        • Set texture binding to FBO texture drawn previously
        • DrawElements call
      • FBOs are used to do post-processing effects
    • Programming
      • Draw a textured rectangle to a FBO
      • Using this FBO as texture, render another rectangle on-screen
      • CheckFramebufferStatus very important
      • Lab #910
    • Lab 910 – Keys for “Render to Texture” lab
      • Location, location, location ! Also note that readpixels doesn’t show anything!
      • // context.clearColor(1.0, 0.0, 0.0, 1.0);
      • // context.clear(context.COLOR_BUFFER_BIT | context.DEPTH_BUFFER_BIT);
      • // context.flush();
    • Platform Integration
    • Setting up the platform - EGL
      • Context, Window, Surface
      • OpenGL ES –
        • EGL_SWAP_BEHAVIOR == “EGL_BUFFER_PRESERVED”
          • Reduces performance
        • Anti-aliasing configurations
          • EGL_SAMPLES (4 to 16 typically, 4 on embedded platforms)
      • WebGL - preserveDrawingBuffer – attribute
        • Optimisations done if it is known that app is clearing the buffer – no dirty region check and whole scene is drawn efficiently
        • Dirty region check made in some systems
      Android
    • Android Integration Details
      • Android composition uses GLES2.0 mostly as a pixel processor, not a vertex processor
        • Uninteresting rectangular windows, treated as a texture
          • 6 vertices
        • Blending of translucent screens/ buttons/ text
      • 3D (GLES2.0) is natively integrated
        • 3D Live wallpaper backgrounds
        • Video morphing during conferencing (?)
        • Use the NDK
      Surfaceflinger
    • Android SurfaceFlinger architecture
      • Introduction to OpenGL interface on Android
        • http://code.google.com/p/gdc2011-android-opengl/wiki/TalkTranscript
      • HW acceleration on Android 3.0 / 4.0
        • http://android-developers.blogspot.com/2011/11/android-40-graphics-and-animations.html
      composition
    • Optimising OpenGL / ES applications
      • Graphics performance is closely tied to a specific HW
        • Size of interface to memory, cache lines
        • HW shared with CPU – ex, dedicated memory banks
        • Power vs Raw performance
        • Intelligent Discarding of vertices/ objects (!)
      • Performance is typically limited by
        • Memory throughput
        • GPU pixel operations per GPU clock
        • CPU throughput for operations involving vertices
        • Load balancing of units – within the GPU
      • GPUs that are integrated into SOCs are more closely tied to the CPU for operations, than separate GPUs
        • Ex, GPU drivers offload some operations to CPU
      debugging
    • Debugging OpenGL
      • Vanishing vertices, Holes
      • Improper lighting
      • Missing objects in complex scenes
      • Windows Tools
        • Perfkit/ GLExpert / gDEBugger
        • Intel GPA
      • Linux Tools
        • PVRTune (IMG)
        • GDebugger
        • Standard kernel tools
        • Intel GPA
      • Pixel vs Vertex throughput, CPU loading, FPS, Memory limited – tuning knobs
    • References
      • Specs - http://khronos.org/opengles
      • CanvasMatrix.js
        • https://github.com/toji/gl-matrix
      • Tools - http://www.iquilezles.org/apps/shadertoy/
        • http://www.inka3d.com/ (from Maya)
        • http://assimp.sourceforge.net/ - Asset importer
      • ARM – Mali – Architecture Recommendations
        • http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0363d/CJAFCCDE.html
      • Optimising games – simple tips
        • http://glenncorpes.blogspot.com/2011/09/topia-optimising-for-opengles20.html
    • Appendix: Video and Graphics
      • Graphics is computed creation
        • Video is recorded as-is
      • Graphics is object – based
        • Video (today) is not
      • Graphics is computed every frame fully
        • Video is mostly delta sequences
          • Motion-detection, construction, compensation
          • But extensions like swap_region (Nokia) exist
    • Q & A, Feedback
      • Feedback
        • http://ewh.ieee.org/r10/bangalore/ces/gfx2011.html