OpenGL ES based UI Development on TI Platforms

Building advanced user interfaces using SGX Graphics core on OMAP3 Prabindh Sundareson ( [email_address] ) DSPS Texas Instruments

Outline
Overview of OpenGL
Developing an UI with 3D components
Advanced 3D features – texturing, rotation, materials
Integrating the whole system
Performance and Power
Call For Action
Q & A

Video vs Graphics It’s a state machine - Setup the stage, lighting, actors… Then draw it ! Based on motion between frames OpenGL based Graphics Video

The OpenGL ES 2.0 Picture OPENGL Full version ES version Common Common-Lite GLSL companion GLSL-ES companion What we miss in ES compared to desktop version: Polygons, Display lists, Accumulation buffers,… Currently in 3.0 Currently in 2.0 Currently in 1.0.16 Currently in 1.20 EGL Currently in 1.3 Core GL Spec Vertex/Frag shaders Platform interface and Configuration setup EGL Currently in 1.3

ES2.0,ES1.1 and Full Function
ES2.0 offers a Programmable pipeline vs Fixed function in ES1.1
Ex, Lighting, Fog functions in ES1.1 not present in ES2.0, so need to write code to generate these effects.
More flexibility, little more work, but lot of available code
Android uses ES 1.1
ES2.0 NOT backward compatible with ES1.1
See Companion paper for definitive comparison of available functions in ES1.1 and ES2.0
See “Difference Specification” documents in Khronos OpenGL website, for summary of changes compared to openGL “full” function specification
Ok, so what exactly is openGL ?

Primitives, Vertices … D B C A *Note the orientation

The pipeline… Vertex shader outputs position of vertex, calculates orientation Fragment shader outputs colour

Sample OpenGL ES 2.0 program Handle = get_platform_specific_window_handle(); eglGetDisplay (handle); eglInitialize (); eglBindAPI ( EGL_OPENGL_ES_API ); eglChooseConfig (); eglCreateWindowSurface (); eglCreateContext (); eglMakeCurrent (); //Compile and Load the shaders before this step … float afVertices [] = {G,H,F, F,H,E, E,A,F, F,A,B, …}; glEnableVertexAttribArray (0); glVertexAttribPointer ( VERTEX_ARRAY, 3, GL_FLOAT, GL_FALSE, 0, (const void*)afVertices); glDrawArrays(GL_TRIANGLE_STRIP, 0, 4) ; eglSwapBuffers(eglDisplay, eglSurface); Setup Actors Show Keep showing the actors, in a loop, change view/position/texture ..

Simple Shader
char* pszFragShader = "
uniform sampler2D sTexture;
varying mediump vec2 TexCoord;
void main (void)
{
gl_FragColor = texture2D(sTexture, TexCoord);
}";
char* pszVertShader = "
attribute highp vec4 inVertex;
attribute mediump vec2 inTexCoord;
uniform mediump mat4 MVPMatrix;
varying mediump vec2 TexCoord;
void main()
{
gl_Position = MVPMatrix * inVertex;
TexCoord = inTexCoord;
}";
Note: Read the GL ES Shading Language specification for details

OMAP3 + SGX530
The OMAP3 Graphics Engine supports
OpenGL ES1.1 + EGL 1.3  Fixed function pipeline
OpenGL ES2.0 + EGL 1.3  Programmable function pipeline (shaders)
OpenVG 1.0.1
Arbitrary texture sizes
Several useful application libraries provided for
Font support (Fixed fonts)
Loading textures
Loading compressed textures and with mip-maps
Loading effects files
See the Graphics SDK for details

OMAP3 from a UI point of view
High performance ARM Cortex-A8
VFP Lite with NEON
Floating point acceleration
Use the below GCC flags on Linux when compiling, to get best performance on Graphics applications
“ -mcpu=cortex-a8 -mfpu=neon -mfloat-abi=softfp”
SGX core
Display subsystem that supports
Output resolution upto 1280 x 720 (24 bit Digital)
Graphics/Video planes

OMAP3 from a UI point of view
Linux/WinCE
QT/GTK based toolkits
Other 3Ps
Browser as an OS independent User Interface – Opera, Firefox, other Webkit based browsers
Many of these can use Cairo (See companion paper) as backend for 2D rendering
Flash, Flash-lite
Java – M3G
How to integrate 3D graphics efficiently into these ?

Typical UI development with OpenGL ES
Shapes 
Convert to meshes based on primitives (ex, using 3DStudio plugins)
Artwork 
Texture, Mip Maps, filtering modes, lighting (in compressed or uncompressed formats)
Animation 
Path description, timing, dynamic changes, rotation (Write shader logic)
Transparency 
Alpha settings
Programming the display path 
Display driver interfaces and HW
Performance tuning –
Pipelining
Compressed textures

Alpha Blending with SGX
glEnable(GL_BLEND); //disabled by default
glBlendEquation(GL_FUNC_ADD); //many other equations available
glBlendFunc(GL_SRC_ALPHA, GL_DST_ALPHA); //many other modes available

Texturing
Gives more realistic views to otherwise “dull” objects
Ex, Lighting effects
Textures can be calculated programmatically, or can be taken from images specified by application at runtime (ex, live video thumbnails)
Can be applied to lines, or surfaces
Coordinates mapped in Vertex shaders
“ Applying the texture” done in Fragment shaders
Can have multiple textures that can be combined
One for lighting effects, one for material properties (ex. Gloss vs Matt …)

Adding Texturing using OpenGL
Pass Texture coordinates
glEnableVertexAttribArray ();
glVertexAttribPointer ();
Use glTexImage2D () to load image buffer for texture
Apply texture in the fragment shader program
gl_FragColor = texture2D( );
OMAP3 supports ARGB 8888 textures

Texturing with SGX - Example An example Window manager using texturing and SGX only

Rotation with OpenGL
Rotation is accomplished by modifying the viewing angle
Update the Model View Projection Matrix before drawing
Update the vertex position in the vertex shader
gl_Position = MVPMatrix * inVertex; // Recall the shader code
Along with texturing, can be used to create “wet floor” effects
Apply texture
Rotate object to make back side visible, or apply texture in reverse way (remember the orientation of primitives ?)
Need to disable CULLing to make back side visible if first method is used

Key Decision Points
Decide - Compositing also on SGX, or only 3D on SGX ?
Decision should be based on ARM loading for the target UI
Benchmark before deciding – What other applications run on ARM while running the UI ?
Use indexed vertices instead of strips for large number of vertices
How many > 500 vertex objects are being submitted for rendering ?
Avoid dependency on CPU for SGX, and SGX for CPU
Mip-Maps increase the size of textures
But increase the quality discernibly, especially for near-far cases

Getting the final image to the display
Case 1 (Partner mode – work with other applications):
OpenGL allows reading back the composited image from the gl frame buffer
glReadPixels(xpos, ypos, xwidth, yheight, format, type, *userdataptr);
Caveat: This flushes the pipeline. Benchmark for your target application
Now the data is in the external application/driver
Regular alpha blending/compositing operations can be performed on this data
OMAP3 Display System HW supports 3 display planes
2 video planes with up/down scaling, 1 Graphics plane
Any of these planes can be used for blending with outputs from other applications
Case 2 (Full ownership mode)
The final image can be directly written to the framebuffer in full screen mode by EGL itself

Application Integration
Multiple options available
Write separate application, directly use EGL for display to frame buffer
Use SGX for all compositing and window operations
No other window management required
Use application framework like Clutter to build your applications. Configure Clutter to use OpenGL-ES as backend
More application support becomes available with this option, ex. Browsers ..
Clutter is still under active development
Use GTK to manage the Graphics output created by SGX application

The SGX core is connected to the L3 interface (Refer SPRU66B)
Internal DMA, MMU for operating without interfering ARM
Tops at 111 MHz, other variants in the future
Power management handled internally in the SGX HW/driver
Power numbers comparable to ARM at equivalent MHz (contact TI support for details)

Target upto 30 fps with 512x512 (window like) textures in 2D
Frame rate goes down as number/size of textures, and complexity of objects and operations goes up
Use compressed PVRTC mode textures for static textures
Rendering full screen with OpenGL allows ARM to do other stuff
Network stack, Browsers, Multimedia, …

Call For Action
Download the OMAP3 Graphics SDK
Use Beagle or EVM or other suitable OMAP3 platform for development
Use the examples provided to start, download examples from net
Use mailing list for support
Engage at the TI open-source symposium (7/11/08)

Support
For support on Graphics related queries on OMAP3, write to
[email_address]
We will be glad to support you to create that great UI !!

References
TRM for SGX core
http://focus.ti.com /lit/ug/spruff6b/spruff6b.pdf
PC version of PowerVR SDK
http://www.imgtec.com/powervr/insider/sdk/KhronosOpenGLES2xSGX.asp
TI DVEVM software (Needs EVM serial #, and my.ti.com account for registration)
https://www-a.ti.com/extranet/cm/product/dvevmsw/dspswext/general/v1_30_00.shtml
The OpenGL Redbook
http://www.glprogramming.com/red/chapter01.html
OpenGL ES2.0 programming handbook
OpenGL® ES 2.0 Programming Guide, Aaftab Munshi ; Dan Ginsburg; Dave Shreiner
Khronos organisation
http://Khronos.org
Brief description of OMAP3 Graphics usage (Needs my.ti.com acc)
https://focus.ti.com/general/docs/event/accesseventaction.tsp?actionId=414

OpenGL ES based UI Development on TI Platforms

by Prabindh Sundareson on Dec 13, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 36

Statistics

OpenGL ES based UI Development on TI Platforms — Presentation Transcript

http://www.slideshare.net	35
http://translate.googleusercontent.com	1