The Android graphics architecture uses SurfaceFlinger to composite surfaces from apps into the final display. Apps draw to surfaces using a canvas, which can target OpenGL or a software renderer. Views define a display list of drawing commands. SurfaceFlinger receives surfaces from apps and window status from WindowManager to composite surfaces in z-order before each VSYNC signal.

1. Android Graphics
Architecture Overview
2015.08.20

3. Android graphics stack overview
● App acquires its own Surface(s) from SurfaceFlinger
○ comes with a Canvas and a graphics buffer
● App uses Canvas to draw the screen
○ underlying implementations may use OpenGL or
software render engine (e.g. SKIA)
● SurfaceFlinger composes final screen from Surfaces
○ get Activity status from ActivityManager
○ get z-order information from WindowManager
● Gralloc maps to the framebuffer device (fb0)

4. Android View component
● Button, TextView… are all Views
● A page layout is composed of a tree of Views
● Actual drawing behavior implemented in onDraw()
method of each View
○ Display List: a set of graphic commands
○ commands can be mapped to OpenGL commands
● Create DeferredDisplayList in order to optimize the
rendering order
○ analyze the overlapping… relationships
○ minimize GPU usage

5. Android View component
● An example of Display List (button)
○ Save 3
○ DrawPatch
○ Save 3
○ ClipRect 20.00, 4.00, 99.00, 44.00, 1
○ Translate 20.00, 12.00
○ DrawText 9, 18, 9, 0.00, 19.00, 0x17e898
○ Restore
○ RestoreToCount 0
● Can be mapped to gl-prefix OpenGL commands

6. 2D rendering path
● Can be rendered by HW or SW
○ HWUI: hardware accelerated
using OpenGL ES 2.0
○ Skia: software render engine

7. 3D rendering path
● App can instead create
GLSurfaceView
● Use OpenGL ES bindings for
Java
● Either use vendor GPU driver or
software-based PixelFlinger to
render the drawing to Surface

8. SurfaceFlinger
Compose, but not render!

9. SurfaceFlinger
● Services connections from activities (client) via Binder
interface ISurfaceComposer (server)
● Activity acquires Surface from SurfaceFlinger
○ created by ISurfaceComposer
● Receives activity status from ActivityManager
● Receives window status (visibility, z-order) from
WindowManager
● Composes multiple Surfaces into a single image
○ by HWComposer: also generates VSYNC

10. SurfaceFlinger
● GraphicBuffer
○ allocated by Gralloc, accessed using mmap
○ pixel format: RGB/RGBA/RGBO
● Project Butter
○ VSYNC
○ Triple Buffer
○ Choreographer
○ http://www.androidpolice.com/2012/07/12/getting-to-know-andr
oid-4-1-part-3-project-butter-how-it-works-and-what-it-added/

11. VSYNC Signal
● Generated by HWComposer
○ HW source: register callback of HAL
○ or SW source: VSyncThread
● Distributed by SurfaceFlinger
○ Client can create an event connection to SurfaceFlinger’s
MessageQueue to receive the VSync event
○ SurfaceFlinger inherits HWComposer::EventHandler to handle
onVSyncRecieved() and notify listeners

12. Application
● Each page has its View Tree
● Need to traverse the tree when
○ App or page being launched
○ External event: touch, key press
○ Internal event: UI control change, View.Visibility
● How to traverse
○ Post a request to Choreographer (if no request left)
○ Choreograpger calls perfromTraversals() when VSYNC signal
triggered

13. Application
● Inside performTraversals()
○ performMeasure(): calculate size of UI controls
○ performLayout(): calculate UI layout
○ performDraw()
● Inside performDraw()
○ High level: draw to Canvas (based on Surface in Java level)
○ Low level
■ Lock Surface (native level) to dequeue a buffer from
BufferQueue
■ Render to buffer by OpenGL ES or SKIA
■ Unlock Surface to enqueue the buffer

14. Surface (native level)
● Each app/page can acquire one or more Surface/Layer
● Done by using ISurfaceComposerClient to connect to
SurfaceFlinger’s Binder service
● ISurfaceComposerClient will return a SurfaceControl, then use it to
create Surface
● One of two native windows in Android graphic systems (another
one is for SurfaceFlinger)
● OpenGL ES render to the buffer and enqueue to BufferQueue of
Surface, act as a “producer”

15. BufferQueue
● A producer/consumer pattern
○ Producer: Application (Surface)
○ Consumer: SurfaceFlinger
● Buffer operation
○ Application: dequeue()->write buffer->enqueue()
○ SurfaceFlinger: acquire()->read buffer->release()
● When a buffer being queued
○ Layer listens to onFrameAvailable() callback
○ Call Layer’s onFrameQueued() to increment counter variable:
mQueuedFrames

16. BufferQueue
read buffer write buffer
ApplicationSurfaceflinger

17. SurfaceFlinger
● When VSYNC signal triggered
○ handleMessageTransaction(): detect Layers’ changes
(including size, z-order, alpha, add/removal…), save as
mDrawingState
○ handleMessageInvalidate(): acquire a buffer from BufferQueue
if available
○ signalRefresh() ---> call handleMessageRefresh()

18. SurfaceFlinger
● handleMessageRefresh()
○ preComposition(): check if there is any layer with available
buffers (mQueuedFrames > 0), signal invalidate if true
○ rebuildLayerStacks(): by calculating opaque region, visible
region, covered region and transparent region
○ ……
○ doComposition(): done by HWComposer or OpenGL ES
○ Swap buffers (front/back buffer)