The document provides an overview of the Android graphics subsystem and related components: - It describes the main components of the graphics subsystem including SurfaceFlinger, Window Manager, and Activity Manager. - It discusses graphics concepts such as surfaces, layers, and composition. - It provides details on the use of OpenGL and hardware acceleration in Android. - It summarizes the roles of various graphics libraries and interfaces like Skia, PixelFlinger, and Gralloc.

1. Android internals
Egor Elizarov
SPbSU 2012

2. Legal info
 Android internals by Egor Elizarov is licensed under a
Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License
 You are free to
– copy, distribute, display, and perform the work
– make derivative works
 Under the following conditions
– Attribution. You must give the original author credit
– Share Alike. If you alter, transform, or build upon this work, you may
distribute the resulting work only under a license identical to this one
 All pictures and trademarks are the property of their respective owners. Use of
these trademarks and pictures is subject to owners permissions.
 Corrections, suggestions, contributions and translations are welcome!
2 Egor Elizarov SPbSU 2012

3. Lecture 7
Graphic Subsystem
OpenGL HW acceleration
yegor.yelizarov(at)gmail.com
Rev: 1.1
Last update: 06/01/2012 http://vk.com/android_internals
3 Egor Elizarov SPbSU 2012

4. Previous time
 Linux IPC primitives
 Binder
 Application layer
 Service manager
 Middleware level
 Kernel level
 Typical Android sybsystem
4 Egor Elizarov SPbSU 2012

5. Graphic subsystem

Provides buffer to application for drawing

Provides interfaces for draw primitives

Controls and composes surfaces (windows)

Provides 2D & 3D primitives

Uses HW acceleration where acceptable
5 Egor Elizarov SPbSU 2012

6. Main players

Surface Flinger

Window Manager

Activity Manager
6 Egor Elizarov SPbSU 2012

7. Surface

Virtual buffer we can draw on

Can be marshalled/unmarlled in Parcel

One Activity can have more then one Surface
7 Egor Elizarov SPbSU 2012

8. Surface Flinger

Allocates surfaces

Composes surfaces

Can combines 2D/3D surfaces

Can use HW acceleration for surface
composition
8 Egor Elizarov SPbSU 2012

9. Window Manager

Dispatches input events to clients

Provides transition animation capabilities

Window Manager Policy handles common key
events in PhoneWindowManager
(Home/Back/Volume)
9 Egor Elizarov SPbSU 2012

10. Activity manager

Manages activity life cycle

Manages activity stacking

Dispatches intents

Spawn processes
10 Egor Elizarov SPbSU 2012

11. Composition example
11 Egor Elizarov SPbSU 2012

12. Simple view
12 Egor Elizarov SPbSU 2012

13. Khronos OpenGL API

Khronos Group - not for profit industry
consortium creating open standards.

OpenGL – 2D/3D graphics API

OpenGLES – 2D/3D graphics API for embedded
systems

EGL – interface between OpenGLES and native
platform window system
13 Egor Elizarov SPbSU 2012

14. Graphic libraries

SKIA - 2D graphic library

PixelFlinger – android software renderer

Codeflinger – simple graphic JIT

Gralloc – android graphics memory allocator

Mesa – open source OpenGL implementation
14 Egor Elizarov SPbSU 2012

15. Renderscript

Renderscript is new API targeted at high-
performance 3D rendering and compute
operations.

Renderscript provides:
• 3D rendering API on top of HW acceleration
• compute API (similar to CUDA)
• familiar language in C99
15 Egor Elizarov SPbSU 2012

16. Java/middleware interaction
16 Egor Elizarov SPbSU 2012

17. Double buffering
17 Egor Elizarov SPbSU 2012

18. Composition approach

Dirty regions – regions that should be redrawn

Use Z ordering to compose layers

Use HW composer to compose layers if possible

frameworks/base/services/surfaceflinger/Surface
Flinger.cpp
18 Egor Elizarov SPbSU 2012

19. SW vs HW OpenGL
19 Egor Elizarov SPbSU 2012

20. Middleware level
20 Egor Elizarov SPbSU 2012

21. 3D acceleration
21 Egor Elizarov SPbSU 2012

22. 2D acceleration
22 Egor Elizarov SPbSU 2012

23. HAL/kernel interaction
23 Egor Elizarov SPbSU 2012

24. Overlay vs Framebuffer surface

Two different type of surfaces in Android < 3.0

Overlay
– Picture in YUV format (Video, Camera)
– Video4Linux devices (/dev/video*)

Framebuffer surfaces
– Picture in RGB format
– Framebuffer device (/dev/fb*)

Composition is performed by HW
24 Egor Elizarov SPbSU 2012

25. HWComposer (Omap4)
25 Egor Elizarov SPbSU 2012

26. Emulator GL acceleration

In ICS Android team added GL acceleration for
Emulator

Emulator uses x86 host GL acceleration relying
on host hardware accelerated libraries
26 Egor Elizarov SPbSU 2012

27. Android-emulator approach
27 Egor Elizarov SPbSU 2012

28. Android-x86 approach
28 Egor Elizarov SPbSU 2012

29. Next Time

System start up

Media subsystem
29 Egor Elizarov SPbSU 2012

30. Useful links

http://vk.com/android_internals

http://www.slideshare.net/jserv/design-and-concepts-of-android-
graphics

http://people.debian.org.tw/~olv/surfaceflinger/surfaceflinger.pdf

http://www.cnblogs.com/xl19862005/archive/2011/11/17/22153
63.html

Http://www.khronos.org

https://plus.google.com/105051985738280261832/posts/2FXDC
z8x93s
30 Egor Elizarov SPbSU 2012

31. Useful links (2)

http://www.ti.com/lit/wp/swpy027/swpy027.pdf

http://www.freescale.com/files/ftf_2010/Americas/FTF10_FT
F-ENT-F0824.pdf

http://elinux.org/images/0/02/Android_Platform_Optimization
s_SNPS_20111027.pdf

http://android-developers.blogspot.com/2011/02/introducing-
renderscript.html

http://android-
developers.blogspot.com/2011/03/renderscript.html
31 Egor Elizarov SPbSU 2012

32. Thanks to

Sergey Matyukevich for review and advices
(www.linkedin.com/pub/sergey-matyukevich/31/889/769)

Nikolay F. Fominykh for review and advices

Nikita Shulga for advices and notes
(http://www.linkedin.com/pub/nikita-shulga/8/582/287)

Grigory Tolstolytkin for advices and notes
(http://www.linkedin.com/pub/grigory-
tolstolytkin/2a/b41/74)
32 Egor Elizarov SPbSU 2012