SlideShare a Scribd company logo
1 of 58
Download to read offline
Android Graphics




Jim Huang ( 黃敬群 )
Developer & Co-Founder, 0xlab
                                         jserv@0xlab.org

                                Sep 24, 2011 / Study-Area
Rights to copy
                                                                    © Copyright 2011 0xlab
                                                                           http://0xlab.org/
                                                                            contact@0xlab.org
Attribution – ShareAlike 3.0
You are free                                                  Corrections, suggestions, contributions and
                                                                               translations are welcome!
   to copy, distribute, display, and perform the work
   to make derivative works                                                  Latest update: Nov 20, 2011
   to make commercial use of the work
Under the following conditions
      Attribution. You must give the original author credit.
      Share Alike. If you alter, transform, or build upon this work, you may distribute the
      resulting work only under a license identical to this one.
   For any reuse or distribution, you must make clear to others the license terms of this
   work.
   Any of these conditions can be waived if you get permission from the copyright holder.
Your fair use and other rights are in no way affected by the above.
License text: http://creativecommons.org/licenses/by-sa/3.0/legalcode
Agenda   (1) Binder IPC
         (2) Android Graphics
         (3) 2D and Accelerations
         (4) OpenGL|ES
Notice:
    Before looking into Android
 Graphics, you should be aware of
the design of Binder IPC, otherwise
        you would get lost!
Binder IPC
Processes running on Android
$ ps
    ...
root        37    1    248      156     c00aef2c   0000875c   S   /sbin/ueventd
system      42    1    768      260     c022950c   afd0b6fc   S   /system/bin/servicemanager
root        43    1    3824     564     ffffffff   afd0bdac   S   /system/bin/vold
root        44    1    3796     560     ffffffff   afd0bdac   S   /system/bin/netd
root        45    1    628      264     c02588c0   afd0c0cc   S   /system/bin/debuggerd
radio       46    1    4336     672     ffffffff   afd0bdac   S   /system/bin/rild
root        47    1    62224    27576   c00aef2c   afd0b844   S   zygote
media       48    1    16828    3736    ffffffff   afd0b6fc   S   /system/bin/mediaserver
bluetooth   49    1    1216     572     c00aef2c   afd0c59c   S   /system/bin/dbus-daemon
root        50    1    776      316     c02a8424   afd0b45c   S   /system/bin/installd
keystore    51    1    1704     432     c02588c0   afd0c0cc   S   /system/bin/keystore
shell       52    1    696      336     c0050934   afd0c3ac   S   /system/bin/sh
root        53    1    3356     160     ffffffff   00008294   S   /sbin/adbd
system      67    47   172464   32596   ffffffff   afd0b6fc   S   system_server
system      115   47   80028    20728   ffffffff   afd0c51c   S   com.android.systemui
app_24      124   47   80732    20720   ffffffff   afd0c51c   S   com.android.inputmethod.latin
radio       135   47   87848    20324   ffffffff   afd0c51c   S   com.android.phone
app_18      144   47   89136    24160   ffffffff   afd0c51c   S   com.android.launcher
app_7       165   47   86136    22736   ffffffff   afd0c51c   S   android.process.acore
app_0       197   47   73996    17472   ffffffff   afd0c51c   S   com.android.deskclock
app_14      208   47   75000    18464   ffffffff   afd0c51c   S   android.process.media
app_3       219   47   72228    17652   ffffffff   afd0c51c   S   com.android.bluetooth
app_25      234   47   85336    17836   ffffffff   afd0c51c   S   com.android.mms
app_26      254   47   74656    19080   ffffffff   afd0c51c   S   com.android.email
app_27      266   47   74912    18100   ffffffff   afd0c51c   S   com.android.providers.calendar
app_1       285   47   71616    16280   ffffffff   afd0c51c   S   com.android.protips
app_19      293   47   72184    16572   ffffffff   afd0c51c   S   com.android.music
app_21      301   47   74728    17208   ffffffff   afd0c51c   S   com.android.quicksearchbox
app_28      311   47   75408    18040   ffffffff   afd0c51c   S   com.cooliris.media
shell       323   52   856      316     00000000   afd0b45c   R   ps
$
                   More than 30 processes (200+ threads).
                                                                                                   6
IPC = Inter-Process Communication




                Activity
     Activity              Window     Alarm
     Manager               Manager   Manager




Kernel




                                               7
IPC Abstraction

                  More abstract
Intent                            • Intent
                                     – The highest level abstraction
    AIDL                          • Inter process method invocation
                                     – AIDL: Android Interface
         Binder                       Definition Language
                                  • binder: kernel driver
                                  • ashmem: shared memory




                                                                  8
Method invocation




      caller




      callee


In the same process


                              9
Inter-process method invocation


                        caller
                                   interface



caller

            interface
                            How?
callee

                                   interface


                        callee
                                               10
Inter-process method invocation


                           caller
                                            interface


                           Proxy
caller

            interface    Binder in kernel


                        Binder Thread
callee
                             Stub
                                            interface


                           callee
                                                        11
android.os.Parcel



            Delivering arguments of method




”flatten”                                    ”unflatten”


                     transmit
                                                      12
UML Representation



        <<interface>>

                   implements


Proxy                   Stub




                                 13
UML Representation

                                 caller



        <<interface>>              calls


                   implements


Proxy                   Stub

                               extends

                        callee

                                           14
AIDL

Auto generated from .aidl file                      caller



                           <<interface>>




                   Proxy                   Stub



                                           callee

                                                               15
Use Case:
             Who calls onPause() in Activity?
                  2
                      Send message
                       by Handler         Activity

                                                     3
                                                         OnPause() is
                      queue               Looper     called in main thread

  Activity
  Manager                                   Main
                        Binder             Thread
                       Thread #1




Kernel        1

                  Call ”schedulePauseActivity”
                         across process
                                                                     16
Binder

• Multi-thread aware
  – Have internal status per thead
  – Compare to UNIX socket: sockets have internal
    status per file descriptor (FD)




                                                    17
Binder




 A pool of threads is associated to each service application to process
  incoming IPC (Inter-Process Communication).
 Binder performs mapping of object between two processes.
 Binder uses an object reference as an address in a process’s memory
  space.
 Synchronous call, reference couting
Binder
Binder is different from UNIX socket

                  socket             binder
internal status   associated to FD   associated to PID
                                     (FD can be shared among
                                     threads in the same
                                     process)


read & write      stream I/O         done at once by
operation                            ioctl

network           Yes                No
transparency                         expected local only




                                                           20
Transaction of Binder

                      binder_write_read
         write_size                        write buffer
    write_consumed
       write_buffer
        read_size
    read_consumed                          read buffer
       read_buffer




if (ioctl(fd, BINDER_WRITE_READ, &bwt ) >= 0)
     err = NO_ERROR;
else
     err = -errno;

                                                          21
Transaction of Binder
                                           Process A and B have different memory space.
                                           They can not see each other.
             Kernel                             Process B
                                             Process A
                       Binder



                                 Copy memory by copy_from _user
                                 Then, wake up process B

             Kernel
                                                  Process B
                       Binder



                                             Process A
                                 Copy memory by copy_to_user



Internally, Android uses Binder for graphics data transaction across processes.           22
                               It is fairly efficient.
Real Case




Binder IPC is used for communicating between Graphics client and server.       23
Taken from http://www.cnblogs.com/xl19862005/archive/2011/11/17/2215363.html
Ashmem
• Android / Anonymous SHared MEMory subsystem
  – system/core/cutils/ashmem.h
  – int ashmem_create_region(const char *name, size_t size)
  – int ashmem_set_prot_region(int fd, int prot)
  – int ashmem_pin_region(int fd, size_t offset, size_t len)
  – int ashmem_unpin_region(int fd, size_t offset, size_t len)

• a named memory block shared between processes
  that the kernel is allowed to free.
   – This is notable as the kernel is not allowed to
     free standard shared memory.
• Similar to weak reference of Java. Useful to
  implement cache.
• Used in android.os.MemoryFile (Java), 2D memory
  allocator, etc.
                                                                     24
Android Graphics
Surface

  Source: frameworks/base/core/java/android/view/Surface.java
• /* Handle on to a raw buffer that is being
  managed by the screen compositor */
  public class Surface implements Parcelable {
     public Surface() {
        mCanvas = new CompatibleCanvas();
      }
      private class CompatibleCanvas
               extends Canvas { /* ... */ }
  }
 Surface instances can be written to and restored from a Parcel.
 Surface instances can be written to and restored from a Parcel.
Delivering arguments of method



”flatten”                                    ”unflatten”


                     transmit
   Properties
                                   Android SurfaceFlinger
        Can combine 2D/3D surfaces and surfaces from multiple applications
        Surfaces passed as buffers via Binder IPC calls
        Can use OpenGL ES and 2D hardware accelerator for its compositions
             Double-buffering using page-flip
from SurfaceFlinger to Framebuffer
PixelFlinger : software renderer
• Render functions: pointx, linex, recti, trianglex
• Texture and color buffer: activeTexture, bindTexture,
  colorBuffer, readBuffer, depthBuffer, BindTextureLod
• …
• Device framebuffer functions: copyPixels,
  rasterPos2x, rasterPos2i
• Optimizer: codeflinger (JIT assembler)



 I/SurfaceFlinger( 1931): OpenGL informations:

 I/SurfaceFlinger( 1931): vendor    : Android

 I/SurfaceFlinger( 1931): renderer : Android PixelFlinger 1.2

 I/SurfaceFlinger( 1931): version   : OpenGL ES­CM 1.0
GGLAssembler –
GGLAssembler –
codeflinger's JIT Assembler
codeflinger's JIT Assembler
2D and Accelerator
Skia

• SkBitmap
• SkBitmap::Allocator
• GPU integration in Android 4.0
Case Study: WebKit rendering

  Fill rectangle           FillRectangles()



 Set clip region

                                          Acquire font cache

  Show glyphs                                                Blit(), memcpy()
               BatchBlit()
                                                  Allocate font cache

Surface composite

              Blit()           Prepare / finish    Clear()
              TileBlit()         composite
2D Compositing

         SRC

                                                                SRC & DST



                               Compositing
                     1 -asrc                  SRC & NO-DST



               DST




                                                             NO-DST & NO-SRC


1-adst                 adst
Enable 2D Accelerator
Android Graphics (HAL view)

                                                Android Framework (Java)

                   libandroid_runtime
                                         Surfaceflinger                   Copybit
EventHub                                   (service)                   (HW accelerated)




                                               LibGLES
           libui
                                                (libagl)




                       libpixelflinger
2D Accelerator for Android Graphics

     • libcopybit provides hareware bitblit operations which
       includes moving, scaling, rotation, mirroring, and
       more effects, like blending, dithering, bluring, etc.
     • Removed since Android 2.3
     • Android has two copybit interfaces:
         – Blit: moving / blending
        – Stretch: scaling besides moving
     • libcopybit is called by libagl which can do
       swapBuffers to do the framebuffer page flipping that
       can also be accelerated by libcopybit.



Copybit could improve the performance of page flipping
Copybit could improve the performance of page flipping
Copybit operations
Copybit: 2D blitter
Copybit: 2D blitter
PMEM: Manipulate physically continuous memory

 • Hardware graphics/bitblt operations (blitter) needs
   physical continuous memory to manipulate.
 • Android use libgralloc to allocate pmem (physical
   continuous memory) for android native buffer.
   pmem driver can support up to 12 devices, we
   have only one for copybit (Android, android native
   buffer)
 • While running 0xbench, the peak size of the
   pmem allocated (mapped) is 25194496 bytes.

    Take Qualcomm MSM7x25 for example:
    Take Qualcomm MSM7x25 for example:
    • /dev/pmem
    • /dev/pmem
    • /dev/pmem_adsp
    • /dev/pmem_adsp
        – For multimedia codec, audio, video, camera
        – For multimedia codec, audio, video, camera
gralloc & pmem



gralloc_alloc             Offset = Alloc(size),
                          PMEM_CONNECT,
                          PMEM_MAP,
                          memset
                          If success,
                          new private_handle_t(fd, size, flags);
                gralloc
                module


 gralloc_free             PMEM_UNMAP,
                          deallocate(offset)
pmem memory

(mmaped virtual address)base                     master_fd (mmap size: 47MB)
0x49422336, c06c4000,
6c4000                         Buffer 1
   buffer base = base +                          fd when being freed, this
   offset                                        will be closed
                               Buffer 2



                               Buffer 3
                                           Quantum


    Debug pmem,                pmem
    /sys/kernel/pmem_regions
                                          pmem can have up to 12
                                          devices..., we have only one for
                                          copybit (Android native buffer)
gralloc/copybit structure

                     android_native_buffer_t
                     buffer_handle_t handle;


  native_handle_t                     copybit_image_t
                                 native_handle_t* handle;



                         buffer_handle_t                libgralloc
private_handle_t
  Fd (pmem dev)                          Android native buffer contains
       Offset                            buffer_handle_t
        Size                             Which is native_handle * ,
       Base                              defined in system/cutils/
       Flags           virtual address
     lockState
                          private_handle_t::PRIV_FLAGS_U
                          SES_PMEM

         private_handle_t::LOCK_STATE_MAPPED
Functions in copybit HAL

• BLIT (moving)
• Stretch (scaling)
• Alpha-blending
   – Unused in real case
• Rotate
  – Unused in real case
OpenGL|ES
Key Concepts

• Android is moving to OpenGL|ES accelerated
  rendering since version 2.x
• Window systems already comprehend z-order
• 3D != 2D with depth
• OpenGL is object based
  – Describes a scene using its components and properties. Output
    quality is dependent on the renderer, not source
OpenGL Terminology

• OpenGL
  An API from Khronos (from SGI), for constructing a 3D object, doing operations on
  the object, and displaying it

• Primitives
  Triangles, Lines, Points, that can be specified through vertices to define an
  arbitrary shape

• Texture
  Small (!) bitmap to make objects more realistic

• EGL
  The EGL API defines a portable mechanism for creating GL contexts and windows
  for rendering into, which may be used in conjunction with different native platform
  window systems using the WSEGL layer
EGL

• EGL (Embedded-System Graphics Library) is an
  interface between Khronos rendering APIs (such as
  OpenGL ES or OpenVG) and the underlying native platform
  window system.
• EGL handles graphics context management,
  surface/buffer binding, and rendering synchronization
  and enables high-performance, accelerated, mixed-
  mode 2D and 3D rendering using other Khronos
  APIs.
• EGL Surfaces
   – windows - on-screen rendering
  – pbuffers - off-screen rendering
  – pixmaps - off-screen rendering
from EGL to SurfaceFlinger




hgl = hardware
hgl = hardware      agl = android software
                    agl = android software
 OpenGL|ES
 OpenGL|ES          OpenGL|ES renderer
                     OpenGL|ES renderer
http://0xlab.org

More Related Content

What's hot

Understaing Android EGL
Understaing Android EGLUnderstaing Android EGL
Understaing Android EGLSuhan Lee
 
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)Nanik Tolaram
 
Native Android Userspace part of the Embedded Android Workshop at Linaro Conn...
Native Android Userspace part of the Embedded Android Workshop at Linaro Conn...Native Android Userspace part of the Embedded Android Workshop at Linaro Conn...
Native Android Userspace part of the Embedded Android Workshop at Linaro Conn...Opersys inc.
 
Using and Customizing the Android Framework / part 4 of Embedded Android Work...
Using and Customizing the Android Framework / part 4 of Embedded Android Work...Using and Customizing the Android Framework / part 4 of Embedded Android Work...
Using and Customizing the Android Framework / part 4 of Embedded Android Work...Opersys inc.
 
Understanding the Android System Server
Understanding the Android System ServerUnderstanding the Android System Server
Understanding the Android System ServerOpersys inc.
 
Android's HIDL: Treble in the HAL
Android's HIDL: Treble in the HALAndroid's HIDL: Treble in the HAL
Android's HIDL: Treble in the HALOpersys inc.
 
Embedded Android Workshop with Pie
Embedded Android Workshop with PieEmbedded Android Workshop with Pie
Embedded Android Workshop with PieOpersys inc.
 
Android Boot Time Optimization
Android Boot Time OptimizationAndroid Boot Time Optimization
Android Boot Time OptimizationKan-Ru Chen
 
Android's Multimedia Framework
Android's Multimedia FrameworkAndroid's Multimedia Framework
Android's Multimedia FrameworkOpersys inc.
 
Android Treble: Blessing or Trouble?
Android Treble: Blessing or Trouble?Android Treble: Blessing or Trouble?
Android Treble: Blessing or Trouble?Opersys inc.
 
Android Binder IPC for Linux
Android Binder IPC for LinuxAndroid Binder IPC for Linux
Android Binder IPC for LinuxYu-Hsin Hung
 
Android internals By Rajesh Khetan
Android internals By Rajesh KhetanAndroid internals By Rajesh Khetan
Android internals By Rajesh KhetanRajesh Khetan
 
Learning AOSP - Android Booting Process
Learning AOSP - Android Booting ProcessLearning AOSP - Android Booting Process
Learning AOSP - Android Booting ProcessNanik Tolaram
 
Tizen 3.0's Window System Integration Layer of OpenGLES/EGL & Vulkan Driver
Tizen 3.0's Window System Integration Layer of OpenGLES/EGL & Vulkan DriverTizen 3.0's Window System Integration Layer of OpenGLES/EGL & Vulkan Driver
Tizen 3.0's Window System Integration Layer of OpenGLES/EGL & Vulkan DriverRyo Jin
 

What's hot (20)

Understaing Android EGL
Understaing Android EGLUnderstaing Android EGL
Understaing Android EGL
 
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
 
Embedded Android : System Development - Part II (HAL)
Embedded Android : System Development - Part II (HAL)Embedded Android : System Development - Part II (HAL)
Embedded Android : System Development - Part II (HAL)
 
Native Android Userspace part of the Embedded Android Workshop at Linaro Conn...
Native Android Userspace part of the Embedded Android Workshop at Linaro Conn...Native Android Userspace part of the Embedded Android Workshop at Linaro Conn...
Native Android Userspace part of the Embedded Android Workshop at Linaro Conn...
 
Porting Android
Porting AndroidPorting Android
Porting Android
 
Using and Customizing the Android Framework / part 4 of Embedded Android Work...
Using and Customizing the Android Framework / part 4 of Embedded Android Work...Using and Customizing the Android Framework / part 4 of Embedded Android Work...
Using and Customizing the Android Framework / part 4 of Embedded Android Work...
 
Android Internals
Android InternalsAndroid Internals
Android Internals
 
Android IPC Mechanism
Android IPC MechanismAndroid IPC Mechanism
Android IPC Mechanism
 
Understanding the Android System Server
Understanding the Android System ServerUnderstanding the Android System Server
Understanding the Android System Server
 
Android's HIDL: Treble in the HAL
Android's HIDL: Treble in the HALAndroid's HIDL: Treble in the HAL
Android's HIDL: Treble in the HAL
 
Embedded Android Workshop with Pie
Embedded Android Workshop with PieEmbedded Android Workshop with Pie
Embedded Android Workshop with Pie
 
Android Boot Time Optimization
Android Boot Time OptimizationAndroid Boot Time Optimization
Android Boot Time Optimization
 
Android's Multimedia Framework
Android's Multimedia FrameworkAndroid's Multimedia Framework
Android's Multimedia Framework
 
Android Treble: Blessing or Trouble?
Android Treble: Blessing or Trouble?Android Treble: Blessing or Trouble?
Android Treble: Blessing or Trouble?
 
Android Binder IPC for Linux
Android Binder IPC for LinuxAndroid Binder IPC for Linux
Android Binder IPC for Linux
 
Android internals By Rajesh Khetan
Android internals By Rajesh KhetanAndroid internals By Rajesh Khetan
Android internals By Rajesh Khetan
 
Introduction to Android Window System
Introduction to Android Window SystemIntroduction to Android Window System
Introduction to Android Window System
 
Learning AOSP - Android Booting Process
Learning AOSP - Android Booting ProcessLearning AOSP - Android Booting Process
Learning AOSP - Android Booting Process
 
Deep Dive into the AOSP
Deep Dive into the AOSPDeep Dive into the AOSP
Deep Dive into the AOSP
 
Tizen 3.0's Window System Integration Layer of OpenGLES/EGL & Vulkan Driver
Tizen 3.0's Window System Integration Layer of OpenGLES/EGL & Vulkan DriverTizen 3.0's Window System Integration Layer of OpenGLES/EGL & Vulkan Driver
Tizen 3.0's Window System Integration Layer of OpenGLES/EGL & Vulkan Driver
 

Similar to Design and Concepts of Android Graphics

Develop Community-based Android Distribution and Upstreaming Experience
Develop Community-based Android Distribution and Upstreaming Experience Develop Community-based Android Distribution and Upstreaming Experience
Develop Community-based Android Distribution and Upstreaming Experience National Cheng Kung University
 
0xdroid -- community-developed Android distribution by 0xlab
0xdroid -- community-developed Android distribution by 0xlab0xdroid -- community-developed Android distribution by 0xlab
0xdroid -- community-developed Android distribution by 0xlabNational Cheng Kung University
 
Inter-process communication of Android
Inter-process communication of AndroidInter-process communication of Android
Inter-process communication of AndroidTetsuyuki Kobayashi
 
Zeelogic android-training-2013
Zeelogic android-training-2013Zeelogic android-training-2013
Zeelogic android-training-2013Zeelogic Solu
 
Android. behind the scenes_programatica 2012
Android. behind the scenes_programatica 2012Android. behind the scenes_programatica 2012
Android. behind the scenes_programatica 2012Agora Group
 
OpenStack and OpenFlow Demos
OpenStack and OpenFlow DemosOpenStack and OpenFlow Demos
OpenStack and OpenFlow DemosBrent Salisbury
 
Build Community Android Distribution and Ensure the Quality
Build Community Android Distribution and Ensure the QualityBuild Community Android Distribution and Ensure the Quality
Build Community Android Distribution and Ensure the QualityNational Cheng Kung University
 
Android containerization in brief
Android containerization in briefAndroid containerization in brief
Android containerization in briefPo-wen Cheng
 
Deep Dive Into Android Security
Deep Dive Into Android SecurityDeep Dive Into Android Security
Deep Dive Into Android SecurityMarakana Inc.
 
VaMoS 2021 - Deep Software Variability: Towards Handling Cross-Layer Configur...
VaMoS 2021 - Deep Software Variability: Towards Handling Cross-Layer Configur...VaMoS 2021 - Deep Software Variability: Towards Handling Cross-Layer Configur...
VaMoS 2021 - Deep Software Variability: Towards Handling Cross-Layer Configur...Luc Lesoil
 

Similar to Design and Concepts of Android Graphics (20)

Improve Android System Component Performance
Improve Android System Component PerformanceImprove Android System Component Performance
Improve Android System Component Performance
 
Android Optimization: Myth and Reality
Android Optimization: Myth and RealityAndroid Optimization: Myth and Reality
Android Optimization: Myth and Reality
 
Android ipm 20110409
Android ipm 20110409Android ipm 20110409
Android ipm 20110409
 
Discover System Facilities inside Your Android Phone
Discover System Facilities inside Your Android Phone Discover System Facilities inside Your Android Phone
Discover System Facilities inside Your Android Phone
 
Develop Community-based Android Distribution and Upstreaming Experience
Develop Community-based Android Distribution and Upstreaming Experience Develop Community-based Android Distribution and Upstreaming Experience
Develop Community-based Android Distribution and Upstreaming Experience
 
Applied Computer Science Concepts in Android
Applied Computer Science Concepts in AndroidApplied Computer Science Concepts in Android
Applied Computer Science Concepts in Android
 
Build Programming Language Runtime with LLVM
Build Programming Language Runtime with LLVMBuild Programming Language Runtime with LLVM
Build Programming Language Runtime with LLVM
 
Practice of Android Reverse Engineering
Practice of Android Reverse EngineeringPractice of Android Reverse Engineering
Practice of Android Reverse Engineering
 
Guides To Analyzing WebKit Performance
Guides To Analyzing WebKit PerformanceGuides To Analyzing WebKit Performance
Guides To Analyzing WebKit Performance
 
Understanding the Dalvik Virtual Machine
Understanding the Dalvik Virtual MachineUnderstanding the Dalvik Virtual Machine
Understanding the Dalvik Virtual Machine
 
0xdroid -- community-developed Android distribution by 0xlab
0xdroid -- community-developed Android distribution by 0xlab0xdroid -- community-developed Android distribution by 0xlab
0xdroid -- community-developed Android distribution by 0xlab
 
Inter-process communication of Android
Inter-process communication of AndroidInter-process communication of Android
Inter-process communication of Android
 
Zeelogic android-training-2013
Zeelogic android-training-2013Zeelogic android-training-2013
Zeelogic android-training-2013
 
Android. behind the scenes_programatica 2012
Android. behind the scenes_programatica 2012Android. behind the scenes_programatica 2012
Android. behind the scenes_programatica 2012
 
OpenStack and OpenFlow Demos
OpenStack and OpenFlow DemosOpenStack and OpenFlow Demos
OpenStack and OpenFlow Demos
 
Build Community Android Distribution and Ensure the Quality
Build Community Android Distribution and Ensure the QualityBuild Community Android Distribution and Ensure the Quality
Build Community Android Distribution and Ensure the Quality
 
Android containerization in brief
Android containerization in briefAndroid containerization in brief
Android containerization in brief
 
ARM
ARMARM
ARM
 
Deep Dive Into Android Security
Deep Dive Into Android SecurityDeep Dive Into Android Security
Deep Dive Into Android Security
 
VaMoS 2021 - Deep Software Variability: Towards Handling Cross-Layer Configur...
VaMoS 2021 - Deep Software Variability: Towards Handling Cross-Layer Configur...VaMoS 2021 - Deep Software Variability: Towards Handling Cross-Layer Configur...
VaMoS 2021 - Deep Software Variability: Towards Handling Cross-Layer Configur...
 

More from National Cheng Kung University

PyPy's approach to construct domain-specific language runtime
PyPy's approach to construct domain-specific language runtimePyPy's approach to construct domain-specific language runtime
PyPy's approach to construct domain-specific language runtimeNational Cheng Kung University
 
進階嵌入式作業系統設計與實做 (2015 年秋季 ) 課程說明
進階嵌入式作業系統設計與實做 (2015 年秋季 ) 課程說明進階嵌入式作業系統設計與實做 (2015 年秋季 ) 課程說明
進階嵌入式作業系統設計與實做 (2015 年秋季 ) 課程說明National Cheng Kung University
 
給自己更好未來的 3 個練習:嵌入式作業系統設計、實做,與移植 (2015 年春季 ) 課程說明
給自己更好未來的 3 個練習:嵌入式作業系統設計、實做,與移植 (2015 年春季 ) 課程說明給自己更好未來的 3 個練習:嵌入式作業系統設計、實做,與移植 (2015 年春季 ) 課程說明
給自己更好未來的 3 個練習:嵌入式作業系統設計、實做,與移植 (2015 年春季 ) 課程說明National Cheng Kung University
 
進階嵌入式系統開發與實做 (2014 年秋季 ) 課程說明
進階嵌入式系統開發與實做 (2014 年秋季 ) 課程說明進階嵌入式系統開發與實做 (2014 年秋季 ) 課程說明
進階嵌入式系統開發與實做 (2014 年秋季 ) 課程說明National Cheng Kung University
 
Develop Your Own Operating Systems using Cheap ARM Boards
Develop Your Own Operating Systems using Cheap ARM BoardsDevelop Your Own Operating Systems using Cheap ARM Boards
Develop Your Own Operating Systems using Cheap ARM BoardsNational Cheng Kung University
 
Lecture notice about Embedded Operating System Design and Implementation
Lecture notice about Embedded Operating System Design and ImplementationLecture notice about Embedded Operating System Design and Implementation
Lecture notice about Embedded Operating System Design and ImplementationNational Cheng Kung University
 
中輟生談教育: 完全用開放原始碼軟體進行 嵌入式系統教學
中輟生談教育: 完全用開放原始碼軟體進行 嵌入式系統教學中輟生談教育: 完全用開放原始碼軟體進行 嵌入式系統教學
中輟生談教育: 完全用開放原始碼軟體進行 嵌入式系統教學National Cheng Kung University
 
F9: A Secure and Efficient Microkernel Built for Deeply Embedded Systems
F9: A Secure and Efficient Microkernel Built for Deeply Embedded SystemsF9: A Secure and Efficient Microkernel Built for Deeply Embedded Systems
F9: A Secure and Efficient Microkernel Built for Deeply Embedded SystemsNational Cheng Kung University
 

More from National Cheng Kung University (20)

PyPy's approach to construct domain-specific language runtime
PyPy's approach to construct domain-specific language runtimePyPy's approach to construct domain-specific language runtime
PyPy's approach to construct domain-specific language runtime
 
Making Linux do Hard Real-time
Making Linux do Hard Real-timeMaking Linux do Hard Real-time
Making Linux do Hard Real-time
 
2016 年春季嵌入式作業系統課程說明
2016 年春季嵌入式作業系統課程說明2016 年春季嵌入式作業系統課程說明
2016 年春季嵌入式作業系統課程說明
 
Interpreter, Compiler, JIT from scratch
Interpreter, Compiler, JIT from scratchInterpreter, Compiler, JIT from scratch
Interpreter, Compiler, JIT from scratch
 
進階嵌入式作業系統設計與實做 (2015 年秋季 ) 課程說明
進階嵌入式作業系統設計與實做 (2015 年秋季 ) 課程說明進階嵌入式作業系統設計與實做 (2015 年秋季 ) 課程說明
進階嵌入式作業系統設計與實做 (2015 年秋季 ) 課程說明
 
Construct an Efficient and Secure Microkernel for IoT
Construct an Efficient and Secure Microkernel for IoTConstruct an Efficient and Secure Microkernel for IoT
Construct an Efficient and Secure Microkernel for IoT
 
The Internals of "Hello World" Program
The Internals of "Hello World" ProgramThe Internals of "Hello World" Program
The Internals of "Hello World" Program
 
How A Compiler Works: GNU Toolchain
How A Compiler Works: GNU ToolchainHow A Compiler Works: GNU Toolchain
How A Compiler Works: GNU Toolchain
 
Virtual Machine Constructions for Dummies
Virtual Machine Constructions for DummiesVirtual Machine Constructions for Dummies
Virtual Machine Constructions for Dummies
 
給自己更好未來的 3 個練習:嵌入式作業系統設計、實做,與移植 (2015 年春季 ) 課程說明
給自己更好未來的 3 個練習:嵌入式作業系統設計、實做,與移植 (2015 年春季 ) 課程說明給自己更好未來的 3 個練習:嵌入式作業系統設計、實做,與移植 (2015 年春季 ) 課程說明
給自己更好未來的 3 個練習:嵌入式作業系統設計、實做,與移植 (2015 年春季 ) 課程說明
 
從線上售票看作業系統設計議題
從線上售票看作業系統設計議題從線上售票看作業系統設計議題
從線上售票看作業系統設計議題
 
進階嵌入式系統開發與實做 (2014 年秋季 ) 課程說明
進階嵌入式系統開發與實做 (2014 年秋季 ) 課程說明進階嵌入式系統開發與實做 (2014 年秋季 ) 課程說明
進階嵌入式系統開發與實做 (2014 年秋季 ) 課程說明
 
Xvisor: embedded and lightweight hypervisor
Xvisor: embedded and lightweight hypervisorXvisor: embedded and lightweight hypervisor
Xvisor: embedded and lightweight hypervisor
 
Making Linux do Hard Real-time
Making Linux do Hard Real-timeMaking Linux do Hard Real-time
Making Linux do Hard Real-time
 
Implement Runtime Environments for HSA using LLVM
Implement Runtime Environments for HSA using LLVMImplement Runtime Environments for HSA using LLVM
Implement Runtime Environments for HSA using LLVM
 
Priority Inversion on Mars
Priority Inversion on MarsPriority Inversion on Mars
Priority Inversion on Mars
 
Develop Your Own Operating Systems using Cheap ARM Boards
Develop Your Own Operating Systems using Cheap ARM BoardsDevelop Your Own Operating Systems using Cheap ARM Boards
Develop Your Own Operating Systems using Cheap ARM Boards
 
Lecture notice about Embedded Operating System Design and Implementation
Lecture notice about Embedded Operating System Design and ImplementationLecture notice about Embedded Operating System Design and Implementation
Lecture notice about Embedded Operating System Design and Implementation
 
中輟生談教育: 完全用開放原始碼軟體進行 嵌入式系統教學
中輟生談教育: 完全用開放原始碼軟體進行 嵌入式系統教學中輟生談教育: 完全用開放原始碼軟體進行 嵌入式系統教學
中輟生談教育: 完全用開放原始碼軟體進行 嵌入式系統教學
 
F9: A Secure and Efficient Microkernel Built for Deeply Embedded Systems
F9: A Secure and Efficient Microkernel Built for Deeply Embedded SystemsF9: A Secure and Efficient Microkernel Built for Deeply Embedded Systems
F9: A Secure and Efficient Microkernel Built for Deeply Embedded Systems
 

Recently uploaded

A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Karmanjay Verma
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Jeffrey Haguewood
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...itnewsafrica
 

Recently uploaded (20)

A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
 

Design and Concepts of Android Graphics

  • 1. Android Graphics Jim Huang ( 黃敬群 ) Developer & Co-Founder, 0xlab jserv@0xlab.org Sep 24, 2011 / Study-Area
  • 2. Rights to copy © Copyright 2011 0xlab http://0xlab.org/ contact@0xlab.org Attribution – ShareAlike 3.0 You are free Corrections, suggestions, contributions and translations are welcome! to copy, distribute, display, and perform the work to make derivative works Latest update: Nov 20, 2011 to make commercial use of the work Under the following conditions Attribution. You must give the original author credit. Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one. For any reuse or distribution, you must make clear to others the license terms of this work. Any of these conditions can be waived if you get permission from the copyright holder. Your fair use and other rights are in no way affected by the above. License text: http://creativecommons.org/licenses/by-sa/3.0/legalcode
  • 3. Agenda (1) Binder IPC (2) Android Graphics (3) 2D and Accelerations (4) OpenGL|ES
  • 4. Notice: Before looking into Android Graphics, you should be aware of the design of Binder IPC, otherwise you would get lost!
  • 6. Processes running on Android $ ps ... root 37 1 248 156 c00aef2c 0000875c S /sbin/ueventd system 42 1 768 260 c022950c afd0b6fc S /system/bin/servicemanager root 43 1 3824 564 ffffffff afd0bdac S /system/bin/vold root 44 1 3796 560 ffffffff afd0bdac S /system/bin/netd root 45 1 628 264 c02588c0 afd0c0cc S /system/bin/debuggerd radio 46 1 4336 672 ffffffff afd0bdac S /system/bin/rild root 47 1 62224 27576 c00aef2c afd0b844 S zygote media 48 1 16828 3736 ffffffff afd0b6fc S /system/bin/mediaserver bluetooth 49 1 1216 572 c00aef2c afd0c59c S /system/bin/dbus-daemon root 50 1 776 316 c02a8424 afd0b45c S /system/bin/installd keystore 51 1 1704 432 c02588c0 afd0c0cc S /system/bin/keystore shell 52 1 696 336 c0050934 afd0c3ac S /system/bin/sh root 53 1 3356 160 ffffffff 00008294 S /sbin/adbd system 67 47 172464 32596 ffffffff afd0b6fc S system_server system 115 47 80028 20728 ffffffff afd0c51c S com.android.systemui app_24 124 47 80732 20720 ffffffff afd0c51c S com.android.inputmethod.latin radio 135 47 87848 20324 ffffffff afd0c51c S com.android.phone app_18 144 47 89136 24160 ffffffff afd0c51c S com.android.launcher app_7 165 47 86136 22736 ffffffff afd0c51c S android.process.acore app_0 197 47 73996 17472 ffffffff afd0c51c S com.android.deskclock app_14 208 47 75000 18464 ffffffff afd0c51c S android.process.media app_3 219 47 72228 17652 ffffffff afd0c51c S com.android.bluetooth app_25 234 47 85336 17836 ffffffff afd0c51c S com.android.mms app_26 254 47 74656 19080 ffffffff afd0c51c S com.android.email app_27 266 47 74912 18100 ffffffff afd0c51c S com.android.providers.calendar app_1 285 47 71616 16280 ffffffff afd0c51c S com.android.protips app_19 293 47 72184 16572 ffffffff afd0c51c S com.android.music app_21 301 47 74728 17208 ffffffff afd0c51c S com.android.quicksearchbox app_28 311 47 75408 18040 ffffffff afd0c51c S com.cooliris.media shell 323 52 856 316 00000000 afd0b45c R ps $ More than 30 processes (200+ threads). 6
  • 7. IPC = Inter-Process Communication Activity Activity Window Alarm Manager Manager Manager Kernel 7
  • 8. IPC Abstraction More abstract Intent • Intent – The highest level abstraction AIDL • Inter process method invocation – AIDL: Android Interface Binder Definition Language • binder: kernel driver • ashmem: shared memory 8
  • 9. Method invocation caller callee In the same process 9
  • 10. Inter-process method invocation caller interface caller interface How? callee interface callee 10
  • 11. Inter-process method invocation caller interface Proxy caller interface Binder in kernel Binder Thread callee Stub interface callee 11
  • 12. android.os.Parcel Delivering arguments of method ”flatten” ”unflatten” transmit 12
  • 13. UML Representation <<interface>> implements Proxy Stub 13
  • 14. UML Representation caller <<interface>> calls implements Proxy Stub extends callee 14
  • 15. AIDL Auto generated from .aidl file caller <<interface>> Proxy Stub callee 15
  • 16. Use Case: Who calls onPause() in Activity? 2 Send message by Handler Activity 3 OnPause() is queue Looper called in main thread Activity Manager Main Binder Thread Thread #1 Kernel 1 Call ”schedulePauseActivity” across process 16
  • 17. Binder • Multi-thread aware – Have internal status per thead – Compare to UNIX socket: sockets have internal status per file descriptor (FD) 17
  • 18. Binder  A pool of threads is associated to each service application to process incoming IPC (Inter-Process Communication).  Binder performs mapping of object between two processes.  Binder uses an object reference as an address in a process’s memory space.  Synchronous call, reference couting
  • 20. Binder is different from UNIX socket socket binder internal status associated to FD associated to PID (FD can be shared among threads in the same process) read & write stream I/O done at once by operation ioctl network Yes No transparency expected local only 20
  • 21. Transaction of Binder binder_write_read write_size write buffer write_consumed write_buffer read_size read_consumed read buffer read_buffer if (ioctl(fd, BINDER_WRITE_READ, &bwt ) >= 0) err = NO_ERROR; else err = -errno; 21
  • 22. Transaction of Binder Process A and B have different memory space. They can not see each other. Kernel Process B Process A Binder Copy memory by copy_from _user Then, wake up process B Kernel Process B Binder Process A Copy memory by copy_to_user Internally, Android uses Binder for graphics data transaction across processes. 22 It is fairly efficient.
  • 23. Real Case Binder IPC is used for communicating between Graphics client and server. 23 Taken from http://www.cnblogs.com/xl19862005/archive/2011/11/17/2215363.html
  • 24. Ashmem • Android / Anonymous SHared MEMory subsystem – system/core/cutils/ashmem.h – int ashmem_create_region(const char *name, size_t size) – int ashmem_set_prot_region(int fd, int prot) – int ashmem_pin_region(int fd, size_t offset, size_t len) – int ashmem_unpin_region(int fd, size_t offset, size_t len) • a named memory block shared between processes that the kernel is allowed to free. – This is notable as the kernel is not allowed to free standard shared memory. • Similar to weak reference of Java. Useful to implement cache. • Used in android.os.MemoryFile (Java), 2D memory allocator, etc. 24
  • 26. Surface Source: frameworks/base/core/java/android/view/Surface.java • /* Handle on to a raw buffer that is being managed by the screen compositor */ public class Surface implements Parcelable { public Surface() { mCanvas = new CompatibleCanvas(); } private class CompatibleCanvas extends Canvas { /* ... */ } } Surface instances can be written to and restored from a Parcel. Surface instances can be written to and restored from a Parcel.
  • 27. Delivering arguments of method ”flatten” ”unflatten” transmit
  • 28. Properties Android SurfaceFlinger  Can combine 2D/3D surfaces and surfaces from multiple applications  Surfaces passed as buffers via Binder IPC calls  Can use OpenGL ES and 2D hardware accelerator for its compositions  Double-buffering using page-flip
  • 29.
  • 30. from SurfaceFlinger to Framebuffer
  • 31.
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
  • 37. PixelFlinger : software renderer • Render functions: pointx, linex, recti, trianglex • Texture and color buffer: activeTexture, bindTexture, colorBuffer, readBuffer, depthBuffer, BindTextureLod • … • Device framebuffer functions: copyPixels, rasterPos2x, rasterPos2i • Optimizer: codeflinger (JIT assembler) I/SurfaceFlinger( 1931): OpenGL informations: I/SurfaceFlinger( 1931): vendor    : Android I/SurfaceFlinger( 1931): renderer : Android PixelFlinger 1.2 I/SurfaceFlinger( 1931): version   : OpenGL ES­CM 1.0
  • 38. GGLAssembler – GGLAssembler – codeflinger's JIT Assembler codeflinger's JIT Assembler
  • 40. Skia • SkBitmap • SkBitmap::Allocator • GPU integration in Android 4.0
  • 41. Case Study: WebKit rendering Fill rectangle FillRectangles() Set clip region Acquire font cache Show glyphs Blit(), memcpy() BatchBlit() Allocate font cache Surface composite Blit() Prepare / finish Clear() TileBlit() composite
  • 42. 2D Compositing SRC SRC & DST Compositing 1 -asrc SRC & NO-DST DST NO-DST & NO-SRC 1-adst adst
  • 44. Android Graphics (HAL view) Android Framework (Java) libandroid_runtime Surfaceflinger Copybit EventHub (service) (HW accelerated) LibGLES libui (libagl) libpixelflinger
  • 45. 2D Accelerator for Android Graphics • libcopybit provides hareware bitblit operations which includes moving, scaling, rotation, mirroring, and more effects, like blending, dithering, bluring, etc. • Removed since Android 2.3 • Android has two copybit interfaces: – Blit: moving / blending – Stretch: scaling besides moving • libcopybit is called by libagl which can do swapBuffers to do the framebuffer page flipping that can also be accelerated by libcopybit. Copybit could improve the performance of page flipping Copybit could improve the performance of page flipping
  • 46. Copybit operations Copybit: 2D blitter Copybit: 2D blitter
  • 47. PMEM: Manipulate physically continuous memory • Hardware graphics/bitblt operations (blitter) needs physical continuous memory to manipulate. • Android use libgralloc to allocate pmem (physical continuous memory) for android native buffer. pmem driver can support up to 12 devices, we have only one for copybit (Android, android native buffer) • While running 0xbench, the peak size of the pmem allocated (mapped) is 25194496 bytes. Take Qualcomm MSM7x25 for example: Take Qualcomm MSM7x25 for example: • /dev/pmem • /dev/pmem • /dev/pmem_adsp • /dev/pmem_adsp – For multimedia codec, audio, video, camera – For multimedia codec, audio, video, camera
  • 48. gralloc & pmem gralloc_alloc Offset = Alloc(size), PMEM_CONNECT, PMEM_MAP, memset If success, new private_handle_t(fd, size, flags); gralloc module gralloc_free PMEM_UNMAP, deallocate(offset)
  • 49. pmem memory (mmaped virtual address)base master_fd (mmap size: 47MB) 0x49422336, c06c4000, 6c4000 Buffer 1 buffer base = base + fd when being freed, this offset will be closed Buffer 2 Buffer 3 Quantum Debug pmem, pmem /sys/kernel/pmem_regions pmem can have up to 12 devices..., we have only one for copybit (Android native buffer)
  • 50. gralloc/copybit structure android_native_buffer_t buffer_handle_t handle; native_handle_t copybit_image_t native_handle_t* handle; buffer_handle_t libgralloc private_handle_t Fd (pmem dev) Android native buffer contains Offset buffer_handle_t Size Which is native_handle * , Base defined in system/cutils/ Flags virtual address lockState private_handle_t::PRIV_FLAGS_U SES_PMEM private_handle_t::LOCK_STATE_MAPPED
  • 51. Functions in copybit HAL • BLIT (moving) • Stretch (scaling) • Alpha-blending – Unused in real case • Rotate – Unused in real case
  • 53. Key Concepts • Android is moving to OpenGL|ES accelerated rendering since version 2.x • Window systems already comprehend z-order • 3D != 2D with depth • OpenGL is object based – Describes a scene using its components and properties. Output quality is dependent on the renderer, not source
  • 54. OpenGL Terminology • OpenGL An API from Khronos (from SGI), for constructing a 3D object, doing operations on the object, and displaying it • Primitives Triangles, Lines, Points, that can be specified through vertices to define an arbitrary shape • Texture Small (!) bitmap to make objects more realistic • EGL The EGL API defines a portable mechanism for creating GL contexts and windows for rendering into, which may be used in conjunction with different native platform window systems using the WSEGL layer
  • 55. EGL • EGL (Embedded-System Graphics Library) is an interface between Khronos rendering APIs (such as OpenGL ES or OpenVG) and the underlying native platform window system. • EGL handles graphics context management, surface/buffer binding, and rendering synchronization and enables high-performance, accelerated, mixed- mode 2D and 3D rendering using other Khronos APIs. • EGL Surfaces – windows - on-screen rendering – pbuffers - off-screen rendering – pixmaps - off-screen rendering
  • 56. from EGL to SurfaceFlinger hgl = hardware hgl = hardware agl = android software agl = android software OpenGL|ES OpenGL|ES OpenGL|ES renderer OpenGL|ES renderer
  • 57.