The Android Build System
Introduction
Ron Munitz - ron@thepscg.com
Updated: November 12th 2015
Updated: June 22nd 2015
Android Marshmallow / master post android-6.0.0_r26
PSCG
The Android Build System
â—Ź Host Tools
â—Ź Target Tools
â—Ź Platform tools
â—Ź Documentation Tools
● The Target’s (Android Platform) code base
â—Ź Build System
PSCG
The Android Build System
â—Ź Build Systems are a huge topic.
â—Ź A build system
â—‹ Takes a series of rules or recipes
â—‹ Knows how to generate an embedded platform
images/artifacts from source code, configuration
files, BLOBs, ...
â—‹ Allows to easily select a premade configuration and
build a ready to use code without working too hard
â—‹ Allows customizing products and defining new onesPSCG
The Android Build System
â—Ź Consists of two essential folders:
â—‹ build/ Contains the definitions of the build system,
along with some predefined devices.
â—‹ device/ Contains definitions for devices. The build
system rules really parse them
An optional, out of source control folder may be
added for vendor specific additions (mainly BLOBs).
It is listed as vendor/ PSCG
build/
â—Ź buildspec.mk.default - template for remake
â—Ź CleanSpec.mk - Build cleanup definitions
â—Ź core - Build System rules
â—Ź envsetup.sh - environment preparation script
â—Ź libs - some host helper libs
â—Ź target - Target definitions.
â—Ź tools - Building, packaging, etc.
● soong, blueprint, kati ⇒ Ninja build systemPSCG
The Android Build System
â—Ź Main folder: build/
â—Ź Based on GNU make / Ninja
â—‹ Makefile on top directory says: #include
build/core/main.mk
â—‹ Then a lot of other files are included.
â—‹ Heavily uses Python and bash.
â—Ź Heavily uses environment variables
â—‹ @see build/envsetup.sh
PSCG
The Android Build System
â—Ź Very easy to use for building:
â—‹ . build/envsetup.sh #sets env vars/functions
â—‹ lunch <config>-<variant> # selects configuration
○ make # That’s gnu make, no customizations**.
â—Ź Very easy for flashing via Software tools
○ With emulator (no need to flash…)
â—‹ With fastboot
â—‹ With other custom bootloaders
** Not true for master as per today. See next.PSCG
**Introducing Ninja
build/
â—Ź blueprint
â—Ź kati
â—Ź soong
The ninja build system supposedly
reduces incremental build times, and simplifies
the “Makefile” configuration.
build/target
â—Ź This is what you should probably care about.
â—Ź Contains two folders:
â—‹ board/ - board definition file
â—‹ product/ - product definition file
â—Ź Build recipes are defined in these folders.
â—Ź Device definitions in device/ include files
from these folders, and derive from recipes.
â—Ź Or include their own.
PSCG
Where (some) board magic is defined
build/target/board
PSCG
build/target/board
â—Ź Android.mk - automatically included by build
â—‹ includes the no longer necessary AndroidBoard.
mk file at TARGET_DEVICE_DIR
â–  What is required is BoardConfig.mk .
â–  @see build/core/config.mk
â—‹ Populates the fastboot read android-info.txt file with
the contents of the devices board.info.txt
â–  Or with "board=$(TARGET_BOOTLOADER_BOARD_NAME)"
â—Ź Templates for predefined boards
â—‹ @see next slide PSCG
build/target/board (cont.)
â—Ź Contains build templates for predefined
boards
â—‹ emulator, generic_<arch>, vbox_x86
â—Ź Each contains:
â—‹ AndroidBoard.mk (obsolete)
â—‹ BoardConfig.mk - Board definitions (see next slide)
â—‹ device.mk - Some Board/Hardware related
pacakges at device.mk and BoardConfig.mk
PSCG
A bit about BoardConfig.mk
â—Ź TARGET_NO_BOOTLOADER - Self Explaining
â—Ź TARGET_NO_KERNEL - use prebuilt kernel
â—Ź TARGET_USE_CAMERA_STUB
â—Ź ...
â—Ź Architecture, ABIs, Partition layout, OpenGL
config, Radio config…
PSCG
BoardConfig.mk search path
â—Ź The build system searches for BoardConfig.
mk at the following locations:
â—‹ build/target/board/$TARGET_DEVICE/
â—‹ device/*/$TARGET_DEVICE/
â—‹ vendor/*/$TARGET_DEVICE/
â—Ź If there is no such file - the build fails.
â—Ź If there is more than one match - the build
fails.
PSCG
Where (product packages) dreams come true
build/target/product
PSCG
build/target/product
â—Ź AndroidProducts.mk defines a list of products to add
to the build system.
â—Ź More products can be added by declaring additional
AndroidProducts.mk files in either
â—‹ device/.../
â—‹ vendor/.../
â—Ź security/ includes prebuilt certificates
â—Ź Quick Start flow:
â—‹ generic_mips.mk INHERITS generic.mk INHERITS
generic_no_telephony.mk AND telephony.mk ...
PSCG
build/target/product example
â—Ź aosp_x86.mk
INHERITS
â—‹ full_x86.mk
â–  INHERITS
â—Ź aosp_base_telephony.mk
â—Ź board/generic_x86/device.mk
â–  INCLUDES
â—Ź $(SRC_TARGET_DIR)/product/emulator.mk
â—Ź Another example + a full walkthrough are
given in the next slides PSCG
product/full.mk
product/aosp_base_telephony.mk
PRODUCT_PACKAGES += 
gralloc.goldfish 
qemud 
libGLES_android 
...
PRODUCT_COPY_FILES +=
device/generic/goldfish/
init.goldfish.rc:root/init.goldfish.rc 
...
product/full_base_telephony.mk
product/aosp_arm.mk
(similiar for aosp_x86.mk and aosp_mips.mk)
PRODUCT_NAME := aosp_arm
board/generic/device.mk
PRODUCT_PROPERTY_OVERRIDES := 
ro.ril.hspxa=1
ro.ril.gprsclass=10
ro.adbd.qemu=1
PRDOUCT_COPY_FILES := 
apns_conf.xml
vold.conf
media_profiles.xml
media_codecs.xml
audio_policy.xml
product/emulator.mk
product/full.mk
product/aosp_base_telephony.mk
product/full_base_telephony.mk
product/aosp_arm.mk
board/generic/device.mk product/emulator.mk
PRODUCT_PACKAGES := 
Dialer 
Mms 
rild
product/telephony.mk
product/aosp_base.mk
product/full_base.mk
PRODUCT_PACKAGES
PRODUCT_PROPERTY_OVERRIDES
PRODUCT_LOCALES
product/locales_full.mk
product/languages_full.mk
PRODUCT_LOCALES
frameworks/base/data/
sounds/AllAudio.mk
…
external/svox/pico/lang/
all_pico_languages.mk
…
product/generic_no_telephony.
mk
PRODUCT_LOCALES
PRODUCT_PROPERTY_OVERRIDES := 
keyguard.no_require_sim=true 
ro.com.android.dataroaming=true
PRODUCT_COPY_FILES := 
device/generic/goldfish/data/etc/apns-conf.xml:...
frameworks/native/data/etc/handheld_core_hardware.xml:...
product/full_base_telephony.mk
product/telephony.mk
product/aosp_base.mk
product/full_base.mk
locales_full.mk
languages_full.mk
AllAudio.mk all_pico_languages.mk
product/generic_no_telephony.mk
.../fonts.mk
# from frameworks/base/data
# from external
.../keyboards.mk
# idc, kcm, kl, ...
chromium.mk product/core.mk
core_base.mk
core_minimal.mk
base.mk
embedded.mk
PRODUCT_PACKAGES += 
app_process pm am wm ...
PRODUCT_BOOT_JARS +=
PRODUCT_RUNTIMES +=
PRODUCT_SYSTEM_SERVER_JARS +=
PRODUCT_DEFAULT_PROPERTY_OVE
RRIDES +=
PRODUCT_PACKAGES
+=...
PRODUCT_COPY_FILE
S += ...
...
PRODUCT_PROPERTY
_OVERRIDES += ...
PRODUCT_DEFAULT_
PROPERTY_OVERRID
ES += ...
…..
runtime_libart.mk
PRODUCT_PACKAGES += … sepolicy  file_contexts
seapp_contexts property_contexts mac_permissions.xml
selinux_version service_contexts 
init ... libc … adbd … toybox
PRODUCT_COPY_FILES +=...
PRODUCT_DEFAULT_PROPERTY_OVERRIDES += ...
…..
PRODUCT_PACKAGES += 
apache-xml bouncycastle cacerts conscrypt core-junit core-
libart
dalvikvm dex2oat dexdeps dexdump dexlist dmtracedump dx
ext hprof-conv libart libcrypto libexpat libicui18n libicuuc
libjavacore libnativehelper libssl libz oatdump okhttp patchoat
PRODUCT_DEFAULT_PROPERTY_OVERRIDES += 
dalvik.vm.image-dex2oat-Xms=64m 
dalvik.vm.image-dex2oat-Xmx=64m 
dalvik.vm.dex2oat-Xms=64m 
dalvik.vm.dex2oat-Xmx=512m 
ro.dalvik.vm.native.bridge=0
build/core/build-system.html
The definitive build system document!
Most recent news!
Draft version
From 2006...
Does give a good overview of the design
criteria.
PSCG
The nails and hammers of the printed, framed
dream
build/core
PSCG
build/core
â—Ź Processes the definitions we have discussed
of the product and board
â—Ź defines the language for them
â—Ź Also applies a lot of other configuration,
compiler rules, and what not...
â—Ź @see config.mk ,main.mk, product.mk for
info
PSCG
Thank You
Training and Consulting requests:
ron@thepscg.com

The Android Build System - Android Marshmallow

  • 1.
    The Android BuildSystem Introduction Ron Munitz - ron@thepscg.com Updated: November 12th 2015 Updated: June 22nd 2015 Android Marshmallow / master post android-6.0.0_r26 PSCG
  • 2.
    The Android BuildSystem ● Host Tools ● Target Tools ● Platform tools ● Documentation Tools ● The Target’s (Android Platform) code base ● Build System PSCG
  • 3.
    The Android BuildSystem â—Ź Build Systems are a huge topic. â—Ź A build system â—‹ Takes a series of rules or recipes â—‹ Knows how to generate an embedded platform images/artifacts from source code, configuration files, BLOBs, ... â—‹ Allows to easily select a premade configuration and build a ready to use code without working too hard â—‹ Allows customizing products and defining new onesPSCG
  • 4.
    The Android BuildSystem â—Ź Consists of two essential folders: â—‹ build/ Contains the definitions of the build system, along with some predefined devices. â—‹ device/ Contains definitions for devices. The build system rules really parse them An optional, out of source control folder may be added for vendor specific additions (mainly BLOBs). It is listed as vendor/ PSCG
  • 5.
    build/ ● buildspec.mk.default -template for remake ● CleanSpec.mk - Build cleanup definitions ● core - Build System rules ● envsetup.sh - environment preparation script ● libs - some host helper libs ● target - Target definitions. ● tools - Building, packaging, etc. ● soong, blueprint, kati ⇒ Ninja build systemPSCG
  • 6.
    The Android BuildSystem â—Ź Main folder: build/ â—Ź Based on GNU make / Ninja â—‹ Makefile on top directory says: #include build/core/main.mk â—‹ Then a lot of other files are included. â—‹ Heavily uses Python and bash. â—Ź Heavily uses environment variables â—‹ @see build/envsetup.sh PSCG
  • 7.
    The Android BuildSystem ● Very easy to use for building: ○ . build/envsetup.sh #sets env vars/functions ○ lunch <config>-<variant> # selects configuration ○ make # That’s gnu make, no customizations**. ● Very easy for flashing via Software tools ○ With emulator (no need to flash…) ○ With fastboot ○ With other custom bootloaders ** Not true for master as per today. See next.PSCG
  • 8.
    **Introducing Ninja build/ ● blueprint ●kati ● soong The ninja build system supposedly reduces incremental build times, and simplifies the “Makefile” configuration.
  • 9.
    build/target â—Ź This iswhat you should probably care about. â—Ź Contains two folders: â—‹ board/ - board definition file â—‹ product/ - product definition file â—Ź Build recipes are defined in these folders. â—Ź Device definitions in device/ include files from these folders, and derive from recipes. â—Ź Or include their own. PSCG
  • 10.
    Where (some) boardmagic is defined build/target/board PSCG
  • 11.
    build/target/board â—Ź Android.mk -automatically included by build â—‹ includes the no longer necessary AndroidBoard. mk file at TARGET_DEVICE_DIR â–  What is required is BoardConfig.mk . â–  @see build/core/config.mk â—‹ Populates the fastboot read android-info.txt file with the contents of the devices board.info.txt â–  Or with "board=$(TARGET_BOOTLOADER_BOARD_NAME)" â—Ź Templates for predefined boards â—‹ @see next slide PSCG
  • 12.
    build/target/board (cont.) â—Ź Containsbuild templates for predefined boards â—‹ emulator, generic_<arch>, vbox_x86 â—Ź Each contains: â—‹ AndroidBoard.mk (obsolete) â—‹ BoardConfig.mk - Board definitions (see next slide) â—‹ device.mk - Some Board/Hardware related pacakges at device.mk and BoardConfig.mk PSCG
  • 13.
    A bit aboutBoardConfig.mk ● TARGET_NO_BOOTLOADER - Self Explaining ● TARGET_NO_KERNEL - use prebuilt kernel ● TARGET_USE_CAMERA_STUB ● ... ● Architecture, ABIs, Partition layout, OpenGL config, Radio config… PSCG
  • 14.
    BoardConfig.mk search path â—ŹThe build system searches for BoardConfig. mk at the following locations: â—‹ build/target/board/$TARGET_DEVICE/ â—‹ device/*/$TARGET_DEVICE/ â—‹ vendor/*/$TARGET_DEVICE/ â—Ź If there is no such file - the build fails. â—Ź If there is more than one match - the build fails. PSCG
  • 15.
    Where (product packages)dreams come true build/target/product PSCG
  • 16.
    build/target/product â—Ź AndroidProducts.mk definesa list of products to add to the build system. â—Ź More products can be added by declaring additional AndroidProducts.mk files in either â—‹ device/.../ â—‹ vendor/.../ â—Ź security/ includes prebuilt certificates â—Ź Quick Start flow: â—‹ generic_mips.mk INHERITS generic.mk INHERITS generic_no_telephony.mk AND telephony.mk ... PSCG
  • 17.
    build/target/product example â—Ź aosp_x86.mk INHERITS â—‹full_x86.mk â–  INHERITS â—Ź aosp_base_telephony.mk â—Ź board/generic_x86/device.mk â–  INCLUDES â—Ź $(SRC_TARGET_DIR)/product/emulator.mk â—Ź Another example + a full walkthrough are given in the next slides PSCG
  • 18.
    product/full.mk product/aosp_base_telephony.mk PRODUCT_PACKAGES += gralloc.goldfish qemud libGLES_android ... PRODUCT_COPY_FILES += device/generic/goldfish/ init.goldfish.rc:root/init.goldfish.rc ... product/full_base_telephony.mk product/aosp_arm.mk (similiar for aosp_x86.mk and aosp_mips.mk) PRODUCT_NAME := aosp_arm board/generic/device.mk PRODUCT_PROPERTY_OVERRIDES := ro.ril.hspxa=1 ro.ril.gprsclass=10 ro.adbd.qemu=1 PRDOUCT_COPY_FILES := apns_conf.xml vold.conf media_profiles.xml media_codecs.xml audio_policy.xml product/emulator.mk
  • 19.
    product/full.mk product/aosp_base_telephony.mk product/full_base_telephony.mk product/aosp_arm.mk board/generic/device.mk product/emulator.mk PRODUCT_PACKAGES := Dialer Mms rild product/telephony.mk product/aosp_base.mk product/full_base.mk PRODUCT_PACKAGES PRODUCT_PROPERTY_OVERRIDES PRODUCT_LOCALES product/locales_full.mk product/languages_full.mk PRODUCT_LOCALES frameworks/base/data/ sounds/AllAudio.mk … external/svox/pico/lang/ all_pico_languages.mk … product/generic_no_telephony. mk PRODUCT_LOCALES PRODUCT_PROPERTY_OVERRIDES := keyguard.no_require_sim=true ro.com.android.dataroaming=true PRODUCT_COPY_FILES := device/generic/goldfish/data/etc/apns-conf.xml:... frameworks/native/data/etc/handheld_core_hardware.xml:...
  • 20.
    product/full_base_telephony.mk product/telephony.mk product/aosp_base.mk product/full_base.mk locales_full.mk languages_full.mk AllAudio.mk all_pico_languages.mk product/generic_no_telephony.mk .../fonts.mk # fromframeworks/base/data # from external .../keyboards.mk # idc, kcm, kl, ... chromium.mk product/core.mk core_base.mk core_minimal.mk base.mk embedded.mk PRODUCT_PACKAGES += app_process pm am wm ... PRODUCT_BOOT_JARS += PRODUCT_RUNTIMES += PRODUCT_SYSTEM_SERVER_JARS += PRODUCT_DEFAULT_PROPERTY_OVE RRIDES += PRODUCT_PACKAGES +=... PRODUCT_COPY_FILE S += ... ... PRODUCT_PROPERTY _OVERRIDES += ... PRODUCT_DEFAULT_ PROPERTY_OVERRID ES += ... ….. runtime_libart.mk PRODUCT_PACKAGES += … sepolicy file_contexts seapp_contexts property_contexts mac_permissions.xml selinux_version service_contexts init ... libc … adbd … toybox PRODUCT_COPY_FILES +=... PRODUCT_DEFAULT_PROPERTY_OVERRIDES += ... ….. PRODUCT_PACKAGES += apache-xml bouncycastle cacerts conscrypt core-junit core- libart dalvikvm dex2oat dexdeps dexdump dexlist dmtracedump dx ext hprof-conv libart libcrypto libexpat libicui18n libicuuc libjavacore libnativehelper libssl libz oatdump okhttp patchoat PRODUCT_DEFAULT_PROPERTY_OVERRIDES += dalvik.vm.image-dex2oat-Xms=64m dalvik.vm.image-dex2oat-Xmx=64m dalvik.vm.dex2oat-Xms=64m dalvik.vm.dex2oat-Xmx=512m ro.dalvik.vm.native.bridge=0
  • 21.
    build/core/build-system.html The definitive buildsystem document! Most recent news! Draft version From 2006... Does give a good overview of the design criteria. PSCG
  • 22.
    The nails andhammers of the printed, framed dream build/core PSCG
  • 23.
    build/core â—Ź Processes thedefinitions we have discussed of the product and board â—Ź defines the language for them â—Ź Also applies a lot of other configuration, compiler rules, and what not... â—Ź @see config.mk ,main.mk, product.mk for info PSCG
  • 24.
    Thank You Training andConsulting requests: ron@thepscg.com