LCE13: Introduction to OpenEmbedded/Yocto Cross-Compile Environment
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

LCE13: Introduction to OpenEmbedded/Yocto Cross-Compile Environment

  • 2,875 views
Uploaded on

Resource: LCE13 ...

Resource: LCE13
Name: Introduction to OpenEmbedded/Yocto Cross-Compile Environment
Date: 10-07-2013
Speaker: Nicolas Dechesne, Riku Voipio, Trevor Woerner
Video: https://www.youtube.com/watch?v=pgYuIzoXDEk

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,875
On Slideshare
2,875
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
58
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. OpenEmbedded and Yocto introduction Nicolas Dechesne <nicolas.dechesne@linaro.org> Riku Voipio <riku.voipio@linaro.org> Trevor Woerner <trevor.woerner@linaro.org> LCE13, Dublin
  • 2. 2 Agenda ● Intro to OpenEmbedded / Yocto ● Overview of Yocto Project, organization and releases ● Yocto Project vs. DIY ● Introduction to OpenEmbedded concepts (recipes, packages, images, layers, configuration, SDK, …) ● OpenEmbedded/Yocto and Linaro ● 'OpenEmbedded/Yocto Lab' (this afternoon, 3PM) ● Beginners practical introduction ● Guide you through a 'qemuarm' console image build ● Q&A
  • 3. 3 What is the Yocto Project? It provides all the basic bits that every Linux product needs, pre-integrated and tested to let you spend more time on differentiators that matter to you!
  • 4. 4 The Yocto Project ● Open source project, hosted by the Linux Foundation. ● Not the “new kid on the block”, OpenEmbedded has been around since 2003! ● A collaboration project, an “umbrella” project. A collection of projects that make up the “Yocto Project” ● Bitbake ● OpenEmbedded-Core ● Poky ● Application Development Toolkit ● And even more... ● It helps you build a 'tailored' Linux distribution/product for your exact needs. ● Support for ARM, PPC, MIPS, x86
  • 5. 5 Yocto Project organization ● Hierarchical technical leadership: ● Richard Purdie as the project architect ● Sub system maintainers ● Somehow similar to Linux kernel technical leadership ● Yocto Project Advisory Board for administrative leadership ● community, finance, infrastructure, advocacy ● From linaro: TI, Enea, Huawei ● Open to all contributors ● Yocto Compliance program ● Yocto project participants ● Yocto project compatible
  • 6. 6 Yocto/OE releases ● 1 new release ~6 months ● Yocto Project releases in sync with OE, bitbake projects releases ● Yocto Project release is the aggregation of oe-core, bitbake and Poky layers/tools/documentation. ● Stable releases ● v1.4 (aka dylan), v1.3 (aka danny) ● No new feature, only bug fixes ● Development / master branch ● New features (systemd, wayland, Qt5, build engine improvements, ...) ● Components version updates ● Does not 'work' every single day
  • 7. 7 Yocto/OpenEmbedded vs DIY ● Spend less time on things that bring no value to your business: ● OE provides and maintains thousands of up-to-date recipes ● Build an entire Linux system from sources in less than 1 hour, using a validated set of packages (toolchain, busybox, libc, init system...) ● Built in support for package management ● Predictable and reproducible builds ● Provides set of standard tools and build guidelines. ● What you build for a product can be reused for other products. ● autobuilder 'bot' for continuous integration ● Minimal dependencies on host, build as standard user (no root) ● Still, you have all the flexibility you need to ● patch as needed any OE component for your needs/product ● Add any unsupported component, even proprietary bits ● Fully configure the system, and make it exactly the way you want.
  • 8. 8 OpenEmbedded metadata ● OpenEmbedded does not contain components source code, only their metadata ● Bitbake is the build engine, a metadata parser, written in Python ● Recipes ● Component (.bb, .bbappend) ● Image (.bb) ● Class (.bbclass) ● Configuration files: behavior largely controlled by variables ● bitbake.conf ● <machine>.conf ● <distro>.conf ● local.conf, site.conf ● bblayers.conf ● Patches
  • 9. 9 OpenEmbedded overview
  • 10. 10 OpenEmbedded packages ● A recipe generates 1 or more binary packages ● Each recipe build is split into 'tasks' ● Tasks are scheduled by bitbake based on parallelism and dependencies ● fetch, unpack, patch, configure, build, install, package ● All packages are built from scratch with same compiler configuration (CPU, optimizations, tuning, …) according to the underlying target CPU. ● Standard packages are created managed automatically (most of the time) ● Binaries are stripped automatically, and debug binaries are placed in -dbg packages ● Development packages with .h , .so, .a, .pc, … ● ... ● Packages dependencies ● build depends (DEPENDS) are mandatory and explicit in recipes ● Run time depends (RDEPENDS) are semi-automatic (shlibdeps, pcdeps)
  • 11. 11 OpenEmbedded images ● Images are standard recipe (e.g. .bb file), with a different 'task set' ● Images are expected to be 'deployed' on target ● Most standard image format are supported (ext2/3, tarball, jffs2, cramfs, ramdisk, …) ● Support for image generation 'post scripting' ● Packages are assembled into the final root file system to make up images ● Image recipe lists all components/applications required in the image ● RDEPENDS used to pull all the needed libraries, ... ● Multiple images can be built from the same 'build', by assembling different set of packages ● OpenEmbedded has (optional) support for package management in the target image: .rpm, .ipk, .deb. ● Can be used to implement a binary package based distro (with package feed) ● However OE always uses packages 'internally' to build images.
  • 12. 12 OpenEmbedded layers ● Couple of years ago, OpenEmbedded used to be a monolithic repo with thousands of recipes, and became impractical to maintain and use ● Layer concept introduced ● Layers contain additional metadata (recipes, machine, distro, …) ● Oe-core is the base layer for the most fundamental components ● layers provide additional features: meta-systemd, meta-gnome, … ● Layers can be inter-dependent ● Layers can be hosted in different 'trees' (git) ● All layers must work with oe-core ● Layers can add, modify content/beahvior, and/or alter, extend metadata from other layers. ● Generally, each BSP or distro has its own layer(s) ● meta-linaro, https://git.linaro.org/gitweb?p=openembedded/meta-linaro.git ● OpenEmbedded metadata on line index: http://layers.openembedded.org/
  • 13. 13 Poky ● Poky is a reference distro implementation. ● Minimal console only image ● Full Linux system with GUI using “sato” ● Generally used as a starting point (or an example) for building one's own (commercial) distro: Angstrom, ELDK, Mentor Linux, Wind River Linux, ENEA Linux, ... ● “it is often just a starting point, not an end goal” ● Built with bitbake and these layers: oe-core, meta-yocto, meta-yocto-bsp (and yes, the name is confusing!) ● Used to test/validate OpenEmbedded by Yocto Project developers ● “Poky-tiny” is a variant that provides size optimized image. ● “Poky-bleeding” is a variant that uses bleeding edge software version
  • 14. 14 SDK ● Application Development Toolkit (ADT) provides custom built, cross development SDK tailored for the image being built ● Cross toolchain ● Matching sysroot ● Eclipse IDE Yocto plugin ● Support SDK for x86 and/or x86-64 hosts ● ADT can be installed on 'OEM', customer hosts without the need to rebuild the entire image packages ● “bitbake meta-toolchain” : includes only the minimal cross toolchain and libs ● “bitbake -c populate_sdk <image>” : includes all the development headers, libs, … for every package included in the image. e.g. build SDK that matches the target image
  • 15. 15 Anatomy of a build folder ● Lots of space … ~20 Gb ● Entirely contained in '<build>' folder ● Sources used are in '<build>/downloads' ● Builds are in '<build>/tmp/work/<arch>' ● 'deploy' has ● Ready to use images ● Packages ● license information ● Buildhistory records information about content of images, and package build after build (with diffs) ● 'flat' rootfs also available in image <work> folder ● INHERIT += “rm_work” cleans up each <work> folder upon successful build
  • 16. OpenEmbedded and Linaro
  • 17. 17 Background ● Originally we started using OE to bootstrap Aarch64 support ● Also used by Toolchain WG to validate compiler ● Extended interest by networking WG and others ● Integrated to our CI loop
  • 18. 18 Quickstart ● Git clone git://git.linaro.org/openembedded/jenkins-setup.git ● $ sudo bash pre-build-root-install-dependencies.sh ● $ init-and-build.sh linaro-image-minimal ● Have a nice cup ● of coffee!
  • 19. 19 Show me the code ● git.l.org/openembedded/jenkins-setup.git ● Setup and build script as used by integration ● git.l.org/openembedded/manifest.git ● Repo manifest, describes used git repositores ● git.l.org/openembedded/meta-linaro.git ● Contains Linaro, toolchain and aarch64 layers ● Branches for head and stable (dylan) ● Poky support being worked on
  • 20. 20 Continuous integration ● Jenkins: builds – LAVA runtime testing ● https://ci.linaro.org/jenkins/job/openembedded-armv8-rootfs/ ● Building the minimal, lamp and Java Images ● https://git.linaro.org/gitweb?p=qa/test- definitions.git;a=tree;f=openembedded ● Test definitions for LAVA ● http://validation.linaro.org/lava-server/dashboard/image- reports/minimal-armv8 ● Test results in LAVA ● Similar pages for LAMP and JAVA images
  • 21. 21 Future plans ● Switching to Poky/Yocto or stable OE ● Changes in OE head disruptive ● Repo manifest tagging for releases ● For reproducibility ● CI integration of ARMv7 and big endian ● Bitbake world -k for toolchain validation ● Try to keep things simple instead of supporting every possible usecase.
  • 22. 22 Resources ● https://www.yoctoproject.org/ ● http://openembedded.org/wiki/Main_Page ● http://www.slideshare.net/rossburton/the-yocto-project ● https://www.yoctoproject.org/documentation/current ● http://layers.openembedded.org/layerindex/ ● http://www.linuxfoundation.org/news-media/blogs/browse/2013/05/new-paper-how-engineering- leaders-can-use-yocto-project-solve ● http://www.linuxfoundation.org/news-media/announcements/2010/10/linux-foundation-and- consumer-electronics-linux-forum-merge ● http://www.linuxfoundation.org/news-media/blogs/browse/2010/10/while-apple-debates-open-vs- integrated-we-want-best-both-worlds
  • 23. 23 Thank you!
  • 24. 24 OpenEmbedded/Yocto 'lab'
  • 25. 25 Demystify OE/Yocto build through a practical exercise ● Build a minimal console image for qemuarm ● Run the image in QEMU ● Explore and understand the OE build directory structure ● Look at some basic OE developers tasks ● Build/run an application with SDK
  • 26. 26 Practical information ● Source and build cache ● “repo” tool ● Sample of recipe ● Lab instructions are here: ● Part1: https://collaborate.linaro.org/x/NYBm ● Part2: https://collaborate.linaro.org/x/q4Bm ● Part3: https://collaborate.linaro.org/x/zIBm ● Ask questions, we are here to answer them!
  • 27. 27 Interesting things to do... ● Explore configuration files ● Checkout some recipes ● Explore 'WORKDIR' and 'deploy' folders (image and package feed) ● Customize your image ● Add a new recipe, create your own layer ● Exercise with .bbappend ● Create a patch, and apply in recipe ● Use/understand bitbake 'tasks' and experiment typical developer workflow (build, debug, edit, build, debug, ...)
  • 28. 28 Sources mirror, build cache ● OpenEmbedded 'trees' contain no source code ● By design, everything is built from scratch. ● Source code for all recipes is fetched during the build ● 'fetchall' target let you download all source first ● PREMIRROR let you configure OE to use an alternative mirror before going to the Internet (can be local/intranet) ● OE implements “shared state code” (aka sstate) to support incremental builds ● OE uses hash/signature on each 'task' input variables to decide if task needs to be rebuilt. ● Prebuilt components are stored in “sstate” cache, which can be local or http, and can be shared across multiple users. ● Even after deleting the entire working folder, a rebuild is 'instantaneous' ● We prepared 'tarballs' for 'download' and 'sstate' to make this workshop more efficient, hopefully you've downloaded everything already ● http://people.linaro.org/~trevor.woerner/LCE2013/ ● Otherwise we have a couple of HDDs
  • 29. 29 “repo” tool ● Implemented initially by the Android team to manage Android source repositories ● Open source tool: https://code.google.com/p/git-repo/ ● Wrapper on top of git ● Manage 'forest' of git trees ● Support multiple 'product' branches and easy switch from one checkout to another ● Trees metadata stored in 'XML manifest' file https://gerrit.googlesource.com/git-repo/+/master/docs/manifest-format.txt ● Manifest can checkout: ● Latest commit on branch (development) ● Fix commit/tag (release) ● All Linaro OE projects are moving to “repo” ● Repo can (optionally) be used with Gerrit code review
  • 30. 30 Manifest sample <?xml version="1.0" encoding="UTF-8"?> <manifest> <remote name="linaro" fetch="git://git.linaro.org" /> <remote name="oe" fetch="git://git.openembedded.org" /> <remote name="beagleboard" fetch="git://github.com/beagleboard" /> <default remote="oe" revision="master" sync-j="2" /> <project name="openembedded-core" /> <project name="meta-openembedded" /> <project path="openembedded-core/bitbake" name="bitbake" /> <project path="meta-linaro" name="openembedded/meta-linaro" remote="linaro" /> <project name="meta-beagleboard" remote="beagleboard"/> </manifest>
  • 31. 31 How to use “repo” # install repo wrapper (once on each machine) $ curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > /tmp/repo $ chmod a+x /tmp/repo $ sudo mv /tmp/repo /usr/local/bin/ # initialize the repo, and download the manifest $ mkdir <mydir> && cd <mydir> $ repo init -u <manifest.git> -b <branch> # download and checkout all projects $ repo sync # build, edit, commit, debug, … # update my working copy $ repo sync # switch to another branch/product $ repo init -b <my other branch> $ repo sync commands: list status info diff grep
  • 32. 32 Hello World! DESCRIPTION = "Simple helloworld application" SECTION = "examples" LICENSE = "MIT" LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" PR = "r0" SRC_URI = "file://helloworld.c" S = "${WORKDIR}" do_compile() { ${CC} helloworld.c -o helloworld } do_install() { install -d ${D}${bindir} install -m 0755 helloworld ${D}${bindir} } Packages: helloworld helloworld-dev helloworld-dbg
  • 33. 33 smartmontools # meta-oe/recipes-extended/smartmontools/smartmontools_6.1.bb SECTION = "console/utils" DESCRIPTION = "Control and monitor storage systems using S.M.A.R.T." HOMEPAGE = "http://smartmontools.sourceforge.net/" LICENSE = "GPLv2" LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263" SRC_URI = "${SOURCEFORGE_MIRROR}/smartmontools/smartmontools-${PV}.tar.gz" SRC_URI[md5sum] = "83a3a681f8183ed858392d550ae1cca6" SRC_URI[sha256sum] = "a9003b8bccc82682f658ce76d70edb1842411e51dc56d4cd6b56618da1d9ce07" inherit autotools http://cgit.openembedded.org/meta-openembedded/commit/?id=8fec906a4584f9cbc467fb0e406bb6a705848b64
  • 34. 34 Using layer to alter recipes $ more meta-linaro/recipes-core/busybox/busybox_1.20.2.bbappend SRC_URI += "file://linaro.cfg" $ more meta-aarch64/recipes-sato/webkit/webkit-gtk_1.8.3.bbappend SRC_URI_append = "file://aarch64.patch” ● “<packagename>_<version>.bbappend” files can be used in layers ● Any variable from the original recipe can be modified ● Very neat for patching! Avoid duplication, fragmentation. ● “bitbake-layers show-appends”
  • 35. 35 Backup slides
  • 36. 36 Tuning compiler ● CPU compiler options, FPU and ABI can be fully tuned from the machine configuration file ● Each 'architecture' gets a set of pre-defined 'tunes' ● For ARM : VFP vs neon, Thumb vs ARM instructions, Little vs big endian, Calling conventions (hardfp vs softfp, …) TUNEVALID[neon] = "Enable Neon SIMD accelerator unit." TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "neon", " -mfpu=neon", "" ,d)}" ARMPKGSFX_FPU .= "${@bb.utils.contains("TUNE_FEATURES", "neon", "-neon", "" ,d)}" TUNEVALID[vfp] = "Enable Vector Floating Point (vfp) unit." ARMPKGSFX_FPU .= "${@bb.utils.contains("TUNE_FEATURES", "vfp", "-vfp", "" ,d)}" TUNEVALID[callconvention-hard] = "Enable EABI hard float call convention, requires VFP." TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "vfp", bb.utils.contains("TUNE_FEATURES", "callconvention-hard", " -mfloat-abi=hard", " -mfloat-abi=softfp", d), "" ,d)}" ARMPKGSFX_EABI .= "${@bb.utils.contains("TUNE_FEATURES", [ "callconvention-hard", "vfp" ], "hf", "", d)}"
  • 37. 37 Layer configuration file # LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf # changes incompatibly LCONF_VERSION = "5" BBPATH = "${TOPDIR}" BBFILES ?= "" BBLAYERS ?= " ${TOPDIR}/openembedded-core/meta ${TOPDIR}/meta-openembedded/toolchain-layer ${TOPDIR}/meta-openembedded/meta-multimedia ${TOPDIR}/meta-openembedded/meta-oe ${TOPDIR}/meta-linaro/meta-linaro ${TOPDIR}/meta-linaro/meta-linaro-toolchain ${TOPDIR}/meta-qt5 " BBLAYERS_NON_REMOVABLE ?= " ${TOPDIR}/openembedded-core/meta "
  • 38. 38 Machine configuration file #@TYPE: Machine #@NAME: common_pc #@DESCRIPTION: Machine configuration for running a common x86 PREFERRED_PROVIDER_virtual/xserver ?= "xserver-xorg" PREFERRED_PROVIDER_virtual/libgl ?= "mesa" PREFERRED_PROVIDER_virtual/libgles1 ?= "mesa" PREFERRED_PROVIDER_virtual/libgles2 ?= "mesa" require conf/machine/include/qemu.inc require conf/machine/include/tune-i586.inc KERNEL_IMAGETYPE = "bzImage" SERIAL_CONSOLE = "115200 ttyS0" XSERVER = "xserver-xorg mesa-driver-swrast xf86-input-vmmouse xf86-input-keyboard xf86-input-evdev xf86-video-vmware" MACHINE_FEATURES += "x86" MACHINE_ESSENTIAL_EXTRA_RDEPENDS += "v86d" qemux86.conf
  • 39. 39 User local configuration file # Parallelism Options #BB_NUMBER_THREADS = "4" #PARALLEL_MAKE = "-j 4" MACHINE = "qemuarm" DISTRO = “poky” PACKAGE_CLASSES ?= "package_ipk" # "dbg-pkgs" - add -dbg packages for all installed packages # (adds symbol information for debugging/profiling) # "tools-testapps" - add useful testing tools (ts_print, aplay, arecord etc.) # "debug-tweaks" - make an image suitable for development # e.g. ssh root access has a blank password EXTRA_IMAGE_FEATURES = "debug-tweaks" # - 'buildstats' collect build statistics # - 'image-prelink' in order to prelink the filesystem image USER_CLASSES ?= "buildstats image-prelink" #SSTATE_MIRRORS ?= " #file://.* http://someserver.tld/share/sstate/PATH;downloadfilename=PATH n #file://.* file:///some/local/dir/sstate/PATH"
  • 40. 40 Yocto Project Branding/Compliance ● Compliance controls the usage of the Yocto project name, logo and brand. ● Yocto Project Participants ● Open source project, non profit, small business ● Committed to promoting and contributing to Yocto Projects (oe-core, bitbake, yocto layers) ● Yocto Project Compatible ● Products, BSP, layers maintained by an open source, non profit, or Yocto Project member organisation ● Compatible with a specific release of Yocto ● Contribute all Yocto Projects patches upstream ● All Yocto recommendations/processes implemented