SlideShare a Scribd company logo
Yocto Project | The Linux Foundation
Marco Cavallini, KOAN
https://koansoftware.com
1. Strenghten your Yocto deployments with
Autobuilder2 CI tool
2 Yocto Project | The Linux Foundation
Who am I
• Founded KOAN on 1996
• Working with software for industrial automation until 1999
• Linux embedded devloper since 2000
• Openembedded board member since 2009
• Yocto Project participant since 2010
• Yocto Project Advocate since 2012
•
• Software development and consulting
• BSP creation
• Device driver kernel development
• Open embedded and Yocto Project training
3 Yocto Project | The Linux Foundation
Agenda
• What is a Continuous Integration (CI) system
• Autobuilder2 History
• Buildbot, the foundations
• Buildbot mechanics
• Buildbot installation
• Autobuilder2 installation
• Autobuilder2 configuration
• Autobuilder2 usage (as-is)
• Autobuilder2 customization
• Autobuilder2 usage for CI on a single machine
4 Yocto Project | The Linux Foundation
What is Autobuilder
• Autobuilder is a project based on Buildbot
• Buildbot is a Python open-source application used to
build, test, and release a wide variety of software.
• Autobuilder and Buildbot are licensed under GPLv2
5 Yocto Project | The Linux Foundation
CI overview
‘master’
CONTROLLER
• Typical CI on a single machine
‘slave’
WORKER
‘slave’
WORKER
‘slave’
WORKER
6 Yocto Project | The Linux Foundation
CI overview
‘master’
CONTROLLER
• Typical CI on a distributed system
‘slave’
WORKER
‘slave’
WORKER
‘slave’
WORKER
7 Yocto Project | The Linux Foundation
Our goal
‘master’
CONTROLLER
• Speed Yocto builds by populating premirrors with
Autobuilder2
‘slave’
WORKER
SSTATE
image
DL_DIR bitbake
8 Yocto Project | The Linux Foundation
Autobuilder overview
yocto-autobuilder2 yocto-autobuilder-helper
9 Yocto Project | The Linux Foundation
Autobuilder history *
• Creation of Autobuilder
• The autobuilder started life as something OpenedHand used
for testing Poky linux.
• Yocto-autobuilder
• It became "yocto-autobuilder" under Beth's stewardship in
December 2012 when it was totally re-implemented.
• Autobuilder2
• In Feburary 2018 it was rewritten again, in particular to move
from the long obsolete "buildbot eight" codebase to the
"buildbot nine" one but also to fix many long running issues
and get back to using an upstream codebase.
* Thanks to Richard Purdie who provided these information
10 Yocto Project | The Linux Foundation
People behind Autobuilder *
• Project Autobuilder
• Richard Purdie, Elizabeth Flanagan, Joshua Lock as well as
contributions from Tracy Graydon, Anibal Limón and Bill
Randle.
• Project Autobuilder2
• Richard Purdie and Joshua Lock.
• Michael Halstead is the project sysadmin who maintains the
infrastructure it all runs on top of.
* Thanks to Richard Purdie who provided these information
11 Yocto Project | The Linux Foundation
Buildbot
12 Yocto Project | The Linux Foundation
Buildbot basics
• Buildbot is a job scheduling system
• it queues jobs, executes the jobs when the required
resources are available, and reports the results
13 Yocto Project | The Linux Foundation
Buildbot basics
• Workers are typically run on separate machines
14 Yocto Project | The Linux Foundation
Buildbot basics
15 Yocto Project | The Linux Foundation
What happens inside the master
builder n
worker 1 worker nworker 2 . . .
↓cmds results↑results
VCS
repos
change source
poll
changes
scheduler
triggers
direct commands
Buildfactory
- step 1
- step 2
.
.
.
reporter
status/results
Build master
checkout
builder 2
builder 1
. . .
starts builder
scheduled events
* artwork by Mauro Salvini
16 Yocto Project | The Linux Foundation
Buildbot installation
• On a native system
• Probably the fastest solution
• In a Python sandbox
• Isolates it from the host system
• Using pip
• In a Docker container
• Isolates it from the host system
https://docs.buildbot.net/2.4.0/full.html
17 Yocto Project | The Linux Foundation
Buildbot installation (in a Python sandbox) [1/3]
• Create a sandbox
• Install master
• Install worker
mkdir abot-sandbox
cd abot-sandbox
python3 -m venv sandbox
source sandbox/bin/activate
mkdir abot-sandbox
cd abot-sandbox
python3 -m venv sandbox
source sandbox/bin/activate
pip install --upgrade pip
pip install 'buildbot[bundle]'
pip install --upgrade pip
pip install 'buildbot[bundle]'
pip install --upgrade pip
pip install buildbot-worker
pip install --upgrade pip
pip install buildbot-worker
18 Yocto Project | The Linux Foundation
Buildbot installation (in a Python sandbox) [2/3]
• Create the master
• Create the worker
buildbot create-master master
mv master/master.cfg.sample master/master.cfg
buildbot create-master master
mv master/master.cfg.sample master/master.cfg
pip install setuptools-trial
buildbot-worker create-worker worker localhost example-worker pass
pip install setuptools-trial
buildbot-worker create-worker worker localhost example-worker pass
19 Yocto Project | The Linux Foundation
Buildbot installation (in a Python sandbox) [3/3]
• Content of the sandbox
(sandbox) koan@amonra:~/abot-sandbox$ tree -L 2
.
├── master
│   ├── buildbot.tac
│   ├── master.cfg → c['workers'] = [worker.Worker("example-worker", "pass")] (*)
│   └── state.sqlite
├── sandbox
│   ├── bin
│   ├── include
│   ├── lib
│   ├── lib64 -> lib
│   ├── pip-selfcheck.json
│   ├── pyvenv.cfg
│   └── share
└── worker
├── buildbot.tac → workername = 'example-worker' (*)
└── info
(sandbox) koan@amonra:~/abot-sandbox$ tree -L 2
.
├── master
│   ├── buildbot.tac
│   ├── master.cfg → c['workers'] = [worker.Worker("example-worker", "pass")] (*)
│   └── state.sqlite
├── sandbox
│   ├── bin
│   ├── include
│   ├── lib
│   ├── lib64 -> lib
│   ├── pip-selfcheck.json
│   ├── pyvenv.cfg
│   └── share
└── worker
├── buildbot.tac → workername = 'example-worker' (*)
└── info
ftp://ftp.koansoftware.com/public/opensource/buildbot/(*)
20 Yocto Project | The Linux Foundation
Buildbot execution
• Execution of master
• Control of the build system (using your browser)
buildbot start masterbuildbot start master
http://localhost:8010/http://localhost:8010/
21 Yocto Project | The Linux Foundation
Buildbot
• To be continued in a dedicated session…
• Now let’s have a look at Autobuilder
22 Yocto Project | The Linux Foundation
Autobuilder2
Ab2
Ab2
23 Yocto Project | The Linux Foundation
Buildbot vs. Autobuilder2 lexicon Ab2
Buildbot Autobuilder2
Master Controller
Worker Worker
24 Yocto Project | The Linux Foundation
Autobuilder2 installation
• After you created the sandbox
• Create the master and worker directories
• yocto-controller is the directory for master
• --umask sets the proper permissions
• yocto-worker is the directory for worker
• localhost is the network address of the master
• example-worker is the name of the worker
• pass is the password (master.cfg)
Ab2
buildbot create-master -r yocto-controller
buildbot-worker create-worker -r /
--umask=0o22 yocto-worker localhost example-worker pass
buildbot create-master -r yocto-controller
buildbot-worker create-worker -r /
--umask=0o22 yocto-worker localhost example-worker pass
25 Yocto Project | The Linux Foundation
Autobuilder2 installation
• Clone yocto-autobuilder2
• yoctoabb is the mandatory Autobuilder2 directory name
• Clone yocto-autobuilder-helper
cd yocto-controller
git clone https://git.yoctoproject.org/git/yocto-autobuilder2 yoctoabb
ln -rs yoctoabb/master.cfg master.cfg
cd yocto-controller
git clone https://git.yoctoproject.org/git/yocto-autobuilder2 yoctoabb
ln -rs yoctoabb/master.cfg master.cfg
cd ..
git clone https://git.yoctoproject.org/git/yocto-autobuilder-helper
cd ..
git clone https://git.yoctoproject.org/git/yocto-autobuilder-helper
Ab2
26 Yocto Project | The Linux Foundation
Autobuilder2 tree
• Content of the Autobuilder2 sandbox
(sandbox) koan@amonra:~/ab2-sandbox$ tree -L 3
.
├── autobuilder
├── git
├── sandbox
├── yocto-autobuilder-helper
├── yocto-controller
│   └── yoctoabb
│   └── master.cfg
└── yocto-worker
└── buildbot.tac
(sandbox) koan@amonra:~/ab2-sandbox$ tree -L 3
.
├── autobuilder
├── git
├── sandbox
├── yocto-autobuilder-helper
├── yocto-controller
│   └── yoctoabb
│   └── master.cfg
└── yocto-worker
└── buildbot.tac
Ab2
* Simplified tree list
27 Yocto Project | The Linux Foundation
Autobuilder2 installation
• Complete installation instructions
Ab2
http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder2/tree/README-Guide.md
html
ftp://ftp.koansoftware.com/public/talks/YoctoSummit-2019/au
tobuilder2/Autobuilder2-README-Guide.pdf
28 Yocto Project | The Linux Foundation
Autobuilder2
Autobuilder default configuration
29 Yocto Project | The Linux Foundation
Autobuilder2 configuration
• Ab2 default configuration
• The default configuration of Ab2 uses a lot of workers to
generate images for several MACHINES
Ab2
30 Yocto Project | The Linux Foundation
Autobuilder2 official website
https://autobuilder.yoctoproject.org/typhoon/
31 Yocto Project | The Linux Foundation
Autobuilder2 website navigation
32 Yocto Project | The Linux Foundation
Autobuilder2 website navigation
33 Yocto Project | The Linux Foundation
Autobuilder2 - builds
34 Yocto Project | The Linux Foundation
Autobuilder2 - workers
35 Yocto Project | The Linux Foundation
Autobuilder2 – worker details
36 Yocto Project | The Linux Foundation
Autobuilder2 – build details
37 Yocto Project | The Linux Foundation
Autobuilder2 – build details
38 Yocto Project | The Linux Foundation
Autobuilder2 – build details
39 Yocto Project | The Linux Foundation
Autobuilder2 – build details
40 Yocto Project | The Linux Foundation
Autobuilder2
Autobuilder custom ‘lighter’
configuration
41 Yocto Project | The Linux Foundation
Autobuilder2 reduced configuration
• Reduce complexity
• The goal is to setup a configuration for an image for a single
MACHINE only
• This will help yo strenghten the deployments thanks to
recurring builds, tipically nightly
Ab2
42 Yocto Project | The Linux Foundation
Autobuilder2 reduced configuration
• Files to be modified
.
├── yocto-autobuilder-helper
│   └── config.json
├── yocto-controller
│   └── yoctoabb
│   ├── builders.py
│   ├── config.py
│   ├── master.cfg
│   └── schedulers.py
└── yocto-worker
└── buildbot.tac
.
├── yocto-autobuilder-helper
│   └── config.json
├── yocto-controller
│   └── yoctoabb
│   ├── builders.py
│   ├── config.py
│   ├── master.cfg
│   └── schedulers.py
└── yocto-worker
└── buildbot.tac
Ab2
43 Yocto Project | The Linux Foundation
Autobuilder2 configuration
• In yocto-autobuilder-helper
• Edit yocto-autobuilder-helper/config.json
• In yocto-controller
• Edit yocto-controller/yoctoabb/master.cfg
"BASE_HOMEDIR" : "/home/koan/ab2-sandbox",
"BASE_SHAREDDIR" : "${BASE_HOMEDIR}/autobuilder",
"BASE_PUBLISHDIR" : "${BASE_HOMEDIR}/downloads",
"BASE_HOMEDIR" : "/home/koan/ab2-sandbox",
"BASE_SHAREDDIR" : "${BASE_HOMEDIR}/autobuilder",
"BASE_PUBLISHDIR" : "${BASE_HOMEDIR}/downloads",
c['title'] = "KOAN lite Yocto Autobuilder"
c['titleURL'] = "http://localhost:8010/"
c['buildbotURL'] = "http://localhost:8010/"
c['title'] = "KOAN lite Yocto Autobuilder"
c['titleURL'] = "http://localhost:8010/"
c['buildbotURL'] = "http://localhost:8010/"
Ab2
44 Yocto Project | The Linux Foundation
Autobuilder2 configuration
• In yocto-controller (again)
• Edit yocto-controller/yoctoabb/config.py
• Specify the helper directory
workers_koan = ["example-worker"]
workers = workers_koan
all_workers = workers,
workers_koan = ["example-worker"]
workers = workers_koan
all_workers = workers,
sharedrepodir = "/home/koan/ab2-sandbox/repos"
publish_dest = "/home/koan/ab2-sandbox/pub"
sharedrepodir = "/home/koan/ab2-sandbox/repos"
publish_dest = "/home/koan/ab2-sandbox/pub"
Ab2
repos = {
"yocto-autobuilder-helper":
["file:///home/koan/ab2-sandbox/yocto-autobuilder-helper",
"master"],
repos = {
"yocto-autobuilder-helper":
["file:///home/koan/ab2-sandbox/yocto-autobuilder-helper",
"master"],
45 Yocto Project | The Linux Foundation
Autobuilder2 ‘lite’ customized
http://localhost:8010/
46 Yocto Project | The Linux Foundation
Autobuilder2 ‘lite’ customized
47 Yocto Project | The Linux Foundation
Autobuilder2 to speed up Yocto build
• Share the artefacts with Yocto
• Edit local.conf
• Share downloads
• Share SSTATE
PREMIRRORS_prepend = "
git://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
ftp://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
http://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
https://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
"
PREMIRRORS_prepend = "
git://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
ftp://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
http://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
https://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
"
Ab2
SSTATE_MIRRORS = "file://.* 
file:///home/koan/ab2-sandbox/autobuilder/pub/sstate/PATH"
SSTATE_MIRRORS = "file://.* 
file:///home/koan/ab2-sandbox/autobuilder/pub/sstate/PATH"
48 Yocto Project | The Linux Foundation
Autobuilder2 to speed up Yocto build
‘master’
CONTROLLER
‘slave’
WORKER
SSTATE
image
DL_DIR bitbake
Yocto Project | The Linux Foundation
Questions?
https://koansoftware.comhttps://yoctoproject.org

More Related Content

What's hot

The Yocto Project
The Yocto ProjectThe Yocto Project
The Yocto Project
rossburton
 
Yocto project
Yocto projectYocto project
yocto_scale_handout-with-notes
yocto_scale_handout-with-notesyocto_scale_handout-with-notes
yocto_scale_handout-with-notes
Steve Arnold
 
Build your own embedded linux distributions by yocto project
Build your own embedded linux distributions by yocto projectBuild your own embedded linux distributions by yocto project
Build your own embedded linux distributions by yocto project
Yen-Chin Lee
 
Yocto and IoT - a retrospective
Yocto and IoT - a retrospectiveYocto and IoT - a retrospective
Yocto and IoT - a retrospective
Open-RnD
 
Yocto: Treinamento em Português
Yocto: Treinamento em PortuguêsYocto: Treinamento em Português
Yocto: Treinamento em Português
Otavio Salvador
 
Eclipse IDE Yocto Plugin
Eclipse IDE Yocto PluginEclipse IDE Yocto Plugin
Eclipse IDE Yocto Plugin
cudma
 
Yocto Project - OSCON 7-17-2012
Yocto Project - OSCON 7-17-2012Yocto Project - OSCON 7-17-2012
Yocto Project - OSCON 7-17-2012
Jeffrey Osier-Mixon
 
Yocto - Embedded Linux Distribution Maker
Yocto - Embedded Linux Distribution MakerYocto - Embedded Linux Distribution Maker
Yocto - Embedded Linux Distribution Maker
Sherif Mousa
 
Debugging embedded devices using GDB
Debugging embedded devices using GDBDebugging embedded devices using GDB
Debugging embedded devices using GDB
Chris Simmonds
 
Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...
Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...
Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...
Anne Nicolas
 
Debian or Yocto Project? Which is the best for your Embedded Linux project?
Debian or Yocto Project? Which is the best for your Embedded Linux project?Debian or Yocto Project? Which is the best for your Embedded Linux project?
Debian or Yocto Project? Which is the best for your Embedded Linux project?
Chris Simmonds
 
Introduction to OpenBricks: an Embedded Linux Framework
Introduction to OpenBricks: an Embedded Linux FrameworkIntroduction to OpenBricks: an Embedded Linux Framework
Introduction to OpenBricks: an Embedded Linux Framework
Benjamin Zores
 
Embedded Linux primer
Embedded Linux primerEmbedded Linux primer
Embedded Linux primer
Drew Fustini
 
The end of embedded Linux (as we know it)
The end of embedded Linux (as we know it)The end of embedded Linux (as we know it)
The end of embedded Linux (as we know it)
Chris Simmonds
 
Reducing the boot time of Linux devices
Reducing the boot time of Linux devicesReducing the boot time of Linux devices
Reducing the boot time of Linux devices
Chris Simmonds
 
Reducing boot time in embedded Linux
Reducing boot time in embedded LinuxReducing boot time in embedded Linux
Reducing boot time in embedded Linux
Chris Simmonds
 
Droidcon 2013 france - The Growth of Android in Embedded Systems
Droidcon 2013 france - The Growth of Android in Embedded SystemsDroidcon 2013 france - The Growth of Android in Embedded Systems
Droidcon 2013 france - The Growth of Android in Embedded Systems
Benjamin Zores
 
Building Emmbedded Linux with Yocto project
Building Emmbedded Linux with Yocto projectBuilding Emmbedded Linux with Yocto project
Building Emmbedded Linux with Yocto project
twcoimbatore
 
UplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processor
UplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processorUplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processor
UplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processor
Satya Harish
 

What's hot (20)

The Yocto Project
The Yocto ProjectThe Yocto Project
The Yocto Project
 
Yocto project
Yocto projectYocto project
Yocto project
 
yocto_scale_handout-with-notes
yocto_scale_handout-with-notesyocto_scale_handout-with-notes
yocto_scale_handout-with-notes
 
Build your own embedded linux distributions by yocto project
Build your own embedded linux distributions by yocto projectBuild your own embedded linux distributions by yocto project
Build your own embedded linux distributions by yocto project
 
Yocto and IoT - a retrospective
Yocto and IoT - a retrospectiveYocto and IoT - a retrospective
Yocto and IoT - a retrospective
 
Yocto: Treinamento em Português
Yocto: Treinamento em PortuguêsYocto: Treinamento em Português
Yocto: Treinamento em Português
 
Eclipse IDE Yocto Plugin
Eclipse IDE Yocto PluginEclipse IDE Yocto Plugin
Eclipse IDE Yocto Plugin
 
Yocto Project - OSCON 7-17-2012
Yocto Project - OSCON 7-17-2012Yocto Project - OSCON 7-17-2012
Yocto Project - OSCON 7-17-2012
 
Yocto - Embedded Linux Distribution Maker
Yocto - Embedded Linux Distribution MakerYocto - Embedded Linux Distribution Maker
Yocto - Embedded Linux Distribution Maker
 
Debugging embedded devices using GDB
Debugging embedded devices using GDBDebugging embedded devices using GDB
Debugging embedded devices using GDB
 
Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...
Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...
Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...
 
Debian or Yocto Project? Which is the best for your Embedded Linux project?
Debian or Yocto Project? Which is the best for your Embedded Linux project?Debian or Yocto Project? Which is the best for your Embedded Linux project?
Debian or Yocto Project? Which is the best for your Embedded Linux project?
 
Introduction to OpenBricks: an Embedded Linux Framework
Introduction to OpenBricks: an Embedded Linux FrameworkIntroduction to OpenBricks: an Embedded Linux Framework
Introduction to OpenBricks: an Embedded Linux Framework
 
Embedded Linux primer
Embedded Linux primerEmbedded Linux primer
Embedded Linux primer
 
The end of embedded Linux (as we know it)
The end of embedded Linux (as we know it)The end of embedded Linux (as we know it)
The end of embedded Linux (as we know it)
 
Reducing the boot time of Linux devices
Reducing the boot time of Linux devicesReducing the boot time of Linux devices
Reducing the boot time of Linux devices
 
Reducing boot time in embedded Linux
Reducing boot time in embedded LinuxReducing boot time in embedded Linux
Reducing boot time in embedded Linux
 
Droidcon 2013 france - The Growth of Android in Embedded Systems
Droidcon 2013 france - The Growth of Android in Embedded SystemsDroidcon 2013 france - The Growth of Android in Embedded Systems
Droidcon 2013 france - The Growth of Android in Embedded Systems
 
Building Emmbedded Linux with Yocto project
Building Emmbedded Linux with Yocto projectBuilding Emmbedded Linux with Yocto project
Building Emmbedded Linux with Yocto project
 
UplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processor
UplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processorUplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processor
UplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processor
 

Similar to Autobuilder2 Yocto Project Summit Lyon 2019

Yocto Project Kernel Lab hands-on
Yocto Project Kernel Lab hands-onYocto Project Kernel Lab hands-on
Yocto Project Kernel Lab hands-on
Trevor Woerner
 
An Introduction to the Yocto Embedded Framework 2018
An Introduction to the Yocto Embedded Framework 2018An Introduction to the Yocto Embedded Framework 2018
An Introduction to the Yocto Embedded Framework 2018
ICS
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
FIDO Alliance
 
Yocto Project Kernel Lab, Hands-On
Yocto Project Kernel Lab, Hands-OnYocto Project Kernel Lab, Hands-On
Yocto Project Kernel Lab, Hands-On
Trevor Woerner
 
Stefano Cordibella - An introduction to Yocto Project
Stefano Cordibella - An introduction to Yocto ProjectStefano Cordibella - An introduction to Yocto Project
Stefano Cordibella - An introduction to Yocto Project
linuxlab_conf
 
IoT Prototyping using BBB and Debian
IoT Prototyping using BBB and DebianIoT Prototyping using BBB and Debian
IoT Prototyping using BBB and Debian
Mender.io
 
IoT: Contrasting Yocto/Buildroot to binary OSes
IoT: Contrasting Yocto/Buildroot to binary OSesIoT: Contrasting Yocto/Buildroot to binary OSes
IoT: Contrasting Yocto/Buildroot to binary OSes
Mender.io
 
FOSDEM_2019_Buildroot_RISCV.pdf
FOSDEM_2019_Buildroot_RISCV.pdfFOSDEM_2019_Buildroot_RISCV.pdf
FOSDEM_2019_Buildroot_RISCV.pdf
ssuser30e7d2
 
Building Embedded Linux UDOONEO
Building Embedded Linux UDOONEOBuilding Embedded Linux UDOONEO
Building Embedded Linux UDOONEO
NEEVEE Technologies
 
Embedded linux build systems
Embedded linux build systems  Embedded linux build systems
Embedded linux build systems
Mender.io
 
[Webinar] An Introduction to the Yocto Embedded Framework
[Webinar] An Introduction to the Yocto Embedded Framework[Webinar] An Introduction to the Yocto Embedded Framework
[Webinar] An Introduction to the Yocto Embedded Framework
ICS
 
Comparing Next-Generation Container Image Building Tools
 Comparing Next-Generation Container Image Building Tools Comparing Next-Generation Container Image Building Tools
Comparing Next-Generation Container Image Building Tools
Akihiro Suda
 
Embedded Linux Build Systems - Texas Linux Fest 2018
Embedded Linux Build Systems - Texas Linux Fest 2018Embedded Linux Build Systems - Texas Linux Fest 2018
Embedded Linux Build Systems - Texas Linux Fest 2018
Mender.io
 
Introduction to the rapid prototyping with python and linux for embedded systems
Introduction to the rapid prototyping with python and linux for embedded systemsIntroduction to the rapid prototyping with python and linux for embedded systems
Introduction to the rapid prototyping with python and linux for embedded systems
Naohiko Shimizu
 
LinuxKit and Moby, News from DockerCon 2017
LinuxKit and Moby, News from DockerCon 2017LinuxKit and Moby, News from DockerCon 2017
LinuxKit and Moby, News from DockerCon 2017
Dieter Reuter
 
LinuxKit and Moby, news from DockerCon 2017 - Austin,TX
LinuxKit and Moby, news from DockerCon 2017 - Austin,TXLinuxKit and Moby, news from DockerCon 2017 - Austin,TX
LinuxKit and Moby, news from DockerCon 2017 - Austin,TX
Dieter Reuter
 
Why the yocto project for my io t project elc_edinburgh_2018
Why the yocto project for my io t project elc_edinburgh_2018Why the yocto project for my io t project elc_edinburgh_2018
Why the yocto project for my io t project elc_edinburgh_2018
Mender.io
 
Luca Ceresoli - Buildroot vs Yocto: Differences for Your Daily Job
Luca Ceresoli - Buildroot vs Yocto: Differences for Your Daily JobLuca Ceresoli - Buildroot vs Yocto: Differences for Your Daily Job
Luca Ceresoli - Buildroot vs Yocto: Differences for Your Daily Job
linuxlab_conf
 
Creating new Tizen profiles using the Yocto Project
Creating new Tizen profiles  using the Yocto ProjectCreating new Tizen profiles  using the Yocto Project
Creating new Tizen profiles using the Yocto Project
Leon Anavi
 
Generating a Reproducible and Maintainable Embedded Linux Environment with Po...
Generating a Reproducible and Maintainable Embedded Linux Environment with Po...Generating a Reproducible and Maintainable Embedded Linux Environment with Po...
Generating a Reproducible and Maintainable Embedded Linux Environment with Po...
LF Events
 

Similar to Autobuilder2 Yocto Project Summit Lyon 2019 (20)

Yocto Project Kernel Lab hands-on
Yocto Project Kernel Lab hands-onYocto Project Kernel Lab hands-on
Yocto Project Kernel Lab hands-on
 
An Introduction to the Yocto Embedded Framework 2018
An Introduction to the Yocto Embedded Framework 2018An Introduction to the Yocto Embedded Framework 2018
An Introduction to the Yocto Embedded Framework 2018
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
 
Yocto Project Kernel Lab, Hands-On
Yocto Project Kernel Lab, Hands-OnYocto Project Kernel Lab, Hands-On
Yocto Project Kernel Lab, Hands-On
 
Stefano Cordibella - An introduction to Yocto Project
Stefano Cordibella - An introduction to Yocto ProjectStefano Cordibella - An introduction to Yocto Project
Stefano Cordibella - An introduction to Yocto Project
 
IoT Prototyping using BBB and Debian
IoT Prototyping using BBB and DebianIoT Prototyping using BBB and Debian
IoT Prototyping using BBB and Debian
 
IoT: Contrasting Yocto/Buildroot to binary OSes
IoT: Contrasting Yocto/Buildroot to binary OSesIoT: Contrasting Yocto/Buildroot to binary OSes
IoT: Contrasting Yocto/Buildroot to binary OSes
 
FOSDEM_2019_Buildroot_RISCV.pdf
FOSDEM_2019_Buildroot_RISCV.pdfFOSDEM_2019_Buildroot_RISCV.pdf
FOSDEM_2019_Buildroot_RISCV.pdf
 
Building Embedded Linux UDOONEO
Building Embedded Linux UDOONEOBuilding Embedded Linux UDOONEO
Building Embedded Linux UDOONEO
 
Embedded linux build systems
Embedded linux build systems  Embedded linux build systems
Embedded linux build systems
 
[Webinar] An Introduction to the Yocto Embedded Framework
[Webinar] An Introduction to the Yocto Embedded Framework[Webinar] An Introduction to the Yocto Embedded Framework
[Webinar] An Introduction to the Yocto Embedded Framework
 
Comparing Next-Generation Container Image Building Tools
 Comparing Next-Generation Container Image Building Tools Comparing Next-Generation Container Image Building Tools
Comparing Next-Generation Container Image Building Tools
 
Embedded Linux Build Systems - Texas Linux Fest 2018
Embedded Linux Build Systems - Texas Linux Fest 2018Embedded Linux Build Systems - Texas Linux Fest 2018
Embedded Linux Build Systems - Texas Linux Fest 2018
 
Introduction to the rapid prototyping with python and linux for embedded systems
Introduction to the rapid prototyping with python and linux for embedded systemsIntroduction to the rapid prototyping with python and linux for embedded systems
Introduction to the rapid prototyping with python and linux for embedded systems
 
LinuxKit and Moby, News from DockerCon 2017
LinuxKit and Moby, News from DockerCon 2017LinuxKit and Moby, News from DockerCon 2017
LinuxKit and Moby, News from DockerCon 2017
 
LinuxKit and Moby, news from DockerCon 2017 - Austin,TX
LinuxKit and Moby, news from DockerCon 2017 - Austin,TXLinuxKit and Moby, news from DockerCon 2017 - Austin,TX
LinuxKit and Moby, news from DockerCon 2017 - Austin,TX
 
Why the yocto project for my io t project elc_edinburgh_2018
Why the yocto project for my io t project elc_edinburgh_2018Why the yocto project for my io t project elc_edinburgh_2018
Why the yocto project for my io t project elc_edinburgh_2018
 
Luca Ceresoli - Buildroot vs Yocto: Differences for Your Daily Job
Luca Ceresoli - Buildroot vs Yocto: Differences for Your Daily JobLuca Ceresoli - Buildroot vs Yocto: Differences for Your Daily Job
Luca Ceresoli - Buildroot vs Yocto: Differences for Your Daily Job
 
Creating new Tizen profiles using the Yocto Project
Creating new Tizen profiles  using the Yocto ProjectCreating new Tizen profiles  using the Yocto Project
Creating new Tizen profiles using the Yocto Project
 
Generating a Reproducible and Maintainable Embedded Linux Environment with Po...
Generating a Reproducible and Maintainable Embedded Linux Environment with Po...Generating a Reproducible and Maintainable Embedded Linux Environment with Po...
Generating a Reproducible and Maintainable Embedded Linux Environment with Po...
 

Recently uploaded

Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
Sven Peters
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
Patrick Weigel
 
What’s New in Odoo 17 – A Complete Roadmap
What’s New in Odoo 17 – A Complete RoadmapWhat’s New in Odoo 17 – A Complete Roadmap
What’s New in Odoo 17 – A Complete Roadmap
Envertis Software Solutions
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Paul Brebner
 
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
kgyxske
 
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
safelyiotech
 
The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024
Yara Milbes
 
Project Management: The Role of Project Dashboards.pdf
Project Management: The Role of Project Dashboards.pdfProject Management: The Role of Project Dashboards.pdf
Project Management: The Role of Project Dashboards.pdf
Karya Keeper
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
Alina Yurenko
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
Peter Muessig
 
ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.
Maitrey Patel
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptxMigration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
ervikas4
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
ICS
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
gapen1
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
dakas1
 
Quarter 3 SLRP grade 9.. gshajsbhhaheabh
Quarter 3 SLRP grade 9.. gshajsbhhaheabhQuarter 3 SLRP grade 9.. gshajsbhhaheabh
Quarter 3 SLRP grade 9.. gshajsbhhaheabh
aisafed42
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Green Software Development
 
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
dakas1
 

Recently uploaded (20)

Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
 
What’s New in Odoo 17 – A Complete Roadmap
What’s New in Odoo 17 – A Complete RoadmapWhat’s New in Odoo 17 – A Complete Roadmap
What’s New in Odoo 17 – A Complete Roadmap
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
 
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
 
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
 
The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024
 
Project Management: The Role of Project Dashboards.pdf
Project Management: The Role of Project Dashboards.pdfProject Management: The Role of Project Dashboards.pdf
Project Management: The Role of Project Dashboards.pdf
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
 
ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptxMigration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
 
Quarter 3 SLRP grade 9.. gshajsbhhaheabh
Quarter 3 SLRP grade 9.. gshajsbhhaheabhQuarter 3 SLRP grade 9.. gshajsbhhaheabh
Quarter 3 SLRP grade 9.. gshajsbhhaheabh
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
 
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
 

Autobuilder2 Yocto Project Summit Lyon 2019

  • 1. Yocto Project | The Linux Foundation Marco Cavallini, KOAN https://koansoftware.com 1. Strenghten your Yocto deployments with Autobuilder2 CI tool
  • 2. 2 Yocto Project | The Linux Foundation Who am I • Founded KOAN on 1996 • Working with software for industrial automation until 1999 • Linux embedded devloper since 2000 • Openembedded board member since 2009 • Yocto Project participant since 2010 • Yocto Project Advocate since 2012 • • Software development and consulting • BSP creation • Device driver kernel development • Open embedded and Yocto Project training
  • 3. 3 Yocto Project | The Linux Foundation Agenda • What is a Continuous Integration (CI) system • Autobuilder2 History • Buildbot, the foundations • Buildbot mechanics • Buildbot installation • Autobuilder2 installation • Autobuilder2 configuration • Autobuilder2 usage (as-is) • Autobuilder2 customization • Autobuilder2 usage for CI on a single machine
  • 4. 4 Yocto Project | The Linux Foundation What is Autobuilder • Autobuilder is a project based on Buildbot • Buildbot is a Python open-source application used to build, test, and release a wide variety of software. • Autobuilder and Buildbot are licensed under GPLv2
  • 5. 5 Yocto Project | The Linux Foundation CI overview ‘master’ CONTROLLER • Typical CI on a single machine ‘slave’ WORKER ‘slave’ WORKER ‘slave’ WORKER
  • 6. 6 Yocto Project | The Linux Foundation CI overview ‘master’ CONTROLLER • Typical CI on a distributed system ‘slave’ WORKER ‘slave’ WORKER ‘slave’ WORKER
  • 7. 7 Yocto Project | The Linux Foundation Our goal ‘master’ CONTROLLER • Speed Yocto builds by populating premirrors with Autobuilder2 ‘slave’ WORKER SSTATE image DL_DIR bitbake
  • 8. 8 Yocto Project | The Linux Foundation Autobuilder overview yocto-autobuilder2 yocto-autobuilder-helper
  • 9. 9 Yocto Project | The Linux Foundation Autobuilder history * • Creation of Autobuilder • The autobuilder started life as something OpenedHand used for testing Poky linux. • Yocto-autobuilder • It became "yocto-autobuilder" under Beth's stewardship in December 2012 when it was totally re-implemented. • Autobuilder2 • In Feburary 2018 it was rewritten again, in particular to move from the long obsolete "buildbot eight" codebase to the "buildbot nine" one but also to fix many long running issues and get back to using an upstream codebase. * Thanks to Richard Purdie who provided these information
  • 10. 10 Yocto Project | The Linux Foundation People behind Autobuilder * • Project Autobuilder • Richard Purdie, Elizabeth Flanagan, Joshua Lock as well as contributions from Tracy Graydon, Anibal Limón and Bill Randle. • Project Autobuilder2 • Richard Purdie and Joshua Lock. • Michael Halstead is the project sysadmin who maintains the infrastructure it all runs on top of. * Thanks to Richard Purdie who provided these information
  • 11. 11 Yocto Project | The Linux Foundation Buildbot
  • 12. 12 Yocto Project | The Linux Foundation Buildbot basics • Buildbot is a job scheduling system • it queues jobs, executes the jobs when the required resources are available, and reports the results
  • 13. 13 Yocto Project | The Linux Foundation Buildbot basics • Workers are typically run on separate machines
  • 14. 14 Yocto Project | The Linux Foundation Buildbot basics
  • 15. 15 Yocto Project | The Linux Foundation What happens inside the master builder n worker 1 worker nworker 2 . . . ↓cmds results↑results VCS repos change source poll changes scheduler triggers direct commands Buildfactory - step 1 - step 2 . . . reporter status/results Build master checkout builder 2 builder 1 . . . starts builder scheduled events * artwork by Mauro Salvini
  • 16. 16 Yocto Project | The Linux Foundation Buildbot installation • On a native system • Probably the fastest solution • In a Python sandbox • Isolates it from the host system • Using pip • In a Docker container • Isolates it from the host system https://docs.buildbot.net/2.4.0/full.html
  • 17. 17 Yocto Project | The Linux Foundation Buildbot installation (in a Python sandbox) [1/3] • Create a sandbox • Install master • Install worker mkdir abot-sandbox cd abot-sandbox python3 -m venv sandbox source sandbox/bin/activate mkdir abot-sandbox cd abot-sandbox python3 -m venv sandbox source sandbox/bin/activate pip install --upgrade pip pip install 'buildbot[bundle]' pip install --upgrade pip pip install 'buildbot[bundle]' pip install --upgrade pip pip install buildbot-worker pip install --upgrade pip pip install buildbot-worker
  • 18. 18 Yocto Project | The Linux Foundation Buildbot installation (in a Python sandbox) [2/3] • Create the master • Create the worker buildbot create-master master mv master/master.cfg.sample master/master.cfg buildbot create-master master mv master/master.cfg.sample master/master.cfg pip install setuptools-trial buildbot-worker create-worker worker localhost example-worker pass pip install setuptools-trial buildbot-worker create-worker worker localhost example-worker pass
  • 19. 19 Yocto Project | The Linux Foundation Buildbot installation (in a Python sandbox) [3/3] • Content of the sandbox (sandbox) koan@amonra:~/abot-sandbox$ tree -L 2 . ├── master │   ├── buildbot.tac │   ├── master.cfg → c['workers'] = [worker.Worker("example-worker", "pass")] (*) │   └── state.sqlite ├── sandbox │   ├── bin │   ├── include │   ├── lib │   ├── lib64 -> lib │   ├── pip-selfcheck.json │   ├── pyvenv.cfg │   └── share └── worker ├── buildbot.tac → workername = 'example-worker' (*) └── info (sandbox) koan@amonra:~/abot-sandbox$ tree -L 2 . ├── master │   ├── buildbot.tac │   ├── master.cfg → c['workers'] = [worker.Worker("example-worker", "pass")] (*) │   └── state.sqlite ├── sandbox │   ├── bin │   ├── include │   ├── lib │   ├── lib64 -> lib │   ├── pip-selfcheck.json │   ├── pyvenv.cfg │   └── share └── worker ├── buildbot.tac → workername = 'example-worker' (*) └── info ftp://ftp.koansoftware.com/public/opensource/buildbot/(*)
  • 20. 20 Yocto Project | The Linux Foundation Buildbot execution • Execution of master • Control of the build system (using your browser) buildbot start masterbuildbot start master http://localhost:8010/http://localhost:8010/
  • 21. 21 Yocto Project | The Linux Foundation Buildbot • To be continued in a dedicated session… • Now let’s have a look at Autobuilder
  • 22. 22 Yocto Project | The Linux Foundation Autobuilder2 Ab2 Ab2
  • 23. 23 Yocto Project | The Linux Foundation Buildbot vs. Autobuilder2 lexicon Ab2 Buildbot Autobuilder2 Master Controller Worker Worker
  • 24. 24 Yocto Project | The Linux Foundation Autobuilder2 installation • After you created the sandbox • Create the master and worker directories • yocto-controller is the directory for master • --umask sets the proper permissions • yocto-worker is the directory for worker • localhost is the network address of the master • example-worker is the name of the worker • pass is the password (master.cfg) Ab2 buildbot create-master -r yocto-controller buildbot-worker create-worker -r / --umask=0o22 yocto-worker localhost example-worker pass buildbot create-master -r yocto-controller buildbot-worker create-worker -r / --umask=0o22 yocto-worker localhost example-worker pass
  • 25. 25 Yocto Project | The Linux Foundation Autobuilder2 installation • Clone yocto-autobuilder2 • yoctoabb is the mandatory Autobuilder2 directory name • Clone yocto-autobuilder-helper cd yocto-controller git clone https://git.yoctoproject.org/git/yocto-autobuilder2 yoctoabb ln -rs yoctoabb/master.cfg master.cfg cd yocto-controller git clone https://git.yoctoproject.org/git/yocto-autobuilder2 yoctoabb ln -rs yoctoabb/master.cfg master.cfg cd .. git clone https://git.yoctoproject.org/git/yocto-autobuilder-helper cd .. git clone https://git.yoctoproject.org/git/yocto-autobuilder-helper Ab2
  • 26. 26 Yocto Project | The Linux Foundation Autobuilder2 tree • Content of the Autobuilder2 sandbox (sandbox) koan@amonra:~/ab2-sandbox$ tree -L 3 . ├── autobuilder ├── git ├── sandbox ├── yocto-autobuilder-helper ├── yocto-controller │   └── yoctoabb │   └── master.cfg └── yocto-worker └── buildbot.tac (sandbox) koan@amonra:~/ab2-sandbox$ tree -L 3 . ├── autobuilder ├── git ├── sandbox ├── yocto-autobuilder-helper ├── yocto-controller │   └── yoctoabb │   └── master.cfg └── yocto-worker └── buildbot.tac Ab2 * Simplified tree list
  • 27. 27 Yocto Project | The Linux Foundation Autobuilder2 installation • Complete installation instructions Ab2 http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder2/tree/README-Guide.md html ftp://ftp.koansoftware.com/public/talks/YoctoSummit-2019/au tobuilder2/Autobuilder2-README-Guide.pdf
  • 28. 28 Yocto Project | The Linux Foundation Autobuilder2 Autobuilder default configuration
  • 29. 29 Yocto Project | The Linux Foundation Autobuilder2 configuration • Ab2 default configuration • The default configuration of Ab2 uses a lot of workers to generate images for several MACHINES Ab2
  • 30. 30 Yocto Project | The Linux Foundation Autobuilder2 official website https://autobuilder.yoctoproject.org/typhoon/
  • 31. 31 Yocto Project | The Linux Foundation Autobuilder2 website navigation
  • 32. 32 Yocto Project | The Linux Foundation Autobuilder2 website navigation
  • 33. 33 Yocto Project | The Linux Foundation Autobuilder2 - builds
  • 34. 34 Yocto Project | The Linux Foundation Autobuilder2 - workers
  • 35. 35 Yocto Project | The Linux Foundation Autobuilder2 – worker details
  • 36. 36 Yocto Project | The Linux Foundation Autobuilder2 – build details
  • 37. 37 Yocto Project | The Linux Foundation Autobuilder2 – build details
  • 38. 38 Yocto Project | The Linux Foundation Autobuilder2 – build details
  • 39. 39 Yocto Project | The Linux Foundation Autobuilder2 – build details
  • 40. 40 Yocto Project | The Linux Foundation Autobuilder2 Autobuilder custom ‘lighter’ configuration
  • 41. 41 Yocto Project | The Linux Foundation Autobuilder2 reduced configuration • Reduce complexity • The goal is to setup a configuration for an image for a single MACHINE only • This will help yo strenghten the deployments thanks to recurring builds, tipically nightly Ab2
  • 42. 42 Yocto Project | The Linux Foundation Autobuilder2 reduced configuration • Files to be modified . ├── yocto-autobuilder-helper │   └── config.json ├── yocto-controller │   └── yoctoabb │   ├── builders.py │   ├── config.py │   ├── master.cfg │   └── schedulers.py └── yocto-worker └── buildbot.tac . ├── yocto-autobuilder-helper │   └── config.json ├── yocto-controller │   └── yoctoabb │   ├── builders.py │   ├── config.py │   ├── master.cfg │   └── schedulers.py └── yocto-worker └── buildbot.tac Ab2
  • 43. 43 Yocto Project | The Linux Foundation Autobuilder2 configuration • In yocto-autobuilder-helper • Edit yocto-autobuilder-helper/config.json • In yocto-controller • Edit yocto-controller/yoctoabb/master.cfg "BASE_HOMEDIR" : "/home/koan/ab2-sandbox", "BASE_SHAREDDIR" : "${BASE_HOMEDIR}/autobuilder", "BASE_PUBLISHDIR" : "${BASE_HOMEDIR}/downloads", "BASE_HOMEDIR" : "/home/koan/ab2-sandbox", "BASE_SHAREDDIR" : "${BASE_HOMEDIR}/autobuilder", "BASE_PUBLISHDIR" : "${BASE_HOMEDIR}/downloads", c['title'] = "KOAN lite Yocto Autobuilder" c['titleURL'] = "http://localhost:8010/" c['buildbotURL'] = "http://localhost:8010/" c['title'] = "KOAN lite Yocto Autobuilder" c['titleURL'] = "http://localhost:8010/" c['buildbotURL'] = "http://localhost:8010/" Ab2
  • 44. 44 Yocto Project | The Linux Foundation Autobuilder2 configuration • In yocto-controller (again) • Edit yocto-controller/yoctoabb/config.py • Specify the helper directory workers_koan = ["example-worker"] workers = workers_koan all_workers = workers, workers_koan = ["example-worker"] workers = workers_koan all_workers = workers, sharedrepodir = "/home/koan/ab2-sandbox/repos" publish_dest = "/home/koan/ab2-sandbox/pub" sharedrepodir = "/home/koan/ab2-sandbox/repos" publish_dest = "/home/koan/ab2-sandbox/pub" Ab2 repos = { "yocto-autobuilder-helper": ["file:///home/koan/ab2-sandbox/yocto-autobuilder-helper", "master"], repos = { "yocto-autobuilder-helper": ["file:///home/koan/ab2-sandbox/yocto-autobuilder-helper", "master"],
  • 45. 45 Yocto Project | The Linux Foundation Autobuilder2 ‘lite’ customized http://localhost:8010/
  • 46. 46 Yocto Project | The Linux Foundation Autobuilder2 ‘lite’ customized
  • 47. 47 Yocto Project | The Linux Foundation Autobuilder2 to speed up Yocto build • Share the artefacts with Yocto • Edit local.conf • Share downloads • Share SSTATE PREMIRRORS_prepend = " git://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n ftp://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n http://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n https://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n " PREMIRRORS_prepend = " git://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n ftp://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n http://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n https://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n " Ab2 SSTATE_MIRRORS = "file://.* file:///home/koan/ab2-sandbox/autobuilder/pub/sstate/PATH" SSTATE_MIRRORS = "file://.* file:///home/koan/ab2-sandbox/autobuilder/pub/sstate/PATH"
  • 48. 48 Yocto Project | The Linux Foundation Autobuilder2 to speed up Yocto build ‘master’ CONTROLLER ‘slave’ WORKER SSTATE image DL_DIR bitbake
  • 49. Yocto Project | The Linux Foundation Questions? https://koansoftware.comhttps://yoctoproject.org