LCA14: LCA14-203: Introduction to CI @ Linaro 1of 3


Published on

Resource: LCA14
Name: LCA14-203: Introduction to CI @ Linaro 1of 3
Date: 04-03-2014
Speaker: F.Boudra, M.Wasilewski

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

LCA14: LCA14-203: Introduction to CI @ Linaro 1of 3

  1. 1. Tue-4-Mar, 10:05am, F.Boudra, M.Wasilewski LCA14-203: Intro to CI@Linaro 1 of 3
  2. 2. Hour 1 ● QA Services will introduce you to existing Linaro Test Plans ● Builds and Baselines will provide an overview to our build system, how to create, manage and close out a build and how to submit the build artifacts to LAVA for a test on lab hardware. Hour 2 ● LAVA will describe the process of adding new test capabilities (New Platforms and/or New external test equipment) ● QA Services will overview how they use LAVA for tests Hour 3 ● LAVA will show you new features to improve test result filtering and Visualization in LAVA ● QA Services will show you how Linaro rolls all the Build results, Test Results (Manual / LAVA) into our test reports Intro to CI @ Linaro
  3. 3. • Engineering builds • Android • OpenEmbedded • Ubuntu • LSK • Working groups • LEG • LNG • Requesting new Test Plan • Contact QA team (Milosz Wasilewski) • File REQ card • Creating new test plans will be covered on Thursday session Existing Test Plans
  4. 4. • Some test plans are not public (like LSK) • Sharing to selected people isn’t easy as it implies google account • Turning ‘paper’ test plan into JSON test definition is done manually by editing Jenkins configs Existing Test Plans - problems
  5. 5. • Jenkins as the build system • for Android builds • for everything else • individual components • acpica, libvirt QEMU, UEFI, etc... • Ubuntu and OpenEmbedded based engineering builds • hardware packs, rootfs, pre-built images • Kernel CI • mainline, ARM SoC (for-next), linux-next, linux-stable • LSK, Linux Linaro, LEG, LNG • LAVA as the automated testing system •, Linaro’s production server • running LAVA jobs on Cambridge lab hardware, restricted to Linaro •, Linaro’s Community server • running LAVA jobs on community hardware, open to everybody Anatomy of CI loop
  6. 6. • Select a view • Click “New job” button • the new job is created on the selected view How to create a build job
  7. 7. • Define a job name • avoid names with spaces, it’s used in the URL • Select the build job type • free-style project • Multijob project • Multi-configuration project • Copy an existing job How to create a build job
  8. 8. • Configure the general options • discard old builds • days to keep build • max number of builds to keep • enabled project based security • users/groups from and “anonymous” account • permissions: configure, read, extended read, build, worskpace • this build is parametrized • file parameter (properties), password parameter • restrict where this project can be run • label expression • various builders (ephemeral vs persistent, regular user vs sudoer) • Amazon EC2 instances • VPS instances • native builders How to create a build job
  9. 9. • Configure the advanced project options • use custom workspace (useful for multi-configuration project) • directory for sub-builds “.” • Configure source code management • git plugin • repository URL • advanced -> refspec • branch specifier • additional behavior • advanced clone behaviours -> shallow clone • check out to a sub-directory • clean after checkout • wipe out repository and force clone How to create a build job
  10. 10. • Configure build triggers • trigger builds remotely (e.g. git hook) • build periodically (cron syntax) • gerrit event • URL trigger (useful in particular use cases) • polling, should be run on persistent slave! • Configure configuration matrix (only for multi- configuration project) • add axis • dynamic axis (e.g. defconfig list) • slaves • user-defined (e.g. custom string for gcc version, rootfs, etc...) • run each configuration sequentially • execute touchstones build first How to create a build job
  11. 11. • Configure build environment • abort the build if it’s stuck • add timestamps to the console output • copy files into the job’s workspace (e.g. Linaro GCC toolchain) • set build name (e.g. GIT_REVISION, require to use git plugin) • tie parent to a build node (only for multi-configuration) • Configure build - 1/3 • “execute shell” • run shell script, written inline or pulled from SCM • can use regular git cli instead of Jenkins git plugin • some environment variables are defined by Jenkins (axis too) • can use Linaro CI build tools to build kernels • can use Linaro jenkins setup scripts to build OpenEmbedded images • generate configuration(s) to submit LAVA job (post-build-lava CI helper) • generate BUILD-INFO.txt (required for publishing on snapshots) • define ACLs How to create a build job
  12. 12. • Configure build - 2/3 • “send files or execute command over ssh” to publish build artifacts on • it’s a two step process, transfer files then move files • SSH server: ■ define “source files” ■ define “remote directory”: ${JOB_NAME} /${BUILD_NUMBER}/path/to/artifact • add server -> SSH server: file-move ■ exec command: “reshuffle-files --job-type prebuilt --job- name ${JOB_NAME} --build-num ${BUILD_NUMBER}” • advanced -> select “always SSH from master” How to create a build job
  13. 13. • Configure build - 3/3 • trigger/call builds on other projects • ubuntu-builder, CI helper used to cross build debian packages • ubuntu-builder-native, CI helper used to build naively debian packages • post-build-repo, CI helper used to upload packages to Linaro repository • post-build-lava, CI helper used to submit a LAVA job (JSON job definition) • support fully custom job definition (CUSTOM_JSON_URL) • support pre-built image (IMAGE_URL) • support custom linaro_deploy_image parameters (DEPLOY_IMAGE_PARAMETERS_URL) • support custom metadata (METADATA_URL) • support custom boot image parameters (BOOT_IMAGE_PARAMETERS_URL) • partial support for multi-node jobs How to create a build job
  14. 14. • Configure post-build actions • scan for compiler warnings (GCC, LLVM, doxygen, etc...) • groovy postbuild • powerful groovy scripting, access to Jenkins API (e.g. used to add a direct link to LAVA job id, or rename a build job dynamically) • Jenkins text finder (use regexp to find pattern) • e-mail notification How to create a build job
  15. 15. • Improve the publishing API • Get latest rootfs/hwpack using API • Support Jenkins CLI • Jenkins job descriptions in human readable format, maintained under SCM (Jenkins job builder) Improvements
  16. 16. Any Questions? Q&A
  17. 17. More about Linaro Connect: More about Linaro: More about Linaro engineering: Linaro members: