T-Dose 2017 presentation, Sunday 19 November 2017. Adventures building a declarative pipeline script for a traditional (non-java/non-cloud) installable windows/Linux application. Video will hopefully be available later.
9. Scripted Pipeline example
node('test') {
// compute complete workspace path, from current node to the allocated disk
exws(extWorkspace) {
try {
// run tests in the same workspace that the project was built
sh 'mvn test'
} catch (e) {
// if any exception occurs, mark the build as failed
currentBuild.result = 'FAILURE'
throw e
} finally {
// perform workspace cleanup only if the build have passed
// if the build has failed, the workspace will be kept
cleanWs cleanWhenFailure: false
}
}
}
10. Declarative Pipeline example
pipeline {
agent any
stages {
stage('Example') {
steps {
echo 'Hello World'
}
}
}
post {
always {
echo 'I will always say Hello again!'
}
}
}
13. Many Simple Examples
✘ Lots of simple small fragments of declarative pipeline @ the web
○ Usually java/maven based
○ And very small fragments
✘ but..
✘ Pipelines get big and complex very quickly
✘ Does Declarative Pipeline scale?
✘ Not all plugins support a pipeline scripting interface..
17. ✘ Mixed teams (60 people)
○ SW developers
○ Mechanical / Math engineers
○ Matlab specialists
✘ Each team has one (or more)
tooling/cm dev´s
✘ Teams prior experience with build
servers is (was) Buildbot or just cron
jobs
✘ Team loves Python … not Java
CM team = One Man Army
✘ Simulation software
○ Requiring dedicated hardware
○ Simulations on raw iron
○ Dedicated test tooling
✘ Big:
○ SCM archives
○ Test sets
○ Test result sets
✘ Build and Test takes hours
18. ✘ A Dev worked on Scripted pipeline for
the first team moving to Jenkins
○ Kept bumping our heads
○ (Jenkins) software felt too
unstable for pipeline usage.
✘ Decided to go for graph mode
○ Got builds running the way we
wanted
○ Shorter learning curve for team
○ Use templates for re-usable
jobs
We did try Scripted Pipeline
… 2 years ago
✘ Number of jobs grew quickly
○ Lots of test jobs added
○ Need for running all tests on all
branches
✘ So the pipelines got Big:
○ Next slide...
20. Need for better solutions for growing pipelines
✘ Test & release team complains having difficulty getting overview
✘ Products still have monolithic long serial builds that take hours
○ Need for splitting up in smaller steps possibly on
dedicated slaves/executors
○ Having sub-products available as artefacts
✘ Scaling up test efforts requires many more automated test steps
✘ For me maintaining templates is getting cumbersome
○ 'Old' and 'new' templates crop up for enabling release branches
to be re-build next to current trunks
○ Changes to build templates not easily visible to teams
21. Eyeing Declarative Pipeline
✘ Spring 2017 Cloudbees announced 1.0 Pipeline
○ Nice overview of pipeline using Blue Ocean interface
○ Cloudbees is pushing this as the main build declaration
interface
✘ Simpler syntax seems to appeal to the dev teams
○ Storing the jenkinsFile in code repository makes jenkins
definition part of the code (branch)
✘ Phase 1 Proof Of Concept rebuilding the build for one team in
Pipeline
24. (Jenkins) Software used
✘ Jenkins Master 2.73 (Community Version)
✘ Pipeline Plugin 2.5
✘ Pipeline Declarative 1.2 (1.3.x just came out this week).
✘ Blue Ocean 2.3
26. Use case: Shorten build environment setup time
✘ The current (old) build system clones all 4 repo's one by one.
○ Setting up build env of 15 Gb took 30 to 40 minutes.
(250.000 files)
○ In my Jenkins 1.6 implementation I could save some time
○ Trying to do incremental builds when possible but clean
Builds are needed often.
✘ Using HG/Mercurial tricks and parallelization to the max
○ Turn on caching and sharing.
○ Start all clones parallel to each other (since HG is a single
threaded application.
✘ Brought full checkout time down do 5 mins.
27. Starting The Pipeline
#!/usr/bin/env groovy
pipeline {
// Agent definition for whole pipe. General vs14 Windows build slave.
agent {
label "Windows && x64 && ${GENERATOR}"
}
// General config for build, timestamps and console coloring.
options {
timestamps()
buildDiscarder(logRotator(daysToKeepStr:'90', artifactDaysToKeepStr:'20'))
}
// Poll SCM every 5 minutes every work day.
triggers {
pollSCM('H/5 * * * 1-5')
}
properties( buildDiscarder(logRotator(artifactDaysToKeepStr: '5', artifactNumToKeepStr: '', daysToKeepStr: '15', numToKeepStr: '')) ])
28. Environment Build Slave (2)
Pipeline {
// Set environment for all steps.
// Spawn all tools here also
environment {
GROOVY_HOME = tool name: 'Groovy-3.3.0', type: 'hudson.plugins.groovy.GroovyInstallation'
CMAKE_HOME = tool name: 'Cmake3.7', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
PYTHON_HOME = tool name: 'Python4.1', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
MATLAB_VERSION = "${MATLAB_R2013B_HOME}"
BuildArch = 'win64'
} Manual Says:
tools {
Maven 'apache-maven-3.0.1'
}
29. First Step Env Prep
Pipeline {
stages {
stage('Prepare Env') {
steps {
echo 'determine if last build was successful'
script {
if(!hudson.model.Result.SUCCESS.equals(
currentBuild.rawBuild.getPreviousBuild()?.getResult())) {
env.LAST_BUILD_FAILED = "true"
bat "echo previous build FAILED"
}
else {
env.LAST_BUILD_FAILED = "false"
bat "echo previous build SUCCESS"
}
}
echo 'Setting up build dirs..'
bat 'mkdir vislibrary_build n exit 0'
30. Parallel checkoutPipeline {
stages {
// Check out stage.. parallel checkout of all repo's.
stage('Check Out') {
parallel {
stage ("CloneCsRepo") {
steps {
echo 'Now let us check out C#Repo'
// Use sleep time preventing different HG threads grab same log file name
sleep 9
checkout changelog: true, poll: true, scm: [$class: 'MercurialSCM',
clean: true, credentialsId: '', installation: 'HG Multibranch',
source: "http://hg.wdm.local/hg/CsRepo/", subdir: 'CsRepo']
}
}
stage ("CloneCRepo") {
steps {
echo 'Now let us check out C Repo'
sleep 15
checkout changelog: true, poll: true,
Failed to parse ...Testing_Pipeline2_TMFC/builds/174/changelog2.xml: '<?xml version="1.0" encoding="UTF-8"?>
<changesets>
JENKINS-43176
32. Build Environment is setup
In the build pipeline the following is done:
✘ Tools installed
✘ Archives checked out
✘ Environment setup
33. Need some simple logic?
Pipeline {
stages {
stage ('name') {
step {
script {
switch(GENERATOR) {
case "vs9sp1":
MATLAB_VERSION = "${MATLAB_R2010BSP2_HOME}"
break;
case "vs11":
MATLAB_VERSION = "${MATLAB_R2010BSP2_HOME}"
break;
case "vs14":
MATLAB_VERSION = "${MATLAB_R2013B_HOME}"
break;
}
bat "Echo MATLAB_VERSION = ${MATLAB_VERSION}"
}
34. Use case2: Don't build C-repo if not needed
✘ The current (old) build system (incremental) builds all even if only
one repo is changed.
○ A common case is only changes in C# repo
○ Cannot rely on correct source change detection in building C
repo
○ Not executing a not needed C Repo build step saves 20 minutes
37. Detect changes in repo's
Pipeline {
stages {
stage('Cmake Gen') {
when {
anyOf {
// Is CRepo already build?
environment name: 'CRepo_IS_BUILD', value: 'false' //no..
// Did the previous build fail?
environment name: 'LAST_BUILD_FAILED', value: 'true' //yes..
// CRepo changes
environment name: 'CRepo_IS_CHANGED', value: 'true' //yes..
}
}
steps {
echo 'Do incremental clean for C Repo'
38. Use case3: Have All tests in one job run
✘ Quite a number of tests require dedicated hardware servers
○ Doing traditional jenkins, these are all separate jobs
○ Number of tests on different slaves will increase significantly
○ All tests will have to run on all dev/stage branches
○ Number of jobs will explode doing this traditional Jenkins way
○ Developers testers should get precise feedback for each branch
44. General Problems Annoyances
Using pipeline / Blue Ocean
✘ Slowness / crash browser on large log files
✘ Snippet generator generates Declarative pipeline and/or scripted
○ Two lists of 'steps' in generator
○ Some generated scripts do not work anymore
✘ Completeness of documentation
48. Conclusion
✘ Declarative pipeline is a much needed extension to Jenkins
✘ Functionality is stable
✘ Documentation spotty
✘ Blue Ocean interface is a great improvement
○ General slowness is worrying
✘ Cloudbees is continuing development
○ Approx every 2 to 3 months new functionality is released
✘ We should give CloudBees credit for releasing all this to the
community
50. Credits
Special thanks to all the people who made and released
these awesome resources for free:
✘ Presentation template by SlidesCarnival
✘ Photographs by Unsplash
Editor's Notes
Serialization
Try catch
Java / Groovy specifics.
Serialization
Try catch
Java / Groovy specifics.
Obviscated
Based on a real pipeline.
Tool version numbers.. names might be wrong
File system is virtual
Not added to Path!!
Elevation!
1.2 syntax
1.2 syntax
13 Gb vs 26Gb
Direct manipulation of env vars.
No external groovy script running.
1.2 pipeline
Announced at Jw 2017
Does not work with paralell.
More efficient than making zip's your self
No unwanted artefacts anymore.