Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Jenkins Automation
Julien Pivotto (@roidelapluie)
Open Source Data Center Conference
Berlin
whoami
Julien "roidelapluie" Pivotto
@roidelapluie
Sysadmin at inuits
Automation, monitoring, HA
Jenkins user for 5+ years
inuits
Jenkins in 2 lines
Open Source
Pluggable
The Jenkins Project
Java
Hudson (2005)
Jenkins (2011)
The evolution of Jenkins
Jenkins is moving fast (there is competition:
travis-ci, gitlab-runners)
Far away from just the j...
What can you do with Jenkins?
Testing, building, deploying
Software
Services
Infrastructure
Mission Critical
Public Domain https://commons.wikimedia.org/wiki/File:Roaddamage59quake.JPG
Without Jenkins...
How do I build this?
Where do I deploy this?
Who knows where to upload this?
When did we deploy this?
Automating Jenkins
Creative Commons Attribution-ShareAlike 2.0 https://www.flickr.com/photos/machu/3131057286
Automating Jenkins
Why is it needed?
Make build processes transparent
Make build processes improvable
Make build process r...
Automating Jenkins
Why is it hard?
Building Software is hard
For long, Jenkins has been strongly UI-Driven
It is "easy" to...
XML everywhere!
Creative Commons Attribution 2.0 https://www.flickr.com/photos/generated/6035308729
What can you automate in
Jenkins?
1. The Operating System Jenkins is
running on
The platform/node/host/.. which runs your
Jenkins damon
2. The Jenkins Service
The start/stop/reload of Jenkins
JAVA_OPTIONS
3. Plugins
Which plugins to install
Which version
Which configuration
4. Jenkins Global Configuration
SMTP Server
Email address
Authentication/Authorization
5. Jobs - Views - Folder
Views - That really help people
Jobs and folders - Creation AND deletion,
6. Jobs definition
Jobs content
Build steps
Deployments configuration
7. Operating system you build your
software on
Jenkins nodes
Where your jobs run
(Linux, Android, ...)
Automating the OS
Creative Commons Attribution-Share Alike 2.0 https://www.flickr.com/photos/timdorr/168925884
Jenkins deserves attention
Jenkins Server should be automated
Receive OS updates
Be monitored like any other server
Automating the service
CC0 Public Domain https://pixabay.com/en/butler-tray-beverages-wine-964007/
Installing Jenkins
Do NOT use war file (yes I know it's easy)
Use Jenkins LTS (3 months lifecycle vs 1
week)
Think about: ...
Automating the Jenkins Service
Chef: Jenkins in the supermarket
Puppet: module rtyler/jenkins
Playbooks/etc for other tool...
Jenkins Plugins
You NEED them (never seen a Jenkins setup
without 20-100 plugins)
Fetched from https://updates.jenkins-ci....
Packaging Jenkins Plugins
Plugins have dependencies (against plugins &
Jenkins core)
They have a fixed download path
They ...
Mirroring Jenkins Plugins
Mirror http://updates.jenkins-ci.org
By default, "latest" will be fetched
Don't cache too much
g...
Global configuration
CC0 Public Domain https://commons.wikimedia.org/wiki/File:Waiting_room_bell.jpg
Modifying XML files
DON'T
system-config-dsl-plugin
Like Job DSL, but for the system
https://github.com/jenkinsci/system-config-
dsl-plugin
JENKINS-3...
Creative Commons Attribution-Share Alike 3.0
https://commons.wikimedia.org/wiki/File:Groovy-logo.svg
Groovy
Yet another language to learn? yes.
Programming language for the Java platform
Scripting language
Fully integrated ...
The Jenkins Script Console
A groovy console is available at /script
http://jenkins.example.com/script
also with curl
Requi...
Sample Groovy scripts
Creative Commons Attribution-Share Alike 2.0
https://www.flickr.com/photos/bjornb/88101376
Set number of executors
import jenkins.model.Jenkins;
Jenkins.instance.setNumExecutors(0)
Set HTML formatter
import jenkins.model.Jenkins;
import hudson.markup.RawHtmlMarkupFormatter;
Jenkins.instance.setMarkupFo...
Set system message
import jenkins.model.Jenkins;
Jenkins.instance.setSystemMessage("""
Welcome to <em>Jenkins</em>.
""");
Configure simple-theme
import jenkins.model.Jenkins;
def simpleThemePlugin = 
  Jenkins.instance.getDescriptor(
  "org.cod...
Email
import jenkins.model.Jenkins
def desc = Jenkins.instance.getDescriptor(
  "hudson.tasks.Mailer")
desc.setSmtpHost("1...
Disable usage statistics
import hudson.model.UsageStatistics;
hudson.model.UsageStatistics.DISABLED = true;
But also...
Setup Authorization/Authentication
Setup prefix url
Setup slaves, clouds
Setup global libraries
Setup credenti...
Bonus
import org.jenkinsci.plugins.pipeline.utility
.steps.shaded.org.yaml.snakeyaml.Yaml
Yaml reader = new Yaml()
Map con...
Scripts sources
https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+Script+Console
https://github.com/jenkinsci/jenkins-sc...
Jenkins Javadocs
https://jenkins.io/doc/developer/guides/
http://javadoc.jenkins.io/archive/jenkins-
2.32/
http://javadoc....
Now what?
Creative Commons Attribution 2.0 https://www.flickr.com/photos/51029297@N00/5275403364
Jenkins init.groovy.d
Upon startup, Jenkins will run
 $JENKINS_HOME/init.groovy.d/*.groovy 
scripts
Meanwhile, "Jenkins is...
init.groovy.d directory
behaviour
Scripts executed sequentally (alphanum
order)
Scripts that throws exceptions make startu...
Inside Jenkins
Creative Commons Attribution-ShareAlike 4.0 International
https://commons.wikimedia.org/wiki/File:Mystery_C...
The Multiple Approaches
GUI .. but this talk is about automation, right?
init.groovy.d: to create your seed job
Jenkins Jo...
Jenkins Job Builder
An Openstack Project
Python (not a Jenkins Plugin!)
Support templates
Extensible
Can do raw xml
Limite...
Jenkins Job DSL
A Jenkins Plugin
2012
Groovy DSL to create views & jobs
Put Jobs config under SCM
init.groovy.d
def jobManagement = new JenkinsJobManagement(
  System.out, [:], new File('.'))
new DslScriptLoader(jobManag...
Creative Commons Attribution-Share Alike 3.0
https://commons.wikimedia.org/wiki/File:Groovy-logo.svg
Job DSL support
Large community of users
Lots of plugins supported
Create a job
job('test') {
 scm {
  git('git://example.com/foo.git' 'master')
 }
 steps {
  shell('make test')
 }
}
Create a Pipeline job
pipelineJob('test'){
 definition {
   cps {
     script(buildScript)
   }
 }
}
Set job properties
pipelineJob('test'){
 description('Test Build')
 parameters {
  booleanParam('verbose', false, 'Be Verb...
Loops
config.each(){ jobConfig ­>
  pipelineJob(jobConfig.name) {
    triggers {
      if (jobConfig.triggers?.scm) {
    ...
Create a view
listView('Project A') {
    recurse()
    jobs {
        regex('myprj/[^/]+')
    }
    columns {
        st...
Work with plugins
buildMonitorView("OnScreen Status") {
 jobs {
  name('WatchA')
 }
}
${JENKINS_URL}/plugin/job-dsl/api-vi...
In Pipeline
jobDsl removedJobAction: 'DELETE',
  removedViewAction: 'DELETE',
  targets: 'seeds/*.groovy'
Unleash the power of groovy
import jenkins.model.Jenkins;
dslCfg = Jenkins.instance.getDescriptor(
  "javaposse.jobdsl.plu...
Load extra libraries
jobDsl additionalClasspath: 'src/yaml.jar',
  removedJobAction: 'DELETE',
  removedViewAction: 'DELET...
Read yaml
import org.yaml.snakeyaml.Yaml
Yaml reader = new Yaml()
Map config = reader.load(
  readFileFromWorkspace('cfg.y...
Jenkins Pipeline
Creative Commons Attribution 2.0 https://www.flickr.com/photos/amerune/9294639633
Jenkins Pipeline
Jenkins Jobs as Code
"Jenkins file"
Imperative (aka scripted) Pipeline
Declarative Pipeline (2017)
What is a Jenkinsfile (aka
Pipeline)
A file that contains the definition of a job
No need of Gui
Defines Steps, Reports, E...
How to write Pipelines?
Visual Pipeline editor (WIP)
"Pipeline Syntax" link in jobs
Scriped pipeline
node ("Ubuntu && amd64") {
  stage('checkout') {
    checkout scm
  }
  stage('build') {
    sh 'make'
  ...
Real World Scripted Pipeline
node ("Ubuntu && amd64") {
  checkout scm
  stage('build') {
    try {
      sh 'make'
    } ...
Declarative Pipeline (1/2)
pipeline {
  agent {
      label("Ubuntu && amd64")
  }
  options {
      timeout(time: 235, un...
Declarative Pipeline (2/2)
pipeline {
 post {
  always {
   junit params.jUnitReports
  }
 }
}
Declarative vs scripted
Declarative checkout code by default
Declarative get a workspace "OOTB"
Declarative enforces every...
Going further with Pipeline
Global Libraries Plugins
Job DSL Plugin
Jenkins Nodes
CC0 Public Domain https://www.flickr.com/photos/133259498@N05/27077266322/
Docker Docker Docker
Run jobs inside containers
Clean, short lived containers
Easy to update
Docker Plugin
Docker nodes pattern
Build Container
Tag it with a tag "candidate"
Push it to your registry
Run normal testing
Run actual ...
In practice
Docker Plugin config automated with groovy
Candidate and Release tags are setup as
slaves
They get two labels:...
In the build Jenkinsfile
pipeline {
 agent {
  label("build­centos­7 && ${params.tag}")
 }
}
In the docker-build Jenkinsfile
dockerImage = docker.build("build­centos­7",
  "­­no­cache")
dockerImage.tag('candidate')
...
Pros/Cons
Updated containers won't block the builds (e.g
on packages updates)
Containers stay up to date
Don't forget that...
Docker Moby Docker
Jenkins Master in Docker
WHY WOULD YOU DO THAT?
Atomic upgrade of all plugins at the same
time
Easy rollback / update
Easy...
Jenkins master in Docker
At which price?????
Do not run ANYTHING on master; slave
everything
Keep a Docker Registry with h...
Docker and Jenkins
Jenkins upstream images: alpine/ubuntu
LTS and weekly
Dockerfile
FROM jenkins:2.46.2­alpine
MAINTAINER dev­team@example.com
USER root
COPY jenkins/userContent/ 
  /var/lib/jenk...
Timezone (if needed)
yay alpine
RUN apk add ­­no­cache tzdata && 
  cp /usr/share/zoneinfo/Europe/Brussels 
    /etc/local...
Install plugins
USER jenkins
COPY jenkins/plugins.txt /tmp/plugins.txt
RUN cat /tmp/plugins.txt | xargs 
 /usr/local/bin/i...
init.groovy.d
COPY jenkins/init.groovy.d/ 
  /usr/share/jenkins/ref/init.groovy.d/
Note: if you keep Jenkins config in a v...
JAVA_OPTS
ENV JAVA_OPTS="­Xmx8192m 
  ­Djenkins.install.runSetupWizard=false"
ENV JENKINS_OPTS="­­handlerCountMax=300 
  ­...
Testing the image
Jenkins Pipeline
init.groovy.d switch:
if (System.getenv()['DO_NOT_BUILD']){
    Jenkins.instance.doQuie...
Block until started
That script will fail until Jenkins is started:
#!/bin/bash ­xe
curl 127.0.0.1:8080/jenkins2/ |
grep '...
Testing init.groovy.d
set ­xe
if grep "SEVERE: Failed GroovyInitScript" 
   /var/log/jenkins/jenkins.log ||
  grep "WARNIN...
Seed jobs
Somewhere in init.groovy.d
cause = new RemoteCause('localhost',
 'Build triggered by init.groovy.d')
job = Jenki...
Benefits
A Pipeline to Deploy Jenkins
Automated checks of init.groovy.d etc
Check that the seed job works
Going further
Will the fun ever stop?
Trash the Jenkins config
Only keep build history
Do notput config in persistent volume
Let init.groovy.d/job dsl build you...
The glue
import jenkins.model.Jenkins;
import static groovy.io.FileType.FILES
def rawBuildsBaseDir = '/srv/jenkins/state'
...
Conclusion
Public Domain https://commons.wikimedia.org/wiki/File:YellowOnions.jpg
Jenkins can be FULLY
automated
My recommendations:
init.groovy.d
Jenkins Job DSL
Pipeline
Try dockerized Jenkins ; might w...
Julien Pivotto
roidelapluie
roidelapluie@inuits.eu
Inuits
https://inuits.eu
info@inuits.eu
Contact
Jenkins Automation
Upcoming SlideShare
Loading in …5
×

Jenkins Automation

2,409 views

Published on

State of the Jenkins automation ; presented at Open Source Datacenter Conference in Berlin in 2017.

Published in: Technology

Jenkins Automation

  1. 1. Jenkins Automation Julien Pivotto (@roidelapluie) Open Source Data Center Conference Berlin
  2. 2. whoami Julien "roidelapluie" Pivotto @roidelapluie Sysadmin at inuits Automation, monitoring, HA Jenkins user for 5+ years
  3. 3. inuits
  4. 4. Jenkins in 2 lines Open Source Pluggable
  5. 5. The Jenkins Project Java Hudson (2005) Jenkins (2011)
  6. 6. The evolution of Jenkins Jenkins is moving fast (there is competition: travis-ci, gitlab-runners) Far away from just the java world (e.g. mvn specifics are gone in Pipeline world)
  7. 7. What can you do with Jenkins? Testing, building, deploying Software Services Infrastructure
  8. 8. Mission Critical Public Domain https://commons.wikimedia.org/wiki/File:Roaddamage59quake.JPG
  9. 9. Without Jenkins... How do I build this? Where do I deploy this? Who knows where to upload this? When did we deploy this?
  10. 10. Automating Jenkins Creative Commons Attribution-ShareAlike 2.0 https://www.flickr.com/photos/machu/3131057286
  11. 11. Automating Jenkins Why is it needed? Make build processes transparent Make build processes improvable Make build process reproducible Lower the barrier to update Jenkins security - audit - bugfixes - plugins
  12. 12. Automating Jenkins Why is it hard? Building Software is hard For long, Jenkins has been strongly UI-Driven It is "easy" to deploy (.war file) Hey I've deployed Jenkins on my Laptop! It has lots of plugins (and you need them)
  13. 13. XML everywhere! Creative Commons Attribution 2.0 https://www.flickr.com/photos/generated/6035308729
  14. 14. What can you automate in Jenkins?
  15. 15. 1. The Operating System Jenkins is running on The platform/node/host/.. which runs your Jenkins damon
  16. 16. 2. The Jenkins Service The start/stop/reload of Jenkins JAVA_OPTIONS
  17. 17. 3. Plugins Which plugins to install Which version Which configuration
  18. 18. 4. Jenkins Global Configuration SMTP Server Email address Authentication/Authorization
  19. 19. 5. Jobs - Views - Folder Views - That really help people Jobs and folders - Creation AND deletion,
  20. 20. 6. Jobs definition Jobs content Build steps Deployments configuration
  21. 21. 7. Operating system you build your software on Jenkins nodes Where your jobs run (Linux, Android, ...)
  22. 22. Automating the OS Creative Commons Attribution-Share Alike 2.0 https://www.flickr.com/photos/timdorr/168925884
  23. 23. Jenkins deserves attention Jenkins Server should be automated Receive OS updates Be monitored like any other server
  24. 24. Automating the service CC0 Public Domain https://pixabay.com/en/butler-tray-beverages-wine-964007/
  25. 25. Installing Jenkins Do NOT use war file (yes I know it's easy) Use Jenkins LTS (3 months lifecycle vs 1 week) Think about: user, directories, backups,... Packages provides: upgrade path, downgrade path, control, checksums, signatures, ...
  26. 26. Automating the Jenkins Service Chef: Jenkins in the supermarket Puppet: module rtyler/jenkins Playbooks/etc for other tools as well Docker
  27. 27. Jenkins Plugins You NEED them (never seen a Jenkins setup without 20-100 plugins) Fetched from https://updates.jenkins-ci.org Can be installed from the UI :( Can be installed from the CLI
  28. 28. Packaging Jenkins Plugins Plugins have dependencies (against plugins & Jenkins core) They have a fixed download path They are listed in updates.jenkins.io/update- center.json https://github.com/roidelapluie/Jenkins- Plugins-RPM
  29. 29. Mirroring Jenkins Plugins Mirror http://updates.jenkins-ci.org By default, "latest" will be fetched Don't cache too much github.com/jenkinsci/docker install- plugins.sh
  30. 30. Global configuration CC0 Public Domain https://commons.wikimedia.org/wiki/File:Waiting_room_bell.jpg
  31. 31. Modifying XML files DON'T
  32. 32. system-config-dsl-plugin Like Job DSL, but for the system https://github.com/jenkinsci/system-config- dsl-plugin JENKINS-31094 Downside: it does not exist yet
  33. 33. Creative Commons Attribution-Share Alike 3.0 https://commons.wikimedia.org/wiki/File:Groovy-logo.svg
  34. 34. Groovy Yet another language to learn? yes. Programming language for the Java platform Scripting language Fully integrated in Jenkins Used by automation tools (chef cookbook, puppet module...)
  35. 35. The Jenkins Script Console A groovy console is available at /script http://jenkins.example.com/script also with curl Requires "Overall/Run Scripts" permission
  36. 36. Sample Groovy scripts Creative Commons Attribution-Share Alike 2.0 https://www.flickr.com/photos/bjornb/88101376
  37. 37. Set number of executors import jenkins.model.Jenkins; Jenkins.instance.setNumExecutors(0)
  38. 38. Set HTML formatter import jenkins.model.Jenkins; import hudson.markup.RawHtmlMarkupFormatter; Jenkins.instance.setMarkupFormatter( new RawHtmlMarkupFormatter(false));
  39. 39. Set system message import jenkins.model.Jenkins; Jenkins.instance.setSystemMessage(""" Welcome to <em>Jenkins</em>. """);
  40. 40. Configure simple-theme import jenkins.model.Jenkins; def simpleThemePlugin =    Jenkins.instance.getDescriptor(   "org.codefirst.SimpleThemeDecorator")    simpleThemePlugin.cssUrl =    "/userContent/example.css" simpleThemePlugin.save() Everything in $JENKINS_HOME/userContent is served under /userContent
  41. 41. Email import jenkins.model.Jenkins def desc = Jenkins.instance.getDescriptor(   "hudson.tasks.Mailer") desc.setSmtpHost("172.17.0.1") desc.save()
  42. 42. Disable usage statistics import hudson.model.UsageStatistics; hudson.model.UsageStatistics.DISABLED = true;
  43. 43. But also... Setup Authorization/Authentication Setup prefix url Setup slaves, clouds Setup global libraries Setup credentials Setup first jobs
  44. 44. Bonus import org.jenkinsci.plugins.pipeline.utility .steps.shaded.org.yaml.snakeyaml.Yaml Yaml reader = new Yaml() Map config = reader.load(text) Thanks to pipeline utility step (readYaml() step), you can easily read yaml files
  45. 45. Scripts sources https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+Script+Console https://github.com/jenkinsci/jenkins-scripts/tree/master/scriptler https://github.com/infOpen/ansible-role-jenkins/tree/master/files/groovy_scripts https://github.com/samrocketman/jenkins-bootstrap-jervis/tree/master/scripts https://github.com/jenkinsci/puppet- jenkins/blob/master/files/puppet_helper.groovy http://pghalliday.com/jenkins/groovy/sonar/chef/configuration/management/2014 /09/21/some-useful-jenkins-groovy-scripts.html
  46. 46. Jenkins Javadocs https://jenkins.io/doc/developer/guides/ http://javadoc.jenkins.io/archive/jenkins- 2.32/ http://javadoc.jenkins.io/ http://javadoc.jenkins.io/plugin/gerrit-trigger/
  47. 47. Now what? Creative Commons Attribution 2.0 https://www.flickr.com/photos/51029297@N00/5275403364
  48. 48. Jenkins init.groovy.d Upon startup, Jenkins will run  $JENKINS_HOME/init.groovy.d/*.groovy  scripts Meanwhile, "Jenkins is getting ready..." message is displayed Drop files, restart Jenkins Allows you to preconfigure everything -- without the GUI
  49. 49. init.groovy.d directory behaviour Scripts executed sequentally (alphanum order) Scripts that throws exceptions make startup fail
  50. 50. Inside Jenkins Creative Commons Attribution-ShareAlike 4.0 International https://commons.wikimedia.org/wiki/File:Mystery_Cave_passage.jpg
  51. 51. The Multiple Approaches GUI .. but this talk is about automation, right? init.groovy.d: to create your seed job Jenkins Job Builder: declarative, python, yaml Jenkins Job DSL: imperative, groovy
  52. 52. Jenkins Job Builder An Openstack Project Python (not a Jenkins Plugin!) Support templates Extensible Can do raw xml Limited support for plugins and pipeline Put Jobs config under SCM
  53. 53. Jenkins Job DSL A Jenkins Plugin 2012 Groovy DSL to create views & jobs Put Jobs config under SCM
  54. 54. init.groovy.d def jobManagement = new JenkinsJobManagement(   System.out, [:], new File('.')) new DslScriptLoader(jobManagement).runScript(""" folder('jenkins') {     displayName('Jenkins') } pipelineJob("jenkins/seed"){   definition {         cpsScm {             scm {                 git {                     remote {                         credentials('jenkins­git­na')                         url('git.example.com/seed.git')                     }                     branches('master')                 }             }             scriptPath('Jenkinsfile')         }     } } """)
  55. 55. Creative Commons Attribution-Share Alike 3.0 https://commons.wikimedia.org/wiki/File:Groovy-logo.svg
  56. 56. Job DSL support Large community of users Lots of plugins supported
  57. 57. Create a job job('test') {  scm {   git('git://example.com/foo.git' 'master')  }  steps {   shell('make test')  } }
  58. 58. Create a Pipeline job pipelineJob('test'){  definition {    cps {      script(buildScript)    }  } }
  59. 59. Set job properties pipelineJob('test'){  description('Test Build')  parameters {   booleanParam('verbose', false, 'Be Verbose')  }  logRotator {   numToKeep(10)  } }
  60. 60. Loops config.each(){ jobConfig ­>   pipelineJob(jobConfig.name) {     triggers {       if (jobConfig.triggers?.scm) {         scm(jobConfig.triggers.scm)       }     }   } }
  61. 61. Create a view listView('Project A') {     recurse()     jobs {         regex('myprj/[^/]+')     }     columns {         status()         weather()         name()         lastSuccess()         lastFailure()         lastDuration()         lastBuildConsole()         buildButton()         progressBar()     } }
  62. 62. Work with plugins buildMonitorView("OnScreen Status") {  jobs {   name('WatchA')  } } ${JENKINS_URL}/plugin/job-dsl/api-viewer/
  63. 63. In Pipeline jobDsl removedJobAction: 'DELETE',   removedViewAction: 'DELETE',   targets: 'seeds/*.groovy'
  64. 64. Unleash the power of groovy import jenkins.model.Jenkins; dslCfg = Jenkins.instance.getDescriptor(   "javaposse.jobdsl.plugin.    GlobalJobDslSecurityConfiguration") dslCfg.useScriptSecurity = false == (sec issue with old versions of the plugin) (or power user trick)
  65. 65. Load extra libraries jobDsl additionalClasspath: 'src/yaml.jar',   removedJobAction: 'DELETE',   removedViewAction: 'DELETE',   targets: 'seeds/*.groovy'
  66. 66. Read yaml import org.yaml.snakeyaml.Yaml Yaml reader = new Yaml() Map config = reader.load(   readFileFromWorkspace('cfg.yaml') Drop the snakeyaml jar file and add it to classpath
  67. 67. Jenkins Pipeline Creative Commons Attribution 2.0 https://www.flickr.com/photos/amerune/9294639633
  68. 68. Jenkins Pipeline Jenkins Jobs as Code "Jenkins file" Imperative (aka scripted) Pipeline Declarative Pipeline (2017)
  69. 69. What is a Jenkinsfile (aka Pipeline) A file that contains the definition of a job No need of Gui Defines Steps, Reports, Environments, Nodes,... Plugins can provide steps Generic "step"
  70. 70. How to write Pipelines? Visual Pipeline editor (WIP) "Pipeline Syntax" link in jobs
  71. 71. Scriped pipeline node ("Ubuntu && amd64") {   stage('checkout') {     checkout scm   }   stage('build') {     sh 'make'   }   stage('test') {     sh 'make test'   } }
  72. 72. Real World Scripted Pipeline node ("Ubuntu && amd64") {   checkout scm   stage('build') {     try {       sh 'make'     } catch(err) {       currentBuild.result = 'UNSTABLE'       publishArtifacts('err.log')       throw err     }   }   stage('test') {     sh 'make test'   }   step([$class: 'JavadocArchiver',   javadocDir: "target/site", keepAll: true]) }
  73. 73. Declarative Pipeline (1/2) pipeline {   agent {       label("Ubuntu && amd64")   }   options {       timeout(time: 235, unit: 'MINUTES')       timestamps()       ansiColor('xterm')   }   stages {     stage('build') {       steps {           sh('make')       }    } }
  74. 74. Declarative Pipeline (2/2) pipeline {  post {   always {    junit params.jUnitReports   }  } }
  75. 75. Declarative vs scripted Declarative checkout code by default Declarative get a workspace "OOTB" Declarative enforces everything in stages Declarative allow Pipeline-wide wrappers (e.g ansiColor, timestamps)
  76. 76. Going further with Pipeline Global Libraries Plugins Job DSL Plugin
  77. 77. Jenkins Nodes CC0 Public Domain https://www.flickr.com/photos/133259498@N05/27077266322/
  78. 78. Docker Docker Docker Run jobs inside containers Clean, short lived containers Easy to update Docker Plugin
  79. 79. Docker nodes pattern Build Container Tag it with a tag "candidate" Push it to your registry Run normal testing Run actual builds with that "candidate" If success -> tag with "release" && push
  80. 80. In practice Docker Plugin config automated with groovy Candidate and Release tags are setup as slaves They get two labels: "image" "tag" (e.g. "build- centos-7" "candidate") Jobs get a parameter "tag"
  81. 81. In the build Jenkinsfile pipeline {  agent {   label("build­centos­7 && ${params.tag}")  } }
  82. 82. In the docker-build Jenkinsfile dockerImage = docker.build("build­centos­7",   "­­no­cache") dockerImage.tag('candidate') docker.withRegistry(url, credentials) {   dockerImage.push('candidate') } node("build­centos­7 && candidate") {   sh('test­script') } build job: 'myjob', parameters:   [string(name:'tag', value:'candidate')]    dockerImage.tag('release') docker.withRegistry(url, credentials) {   dockerImage.push('release') }
  83. 83. Pros/Cons Updated containers won't block the builds (e.g on packages updates) Containers stay up to date Don't forget that builds release artifacts (sometime you don't want that in nodes tests)
  84. 84. Docker Moby Docker
  85. 85. Jenkins Master in Docker WHY WOULD YOU DO THAT? Atomic upgrade of all plugins at the same time Easy rollback / update Easy to test Reduce the cost to upgrade
  86. 86. Jenkins master in Docker At which price????? Do not run ANYTHING on master; slave everything Keep a Docker Registry with history Assume Docker instability Think docker deployment etc... (A pipeline for your Jenkins master - how cool is this?)
  87. 87. Docker and Jenkins Jenkins upstream images: alpine/ubuntu LTS and weekly
  88. 88. Dockerfile FROM jenkins:2.46.2­alpine MAINTAINER dev­team@example.com USER root COPY jenkins/userContent/    /var/lib/jenkins/userContent RUN mkdir /var/log/jenkins &&    mkdir /var/lib/jenkins &&    mkdir /var/cache/jenkins   chown ­R jenkins: /var/log/jenkins &&    chown ­R jenkins: /var/lib/jenkins &&    chown ­R jenkins: /var/cache/jenkins
  89. 89. Timezone (if needed) yay alpine RUN apk add ­­no­cache tzdata &&    cp /usr/share/zoneinfo/Europe/Brussels      /etc/localtime &&    echo "Europe/Brussels" > /etc/timezone &&    apk del tzdata
  90. 90. Install plugins USER jenkins COPY jenkins/plugins.txt /tmp/plugins.txt RUN cat /tmp/plugins.txt | xargs   /usr/local/bin/install­plugins.sh /usr/local/bin/install-plugins.sh is provided in the upstream docker image plugins.txt cloudbees­folder cucumber­reports:4.3.2 docker
  91. 91. init.groovy.d COPY jenkins/init.groovy.d/    /usr/share/jenkins/ref/init.groovy.d/ Note: if you keep Jenkins config in a volume, you need to suffix your files .override to update them
  92. 92. JAVA_OPTS ENV JAVA_OPTS="­Xmx8192m    ­Djenkins.install.runSetupWizard=false" ENV JENKINS_OPTS="­­handlerCountMax=300    ­­logfile=/var/log/jenkins/jenkins.log    ­­webroot=/var/cache/jenkins/war"
  93. 93. Testing the image Jenkins Pipeline init.groovy.d switch: if (System.getenv()['DO_NOT_BUILD']){     Jenkins.instance.doQuietDown() }
  94. 94. Block until started That script will fail until Jenkins is started: #!/bin/bash ­xe curl 127.0.0.1:8080/jenkins2/ | grep 'All [Jenkins]' ­q || grep "WARNING: Failed to run script file"  /var/log/jenkins/jenkins.log || grep "SEVERE: Failed GroovyInitScript.init"  /var/log/jenkins/jenkins.log retry(30) {   sleep 10   sh("""docker exec ${dockerContainer.id}      bash ­xe /tmp/block­until­started.sh""") }
  95. 95. Testing init.groovy.d set ­xe if grep "SEVERE: Failed GroovyInitScript"     /var/log/jenkins/jenkins.log ||   grep "WARNING: Failed to run script file"     /var/log/jenkins/jenkins.log then     sleep 600 # Let some time to investigate     echo "FOUND ERRORS IN THE LOGS"     false fi
  96. 96. Seed jobs Somewhere in init.groovy.d cause = new RemoteCause('localhost',  'Build triggered by init.groovy.d') job = Jenkins.instance.getItem("seed") cAction = new CauseAction(cause) build = job.scheduleBuild2(0, cAction) res = build.get().getResult().toString() assert res == 'SUCCESS' Last lines check for completion of seed job
  97. 97. Benefits A Pipeline to Deploy Jenkins Automated checks of init.groovy.d etc Check that the seed job works
  98. 98. Going further Will the fun ever stop?
  99. 99. Trash the Jenkins config Only keep build history Do notput config in persistent volume Let init.groovy.d/job dsl build your config only from code Still, keep job history
  100. 100. The glue import jenkins.model.Jenkins; import static groovy.io.FileType.FILES def rawBuildsBaseDir = '/srv/jenkins/state' Jenkins.instance.setRawBuildsDir(   "${rawBuildsBaseDir}/${ITEM_FULL_NAME}") /srv/jenkins/state is a persistent volume it will keep build history
  101. 101. Conclusion Public Domain https://commons.wikimedia.org/wiki/File:YellowOnions.jpg
  102. 102. Jenkins can be FULLY automated My recommendations: init.groovy.d Jenkins Job DSL Pipeline Try dockerized Jenkins ; might work for you (spoiler: works for us)
  103. 103. Julien Pivotto roidelapluie roidelapluie@inuits.eu Inuits https://inuits.eu info@inuits.eu Contact

×