Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
State of the Jenkins Automation
Julien Pivotto (@roidelapluie)
Floss Spring 2017
Manchester - March 15, 2017
whoami
Julien "roidelapluie" Pivotto
@roidelapluie
Sysadmin at Inuits
Automation, monitoring, HA
Jenkins user for 5+ years
inuits
Jenkins
Open Souce CI Server
Written in Java
First release in 2011
Fork of Hudson (2005)
What can you do with Jenkins?
Testing, building, deploying software
Testing, building, deploying services
Testing, buildin...
Mission Critical
Public Domain https://commons.wikimedia.org/wiki/File:Roaddamage59quake.JPG
Automating Jenkins
Creative Commons Attribution-ShareAlike 2.0 https://www.flickr.com/photos/machu/3131057286
Why is it hard?
For long, Jenkins has been UI-Driven
It is "easy" to deploy (.war file)
It has lots of plugins (and you ne...
Why is it needed?
Security (high value target)
Bugfixes
Plugins
XML everywhere!
Creative Commons Attribution 2.0 https://www.flickr.com/photos/generated/6035308729
Lot of things to automate
Jenkins Service
Jenkins configuration: security, plugins
Jenkins "data": Jobs/Views
Inside the j...
Automating the service
CC0 Public Domain https://pixabay.com/en/butler-tray-beverages-wine-964007/
Automating the Jenkins Service
Chef: Jenkins in the supermarket
Puppet: module rtyler/jenkins
Playbooks/etc for other tool...
Jenkins Master in Docker
Jenkins upstream images: alpine/ubuntu
Please no executors on master
How do you deploy the contai...
Jenkins Plugins
Fetched from https://updates.jenkins-ci.org
Can be installed from the UI :(
Can be installed from the CLI
Packaging Jenkins Plugins
Plugins have dependencies (against plugins &
Jenkins core)
They have a fixed download path
They ...
Mirroring Jenkins Plugins
Mirror http://updates.jenkins-ci.org
By default, "latest" will be fetched
Don't cache too much
g...
Global configuration
CC0 Public Domain https://commons.wikimedia.org/wiki/File:Waiting_room_bell.jpg
Modifying XML files
DON'T
system-config-dsl-plugin
Like Job DSL, but for the system
https://github.com/jenkinsci/system-config-
dsl-plugin
JENKINS-3...
Creative Commons Attribution-Share Alike 3.0
https://commons.wikimedia.org/wiki/File:Groovy-logo.svg
Groovy
Yet another language to learn? yes.
Programming language for the Java platform
Scripting language
Fully integrated ...
The Jenkins Script Console
A groovy console is available at /script
http://jenkins.example.com/script
also with curl
Requi...
Sample Groovy scripts
Creative Commons Attribution-Share Alike 2.0
https://www.flickr.com/photos/bjornb/88101376
Set number of executors
import jenkins.model.Jenkins;
Jenkins.instance.setNumExecutors(0)
Set HTML formatter
import jenkins.model.Jenkins;
import hudson.markup.RawHtmlMarkupFormatter;
Jenkins.instance.setMarkupFo...
Set system message
import jenkins.model.Jenkins;
Jenkins.instance.setSystemMessage("""
Welcome to <em>Jenkins</em>.
""");
Configure simple-theme
import jenkins.model.Jenkins;
def simpleThemePlugin = 
  Jenkins.instance.getDescriptor(
  "org.cod...
Email
import jenkins.model.Jenkins
def desc = Jenkins.instance.getDescriptor(
  "hudson.tasks.Mailer")
desc.setSmtpHost("1...
Disable usage statistics
import hudson.model.UsageStatistics;
hudson.model.UsageStatistics.DISABLED = true;
But also...
Setup Authorization/Authentication
Setup prefix url
Setup slaves, clouds
Setup global libraries
Setup credenti...
Bonus
import org.jenkinsci.plugins.pipeline.utility
.steps.shaded.org.yaml.snakeyaml.Yaml
Yaml reader = new Yaml()
Map con...
Scripts sources
https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+Script+Console
https://github.com/jenkinsci/jenkins-sc...
Jenkins Javadocs
https://jenkins.io/doc/developer/guides/
http://javadoc.jenkins.io/archive/jenkins-
2.32/
http://javadoc....
Now what?
Creative Commons Attribution 2.0 https://www.flickr.com/photos/51029297@N00/5275403364
Jenkins init.groovy.d
Upon startup, Jenkins will run
 $JENKINS_HOME/init.groovy.d/*.groovy 
scripts
Meanwhile, "Jenkins is...
init.groovy.d directory
behaviour
Scripts executed sequentally (alphanum
order)
Scripts that throws exceptions make startu...
Inside Jenkins
The Multiple Approaches
GUI .. but this talk is about automation, right?
init.groovy.d: to create your seed job
Jenkins Jo...
Jenkins Job Builder
An Openstack Project
Python (not a Jenkins Plugin!)
Support templates
Extensible
Can do raw xml
Limite...
init.groovy.d
def jobManagement = new JenkinsJobManagement(
  System.out, [:], new File('.'))
new DslScriptLoader(jobManag...
Jenkins Job DSL
A Jenkins Plugin
2012
Groovy DSL to create views & jobs
Put Jobs config under SCM
Creative Commons Attribution-Share Alike 3.0
https://commons.wikimedia.org/wiki/File:Groovy-logo.svg
Job DSL support
Large community of users
Lots of plugins supported
Create a job
job('test') {
 scm {
  git('git://example.com/foo.git' 'master')
 }
 steps {
  shell('make test')
 }
}
Create a Pipeline job
pipelineJob('test'){
 definition {
   cps {
     script(buildScript)
   }
 }
}
Set job properties
pipelineJob('test'){
 description('Test Build')
 parameters {
  booleanParam('verbose', false, 'Be Verb...
Read yaml
import org.jenkinsci.plugins.pipeline.utility
.steps.shaded.org.yaml.snakeyaml.Yaml
Yaml reader = new Yaml()
Map...
Loops
config.each(){ jobConfig ­>
  pipelineJob(jobConfig.name) {
    triggers {
      if (jobConfig.triggers?.scm) {
    ...
Create a view
listView('Project A') {
    recurse()
    jobs {
        regex('myprj/[^/]+')
    }
    columns {
        st...
Work with plugins
buildMonitorView("OnScreen Status") {
 jobs {
  name('WatchA')
 }
}
${JENKINS_URL}/plugin/job-dsl/api-vi...
In Pipeline
jobDsl removedJobAction: 'DELETE',
  removedViewAction: 'DELETE',
  targets: 'seeds/*.groovy'
Load extra libraries
jobDsl additionalClasspath: 'src/*.jar',
  removedJobAction: 'DELETE',
  removedViewAction: 'DELETE',...
Jenkins Pipeline
Creative Commons Attribution 2.0 https://www.flickr.com/photos/amerune/9294639633
Jenkins Pipeline
Jenkins Jobs as Code
"Jenkins file"
Imperative (aka scripted) Pipeline
Declarative Pipeline (2017)
What is a Jenkinsfile (aka
Pipeline)
A file that contains the definition of a job
No need of Gui
Defines Steps, Reports, E...
How to write Pipelines?
Visual Pipeline editor (WIP)
"Pipeline Syntax" link in jobs
Scriped pipeline
node ("Ubuntu && amd64") {
  stage('checkout') {
    checkout scm
  }
  stage('build') {
    sh 'make'
  ...
Real World Scripted Pipeline
node ("Ubuntu && amd64") {
  checkout scm
  stage('build') {
    try {
      sh 'make'
    } ...
Declarative Pipeline (1/2)
pipeline {
  agent {
      label("Ubuntu && amd64")
  }
  options {
      timeout(time: 235, un...
Declarative Pipeline (2/2)
pipeline {
 post {
  always {
   junit params.jUnitReports
  }
 }
}
Declarative vs scripted
Declarative checkout code by default
Declarative get a workspace "OOTB"
Declarative enforces every...
Going further with Pipeline
Global Libraries Plugins
Job DSL Plugin
Jenkins Nodes
CC0 Public Domain https://www.flickr.com/photos/133259498@N05/27077266322/
Docker Docker Docker
Run jobs inside containers
Clean, short lived containers
Easy to update
Docker Plugin
Docker nodes pattern
Build Container
Tag it with a tag "candidate"
Push it to your registry
Run normal testing
Run actual ...
In practice
Docker Plugin config automated with groovy
Candidate and Release tags are setup as
slaves
They get two labels:...
In the build Jenkinsfile
pipeline {
 agent {
  label("build­centos­7 && ${params.tag}")
 }
}
In the docker-build Jenkinsfile
dockerImage = docker.build("build­centos­7",
  "­­no­cache")
dockerImage.tag('candidate')
...
Pros/Cons
Updated containers won't block the builds (e.g
on packages updates)
Containers stay up to date
Don't forget that...
Conclusion
Public Domain https://commons.wikimedia.org/wiki/File:YellowOnions.jpg
Jenkins can be FULLY
automated
My recommendations:
init.groovy.d
Jenkins Job DSL
Pipeline
You can go further
I am happy to talk about:
Jenkins master in Docker container
BlueOcean
... I use that but did not fit i...
Julien Pivotto
roidelapluie
roidelapluie@inuits.eu
Inuits
https://inuits.eu
info@inuits.eu
Contact
Upcoming SlideShare
Loading in …5
×

State of the Jenkins Automation

3,663 views

Published on

Talk given at floss uk 2017 about how to fully automate your Jenkins setup, from the service to the jobs.

Published in: Technology
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

State of the Jenkins Automation

  1. 1. State of the Jenkins Automation Julien Pivotto (@roidelapluie) Floss Spring 2017 Manchester - March 15, 2017
  2. 2. whoami Julien "roidelapluie" Pivotto @roidelapluie Sysadmin at Inuits Automation, monitoring, HA Jenkins user for 5+ years
  3. 3. inuits
  4. 4. Jenkins Open Souce CI Server Written in Java First release in 2011 Fork of Hudson (2005)
  5. 5. What can you do with Jenkins? Testing, building, deploying software Testing, building, deploying services Testing, building, deploying infra (IaC)
  6. 6. Mission Critical Public Domain https://commons.wikimedia.org/wiki/File:Roaddamage59quake.JPG
  7. 7. Automating Jenkins Creative Commons Attribution-ShareAlike 2.0 https://www.flickr.com/photos/machu/3131057286
  8. 8. Why is it hard? For long, Jenkins has been UI-Driven It is "easy" to deploy (.war file) It has lots of plugins (and you need them)
  9. 9. Why is it needed? Security (high value target) Bugfixes Plugins
  10. 10. XML everywhere! Creative Commons Attribution 2.0 https://www.flickr.com/photos/generated/6035308729
  11. 11. Lot of things to automate Jenkins Service Jenkins configuration: security, plugins Jenkins "data": Jobs/Views Inside the jobs Jenkins nodes
  12. 12. Automating the service CC0 Public Domain https://pixabay.com/en/butler-tray-beverages-wine-964007/
  13. 13. Automating the Jenkins Service Chef: Jenkins in the supermarket Puppet: module rtyler/jenkins Playbooks/etc for other tools as well Docker
  14. 14. Jenkins Master in Docker Jenkins upstream images: alpine/ubuntu Please no executors on master How do you deploy the container? Pipeline to build&upgrade it
  15. 15. Jenkins Plugins Fetched from https://updates.jenkins-ci.org Can be installed from the UI :( Can be installed from the CLI
  16. 16. Packaging Jenkins Plugins Plugins have dependencies (against plugins & Jenkins core) They have a fixed download path They are listed in updates.jenkins.io/update- center.json https://github.com/roidelapluie/Jenkins- Plugins-RPM
  17. 17. Mirroring Jenkins Plugins Mirror http://updates.jenkins-ci.org By default, "latest" will be fetched Don't cache too much github.com/jenkinsci/docker install- plugins.sh
  18. 18. Global configuration CC0 Public Domain https://commons.wikimedia.org/wiki/File:Waiting_room_bell.jpg
  19. 19. Modifying XML files DON'T
  20. 20. system-config-dsl-plugin Like Job DSL, but for the system https://github.com/jenkinsci/system-config- dsl-plugin JENKINS-31094 Downside: it does not exist yet
  21. 21. Creative Commons Attribution-Share Alike 3.0 https://commons.wikimedia.org/wiki/File:Groovy-logo.svg
  22. 22. Groovy Yet another language to learn? yes. Programming language for the Java platform Scripting language Fully integrated in Jenkins Used by automation tools (chef cookbook, puppet module...)
  23. 23. The Jenkins Script Console A groovy console is available at /script http://jenkins.example.com/script also with curl Requires "Overall/Run Scripts" permission
  24. 24. Sample Groovy scripts Creative Commons Attribution-Share Alike 2.0 https://www.flickr.com/photos/bjornb/88101376
  25. 25. Set number of executors import jenkins.model.Jenkins; Jenkins.instance.setNumExecutors(0)
  26. 26. Set HTML formatter import jenkins.model.Jenkins; import hudson.markup.RawHtmlMarkupFormatter; Jenkins.instance.setMarkupFormatter( new RawHtmlMarkupFormatter(false));
  27. 27. Set system message import jenkins.model.Jenkins; Jenkins.instance.setSystemMessage(""" Welcome to <em>Jenkins</em>. """);
  28. 28. Configure simple-theme import jenkins.model.Jenkins; def simpleThemePlugin =    Jenkins.instance.getDescriptor(   "org.codefirst.SimpleThemeDecorator")    simpleThemePlugin.cssUrl =    "/userContent/example.css" simpleThemePlugin.save() Everything in $JENKINS_HOME/userContent is served under /userContent
  29. 29. Email import jenkins.model.Jenkins def desc = Jenkins.instance.getDescriptor(   "hudson.tasks.Mailer") desc.setSmtpHost("172.17.0.1") desc.save()
  30. 30. Disable usage statistics import hudson.model.UsageStatistics; hudson.model.UsageStatistics.DISABLED = true;
  31. 31. But also... Setup Authorization/Authentication Setup prefix url Setup slaves, clouds Setup global libraries Setup credentials Setup first jobs
  32. 32. Bonus import org.jenkinsci.plugins.pipeline.utility .steps.shaded.org.yaml.snakeyaml.Yaml Yaml reader = new Yaml() Map config = reader.load(text) Thanks to pipeline utility step (readYaml() step), you can easily read yaml files
  33. 33. Scripts sources https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+Script+Console https://github.com/jenkinsci/jenkins-scripts/tree/master/scriptler https://github.com/infOpen/ansible-role-jenkins/tree/master/files/groovy_scripts https://github.com/samrocketman/jenkins-bootstrap-jervis/tree/master/scripts https://github.com/jenkinsci/puppet- jenkins/blob/master/files/puppet_helper.groovy http://pghalliday.com/jenkins/groovy/sonar/chef/configuration/management/2014 /09/21/some-useful-jenkins-groovy-scripts.html
  34. 34. Jenkins Javadocs https://jenkins.io/doc/developer/guides/ http://javadoc.jenkins.io/archive/jenkins- 2.32/ http://javadoc.jenkins.io/ http://javadoc.jenkins.io/plugin/gerrit-trigger/
  35. 35. Now what? Creative Commons Attribution 2.0 https://www.flickr.com/photos/51029297@N00/5275403364
  36. 36. Jenkins init.groovy.d Upon startup, Jenkins will run  $JENKINS_HOME/init.groovy.d/*.groovy  scripts Meanwhile, "Jenkins is getting ready..." message is displayed Drop files, restart Jenkins Allows you to preconfigure everything -- without the GUI
  37. 37. init.groovy.d directory behaviour Scripts executed sequentally (alphanum order) Scripts that throws exceptions make startup fail
  38. 38. Inside Jenkins
  39. 39. The Multiple Approaches GUI .. but this talk is about automation, right? init.groovy.d: to create your seed job Jenkins Job Builder: declarative, python, yaml Jenkins Job DSL: imperative, groovy
  40. 40. Jenkins Job Builder An Openstack Project Python (not a Jenkins Plugin!) Support templates Extensible Can do raw xml Limited support for plugins and pipeline Put Jobs config under SCM
  41. 41. init.groovy.d def jobManagement = new JenkinsJobManagement(   System.out, [:], new File('.')) new DslScriptLoader(jobManagement).runScript(""" folder('jenkins') {     displayName('Jenkins') } pipelineJob("jenkins/seed"){   definition {         cpsScm {             scm {                 git {                     remote {                         credentials('jenkins­git­na')                         url('git.example.com/seed.git')                     }                     branches('master')                 }             }             scriptPath('Jenkinsfile')         }     } } """)
  42. 42. Jenkins Job DSL A Jenkins Plugin 2012 Groovy DSL to create views & jobs Put Jobs config under SCM
  43. 43. Creative Commons Attribution-Share Alike 3.0 https://commons.wikimedia.org/wiki/File:Groovy-logo.svg
  44. 44. Job DSL support Large community of users Lots of plugins supported
  45. 45. Create a job job('test') {  scm {   git('git://example.com/foo.git' 'master')  }  steps {   shell('make test')  } }
  46. 46. Create a Pipeline job pipelineJob('test'){  definition {    cps {      script(buildScript)    }  } }
  47. 47. Set job properties pipelineJob('test'){  description('Test Build')  parameters {   booleanParam('verbose', false, 'Be Verbose')  }  logRotator {   numToKeep(10)  } }
  48. 48. Read yaml import org.jenkinsci.plugins.pipeline.utility .steps.shaded.org.yaml.snakeyaml.Yaml Yaml reader = new Yaml() Map config = reader.load(   readFileFromWorkspace('cfg.yaml') Just like in init.groovy.d scripts readFileFromWorkspace()
  49. 49. Loops config.each(){ jobConfig ­>   pipelineJob(jobConfig.name) {     triggers {       if (jobConfig.triggers?.scm) {         scm(jobConfig.triggers.scm)       }     }   } }
  50. 50. Create a view listView('Project A') {     recurse()     jobs {         regex('myprj/[^/]+')     }     columns {         status()         weather()         name()         lastSuccess()         lastFailure()         lastDuration()         lastBuildConsole()         buildButton()         progressBar()     } }
  51. 51. Work with plugins buildMonitorView("OnScreen Status") {  jobs {   name('WatchA')  } } ${JENKINS_URL}/plugin/job-dsl/api-viewer/
  52. 52. In Pipeline jobDsl removedJobAction: 'DELETE',   removedViewAction: 'DELETE',   targets: 'seeds/*.groovy'
  53. 53. Load extra libraries jobDsl additionalClasspath: 'src/*.jar',   removedJobAction: 'DELETE',   removedViewAction: 'DELETE',   targets: 'seeds/*.groovy'
  54. 54. Jenkins Pipeline Creative Commons Attribution 2.0 https://www.flickr.com/photos/amerune/9294639633
  55. 55. Jenkins Pipeline Jenkins Jobs as Code "Jenkins file" Imperative (aka scripted) Pipeline Declarative Pipeline (2017)
  56. 56. What is a Jenkinsfile (aka Pipeline) A file that contains the definition of a job No need of Gui Defines Steps, Reports, Environments, Nodes,... Plugins can provide steps Generic "step"
  57. 57. How to write Pipelines? Visual Pipeline editor (WIP) "Pipeline Syntax" link in jobs
  58. 58. Scriped pipeline node ("Ubuntu && amd64") {   stage('checkout') {     checkout scm   }   stage('build') {     sh 'make'   }   stage('test') {     sh 'make test'   } }
  59. 59. Real World Scripted Pipeline node ("Ubuntu && amd64") {   checkout scm   stage('build') {     try {       sh 'make'     } catch(err) {       currentBuild.result = 'UNSTABLE'       publishArtifacts('err.log')       throw err     }   }   stage('test') {     sh 'make test'   }   step([$class: 'JavadocArchiver',   javadocDir: "target/site", keepAll: true]) }
  60. 60. Declarative Pipeline (1/2) pipeline {   agent {       label("Ubuntu && amd64")   }   options {       timeout(time: 235, unit: 'MINUTES')       timestamps()       ansiColor('xterm')   }   stages {     stage('build') {       steps {           sh('make')       }    } }
  61. 61. Declarative Pipeline (2/2) pipeline {  post {   always {    junit params.jUnitReports   }  } }
  62. 62. Declarative vs scripted Declarative checkout code by default Declarative get a workspace "OOTB" Declarative enforces everything in stages Declarative allow Pipeline-wide wrappers (e.g ansiColor, timestamps)
  63. 63. Going further with Pipeline Global Libraries Plugins Job DSL Plugin
  64. 64. Jenkins Nodes CC0 Public Domain https://www.flickr.com/photos/133259498@N05/27077266322/
  65. 65. Docker Docker Docker Run jobs inside containers Clean, short lived containers Easy to update Docker Plugin
  66. 66. Docker nodes pattern Build Container Tag it with a tag "candidate" Push it to your registry Run normal testing Run actual builds with that "candidate" If success -> tag with "release" && push
  67. 67. In practice Docker Plugin config automated with groovy Candidate and Release tags are setup as slaves They get two labels: "image" "tag" (e.g. "build- centos-7" "candidate") Jobs get a parameter "tag"
  68. 68. In the build Jenkinsfile pipeline {  agent {   label("build­centos­7 && ${params.tag}")  } }
  69. 69. In the docker-build Jenkinsfile dockerImage = docker.build("build­centos­7",   "­­no­cache") dockerImage.tag('candidate') docker.withRegistry(url, credentials) {   dockerImage.push('candidate') } node("build­centos­7 && candidate") {   sh('test­script') } build job: 'myjob', parameters:   [string(name:'tag', value:'candidate')]    dockerImage.tag('release') docker.withRegistry(url, credentials) {   dockerImage.push('release') }
  70. 70. Pros/Cons Updated containers won't block the builds (e.g on packages updates) Containers stay up to date Don't forget that builds release artifacts (sometime you don't want that in nodes tests)
  71. 71. Conclusion Public Domain https://commons.wikimedia.org/wiki/File:YellowOnions.jpg
  72. 72. Jenkins can be FULLY automated My recommendations: init.groovy.d Jenkins Job DSL Pipeline
  73. 73. You can go further I am happy to talk about: Jenkins master in Docker container BlueOcean ... I use that but did not fit in this timeslot
  74. 74. Julien Pivotto roidelapluie roidelapluie@inuits.eu Inuits https://inuits.eu info@inuits.eu Contact

×