Docker and Jenkins [as code]
Docker and Jenkins
[as code]
CI/CD TrackCI/CD Track
© 2019 All Rights Reserved. 3
Today…
3
© 2019 All Rights Reserved. 4
CI/CD. Why containers?
Config Management
Fast provisioning
Clean environments
Disposability
Clustering 4
© 2019 All Rights Reserved. 5
Jenkins and Docker
© 2019 CloudBees, Inc. All Rights Reserved.
Jenkins & Docker. Packaging
https://hub.docker.com/r/jenkins/
© 2019 All Rights Reserved. 7
Jenkins and Docker
• Build agents and steps
© 2019 CloudBees, Inc. All Rights Reserved.
Official agent images
• jenkins/slave – base image with Remoting
• jenkins/jnlp-slave – JNLP agents
• jenkins/ssh-slave – SSH agents
• jenkins/remoting-kafka-agent – Remoting Kafka Plugin
https://hub.docker.com/r/jenkins/
© 2019 All Rights Reserved. 9
Jenkins & Docker. Plugins
Docker Plugin
Docker Pipeline Plugin
Docker Custom Build Environment Plugin
…
Yet Another Docker Plugin
Kubernetes Plugin
Kubernetes Pipeline Plugin
…
© 2019 All Rights Reserved. 10
Example: Docker Pipeline Plugin
https://plugins.jenkins.io/docker-workflow
docker.image('onenashev/gcc-riscv:6.4-rc').inside {
checkout scm
sh "make clean test"
}
© 2019 All Rights Reserved. 11
Jenkins and Docker
• Build agents and steps
• Master images
© 2019 All Rights Reserved. 12
Jenkins masters in Docker
• Production - jenkins/jenkins
• Experimental - jenkins4eval
• ARM, ARM 64, IBM s/390, PowerPC 64 LE
© 2019 All Rights Reserved. 13
Official image: jenkins/jenkins
docker run --rm 
-p 8080:8080 
-p 50000:50000 
-v jenkins_home:/var/jenkins_home 
jenkins/jenkins:lts
© 2019 CloudBees, Inc. All Rights Reserved.
Official image: jenkins/jenkins
• Weekly and LTS releases
• No bundled plugins
• Debian, Alpine, Slim, CentOS
• Java 8, Java 11
https://github.com/jenkinsci/docker
© 2019 CloudBees, Inc. All Rights Reserved.
© 2019 All Rights Reserved. 16
Configuration as Code in Jenkins
16
JobsSystem
configuration
© 2019 All Rights Reserved. 17
Extending images
Official images are designed to be extended
Managed as code
© 2019 CloudBees, Inc. All Rights Reserved.
Extending images
Image source:
http://container.farm/special-extended-container/
© 2019 All Rights Reserved. 19
Extending Docker images
Dockerfile
Plugin bundling
Groovy Hooks
Environment and Java Options
Docs: https://github.com/jenkinsci/docker
© 2019 All Rights Reserved. 20
Dockerfile
© 2019 All Rights Reserved. 21
Plugin bundling
21
© 2019 CloudBees, Inc. All Rights Reserved.
Plugin bundling
22
# Pinned version
blueocean:1.8.2
# Latest
antisamy-markup-formatter
matrix-auth:latest
# Latest from the Experimental update center
git-client:experimental
# Incrementals (JEP-305)
workflow-support:incrementals;org.jenkins-ci.plugins.workflow;2.21-rc591.43d37d4d080a
plugins.txt
© 2019 All Rights Reserved. 23
Groovy Hooks [1/2]
• Standard Groovy engine
• Triggered by Jenkins
• Direct access to the runtime
• Extensible hook engine
• 2 hook types in the core: init, boot-failure
https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script
© 2019 All Rights Reserved. 24
Groovy Hooks [2/2]
Script locations:
• WEB-INF/HOOK.groovy in jenkins.war
• WEB-INF/HOOK.groovy.d/*.groovy in jenkins.war
• $JENKINS_HOME/HOOK.groovy
• $JENKINS_HOME/HOOK.groovy.d/*.groovy
Execution in alphabetical order
https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script
© 2019 All Rights Reserved. 25
Jenkins Startup Flow
JOBS_LOADED
EXTENSIONS_AUGMENTED
COMPLETED
Groovy Hooks: init()
PLUGINS_STARTED
. . .
© 2019 All Rights Reserved. 26
Example
https://github.com/oleg-nenashev/demo-jenkins-config-as-code
© 2019 All Rights Reserved. 27
Adding Groovy hooks to the standard image
Just put Groovy scripts in the right place!
https://github.com/oleg-nenashev/demo-jenkins-config-as-code
© 2019 All Rights Reserved. 28
More examples
oleg-nenashev/demo-jenkins-config-as-code
Praqma/JenkinsAsCodeReference
librecores/librecores-ci
© 2019 CloudBees, Inc. All Rights Reserved.
Add JCasC Plugin!
Configuration-as-Code Plugin
https://plugins.jenkins.io/configuration-as-code
© 2019 CloudBees, Inc. All Rights Reserved.
JCasC Plugin
● Sub-project in Jenkins
● Feb 2018 - first alpha
● Sep 2018 - 1.0
● Now - 1.27
● Configuration via YAML
jenkinsci/configuration-as-code-plugin
gitter.im/jenkinsci/configuration-as-code-plugin
© 2019 CloudBees, Inc. All Rights Reserved.
jenkins:
systemMessage: "JCasC Demo"
numExecutors: 1
scmCheckoutRetryCount: 4
mode: NORMAL
securityRealm:
local:
allowsSignup: false
users:
- id: demoAdmin
password: ${adminpw}
jenkins.yaml
© 2019 CloudBees, Inc. All Rights Reserved.
• YAML specification
• Configuration export from Jenkins
• CLI and REST API for configuration management
• Pre-flight validations and dry-run
• Support of many plugins out-of-the-box
Features
© 2019 CloudBees, Inc. All Rights Reserved.
Adding JCasC to the Docker image
YAML auto-discovery in 1.0+
Needs JCasC Plugin
…
configuration-as-code:1.27
…
© 2019 CloudBees, Inc. All Rights Reserved.
Comparison
JCasC Plugin Groovy Hooks
YAML – standard declarative
definition
Low barrier to entry
Limited integrations
Forward compatibility risks
External rollback
Apply changes w/o restart
No debugging, YAGNI?
Dry run mode + validation
Export from UI
CLI and REST API for CM
Plugin management
Groovy – general-purpose OOP
language
High barrier to entry
Can manage everything
Compatible if API is stable
External rollback
Restart is required
Debugging support
Plugin Management
© 2019 CloudBees, Inc. All Rights Reserved.
What about CloudBees?
© 2019 CloudBees, Inc. All Rights Reserved.
Products
● All products are available as Docker images
○ https://hub.docker.com/u/cloudbees
● Helm charts for CloudBees Jenkins Distribution
● CloudBees Core runs on-premise or in the cloud
○ Kubernetes
○ Docker
○ Virtual machines
○ Bare metal
© 2019 CloudBees, Inc. All Rights Reserved.
Plugins
● All Jenkins Docker plugins are available to customers
● Extended SLA for several plugins
https://go.cloudbees.com/plugins/
© 2019 CloudBees, Inc. All Rights Reserved.
Plugin Management
● Added value features for Docker
● CloudBees Assurance Program and
Beekeper in all products
○ https://go.cloudbees.com/docs/cloudbees-
documentation/assurance-program/
● Plugin catalog for CloudBees Core
○ https://go.cloudbees.com/docs/plugins/configur
ing-plugin-catalogs/
© 2019 CloudBees, Inc. All Rights Reserved.
CloudBees Assurance
Program
● Verified plugin sets
● Upgrade assistant
● Administrative monitors
● Opt-in automatic upgrade
© 2019 CloudBees, Inc. All Rights Reserved.
Lists the set of plugins, plugin versions and
plugin dependencies that CloudBees has
tested for compatibility and security.
Administrative dashboard provides a
centralized view of the monitored
Jenkins plugins, recommended actions,
and configuration options available.
© 2019 CloudBees, Inc. All Rights Reserved.
JCasC in CloudBees products
● We are working on it! (c)
● Technical Preview for…
○ CloudBees Jenkins Support
○ CloudBees Jenkins Distribution
● Other products - http://bit.ly/cjd-jcasc
http://bit.ly/jcasc-cloudbees
© 2019 CloudBees, Inc. All Rights Reserved.
JCasC in CloudBees Jenkins Distribution
● Combines with CloudBees Assurance Program
● Documentation and Quick Start
○ http://bit.ly/cjd-jcasc
● Demo
○ https://github.com/cloudbees-oss/cjd-jcasc-demo
© 2019 CloudBees, Inc. All Rights Reserved.
https://github.com/cloudbees-oss/cjd-jcasc-demo
© 2019 CloudBees, Inc. All Rights Reserved.
Takeaways
1. Dockerize Jenkins
2. Use & extend official images
3. Use static configurations
4. Use Configuration-as-Code
5. Use Custom WAR/Docker Packager
6. Try CloudBees Jenkins Distribution
© 2019 CloudBees, Inc. All Rights Reserved.
DevOptics
Software Delivery
Visibility & Insights
Core
Unified Software
Delivery &
Governance
CodeShip
CI/CD as a
Service
Flow
Adaptive
Release
Orchestration
DevOps ExcellenceJenkins
CloudBees Jenkins
Distribution
CloudBees Jenkins X
Distribution
24x7 Technical Support
Assisted Updates
Support
Accelerator
Training
Customer Success Managers
DevOps Consultants
Rollout
Feature
Flag
Management
Continuous Delivery Products and Services
Docker and Jenkins [as code]

Docker and Jenkins [as code]

  • 1.
  • 2.
    Docker and Jenkins [ascode] CI/CD TrackCI/CD Track
  • 3.
    © 2019 AllRights Reserved. 3 Today… 3
  • 4.
    © 2019 AllRights Reserved. 4 CI/CD. Why containers? Config Management Fast provisioning Clean environments Disposability Clustering 4
  • 5.
    © 2019 AllRights Reserved. 5 Jenkins and Docker
  • 6.
    © 2019 CloudBees,Inc. All Rights Reserved. Jenkins & Docker. Packaging https://hub.docker.com/r/jenkins/
  • 7.
    © 2019 AllRights Reserved. 7 Jenkins and Docker • Build agents and steps
  • 8.
    © 2019 CloudBees,Inc. All Rights Reserved. Official agent images • jenkins/slave – base image with Remoting • jenkins/jnlp-slave – JNLP agents • jenkins/ssh-slave – SSH agents • jenkins/remoting-kafka-agent – Remoting Kafka Plugin https://hub.docker.com/r/jenkins/
  • 9.
    © 2019 AllRights Reserved. 9 Jenkins & Docker. Plugins Docker Plugin Docker Pipeline Plugin Docker Custom Build Environment Plugin … Yet Another Docker Plugin Kubernetes Plugin Kubernetes Pipeline Plugin …
  • 10.
    © 2019 AllRights Reserved. 10 Example: Docker Pipeline Plugin https://plugins.jenkins.io/docker-workflow docker.image('onenashev/gcc-riscv:6.4-rc').inside { checkout scm sh "make clean test" }
  • 11.
    © 2019 AllRights Reserved. 11 Jenkins and Docker • Build agents and steps • Master images
  • 12.
    © 2019 AllRights Reserved. 12 Jenkins masters in Docker • Production - jenkins/jenkins • Experimental - jenkins4eval • ARM, ARM 64, IBM s/390, PowerPC 64 LE
  • 13.
    © 2019 AllRights Reserved. 13 Official image: jenkins/jenkins docker run --rm -p 8080:8080 -p 50000:50000 -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts
  • 14.
    © 2019 CloudBees,Inc. All Rights Reserved. Official image: jenkins/jenkins • Weekly and LTS releases • No bundled plugins • Debian, Alpine, Slim, CentOS • Java 8, Java 11 https://github.com/jenkinsci/docker
  • 15.
    © 2019 CloudBees,Inc. All Rights Reserved.
  • 16.
    © 2019 AllRights Reserved. 16 Configuration as Code in Jenkins 16 JobsSystem configuration
  • 17.
    © 2019 AllRights Reserved. 17 Extending images Official images are designed to be extended Managed as code
  • 18.
    © 2019 CloudBees,Inc. All Rights Reserved. Extending images Image source: http://container.farm/special-extended-container/
  • 19.
    © 2019 AllRights Reserved. 19 Extending Docker images Dockerfile Plugin bundling Groovy Hooks Environment and Java Options Docs: https://github.com/jenkinsci/docker
  • 20.
    © 2019 AllRights Reserved. 20 Dockerfile
  • 21.
    © 2019 AllRights Reserved. 21 Plugin bundling 21
  • 22.
    © 2019 CloudBees,Inc. All Rights Reserved. Plugin bundling 22 # Pinned version blueocean:1.8.2 # Latest antisamy-markup-formatter matrix-auth:latest # Latest from the Experimental update center git-client:experimental # Incrementals (JEP-305) workflow-support:incrementals;org.jenkins-ci.plugins.workflow;2.21-rc591.43d37d4d080a plugins.txt
  • 23.
    © 2019 AllRights Reserved. 23 Groovy Hooks [1/2] • Standard Groovy engine • Triggered by Jenkins • Direct access to the runtime • Extensible hook engine • 2 hook types in the core: init, boot-failure https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script
  • 24.
    © 2019 AllRights Reserved. 24 Groovy Hooks [2/2] Script locations: • WEB-INF/HOOK.groovy in jenkins.war • WEB-INF/HOOK.groovy.d/*.groovy in jenkins.war • $JENKINS_HOME/HOOK.groovy • $JENKINS_HOME/HOOK.groovy.d/*.groovy Execution in alphabetical order https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script
  • 25.
    © 2019 AllRights Reserved. 25 Jenkins Startup Flow JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED Groovy Hooks: init() PLUGINS_STARTED . . .
  • 26.
    © 2019 AllRights Reserved. 26 Example https://github.com/oleg-nenashev/demo-jenkins-config-as-code
  • 27.
    © 2019 AllRights Reserved. 27 Adding Groovy hooks to the standard image Just put Groovy scripts in the right place! https://github.com/oleg-nenashev/demo-jenkins-config-as-code
  • 28.
    © 2019 AllRights Reserved. 28 More examples oleg-nenashev/demo-jenkins-config-as-code Praqma/JenkinsAsCodeReference librecores/librecores-ci
  • 29.
    © 2019 CloudBees,Inc. All Rights Reserved. Add JCasC Plugin! Configuration-as-Code Plugin https://plugins.jenkins.io/configuration-as-code
  • 30.
    © 2019 CloudBees,Inc. All Rights Reserved. JCasC Plugin ● Sub-project in Jenkins ● Feb 2018 - first alpha ● Sep 2018 - 1.0 ● Now - 1.27 ● Configuration via YAML jenkinsci/configuration-as-code-plugin gitter.im/jenkinsci/configuration-as-code-plugin
  • 31.
    © 2019 CloudBees,Inc. All Rights Reserved. jenkins: systemMessage: "JCasC Demo" numExecutors: 1 scmCheckoutRetryCount: 4 mode: NORMAL securityRealm: local: allowsSignup: false users: - id: demoAdmin password: ${adminpw} jenkins.yaml
  • 32.
    © 2019 CloudBees,Inc. All Rights Reserved. • YAML specification • Configuration export from Jenkins • CLI and REST API for configuration management • Pre-flight validations and dry-run • Support of many plugins out-of-the-box Features
  • 33.
    © 2019 CloudBees,Inc. All Rights Reserved. Adding JCasC to the Docker image YAML auto-discovery in 1.0+ Needs JCasC Plugin … configuration-as-code:1.27 …
  • 34.
    © 2019 CloudBees,Inc. All Rights Reserved. Comparison JCasC Plugin Groovy Hooks YAML – standard declarative definition Low barrier to entry Limited integrations Forward compatibility risks External rollback Apply changes w/o restart No debugging, YAGNI? Dry run mode + validation Export from UI CLI and REST API for CM Plugin management Groovy – general-purpose OOP language High barrier to entry Can manage everything Compatible if API is stable External rollback Restart is required Debugging support Plugin Management
  • 35.
    © 2019 CloudBees,Inc. All Rights Reserved. What about CloudBees?
  • 36.
    © 2019 CloudBees,Inc. All Rights Reserved. Products ● All products are available as Docker images ○ https://hub.docker.com/u/cloudbees ● Helm charts for CloudBees Jenkins Distribution ● CloudBees Core runs on-premise or in the cloud ○ Kubernetes ○ Docker ○ Virtual machines ○ Bare metal
  • 37.
    © 2019 CloudBees,Inc. All Rights Reserved. Plugins ● All Jenkins Docker plugins are available to customers ● Extended SLA for several plugins https://go.cloudbees.com/plugins/
  • 38.
    © 2019 CloudBees,Inc. All Rights Reserved. Plugin Management ● Added value features for Docker ● CloudBees Assurance Program and Beekeper in all products ○ https://go.cloudbees.com/docs/cloudbees- documentation/assurance-program/ ● Plugin catalog for CloudBees Core ○ https://go.cloudbees.com/docs/plugins/configur ing-plugin-catalogs/
  • 39.
    © 2019 CloudBees,Inc. All Rights Reserved. CloudBees Assurance Program ● Verified plugin sets ● Upgrade assistant ● Administrative monitors ● Opt-in automatic upgrade
  • 40.
    © 2019 CloudBees,Inc. All Rights Reserved. Lists the set of plugins, plugin versions and plugin dependencies that CloudBees has tested for compatibility and security. Administrative dashboard provides a centralized view of the monitored Jenkins plugins, recommended actions, and configuration options available.
  • 41.
    © 2019 CloudBees,Inc. All Rights Reserved. JCasC in CloudBees products ● We are working on it! (c) ● Technical Preview for… ○ CloudBees Jenkins Support ○ CloudBees Jenkins Distribution ● Other products - http://bit.ly/cjd-jcasc http://bit.ly/jcasc-cloudbees
  • 42.
    © 2019 CloudBees,Inc. All Rights Reserved. JCasC in CloudBees Jenkins Distribution ● Combines with CloudBees Assurance Program ● Documentation and Quick Start ○ http://bit.ly/cjd-jcasc ● Demo ○ https://github.com/cloudbees-oss/cjd-jcasc-demo
  • 43.
    © 2019 CloudBees,Inc. All Rights Reserved. https://github.com/cloudbees-oss/cjd-jcasc-demo
  • 44.
    © 2019 CloudBees,Inc. All Rights Reserved. Takeaways 1. Dockerize Jenkins 2. Use & extend official images 3. Use static configurations 4. Use Configuration-as-Code 5. Use Custom WAR/Docker Packager 6. Try CloudBees Jenkins Distribution
  • 45.
    © 2019 CloudBees,Inc. All Rights Reserved. DevOptics Software Delivery Visibility & Insights Core Unified Software Delivery & Governance CodeShip CI/CD as a Service Flow Adaptive Release Orchestration DevOps ExcellenceJenkins CloudBees Jenkins Distribution CloudBees Jenkins X Distribution 24x7 Technical Support Assisted Updates Support Accelerator Training Customer Success Managers DevOps Consultants Rollout Feature Flag Management Continuous Delivery Products and Services