SlideShare a Scribd company logo
Jenkins Pipelines
Steffen Gebert (@StGebert)
DevOps Meetup Frankfurt, 20.10.2016
About Me
Researcher / PhD Student
(Software-based Networks)
2011 - 2016
Core Team Member
2010 - 2013
Server Admin Team Member
since 2011
2
Co-Organizer DevOps Meetup Würzburg
since last week
Continuous Delivery
4
Agile Development
• Sprints of ~1-4 weeks duration
• Result: Product that can be
shown/given to customer
• Return: Feedback
Image	credits:	Marekventur,
Wikimedia, CC-BY-SA-3.0
5
Continuous Delivery
• Reduce cycle times (even more)
• cf. “DevOps is not enough” talk
• “How long does it take to release a one-line change?”
• Potentially deliverable code with every commit
• Automated tests decide about acceptance
• You don’t have to release/deploy it, but you could
à Continuous Deployment: Deploy every successfully
tested commit automatically
6
Pipelines
• Every check-in triggers
pipeline execution
• Feedback to the team in
every stage
• “Bring the pain forward”
• “Fail fast, fail often”
• Minimize execution time
• Always aware of
latest stable release
CI & CD Tools
8
CI/CD Tools
• On-premise
• Jenkins
• Thoughtworks Go
• Gitlab CI
• SaaS
• TravisCI
• CircleCI
• AppVeyor
• Codeship
• Visual Studio Team Services
9
Why (I like) Jenkins
• Established open source project
• On premise installation
• Thousands of plugins
• Integrates with many tools/services
• Tailored to CI/CD
10
History of CI/CD with Jenkins
• Downstream Jobs
• Job A triggers job B triggers job C triggers job D trig...
• If one job fails, fail all
• Build Pipeline View
But that’s awkward
… and what I want instead
12
Configuration as Code
• Define jobs/pipelines as code
• Avoid point & click
• In version control
• Can live with the application
• Scales better
• Example: .travis.yml (from TravisCI)
• Similar to GitlabCI
language: php
services:
- redis-server
before_script:
- composer install
script:
- ./bin/phpunit -c …
notifications:
slack:
…
13
Code-driven Approaches in Jenkins
• Jenkins Job Builder
• Python / YAML based, from the OpenStack project
• Job DSL plugin
• Groovy DSL!
(e.g. query Github API and create
jobs for all branches)
• Great thing!
• But creates single jobs
job('my-project-main') {
scm {
git('https://github.com/...')
}
triggers {
scm('H/15 * * * *')
}
publishers {
downstream('my-project-unit')
}
}
Jenkins Pipeline Plugins
15
Jenkins Pipeline Plugins
• Whole suite of plugins (10+), open-sourced earlier this year
• Shipped with Jenkins 2.0
• Formerly commercially available by CloudBees, called Workflow
• Define pipeline as code (again Groovy DSL)
stage("Hello") {
echo "*Hello*"
}
stage("World") {
echo "*World*"
}
16
Pipeline Example
node {
stage("Build") {
sh "echo Could run 'composer install' now"
}
stage("Unit") {
sh "echo Could run 'phpunit' now"
}
// ...
}
17
Pipeline Execution
• node step allocates executor slot (“heavyweight executor”)
• As other Jenkins jobs
• Filter node by labels (i.e. node('php7'), node('master'))
• Required for heavy lifting
• Avoid many sequential allocations (slows down pipeline progress)
• Code outside of node
• Flyweight executor
• Running on master, "for free”
• Pipeline execution survives Jenkins restarts (CPS)
18
Pipeline DSL Steps
• Shellout
• *nix systems: sh
• sh("make"), sh("rm -rf /")
• Windows systems: bat
• bat("C:Program Files...")
• SCM
• checkout("https://github.com/..git")
• File handling
• readFile("my.config")
• writeFile(file: "README.md", text: "Hello World")
• fileExists
19
Pipeline DSL Steps (2)
• Build another job:
• Will not go further into detail J
build("jobname")
build("jobname", wait: false, propagate: false)
Build(job: 'jobname', parameters: [
[$class: 'StringParameterValue', name: 'target', value: target]
[$class: 'ListSubversionTagsParameterValue', name: 'release', tag: release],
[$class: 'BooleanParameterValue', name: 'update_composer', value:
update_composer.to_boolean()])
20
Pipeline DSL Steps (3)
• Copy workspace to next executor
• Will not go further into detail J
stage("build") {
node {
sh("make")
stash("my-workspace")
}
}
stage("release") {
node('magic-release-tool') {
unstash("my-workspace")
sh("release")
}
}
21
Pipeline DSL Steps (4)
• Specify custom environment variables
• Use variables provided by Jenkins
• Global Groovy variables
withEnv(['MYTOOL_HOME=/usr/local/mytool']) {
sh '$MYTOOL_HOME/bin/start‘
}
env.FOO = "bar"
sh 'echo $BAR'
sh 'echo $BUILD_NUMBER > version.txt'
currentBuild.result = 'FAILED'
22
Even More Pipeline Steps
• Plugins contribute additional steps
• Steps available in this Jenkins instance via
• Online reference: https://jenkins.io/doc/pipeline/steps/
23
Example: Manually Confirm Deployment
stage("Signoff") {
input("Deploy that to prod?")
milestone()
}
stage(name: "Deploy", concurrency: 1) { node {..} }
24
Docker
• Run build jobs within Docker containers
• No need to install software on Jenkins master/slave
• Use multiple versions of the same tool
• Containers can be existing ones
or built on demand
• .. and Kubernetes
stage("run in docker") {
node {
withDockerContainer("php:7-fpm") {
sh "php -v”
}
}
}
25
Snipped Editor & Docs
• Because first steps are hard..
• Auto-generated DSL documentation
(Pipeline Syntax → Step Reference)
Where to put that config?
27
Pipeline Configuration
• Paste code into job config (fine for testing)
• Create pipelines via JobDSL
• Commit it to your repo
• File called Jenkinsfile
• It evolves with the application
• It is versioned
• Everybody can read (and modify) it
• You can throw away your Jenkins master at any time
28
Multibranch & Organization Folder Plugins
• Scans a complete GitHub/Bitbucket organisation for Jenkinsfile
• Triggered by Webhook and/or runs periodically
• Automatically adds pipeline jobs per repo/branch/PR
29
DRY: Jenkins Global Library
• Provides shared functionality available for all jobs
• Stored on Jenkins master, available to all slaves
• Available via Jenkins-integrated Git server
• Can be loaded from remote Git repos, specified in global configuration
and Jenkinsfile
@Library("https://github.com/..")
node { deployToProd() }
Jenkins Pipelines for
Chef Cookbooks
Real-World Example
31
Chef CI/CD at TYPO3.org
• Code that runs the *.typo3.org infrastructure, chef-ci.typo3.org
• Objective: Chef cookbooks
• Server provisioning (installs packages, configures services)
• Code: github.com/TYPO3-cookbooks
32
Many Cookbooks, Many Pipelines
• Scans our GitHub organization TYPO3-cookbooks
• Automatically adds/removes pipelines for branches and pull requests*
• Triggered via Webhooks
• Contents of Jenkinsfile: def pipe = new org.typo3.chefci.v1.Pipeline()
pipe.execute()
* Currently suspicious to arbitrary code execution
33
Jenkins Global Library
• Pipelines implemented in Global Library
TYPO3-infrastructure/jenkins-pipeline-global-library-chefci
34
Parallelize Integration Tests
• Run Test-Kitchen (integration test for Chef cookbooks)
• Run all instances in parallel (by Jenkins)
$ kitchen status
Instance Driver Provisioner [..] Last Action
default-debian-78 Docker ChefZero <Not Created>
default-debian-82 Docker ChefZero <Not Created>
physical-debian-78 Docker ChefZero <Not Created>
physical-debian-82 Docker ChefZero <Not Created>
production-debian-78 Docker ChefZero <Not Created>
production-debian-82 Docker ChefZero <Not Created>
35
Parallelize Integration Tests (2)
• Goal: Extract instance list, run kitchen commands in parallel
• Expected result:
parallel(
'default-debian-82': {
node {
unstash('cookbook-tk')
sh('kitchen test --destroy always default-debian-82')
}
},
'physical-debian-82': {
node {
unstash('cookbook-tk')
sh('kitchen test --destroy always physical-debian-82')
}...
36
Parallelize Integration Tests (3)
• Grab list of instance names
def ArrayList<String> getInstances(){
def instanceNames = []
node {
def lines = sh(script: 'kitchen status', returnStdout: true).split('n')
for (int i = 1; i < lines.size(); i++) {
instanceNames << lines[i].tokenize(' ')[0]
}
}
return instanceNames
}
* Closures don’t always work well within Pipeline code (cf. @NonCPS)
37
Parallelize Integration Tests (4)
def Closure getNodeForInstance(String instanceName) {
return {
// this node (one per instance) is later executed in parallel
node {
// restore workspace
unstash('cookbook-tk')
sh('kitchen test --destroy always ' + instanceName)
}}}
for (int i = 0; i < instanceNames.size(); i++) {
def instanceName = instanceNames.get(i)
plNodes[instanceName] = this.getNodeForInstance(instanceName)
}
parallel plNodes
38
Failure Notification
• Pipeline stops, when any step fails
• But.. I want that info in Slack!
def run(Object step){
try {
step.execute()
} catch (err) {
this.postBuildNotify
failTheBuild("Build failed")
}
}
def execute() {
this.prepare()
this.run(new Lint())
this.run(new BerkshelfInstall())
this.run(new TestKitchen())
this.run(new ArchiveArtifacts())
// erm… git flow is a bad idea
if (env.BRANCH_NAME == "master") {
this.run(new BerkshelfUpload())
}
}
Blue Ocean & Other News
A new Jenkins UI
40
Blue Ocean
• Tailored to the pipeline plugins
41
Blue Ocean
• Tailored to the pipeline plugins
42
Pipeline Editor*
* Recently announced, not yet available
43
Declarative Pipelines
• Sorry, the code I’ve
shown might be already
outdated.. J
• Declarative approach
solves some issues
(failure handling, post-
build steps)
pipeline {
agent docker:'node:6.3'
stages {
stage('build') {
sh '..'
}
stage ('test') {
sh 'npm test'
}
}
postBuild {
always {
sh 'echo "This will always run"'
}
failure {
sh 'echo "This will run only if failed"'
}}}
https://jenkins.io/blog/2016/09/19/blueocean
-beta-declarative-pipeline-pipeline-editor/
44
Summary
• Jenkins pipeline suite modernizes its CI/CD capabilities
• CloudBees, Inc. is very actively pushing development
• Many Jenkins plugins already compatible
• Pipeline defined as code
• Versioned
• Doesn't mess up Jenkins jobs
• Code sharing
• Automated pipeline creation based on GitHub/Bitbucket APIs
• Blue Ocean refreshes Jenkins' UI
• Still couple of rough edges (failure handling, frequent changes)
45
Further Reading
• Pipeline Tutorial:
https://github.com/jenkinsci/pipeline-plugin/blob/master/TUTORIAL.md
• Getting started with pipelines:
https://jenkins.io/pipeline/getting-started-pipelines/
• Jenkins World 2016 Wrap Up - Pipelines:
https://jenkins.io/blog/2016/09/24/jenkins-world-2016-wrap-up-pipeline/
• Step documentation:
https://jenkins.io/doc/pipeline/steps/
• Pipeline global library:
https://github.com/jenkinsci/workflow-cps-global-lib-plugin/blob/master/README.md
• Docker in Jenkins pipelines:
https://jenkins.io/blog/2016/08/08/docker-pipeline-environments/
• Notifications (Mail, Slack, etc.):
https://jenkins.io/blog/2016/07/18/pipline-notifications/
• Parallel execution:
https://jenkins.io/blog/2016/06/16/parallel-test-executor-plugin/
• Extending pipeline DSL:
https://jenkins.io/blog/2016/04/21/dsl-plugins/
• Controlling the Flow with Stage, Lock, and Milestone:
https://jenkins.io/blog/2016/10/16/stage-lock-milestone/
• TYPO3's Chef CI:
https://chef-ci.typo3.org
46

More Related Content

What's hot

Jenkins
JenkinsJenkins
Jenkins tutorial
Jenkins tutorialJenkins tutorial
Jenkins tutorial
HarikaReddy115
 
Jenkins-CI
Jenkins-CIJenkins-CI
Jenkins-CI
Gong Haibing
 
Jenkins tutorial for beginners
Jenkins tutorial for beginnersJenkins tutorial for beginners
Jenkins tutorial for beginners
BugRaptors
 
CICD with Jenkins
CICD with JenkinsCICD with Jenkins
What is Jenkins | Jenkins Tutorial for Beginners | Edureka
What is Jenkins | Jenkins Tutorial for Beginners | EdurekaWhat is Jenkins | Jenkins Tutorial for Beginners | Edureka
What is Jenkins | Jenkins Tutorial for Beginners | Edureka
Edureka!
 
Jenkins
JenkinsJenkins
Jenkins Introduction
Jenkins IntroductionJenkins Introduction
Jenkins Introduction
Pavan Gupta
 
Dev ops using Jenkins
Dev ops using JenkinsDev ops using Jenkins
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Edureka!
 
Getting started with Jenkins
Getting started with JenkinsGetting started with Jenkins
Getting started with Jenkins
Edureka!
 
Introduction to jenkins
Introduction to jenkinsIntroduction to jenkins
Introduction to jenkins
Abe Diaz
 
Jenkins for java world
Jenkins for java worldJenkins for java world
Jenkins for java world
Ashok Kumar
 
Jenkins Tutorial.pdf
Jenkins Tutorial.pdfJenkins Tutorial.pdf
Jenkins Tutorial.pdf
devtestervicky
 
What is Continuous Integration? | Continuous Integration with Jenkins | DevOp...
What is Continuous Integration? | Continuous Integration with Jenkins | DevOp...What is Continuous Integration? | Continuous Integration with Jenkins | DevOp...
What is Continuous Integration? | Continuous Integration with Jenkins | DevOp...
Edureka!
 
DevOps 101 - an Introduction to DevOps
DevOps 101  - an Introduction to DevOpsDevOps 101  - an Introduction to DevOps
DevOps 101 - an Introduction to DevOps
Red Gate Software
 
GitLab.pptx
GitLab.pptxGitLab.pptx
GitLab.pptx
LeoulZewelde1
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
Aditya Konarde
 
CI and CD with Jenkins
CI and CD with JenkinsCI and CD with Jenkins
CI and CD with Jenkins
Martin Málek
 
CI/CD Pipeline as a Code using Jenkins 2
CI/CD Pipeline as a Code using Jenkins 2CI/CD Pipeline as a Code using Jenkins 2
CI/CD Pipeline as a Code using Jenkins 2
Mayank Patel
 

What's hot (20)

Jenkins
JenkinsJenkins
Jenkins
 
Jenkins tutorial
Jenkins tutorialJenkins tutorial
Jenkins tutorial
 
Jenkins-CI
Jenkins-CIJenkins-CI
Jenkins-CI
 
Jenkins tutorial for beginners
Jenkins tutorial for beginnersJenkins tutorial for beginners
Jenkins tutorial for beginners
 
CICD with Jenkins
CICD with JenkinsCICD with Jenkins
CICD with Jenkins
 
What is Jenkins | Jenkins Tutorial for Beginners | Edureka
What is Jenkins | Jenkins Tutorial for Beginners | EdurekaWhat is Jenkins | Jenkins Tutorial for Beginners | Edureka
What is Jenkins | Jenkins Tutorial for Beginners | Edureka
 
Jenkins
JenkinsJenkins
Jenkins
 
Jenkins Introduction
Jenkins IntroductionJenkins Introduction
Jenkins Introduction
 
Dev ops using Jenkins
Dev ops using JenkinsDev ops using Jenkins
Dev ops using Jenkins
 
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
 
Getting started with Jenkins
Getting started with JenkinsGetting started with Jenkins
Getting started with Jenkins
 
Introduction to jenkins
Introduction to jenkinsIntroduction to jenkins
Introduction to jenkins
 
Jenkins for java world
Jenkins for java worldJenkins for java world
Jenkins for java world
 
Jenkins Tutorial.pdf
Jenkins Tutorial.pdfJenkins Tutorial.pdf
Jenkins Tutorial.pdf
 
What is Continuous Integration? | Continuous Integration with Jenkins | DevOp...
What is Continuous Integration? | Continuous Integration with Jenkins | DevOp...What is Continuous Integration? | Continuous Integration with Jenkins | DevOp...
What is Continuous Integration? | Continuous Integration with Jenkins | DevOp...
 
DevOps 101 - an Introduction to DevOps
DevOps 101  - an Introduction to DevOpsDevOps 101  - an Introduction to DevOps
DevOps 101 - an Introduction to DevOps
 
GitLab.pptx
GitLab.pptxGitLab.pptx
GitLab.pptx
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
 
CI and CD with Jenkins
CI and CD with JenkinsCI and CD with Jenkins
CI and CD with Jenkins
 
CI/CD Pipeline as a Code using Jenkins 2
CI/CD Pipeline as a Code using Jenkins 2CI/CD Pipeline as a Code using Jenkins 2
CI/CD Pipeline as a Code using Jenkins 2
 

Similar to Jenkins Pipelines

(Declarative) Jenkins Pipelines
(Declarative) Jenkins Pipelines(Declarative) Jenkins Pipelines
(Declarative) Jenkins Pipelines
Steffen Gebert
 
An Open-Source Chef Cookbook CI/CD Implementation Using Jenkins Pipelines
An Open-Source Chef Cookbook CI/CD Implementation Using Jenkins PipelinesAn Open-Source Chef Cookbook CI/CD Implementation Using Jenkins Pipelines
An Open-Source Chef Cookbook CI/CD Implementation Using Jenkins Pipelines
Steffen Gebert
 
Atlanta Jenkins Area Meetup October 22nd 2015
Atlanta Jenkins Area Meetup October 22nd 2015Atlanta Jenkins Area Meetup October 22nd 2015
Atlanta Jenkins Area Meetup October 22nd 2015
Kurt Madel
 
DevOps Odessa #TechTalks 21.01.2020
DevOps Odessa #TechTalks 21.01.2020DevOps Odessa #TechTalks 21.01.2020
DevOps Odessa #TechTalks 21.01.2020
Lohika_Odessa_TechTalks
 
Docker and Puppet for Continuous Integration
Docker and Puppet for Continuous IntegrationDocker and Puppet for Continuous Integration
Docker and Puppet for Continuous Integration
Giacomo Vacca
 
Jenkins days workshop pipelines - Eric Long
Jenkins days workshop  pipelines - Eric LongJenkins days workshop  pipelines - Eric Long
Jenkins days workshop pipelines - Eric Long
ericlongtx
 
Dev ops meetup
Dev ops meetupDev ops meetup
Dev ops meetup
Bigdata Meetup Kochi
 
CIbox - OpenSource solution for making your #devops better
CIbox - OpenSource solution for making your #devops betterCIbox - OpenSource solution for making your #devops better
CIbox - OpenSource solution for making your #devops better
Andrii Podanenko
 
OpenShift Build Pipelines @ Lightweight Java User Group Meetup
OpenShift Build Pipelines @ Lightweight Java User Group MeetupOpenShift Build Pipelines @ Lightweight Java User Group Meetup
OpenShift Build Pipelines @ Lightweight Java User Group Meetup
Tobias Schneck
 
Moving from Jenkins 1 to 2 declarative pipeline adventures
Moving from Jenkins 1 to 2 declarative pipeline adventuresMoving from Jenkins 1 to 2 declarative pipeline adventures
Moving from Jenkins 1 to 2 declarative pipeline adventures
Frits Van Der Holst
 
DevOPS training - Day 2/2
DevOPS training - Day 2/2DevOPS training - Day 2/2
DevOPS training - Day 2/2
Vincent Mercier
 
Pipeline as code - new feature in Jenkins 2
Pipeline as code - new feature in Jenkins 2Pipeline as code - new feature in Jenkins 2
Pipeline as code - new feature in Jenkins 2
Michal Ziarnik
 
Jenkins Days - Workshop - Let's Build a Pipeline - Los Angeles
Jenkins Days - Workshop - Let's Build a Pipeline - Los AngelesJenkins Days - Workshop - Let's Build a Pipeline - Los Angeles
Jenkins Days - Workshop - Let's Build a Pipeline - Los Angeles
Andy Pemberton
 
MoldCamp - multidimentional testing workflow. CIBox.
MoldCamp  - multidimentional testing workflow. CIBox.MoldCamp  - multidimentional testing workflow. CIBox.
MoldCamp - multidimentional testing workflow. CIBox.
Andrii Podanenko
 
Building an Extensible, Resumable DSL on Top of Apache Groovy
Building an Extensible, Resumable DSL on Top of Apache GroovyBuilding an Extensible, Resumable DSL on Top of Apache Groovy
Building an Extensible, Resumable DSL on Top of Apache Groovy
jgcloudbees
 
Continuous Integration With Jenkins Docker SQL Server
Continuous Integration With Jenkins Docker SQL ServerContinuous Integration With Jenkins Docker SQL Server
Continuous Integration With Jenkins Docker SQL Server
Chris Adkin
 
Continuous Deployment with Kubernetes, Docker and GitLab CI
Continuous Deployment with Kubernetes, Docker and GitLab CIContinuous Deployment with Kubernetes, Docker and GitLab CI
Continuous Deployment with Kubernetes, Docker and GitLab CI
alexanderkiel
 
Continuous Integration & Development with Gitlab
Continuous Integration & Development with GitlabContinuous Integration & Development with Gitlab
Continuous Integration & Development with Gitlab
Ayush Sharma
 
Practical introduction to dev ops with chef
Practical introduction to dev ops with chefPractical introduction to dev ops with chef
Practical introduction to dev ops with chef
LeanDog
 
What to expect from Java 9
What to expect from Java 9What to expect from Java 9
What to expect from Java 9
Ivan Krylov
 

Similar to Jenkins Pipelines (20)

(Declarative) Jenkins Pipelines
(Declarative) Jenkins Pipelines(Declarative) Jenkins Pipelines
(Declarative) Jenkins Pipelines
 
An Open-Source Chef Cookbook CI/CD Implementation Using Jenkins Pipelines
An Open-Source Chef Cookbook CI/CD Implementation Using Jenkins PipelinesAn Open-Source Chef Cookbook CI/CD Implementation Using Jenkins Pipelines
An Open-Source Chef Cookbook CI/CD Implementation Using Jenkins Pipelines
 
Atlanta Jenkins Area Meetup October 22nd 2015
Atlanta Jenkins Area Meetup October 22nd 2015Atlanta Jenkins Area Meetup October 22nd 2015
Atlanta Jenkins Area Meetup October 22nd 2015
 
DevOps Odessa #TechTalks 21.01.2020
DevOps Odessa #TechTalks 21.01.2020DevOps Odessa #TechTalks 21.01.2020
DevOps Odessa #TechTalks 21.01.2020
 
Docker and Puppet for Continuous Integration
Docker and Puppet for Continuous IntegrationDocker and Puppet for Continuous Integration
Docker and Puppet for Continuous Integration
 
Jenkins days workshop pipelines - Eric Long
Jenkins days workshop  pipelines - Eric LongJenkins days workshop  pipelines - Eric Long
Jenkins days workshop pipelines - Eric Long
 
Dev ops meetup
Dev ops meetupDev ops meetup
Dev ops meetup
 
CIbox - OpenSource solution for making your #devops better
CIbox - OpenSource solution for making your #devops betterCIbox - OpenSource solution for making your #devops better
CIbox - OpenSource solution for making your #devops better
 
OpenShift Build Pipelines @ Lightweight Java User Group Meetup
OpenShift Build Pipelines @ Lightweight Java User Group MeetupOpenShift Build Pipelines @ Lightweight Java User Group Meetup
OpenShift Build Pipelines @ Lightweight Java User Group Meetup
 
Moving from Jenkins 1 to 2 declarative pipeline adventures
Moving from Jenkins 1 to 2 declarative pipeline adventuresMoving from Jenkins 1 to 2 declarative pipeline adventures
Moving from Jenkins 1 to 2 declarative pipeline adventures
 
DevOPS training - Day 2/2
DevOPS training - Day 2/2DevOPS training - Day 2/2
DevOPS training - Day 2/2
 
Pipeline as code - new feature in Jenkins 2
Pipeline as code - new feature in Jenkins 2Pipeline as code - new feature in Jenkins 2
Pipeline as code - new feature in Jenkins 2
 
Jenkins Days - Workshop - Let's Build a Pipeline - Los Angeles
Jenkins Days - Workshop - Let's Build a Pipeline - Los AngelesJenkins Days - Workshop - Let's Build a Pipeline - Los Angeles
Jenkins Days - Workshop - Let's Build a Pipeline - Los Angeles
 
MoldCamp - multidimentional testing workflow. CIBox.
MoldCamp  - multidimentional testing workflow. CIBox.MoldCamp  - multidimentional testing workflow. CIBox.
MoldCamp - multidimentional testing workflow. CIBox.
 
Building an Extensible, Resumable DSL on Top of Apache Groovy
Building an Extensible, Resumable DSL on Top of Apache GroovyBuilding an Extensible, Resumable DSL on Top of Apache Groovy
Building an Extensible, Resumable DSL on Top of Apache Groovy
 
Continuous Integration With Jenkins Docker SQL Server
Continuous Integration With Jenkins Docker SQL ServerContinuous Integration With Jenkins Docker SQL Server
Continuous Integration With Jenkins Docker SQL Server
 
Continuous Deployment with Kubernetes, Docker and GitLab CI
Continuous Deployment with Kubernetes, Docker and GitLab CIContinuous Deployment with Kubernetes, Docker and GitLab CI
Continuous Deployment with Kubernetes, Docker and GitLab CI
 
Continuous Integration & Development with Gitlab
Continuous Integration & Development with GitlabContinuous Integration & Development with Gitlab
Continuous Integration & Development with Gitlab
 
Practical introduction to dev ops with chef
Practical introduction to dev ops with chefPractical introduction to dev ops with chef
Practical introduction to dev ops with chef
 
What to expect from Java 9
What to expect from Java 9What to expect from Java 9
What to expect from Java 9
 

More from Steffen Gebert

Building an IoT SuperNetwork on top of the AWS Global Infrastructure
Building an IoT SuperNetwork on top of the AWS Global InfrastructureBuilding an IoT SuperNetwork on top of the AWS Global Infrastructure
Building an IoT SuperNetwork on top of the AWS Global Infrastructure
Steffen Gebert
 
Wenn selbst ‘erlaube allen Verkehr von 0.0.0.0/0’ nicht hilft - Verbindungspr...
Wenn selbst ‘erlaube allen Verkehr von 0.0.0.0/0’ nicht hilft - Verbindungspr...Wenn selbst ‘erlaube allen Verkehr von 0.0.0.0/0’ nicht hilft - Verbindungspr...
Wenn selbst ‘erlaube allen Verkehr von 0.0.0.0/0’ nicht hilft - Verbindungspr...
Steffen Gebert
 
Feature Management Platforms
Feature Management PlatformsFeature Management Platforms
Feature Management Platforms
Steffen Gebert
 
Serverless Networking - How We Provide Cloud-Native Connectivity for IoT Devices
Serverless Networking - How We Provide Cloud-Native Connectivity for IoT DevicesServerless Networking - How We Provide Cloud-Native Connectivity for IoT Devices
Serverless Networking - How We Provide Cloud-Native Connectivity for IoT Devices
Steffen Gebert
 
How our Cloudy Mindsets Approached Physical Routers
How our Cloudy Mindsets Approached Physical RoutersHow our Cloudy Mindsets Approached Physical Routers
How our Cloudy Mindsets Approached Physical Routers
Steffen Gebert
 
Jenkins vs. AWS CodePipeline (AWS User Group Berlin)
Jenkins vs. AWS CodePipeline (AWS User Group Berlin)Jenkins vs. AWS CodePipeline (AWS User Group Berlin)
Jenkins vs. AWS CodePipeline (AWS User Group Berlin)
Steffen Gebert
 
Jenkins vs. AWS CodePipeline
Jenkins vs. AWS CodePipelineJenkins vs. AWS CodePipeline
Jenkins vs. AWS CodePipeline
Steffen Gebert
 
Monitoring Akka with Kamon 1.0
Monitoring Akka with Kamon 1.0Monitoring Akka with Kamon 1.0
Monitoring Akka with Kamon 1.0
Steffen Gebert
 
Continuous Delivery
Continuous DeliveryContinuous Delivery
Continuous Delivery
Steffen Gebert
 
Let's go HTTPS-only! - More Than Buying a Certificate
Let's go HTTPS-only! - More Than Buying a CertificateLet's go HTTPS-only! - More Than Buying a Certificate
Let's go HTTPS-only! - More Than Buying a Certificate
Steffen Gebert
 
Cleaning Up the Dirt of the Nineties - How New Protocols are Modernizing the Web
Cleaning Up the Dirt of the Nineties - How New Protocols are Modernizing the WebCleaning Up the Dirt of the Nineties - How New Protocols are Modernizing the Web
Cleaning Up the Dirt of the Nineties - How New Protocols are Modernizing the Web
Steffen Gebert
 
Investigating the Impact of Network Topology on the Processing Times of SDN C...
Investigating the Impact of Network Topology on the Processing Times of SDN C...Investigating the Impact of Network Topology on the Processing Times of SDN C...
Investigating the Impact of Network Topology on the Processing Times of SDN C...
Steffen Gebert
 
SDN interfaces and performance analysis of SDN components
SDN interfaces and performance analysis of SDN componentsSDN interfaces and performance analysis of SDN components
SDN interfaces and performance analysis of SDN components
Steffen Gebert
 
Git Power-Workshop
Git Power-WorkshopGit Power-Workshop
Git Power-Workshop
Steffen Gebert
 
The Development Infrastructure of the TYPO3 Project
The Development Infrastructure of the TYPO3 ProjectThe Development Infrastructure of the TYPO3 Project
The Development Infrastructure of the TYPO3 Project
Steffen Gebert
 
Der Weg zu TYPO3 CMS 6.0 und Einblicke in die TYPO3-Entwicklung
Der Weg zu TYPO3 CMS 6.0 und Einblicke in die TYPO3-EntwicklungDer Weg zu TYPO3 CMS 6.0 und Einblicke in die TYPO3-Entwicklung
Der Weg zu TYPO3 CMS 6.0 und Einblicke in die TYPO3-Entwicklung
Steffen Gebert
 
Official typo3.org infrastructure &
the TYPO3 Server Admin Team
Official typo3.org infrastructure &
the TYPO3 Server Admin TeamOfficial typo3.org infrastructure &
the TYPO3 Server Admin Team
Official typo3.org infrastructure &
the TYPO3 Server Admin Team
Steffen Gebert
 
Neuigkeiten aus dem TYPO3-Projekt
Neuigkeiten aus dem TYPO3-ProjektNeuigkeiten aus dem TYPO3-Projekt
Neuigkeiten aus dem TYPO3-Projekt
Steffen Gebert
 
The TYPO3 Server Admin Team
The TYPO3 Server Admin TeamThe TYPO3 Server Admin Team
The TYPO3 Server Admin Team
Steffen Gebert
 
Gerrit Workshop
Gerrit WorkshopGerrit Workshop
Gerrit Workshop
Steffen Gebert
 

More from Steffen Gebert (20)

Building an IoT SuperNetwork on top of the AWS Global Infrastructure
Building an IoT SuperNetwork on top of the AWS Global InfrastructureBuilding an IoT SuperNetwork on top of the AWS Global Infrastructure
Building an IoT SuperNetwork on top of the AWS Global Infrastructure
 
Wenn selbst ‘erlaube allen Verkehr von 0.0.0.0/0’ nicht hilft - Verbindungspr...
Wenn selbst ‘erlaube allen Verkehr von 0.0.0.0/0’ nicht hilft - Verbindungspr...Wenn selbst ‘erlaube allen Verkehr von 0.0.0.0/0’ nicht hilft - Verbindungspr...
Wenn selbst ‘erlaube allen Verkehr von 0.0.0.0/0’ nicht hilft - Verbindungspr...
 
Feature Management Platforms
Feature Management PlatformsFeature Management Platforms
Feature Management Platforms
 
Serverless Networking - How We Provide Cloud-Native Connectivity for IoT Devices
Serverless Networking - How We Provide Cloud-Native Connectivity for IoT DevicesServerless Networking - How We Provide Cloud-Native Connectivity for IoT Devices
Serverless Networking - How We Provide Cloud-Native Connectivity for IoT Devices
 
How our Cloudy Mindsets Approached Physical Routers
How our Cloudy Mindsets Approached Physical RoutersHow our Cloudy Mindsets Approached Physical Routers
How our Cloudy Mindsets Approached Physical Routers
 
Jenkins vs. AWS CodePipeline (AWS User Group Berlin)
Jenkins vs. AWS CodePipeline (AWS User Group Berlin)Jenkins vs. AWS CodePipeline (AWS User Group Berlin)
Jenkins vs. AWS CodePipeline (AWS User Group Berlin)
 
Jenkins vs. AWS CodePipeline
Jenkins vs. AWS CodePipelineJenkins vs. AWS CodePipeline
Jenkins vs. AWS CodePipeline
 
Monitoring Akka with Kamon 1.0
Monitoring Akka with Kamon 1.0Monitoring Akka with Kamon 1.0
Monitoring Akka with Kamon 1.0
 
Continuous Delivery
Continuous DeliveryContinuous Delivery
Continuous Delivery
 
Let's go HTTPS-only! - More Than Buying a Certificate
Let's go HTTPS-only! - More Than Buying a CertificateLet's go HTTPS-only! - More Than Buying a Certificate
Let's go HTTPS-only! - More Than Buying a Certificate
 
Cleaning Up the Dirt of the Nineties - How New Protocols are Modernizing the Web
Cleaning Up the Dirt of the Nineties - How New Protocols are Modernizing the WebCleaning Up the Dirt of the Nineties - How New Protocols are Modernizing the Web
Cleaning Up the Dirt of the Nineties - How New Protocols are Modernizing the Web
 
Investigating the Impact of Network Topology on the Processing Times of SDN C...
Investigating the Impact of Network Topology on the Processing Times of SDN C...Investigating the Impact of Network Topology on the Processing Times of SDN C...
Investigating the Impact of Network Topology on the Processing Times of SDN C...
 
SDN interfaces and performance analysis of SDN components
SDN interfaces and performance analysis of SDN componentsSDN interfaces and performance analysis of SDN components
SDN interfaces and performance analysis of SDN components
 
Git Power-Workshop
Git Power-WorkshopGit Power-Workshop
Git Power-Workshop
 
The Development Infrastructure of the TYPO3 Project
The Development Infrastructure of the TYPO3 ProjectThe Development Infrastructure of the TYPO3 Project
The Development Infrastructure of the TYPO3 Project
 
Der Weg zu TYPO3 CMS 6.0 und Einblicke in die TYPO3-Entwicklung
Der Weg zu TYPO3 CMS 6.0 und Einblicke in die TYPO3-EntwicklungDer Weg zu TYPO3 CMS 6.0 und Einblicke in die TYPO3-Entwicklung
Der Weg zu TYPO3 CMS 6.0 und Einblicke in die TYPO3-Entwicklung
 
Official typo3.org infrastructure &
the TYPO3 Server Admin Team
Official typo3.org infrastructure &
the TYPO3 Server Admin TeamOfficial typo3.org infrastructure &
the TYPO3 Server Admin Team
Official typo3.org infrastructure &
the TYPO3 Server Admin Team
 
Neuigkeiten aus dem TYPO3-Projekt
Neuigkeiten aus dem TYPO3-ProjektNeuigkeiten aus dem TYPO3-Projekt
Neuigkeiten aus dem TYPO3-Projekt
 
The TYPO3 Server Admin Team
The TYPO3 Server Admin TeamThe TYPO3 Server Admin Team
The TYPO3 Server Admin Team
 
Gerrit Workshop
Gerrit WorkshopGerrit Workshop
Gerrit Workshop
 

Recently uploaded

The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
operationspcvita
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
Fwdays
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
saastr
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 

Recently uploaded (20)

The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Artificial Intelligence and Electronic Warfare
Artificial Intelligence and Electronic WarfareArtificial Intelligence and Electronic Warfare
Artificial Intelligence and Electronic Warfare
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 

Jenkins Pipelines

  • 1. Jenkins Pipelines Steffen Gebert (@StGebert) DevOps Meetup Frankfurt, 20.10.2016
  • 2. About Me Researcher / PhD Student (Software-based Networks) 2011 - 2016 Core Team Member 2010 - 2013 Server Admin Team Member since 2011 2 Co-Organizer DevOps Meetup Würzburg since last week
  • 4. 4 Agile Development • Sprints of ~1-4 weeks duration • Result: Product that can be shown/given to customer • Return: Feedback Image credits: Marekventur, Wikimedia, CC-BY-SA-3.0
  • 5. 5 Continuous Delivery • Reduce cycle times (even more) • cf. “DevOps is not enough” talk • “How long does it take to release a one-line change?” • Potentially deliverable code with every commit • Automated tests decide about acceptance • You don’t have to release/deploy it, but you could à Continuous Deployment: Deploy every successfully tested commit automatically
  • 6. 6 Pipelines • Every check-in triggers pipeline execution • Feedback to the team in every stage • “Bring the pain forward” • “Fail fast, fail often” • Minimize execution time • Always aware of latest stable release
  • 7. CI & CD Tools
  • 8. 8 CI/CD Tools • On-premise • Jenkins • Thoughtworks Go • Gitlab CI • SaaS • TravisCI • CircleCI • AppVeyor • Codeship • Visual Studio Team Services
  • 9. 9 Why (I like) Jenkins • Established open source project • On premise installation • Thousands of plugins • Integrates with many tools/services • Tailored to CI/CD
  • 10. 10 History of CI/CD with Jenkins • Downstream Jobs • Job A triggers job B triggers job C triggers job D trig... • If one job fails, fail all • Build Pipeline View
  • 11. But that’s awkward … and what I want instead
  • 12. 12 Configuration as Code • Define jobs/pipelines as code • Avoid point & click • In version control • Can live with the application • Scales better • Example: .travis.yml (from TravisCI) • Similar to GitlabCI language: php services: - redis-server before_script: - composer install script: - ./bin/phpunit -c … notifications: slack: …
  • 13. 13 Code-driven Approaches in Jenkins • Jenkins Job Builder • Python / YAML based, from the OpenStack project • Job DSL plugin • Groovy DSL! (e.g. query Github API and create jobs for all branches) • Great thing! • But creates single jobs job('my-project-main') { scm { git('https://github.com/...') } triggers { scm('H/15 * * * *') } publishers { downstream('my-project-unit') } }
  • 15. 15 Jenkins Pipeline Plugins • Whole suite of plugins (10+), open-sourced earlier this year • Shipped with Jenkins 2.0 • Formerly commercially available by CloudBees, called Workflow • Define pipeline as code (again Groovy DSL) stage("Hello") { echo "*Hello*" } stage("World") { echo "*World*" }
  • 16. 16 Pipeline Example node { stage("Build") { sh "echo Could run 'composer install' now" } stage("Unit") { sh "echo Could run 'phpunit' now" } // ... }
  • 17. 17 Pipeline Execution • node step allocates executor slot (“heavyweight executor”) • As other Jenkins jobs • Filter node by labels (i.e. node('php7'), node('master')) • Required for heavy lifting • Avoid many sequential allocations (slows down pipeline progress) • Code outside of node • Flyweight executor • Running on master, "for free” • Pipeline execution survives Jenkins restarts (CPS)
  • 18. 18 Pipeline DSL Steps • Shellout • *nix systems: sh • sh("make"), sh("rm -rf /") • Windows systems: bat • bat("C:Program Files...") • SCM • checkout("https://github.com/..git") • File handling • readFile("my.config") • writeFile(file: "README.md", text: "Hello World") • fileExists
  • 19. 19 Pipeline DSL Steps (2) • Build another job: • Will not go further into detail J build("jobname") build("jobname", wait: false, propagate: false) Build(job: 'jobname', parameters: [ [$class: 'StringParameterValue', name: 'target', value: target] [$class: 'ListSubversionTagsParameterValue', name: 'release', tag: release], [$class: 'BooleanParameterValue', name: 'update_composer', value: update_composer.to_boolean()])
  • 20. 20 Pipeline DSL Steps (3) • Copy workspace to next executor • Will not go further into detail J stage("build") { node { sh("make") stash("my-workspace") } } stage("release") { node('magic-release-tool') { unstash("my-workspace") sh("release") } }
  • 21. 21 Pipeline DSL Steps (4) • Specify custom environment variables • Use variables provided by Jenkins • Global Groovy variables withEnv(['MYTOOL_HOME=/usr/local/mytool']) { sh '$MYTOOL_HOME/bin/start‘ } env.FOO = "bar" sh 'echo $BAR' sh 'echo $BUILD_NUMBER > version.txt' currentBuild.result = 'FAILED'
  • 22. 22 Even More Pipeline Steps • Plugins contribute additional steps • Steps available in this Jenkins instance via • Online reference: https://jenkins.io/doc/pipeline/steps/
  • 23. 23 Example: Manually Confirm Deployment stage("Signoff") { input("Deploy that to prod?") milestone() } stage(name: "Deploy", concurrency: 1) { node {..} }
  • 24. 24 Docker • Run build jobs within Docker containers • No need to install software on Jenkins master/slave • Use multiple versions of the same tool • Containers can be existing ones or built on demand • .. and Kubernetes stage("run in docker") { node { withDockerContainer("php:7-fpm") { sh "php -v” } } }
  • 25. 25 Snipped Editor & Docs • Because first steps are hard.. • Auto-generated DSL documentation (Pipeline Syntax → Step Reference)
  • 26. Where to put that config?
  • 27. 27 Pipeline Configuration • Paste code into job config (fine for testing) • Create pipelines via JobDSL • Commit it to your repo • File called Jenkinsfile • It evolves with the application • It is versioned • Everybody can read (and modify) it • You can throw away your Jenkins master at any time
  • 28. 28 Multibranch & Organization Folder Plugins • Scans a complete GitHub/Bitbucket organisation for Jenkinsfile • Triggered by Webhook and/or runs periodically • Automatically adds pipeline jobs per repo/branch/PR
  • 29. 29 DRY: Jenkins Global Library • Provides shared functionality available for all jobs • Stored on Jenkins master, available to all slaves • Available via Jenkins-integrated Git server • Can be loaded from remote Git repos, specified in global configuration and Jenkinsfile @Library("https://github.com/..") node { deployToProd() }
  • 30. Jenkins Pipelines for Chef Cookbooks Real-World Example
  • 31. 31 Chef CI/CD at TYPO3.org • Code that runs the *.typo3.org infrastructure, chef-ci.typo3.org • Objective: Chef cookbooks • Server provisioning (installs packages, configures services) • Code: github.com/TYPO3-cookbooks
  • 32. 32 Many Cookbooks, Many Pipelines • Scans our GitHub organization TYPO3-cookbooks • Automatically adds/removes pipelines for branches and pull requests* • Triggered via Webhooks • Contents of Jenkinsfile: def pipe = new org.typo3.chefci.v1.Pipeline() pipe.execute() * Currently suspicious to arbitrary code execution
  • 33. 33 Jenkins Global Library • Pipelines implemented in Global Library TYPO3-infrastructure/jenkins-pipeline-global-library-chefci
  • 34. 34 Parallelize Integration Tests • Run Test-Kitchen (integration test for Chef cookbooks) • Run all instances in parallel (by Jenkins) $ kitchen status Instance Driver Provisioner [..] Last Action default-debian-78 Docker ChefZero <Not Created> default-debian-82 Docker ChefZero <Not Created> physical-debian-78 Docker ChefZero <Not Created> physical-debian-82 Docker ChefZero <Not Created> production-debian-78 Docker ChefZero <Not Created> production-debian-82 Docker ChefZero <Not Created>
  • 35. 35 Parallelize Integration Tests (2) • Goal: Extract instance list, run kitchen commands in parallel • Expected result: parallel( 'default-debian-82': { node { unstash('cookbook-tk') sh('kitchen test --destroy always default-debian-82') } }, 'physical-debian-82': { node { unstash('cookbook-tk') sh('kitchen test --destroy always physical-debian-82') }...
  • 36. 36 Parallelize Integration Tests (3) • Grab list of instance names def ArrayList<String> getInstances(){ def instanceNames = [] node { def lines = sh(script: 'kitchen status', returnStdout: true).split('n') for (int i = 1; i < lines.size(); i++) { instanceNames << lines[i].tokenize(' ')[0] } } return instanceNames } * Closures don’t always work well within Pipeline code (cf. @NonCPS)
  • 37. 37 Parallelize Integration Tests (4) def Closure getNodeForInstance(String instanceName) { return { // this node (one per instance) is later executed in parallel node { // restore workspace unstash('cookbook-tk') sh('kitchen test --destroy always ' + instanceName) }}} for (int i = 0; i < instanceNames.size(); i++) { def instanceName = instanceNames.get(i) plNodes[instanceName] = this.getNodeForInstance(instanceName) } parallel plNodes
  • 38. 38 Failure Notification • Pipeline stops, when any step fails • But.. I want that info in Slack! def run(Object step){ try { step.execute() } catch (err) { this.postBuildNotify failTheBuild("Build failed") } } def execute() { this.prepare() this.run(new Lint()) this.run(new BerkshelfInstall()) this.run(new TestKitchen()) this.run(new ArchiveArtifacts()) // erm… git flow is a bad idea if (env.BRANCH_NAME == "master") { this.run(new BerkshelfUpload()) } }
  • 39. Blue Ocean & Other News A new Jenkins UI
  • 40. 40 Blue Ocean • Tailored to the pipeline plugins
  • 41. 41 Blue Ocean • Tailored to the pipeline plugins
  • 42. 42 Pipeline Editor* * Recently announced, not yet available
  • 43. 43 Declarative Pipelines • Sorry, the code I’ve shown might be already outdated.. J • Declarative approach solves some issues (failure handling, post- build steps) pipeline { agent docker:'node:6.3' stages { stage('build') { sh '..' } stage ('test') { sh 'npm test' } } postBuild { always { sh 'echo "This will always run"' } failure { sh 'echo "This will run only if failed"' }}} https://jenkins.io/blog/2016/09/19/blueocean -beta-declarative-pipeline-pipeline-editor/
  • 44. 44 Summary • Jenkins pipeline suite modernizes its CI/CD capabilities • CloudBees, Inc. is very actively pushing development • Many Jenkins plugins already compatible • Pipeline defined as code • Versioned • Doesn't mess up Jenkins jobs • Code sharing • Automated pipeline creation based on GitHub/Bitbucket APIs • Blue Ocean refreshes Jenkins' UI • Still couple of rough edges (failure handling, frequent changes)
  • 45. 45 Further Reading • Pipeline Tutorial: https://github.com/jenkinsci/pipeline-plugin/blob/master/TUTORIAL.md • Getting started with pipelines: https://jenkins.io/pipeline/getting-started-pipelines/ • Jenkins World 2016 Wrap Up - Pipelines: https://jenkins.io/blog/2016/09/24/jenkins-world-2016-wrap-up-pipeline/ • Step documentation: https://jenkins.io/doc/pipeline/steps/ • Pipeline global library: https://github.com/jenkinsci/workflow-cps-global-lib-plugin/blob/master/README.md • Docker in Jenkins pipelines: https://jenkins.io/blog/2016/08/08/docker-pipeline-environments/ • Notifications (Mail, Slack, etc.): https://jenkins.io/blog/2016/07/18/pipline-notifications/ • Parallel execution: https://jenkins.io/blog/2016/06/16/parallel-test-executor-plugin/ • Extending pipeline DSL: https://jenkins.io/blog/2016/04/21/dsl-plugins/ • Controlling the Flow with Stage, Lock, and Milestone: https://jenkins.io/blog/2016/10/16/stage-lock-milestone/ • TYPO3's Chef CI: https://chef-ci.typo3.org
  • 46. 46