More Related Content Similar to Pimp your Continuous Delivery Pipeline with Jenkins workflow (W-JAX 14) (20) Pimp your Continuous Delivery Pipeline with Jenkins workflow (W-JAX 14)1. Pimp your Continuous Delivery Pipeline
with Jenkins workflow
Cyrille Le Clerc, CloudBees
©2014 CloudBees, Inc. All Rights Reserved
2. ©2014 CloudBees, Inc. All Rights Reserved
About Me
@cyrilleleclerc
Cyrille Le Clerc
Open
Source
CTO
Product
Management
DevOps,
Infra
as
Code,
Con9nuous
Delivery
4. ©2014 CloudBees, Inc. All Rights Reserved
Agenda
• Con9nuous
Delivery
• Jenkins
&
Con9nuous
Delivery
• Workflows
in
Jenkins
today
• The
new
Jenkins
Workflow
Engine
• Jenkins
Workflow
Syntax
Card
• Possible
future
6. Continuous Delivery
©2014 CloudBees, Inc. All Rights Reserved
• “Reliable
So+ware
Releases
through
Build,
Test,
and
Deployment
Automa=on”
• DONE
=
Shippable
into
Produc9on
7. Continuous Integration Steps
©2014 CloudBees, Inc. All Rights Reserved
Con$nuous
Integra$on
Source Code
Quality Analysis
Compile &
Unit Tests
Binaries
8. Continuous Delivery Steps
QA
©2014 CloudBees, Inc. All Rights Reserved
Con$nuous
Delivery
DEV
DEV
OPS
Source Code
Quality Analysis
Compile &
Unit Tests
Binaries
Perfs Environment
Func Test Environment
UAT Environment
Production Environment
GO LIVE!
Release!
Staging Environment
9. Continuous Delivery Pipeline
©2014 CloudBees, Inc. All Rights Reserved
Continuous Delivery Pipeline
Compile &
Unit Tests
Perfs Tests
Selenium Tests Deploy UAT
Sonar Analysis
Deploy Staging Deploy Production
10. Jenkins: Hub of Continuous Delivery
©2014 CloudBees, Inc. All Rights Reserved
• Connect
the
dots
• It
is
the
process
Source Code
Perfs Environment
Func Test Environment
UAT Environment
Binaries
Quality Analysis
Compile &
Unit Tests
11. CD Pipeline è Workflow
©2014 CloudBees, Inc. All Rights Reserved
Continuous Delivery Pipeline
Compile &
Unit Tests
Perfs Tests
Selenium Tests Deploy UAT
Sonar Analysis
Deploy Staging Deploy Production
12. ©2014 CloudBees, Inc. All Rights Reserved
What we need
• Complex
pipelines
involving
mul9ple
stages
• Non-‐sequen$al
logic
such
as
loops,
forks
…
• Long-‐running
builds
must
survive
outages
• Interac$ons
with
humans
including
pauses,
input
• Restartable
builds
in
case
of
transient
errors
• Reusable
defini$ons
to
avoid
duplica9on
• Comprehensible
scripts
with
one
clear
defini9on
13. Workflow until today
©2014 CloudBees, Inc. All Rights Reserved
• Job
chaining
• Build
Flow
Plugin
• Build
Pipeline
Plugin
• Build
Delivery
Pipeline
14. Workflow until today
• Many
atomic
jobs
©2014 CloudBees, Inc. All Rights Reserved
• Hard
to
share
variables/state
between
jobs
• Limited
logic
• Mix
build
triggers,
parameterized
build
…
15. Build Flow Plugin
• Did
have
scriptability
and
extensibility
©2014 CloudBees, Inc. All Rights Reserved
• Did
not
address
configura9on
“sprawl”
• Disjointed
view
of
what
really
ran
• No
ability
to
survive
restarts
• Almost
good
enough
but
could
not
go
further
17. Simplifying our sample workflow
©2014 CloudBees, Inc. All Rights Reserved
Compile &
Unit Tests
Perfs Tests
Selenium Tests Deploy UAT
Sonar Analysis
Deploy Staging Deploy Production
18. Simplifying our sample workflow
©2014 CloudBees, Inc. All Rights Reserved
Compile &
Unit Tests
Perfs Tests
Selenium Tests Deploy UAT
Sonar Analysis
Deploy Staging Deploy Production
19. ©2014 CloudBees, Inc. All Rights Reserved
What we need
• Complex
pipelines
involving
mul9ple
stages
• Non-‐sequen$al
logic
such
as
loops,
forks
…
• Long-‐running
builds
must
survive
outages
• Interac$ons
with
humans
including
pauses,
input
• Restartable
builds
in
case
of
transient
errors
• Reusable
defini$ons
to
avoid
duplica9on
• Comprehensible
scripts
with
one
clear
defini9on
21. ©2014 CloudBees, Inc. All Rights Reserved
Key features
• En9re
flow
is
one
concise
Groovy
script
– For
loops,
try-‐finally,
fork-‐join
…
• Can
restart
Jenkins
while
flow
is
running
• Allocate
slave
nodes
and
workspaces
– As
many
as
you
want,
when
you
want
• Stages
throYle
concurrency
of
builds
• Human
input/approval
integrated
into
flow
• Standard
project
concepts:
SCM,
ar9facts,
plugins
22. Groovy DSL vs. Graphical Workflow
• Familiar
control
flow
construc9on
©2014 CloudBees, Inc. All Rights Reserved
• Familiar
“tools”
for
building
abstrac9ons
Func9ons,
classes,
variables,
…
• Workflow
in
version
control
• As
opposed
to:
Graphical
workflow
designer
23. ©2014 CloudBees, Inc. All Rights Reserved
Project setup
• One
workflow
is
defined
as
a
job
• Single
script
for
all
steps
• Build
triggers
&
parameters
like
regular
projects
• Each
workflow
execu9on
is
a
regular
Jenkins
build
displayed
in
regular
Jenkins
views
• Graphical
visualiza9on
of
actual
build
possible
Not
of
visualiza9on
job
defini9on
but
of
build
execu9on
32. Workflow Syntax Card
• stage:
Enter
a
new
stage
©2014 CloudBees, Inc. All Rights Reserved
• node:
Allocate
node
• ws:
Allocate
workspace
• parallel:
Execute
sub-‐workflow
in
parallel
• retry:
Retry
the
body
up
to
N
9mes
• catchError:
Catch
error
and
con9nue
• input:
Input
/
human
interac9on
• load:
Evaluate
a
Groovy
source
file
into
the
workflow
script
33. Workflow Syntax Card
• step:
General
build
step
©2014 CloudBees, Inc. All Rights Reserved
• sh:
Shell
script
• bat:
Windows
batch
script
34. Workflow Syntax Card
• archive:
Archive
ar9facts
©2014 CloudBees, Inc. All Rights Reserved
• unarchive:
Copy
archived
ar9fact
into
the
workspace
• echo:
Print
message
• dir:
Change
directory
• readFile:
Read
file
from
workspace
• writeFile:
Write
file
to
workspace
35. Workflow Syntax Card
• git:
©2014 CloudBees, Inc. All Rights Reserved
Git
• svn:
Subversion
• scm:
General
SCM
• tool:
install
a
tool
• build:
Build
a
job
36. Workflow Syntax Card
• checkpoint:
capture
the
state
of
the
execu9on
so
that
it
can
be
restarted
later
©2014 CloudBees, Inc. All Rights Reserved
38. Possible Future: probably OSS
• More
plugin
interoperability
©2014 CloudBees, Inc. All Rights Reserved
• Mul9-‐branch
project
(&
SCM
API)
integra9on
• More
advanced
SCM
• In-‐IDE
edi9ng
support
• Concise
syntax
39. Possible Future: probably Jenkins Enterprise
• More
visualiza9ons
©2014 CloudBees, Inc. All Rights Reserved
• Validated
merge
integra9on
• Deployment,
incl.
blue/green
with
rollback
• Freestyle
and/or
Build
Flow
import
• Templates
integra9on
40. Get the code
• hYps://gist.github.com/cyrille-‐leclerc/
796085e19d9cec4a71ef
– workflow.groovy
©2014 CloudBees, Inc. All Rights Reserved