1. Pipeline as CodePipeline as Code
Kris Buytaert
@krisbuytaert

2. Kris BuytaertKris Buytaert
● I used to be a Dev,I used to be a Dev,
● Then Became an OpThen Became an Op
● CTO and Open Source ConsultantCTO and Open Source Consultant
@@inuits.euinuits.eu
● Everything is a freaking DNSEverything is a freaking DNS
ProblemProblem
● Evangelizing devopsEvangelizing devops
● Organiser of #devopsdays,Organiser of #devopsdays,
#cfgmgmtcamp, #loadays, ….#cfgmgmtcamp, #loadays, ….

3. cAmscAms
● Build AutomationBuild Automation
● Test AutomationTest Automation
● Infrastructure as CodeInfrastructure as Code
● Monitoring as CodeMonitoring as Code
● Security as CodeSecurity as Code
● Pipeline as CodePipeline as Code

4. Jenkins as ExampleJenkins as Example
•
Open Source toolOpen Source tool
•
Widely adopted in the EnterpriseWidely adopted in the Enterprise
•
Plenty of plugins availablePlenty of plugins available
•
UI FocusedUI Focused

5. PipelinesPipelines
● Lots of themLots of them
● Similar ones, but not identical onesSimilar ones, but not identical ones
● One project = different deploymentOne project = different deployment
targetstargets
● People move teams expect samePeople move teams expect same
patternspatterns
● Mostly unmaintanedMostly unmaintaned

6. Dirty ClickersDirty Clickers

7. Complex PipelinesComplex Pipelines

8. Scaling PipelinesScaling Pipelines
● Create a Pipeline,Create a Pipeline,
● For job in PipelineFor job in Pipeline
•
Create new Job Based on OldJobCreate new Job Based on OldJob
● Update One JobUpdate One Job
● Never refactor the restNever refactor the rest

9. Generating Jenkins PipelinesGenerating Jenkins Pipelines
● Template the XMLTemplate the XML
● Put it in PuppetPut it in Puppet
•
Worked for stable pipelinesWorked for stable pipelines
•
KindaKinda
● XML gets rewritten on the flyXML gets rewritten on the fly

10. JenkinsJobBuilderJenkinsJobBuilder
● First Usable AttemptFirst Usable Attempt
● Python BasedPython Based
● Openstack CommunityOpenstack Community
● Limited FunctionalityLimited Functionality
● Little Adoption / OpenstackLittle Adoption / Openstack
Abandonned itAbandonned it

11. PipelineDSLPipelineDSL
● JenkinsfileJenkinsfile
● Inside a repoInside a repo
● GroovyGroovy
● Limited functionalityLimited functionality
● Initially buggyInitially buggy
● Popular for Easy tasksPopular for Easy tasks
•
(straight, no splits etc)(straight, no splits etc)

12. Jenkins Job DSLJenkins Job DSL
● GroovyGroovy
● FlexibleFlexible
● Well SupportedWell Supported
● Suitable for more complex PipelinesSuitable for more complex Pipelines

13. SeedjobsSeedjobs
● GroovyGroovy
● GitGit
● Rebuild jobs onRebuild jobs on
commitcommit
● Projects inProjects in
foldersfolders

14. ExampleExample

15. Example dsl projectExample dsl project
job("${foldername}/dashing-dashboard-build")job("${foldername}/dashing-dashboard-build")
job("${foldername}/dashing-dashboard-test")job("${foldername}/dashing-dashboard-test")
job("${foldername}/dashing-upload2repo")job("${foldername}/dashing-upload2repo")
job("${foldername}/dashing-dashboard-job("${foldername}/dashing-dashboard-
deploy")deploy")
job("${foldername}/dashing-dashboard-job("${foldername}/dashing-dashboard-
deploy-prd")deploy-prd")
job("${foldername}/dashing-build")job("${foldername}/dashing-build")

16. Larger CI StacksLarger CI Stacks
● Generate Pipelines / Jobs based onGenerate Pipelines / Jobs based on
config files ,config files ,
● Build librariesBuild libraries
•
CheckoutJobCheckoutJob
•
DeployJobDeployJob
•
PackageJobPackageJob
● Use Groovy / JobDSL to generateUse Groovy / JobDSL to generate
PipelineDSLPipelineDSL

17. Other Tools :Other Tools :

18. Problems solvedProblems solved
● One job per task, no reuse of jobs withOne job per task, no reuse of jobs with
different parametersdifferent parameters
● All Jobs are in sync, update one =All Jobs are in sync, update one =
update allupdate all
● Centrally managed jobs (git)Centrally managed jobs (git)

19. Stop Clicking , Write CodeStop Clicking , Write Code

20. ContactContact
Kris BuytaertKris Buytaert Kris.Buytaert@inuits.beKris.Buytaert@inuits.be
Further ReadingFurther Reading
@krisbuytaert@krisbuytaert
http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/
http://www.inuits.be/http://www.inuits.be/
InuitsInuits
EssensteenwegEssensteenweg
3131
BrasschaatBrasschaat
BelgiumBelgium
891.514.231891.514.231