Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Pipeline as Code

384 views

Published on

Ignite about Pipeline as Code for #devopsdays London 2017

Published in: Software
  • Be the first to comment

Pipeline as Code

  1. 1. Pipeline as CodePipeline as Code Kris Buytaert @krisbuytaert
  2. 2. Kris BuytaertKris Buytaert ● I used to be a Dev,I used to be a Dev, ● Then Became an OpThen Became an Op ● CTO and Open Source ConsultantCTO and Open Source Consultant @@inuits.euinuits.eu ● Everything is a freaking DNSEverything is a freaking DNS ProblemProblem ● Evangelizing devopsEvangelizing devops ● Organiser of #devopsdays,Organiser of #devopsdays, #cfgmgmtcamp, #loadays, ….#cfgmgmtcamp, #loadays, ….
  3. 3. cAmscAms ● Build AutomationBuild Automation ● Test AutomationTest Automation ● Infrastructure as CodeInfrastructure as Code ● Monitoring as CodeMonitoring as Code ● Security as CodeSecurity as Code ● Pipeline as CodePipeline as Code
  4. 4. Jenkins as ExampleJenkins as Example • Open Source toolOpen Source tool • Widely adopted in the EnterpriseWidely adopted in the Enterprise • Plenty of plugins availablePlenty of plugins available • UI FocusedUI Focused
  5. 5. PipelinesPipelines ● Lots of themLots of them ● Similar ones, but not identical onesSimilar ones, but not identical ones ● One project = different deploymentOne project = different deployment targetstargets ● People move teams expect samePeople move teams expect same patternspatterns ● Mostly unmaintanedMostly unmaintaned
  6. 6. Dirty ClickersDirty Clickers
  7. 7. Complex PipelinesComplex Pipelines
  8. 8. Scaling PipelinesScaling Pipelines ● Create a Pipeline,Create a Pipeline, ● For job in PipelineFor job in Pipeline • Create new Job Based on OldJobCreate new Job Based on OldJob ● Update One JobUpdate One Job ● Never refactor the restNever refactor the rest
  9. 9. Generating Jenkins PipelinesGenerating Jenkins Pipelines ● Template the XMLTemplate the XML ● Put it in PuppetPut it in Puppet • Worked for stable pipelinesWorked for stable pipelines • KindaKinda ● XML gets rewritten on the flyXML gets rewritten on the fly
  10. 10. JenkinsJobBuilderJenkinsJobBuilder ● First Usable AttemptFirst Usable Attempt ● Python BasedPython Based ● Openstack CommunityOpenstack Community ● Limited FunctionalityLimited Functionality ● Little Adoption / OpenstackLittle Adoption / Openstack Abandonned itAbandonned it
  11. 11. PipelineDSLPipelineDSL ● JenkinsfileJenkinsfile ● Inside a repoInside a repo ● GroovyGroovy ● Limited functionalityLimited functionality ● Initially buggyInitially buggy ● Popular for Easy tasksPopular for Easy tasks • (straight, no splits etc)(straight, no splits etc)
  12. 12. Jenkins Job DSLJenkins Job DSL ● GroovyGroovy ● FlexibleFlexible ● Well SupportedWell Supported ● Suitable for more complex PipelinesSuitable for more complex Pipelines
  13. 13. SeedjobsSeedjobs ● GroovyGroovy ● GitGit ● Rebuild jobs onRebuild jobs on commitcommit ● Projects inProjects in foldersfolders
  14. 14. ExampleExample
  15. 15. Example dsl projectExample dsl project job("${foldername}/dashing-dashboard-build")job("${foldername}/dashing-dashboard-build") job("${foldername}/dashing-dashboard-test")job("${foldername}/dashing-dashboard-test") job("${foldername}/dashing-upload2repo")job("${foldername}/dashing-upload2repo") job("${foldername}/dashing-dashboard-job("${foldername}/dashing-dashboard- deploy")deploy") job("${foldername}/dashing-dashboard-job("${foldername}/dashing-dashboard- deploy-prd")deploy-prd") job("${foldername}/dashing-build")job("${foldername}/dashing-build")
  16. 16. Larger CI StacksLarger CI Stacks ● Generate Pipelines / Jobs based onGenerate Pipelines / Jobs based on config files ,config files , ● Build librariesBuild libraries • CheckoutJobCheckoutJob • DeployJobDeployJob • PackageJobPackageJob ● Use Groovy / JobDSL to generateUse Groovy / JobDSL to generate PipelineDSLPipelineDSL
  17. 17. Other Tools :Other Tools :
  18. 18. Problems solvedProblems solved ● One job per task, no reuse of jobs withOne job per task, no reuse of jobs with different parametersdifferent parameters ● All Jobs are in sync, update one =All Jobs are in sync, update one = update allupdate all ● Centrally managed jobs (git)Centrally managed jobs (git)
  19. 19. Stop Clicking , Write CodeStop Clicking , Write Code
  20. 20. ContactContact Kris BuytaertKris Buytaert Kris.Buytaert@inuits.beKris.Buytaert@inuits.be Further ReadingFurther Reading @krisbuytaert@krisbuytaert http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/ http://www.inuits.be/http://www.inuits.be/ InuitsInuits EssensteenwegEssensteenweg 3131 BrasschaatBrasschaat BelgiumBelgium 891.514.231891.514.231

×