Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Pimp your Continuous Delivery Pipeline with Jenkins workflow (W-JAX 14)

32,914 views

Published on

Continuous delivery pipelines are, by definition, workflows with parallel job executions, join points, retries of jobs (Selenium tests are fragile) and manual steps (validation by a QA team). Come and discover how the new workflow engine of Jenkins CI and its Groovy-based DSL will give another dimension to your continuous delivery pipelines and greatly simplify your life.

Sample workflow groovy script used in this presentation: https://gist.github.com/cyrille-leclerc/796085e19d9cec4a71ef

Jenkins workflow syntax reference card: https://github.com/cyrille-leclerc/workflow-plugin/blob/master/SYNTAX-REFERENCE-CARD.md

Published in: Technology

Pimp your Continuous Delivery Pipeline with Jenkins workflow (W-JAX 14)

  1. 1. Pimp your Continuous Delivery Pipeline with Jenkins workflow Cyrille Le Clerc, CloudBees ©2014 CloudBees, Inc. All Rights Reserved
  2. 2. ©2014 CloudBees, Inc. All Rights Reserved About Me @cyrilleleclerc Cyrille Le Clerc Open Source CTO Product Management DevOps, Infra as Code, Con9nuous Delivery
  3. 3. ©2014 CloudBees, Inc. All Rights Reserved About you Jenkins? CI? CD? Code Jenkins plugin?
  4. 4. ©2014 CloudBees, Inc. All Rights Reserved Agenda • Con9nuous Delivery • Jenkins & Con9nuous Delivery • Workflows in Jenkins today • The new Jenkins Workflow Engine • Jenkins Workflow Syntax Card • Possible future
  5. 5. Continuous Delivery ©2014 CloudBees, Inc. All Rights Reserved
  6. 6. Continuous Delivery ©2014 CloudBees, Inc. All Rights Reserved • “Reliable So+ware Releases through Build, Test, and Deployment Automa=on” • DONE = Shippable into Produc9on
  7. 7. Continuous Integration Steps ©2014 CloudBees, Inc. All Rights Reserved Con$nuous Integra$on Source Code Quality Analysis Compile & Unit Tests Binaries
  8. 8. Continuous Delivery Steps QA ©2014 CloudBees, Inc. All Rights Reserved Con$nuous Delivery DEV DEV OPS Source Code Quality Analysis Compile & Unit Tests Binaries Perfs Environment Func Test Environment UAT Environment Production Environment GO LIVE! Release! Staging Environment
  9. 9. Continuous Delivery Pipeline ©2014 CloudBees, Inc. All Rights Reserved Continuous Delivery Pipeline Compile & Unit Tests Perfs Tests Selenium Tests Deploy UAT Sonar Analysis Deploy Staging Deploy Production
  10. 10. Jenkins: Hub of Continuous Delivery ©2014 CloudBees, Inc. All Rights Reserved • Connect the dots • It is the process Source Code Perfs Environment Func Test Environment UAT Environment Binaries Quality Analysis Compile & Unit Tests
  11. 11. CD Pipeline è Workflow ©2014 CloudBees, Inc. All Rights Reserved Continuous Delivery Pipeline Compile & Unit Tests Perfs Tests Selenium Tests Deploy UAT Sonar Analysis Deploy Staging Deploy Production
  12. 12. ©2014 CloudBees, Inc. All Rights Reserved What we need • Complex pipelines involving mul9ple stages • Non-­‐sequen$al logic such as loops, forks … • Long-­‐running builds must survive outages • Interac$ons with humans including pauses, input • Restartable builds in case of transient errors • Reusable defini$ons to avoid duplica9on • Comprehensible scripts with one clear defini9on
  13. 13. Workflow until today ©2014 CloudBees, Inc. All Rights Reserved • Job chaining • Build Flow Plugin • Build Pipeline Plugin • Build Delivery Pipeline
  14. 14. Workflow until today • Many atomic jobs ©2014 CloudBees, Inc. All Rights Reserved • Hard to share variables/state between jobs • Limited logic • Mix build triggers, parameterized build …
  15. 15. Build Flow Plugin • Did have scriptability and extensibility ©2014 CloudBees, Inc. All Rights Reserved • Did not address configura9on “sprawl” • Disjointed view of what really ran • No ability to survive restarts • Almost good enough but could not go further
  16. 16. Jenkins Workflow Engine ©2014 CloudBees, Inc. All Rights Reserved
  17. 17. Simplifying our sample workflow ©2014 CloudBees, Inc. All Rights Reserved Compile & Unit Tests Perfs Tests Selenium Tests Deploy UAT Sonar Analysis Deploy Staging Deploy Production
  18. 18. Simplifying our sample workflow ©2014 CloudBees, Inc. All Rights Reserved Compile & Unit Tests Perfs Tests Selenium Tests Deploy UAT Sonar Analysis Deploy Staging Deploy Production
  19. 19. ©2014 CloudBees, Inc. All Rights Reserved What we need • Complex pipelines involving mul9ple stages • Non-­‐sequen$al logic such as loops, forks … • Long-­‐running builds must survive outages • Interac$ons with humans including pauses, input • Restartable builds in case of transient errors • Reusable defini$ons to avoid duplica9on • Comprehensible scripts with one clear defini9on
  20. 20. New Workflow Syntax ©2014 CloudBees, Inc. All Rights Reserved
  21. 21. ©2014 CloudBees, Inc. All Rights Reserved Key features • En9re flow is one concise Groovy script – For loops, try-­‐finally, fork-­‐join … • Can restart Jenkins while flow is running • Allocate slave nodes and workspaces – As many as you want, when you want • Stages throYle concurrency of builds • Human input/approval integrated into flow • Standard project concepts: SCM, ar9facts, plugins
  22. 22. Groovy DSL vs. Graphical Workflow • Familiar control flow construc9on ©2014 CloudBees, Inc. All Rights Reserved • Familiar “tools” for building abstrac9ons Func9ons, classes, variables, … • Workflow in version control • As opposed to: Graphical workflow designer
  23. 23. ©2014 CloudBees, Inc. All Rights Reserved Project setup • One workflow is defined as a job • Single script for all steps • Build triggers & parameters like regular projects • Each workflow execu9on is a regular Jenkins build displayed in regular Jenkins views • Graphical visualiza9on of actual build possible Not of visualiza9on job defini9on but of build execu9on
  24. 24. Non-sequential logic try {} catch {} ©2014 CloudBees, Inc. All Rights Reserved
  25. 25. Interaction with humans ©2014 CloudBees, Inc. All Rights Reserved
  26. 26. Restartable build / checkpoint ©2014 CloudBees, Inc. All Rights Reserved
  27. 27. ©2014 CloudBees, Inc. All Rights Reserved Visualization
  28. 28. ©2014 CloudBees, Inc. All Rights Reserved Stages • Semaphore • Visualiza9on
  29. 29. Comprehensible scripts ©2014 CloudBees, Inc. All Rights Reserved
  30. 30. ©2014 CloudBees, Inc. All Rights Reserved Demo time!
  31. 31. Workflow Syntax Card ©2014 CloudBees, Inc. All Rights Reserved
  32. 32. Workflow Syntax Card • stage: Enter a new stage ©2014 CloudBees, Inc. All Rights Reserved • node: Allocate node • ws: Allocate workspace • parallel: Execute sub-­‐workflow in parallel • retry: Retry the body up to N 9mes • catchError: Catch error and con9nue • input: Input / human interac9on • load: Evaluate a Groovy source file into the workflow script
  33. 33. Workflow Syntax Card • step: General build step ©2014 CloudBees, Inc. All Rights Reserved • sh: Shell script • bat: Windows batch script
  34. 34. Workflow Syntax Card • archive: Archive ar9facts ©2014 CloudBees, Inc. All Rights Reserved • unarchive: Copy archived ar9fact into the workspace • echo: Print message • dir: Change directory • readFile: Read file from workspace • writeFile: Write file to workspace
  35. 35. Workflow Syntax Card • git: ©2014 CloudBees, Inc. All Rights Reserved Git • svn: Subversion • scm: General SCM • tool: install a tool • build: Build a job
  36. 36. Workflow Syntax Card • checkpoint: capture the state of the execu9on so that it can be restarted later ©2014 CloudBees, Inc. All Rights Reserved
  37. 37. Possible Future ©2014 CloudBees, Inc. All Rights Reserved
  38. 38. Possible Future: probably OSS • More plugin interoperability ©2014 CloudBees, Inc. All Rights Reserved • Mul9-­‐branch project (& SCM API) integra9on • More advanced SCM • In-­‐IDE edi9ng support • Concise syntax
  39. 39. Possible Future: probably Jenkins Enterprise • More visualiza9ons ©2014 CloudBees, Inc. All Rights Reserved • Validated merge integra9on • Deployment, incl. blue/green with rollback • Freestyle and/or Build Flow import • Templates integra9on
  40. 40. Get the code • hYps://gist.github.com/cyrille-­‐leclerc/ 796085e19d9cec4a71ef – workflow.groovy ©2014 CloudBees, Inc. All Rights Reserved
  41. 41. ©2014 CloudBees, Inc. All Rights Reserved

×