Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continuous delivery with jenkins pipelines (@devfest Vienna)

499 views

Published on

Presentation at DevFest Vienna 25.11.2017:
This talk demonstrates how a continuous delivery deployment pipeline can be set up harnessing jenkins 2’s Pipeline as Code features as well as its new Blue Ocean User Experience.

Published in: Software
  • Be the first to comment

Continuous delivery with jenkins pipelines (@devfest Vienna)

  1. 1. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Continuous Delivery with Jenkins Pipelines (incl. Advanced Topics) Roman Pickl
  2. 2. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com • 10+ years’ experience in integrating transport systems • 22 mobility Apps in the stores • 43+ million requests / month • 500.000+ unique users / month At a glance qando Vienna, Linz, Graz, Salzburg, Klagenfurt. SMILE Lighthouse project for integrated mobility Wien Mobil Lab Vienna FluidHub Powering the Integrated Mobility and MaaS Ecosystem 2009 avv connect Aachen Region, North Rhine-Westphalia. 2014 NUMO Vienna 2012 2015 2016
  3. 3. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com How did I end up here? • Roman Pickl (@rompic) • CTO @ Fluidtime • In charge of the technical development • Uses Jenkins since 2012 • Loves CI/CD/DevOps • Here to learn
  4. 4. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Continuous Delivery (CD) Automated implementation of your system‘s build, deploy, test, release process •Every change results in a build •Every build is a release candidate •Delivery can be done at any time, on any environment à Make releases a non-event Deployment Pipeline provides: •Visibility •Feedback •Control Read these books! Jez Humble on Continuous Delivery(2012): https://www.youtube.com/watch?v=skLJuksCRTw
  5. 5. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Deployment Pipelines (Let’s build it with jenkins) (Jez Humble D. F., Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation, 2010)
  6. 6. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Jenkins #1 Continuous Integration and Delivery server •Created by Kohsuke Kawaguchi •Initial Release 2005 (Hudson) •Open Source (MIT License) •Active and independent community (https://jenkins.io) •155,000 active installations •1,500+ plugins (!) •Since 2.0 Pipelines (April 2016) are first class citizens •Pipeline as Code (Jenkinsfile). •New User Experience “Blue Ocean” with Blue Ocean Pipeline Editor http://dduportal.github.io/presentations/mixit-2017-jenkins-pipeline/slides.html#/meet_jenkins https://jenkins.io/blog/2017/04/05/say-hello-blueocean-1-0/ http://stats.jenkins.io/
  7. 7. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Welcome Blue Ocean!
  8. 8. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Create a Pipeline Wizard
  9. 9. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Jenkinsfile Written in a Groovy DSL “Jenkinsfile” in top level folder (different path possible since June 2017 https://issues.jenkins- ci.org/browse/JENKINS-34561) Store in SCM (e.g. GIT) for additional benefits •Code review/iteration •Audit trail •Single source of truth Supports two syntaxes (can be mixed) •Declarative pipelines (easier; new; 1.0 Feb 2017) •Scripted pipelines (more powerful) https://jenkins.io/doc/book/pipeline/jenkinsfile/
  10. 10. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Scripted vs. Declarative Scripted Pipelines: Declarative Pipeline: https://jenkins.io/doc/book/pipeline/syntax/
  11. 11. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Scripted vs. Declarative Scripted Pipeline: Declarative Pipeline: https://jenkins.io/doc/book/pipeline/syntax/ • imperative programming model • fully featured programming environment, • higher flexibility and extensibility • very few limits => for power-users and more complex requirements • declarative programming model • simpler and more opinionated syntax for authoring Jenkins Pipeline. • Allows for validation and a visual editor • limits what is available to the user => ideal choice for simpler continuous delivery pipelinesBoth: • use Groovy • same Pipeline sub-system underneath • mostly use same steps • able to utilize Shared Libraries => can be mixed using the script step See https://jenkins.io/blog/2017/01/19/converting-conditional-to-pipeline/ for a more complex example of migrating a freestyle job to a declarative/scripted pipeline.
  12. 12. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com script Step • takes a block of Scripted Pipeline & executes that in the Declarative Pipeline • can provide a useful "escape hatch". • script blocks of non-trivial size and/or complexity should be moved into Shared Libraries https://jenkins.io/doc/book/pipeline/syntax/
  13. 13. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Settings
  14. 14. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Stages
  15. 15. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Parallel Execution With declarative1.2
  16. 16. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Approval ... Deploy to Stage, Deploy Live, Release omitted here
  17. 17. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Post Build Notifications
  18. 18. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Demo 1. docker run -p 8080:8080 jenkinsci/blueocean (add –p 44444:44444 or any other port to try the ssh linter) 2. Go to http://localhost:8080/ 3. Unlock jenkins with initialpw from log file 4. Install suggested plugins (you may have to continue the process in case any plugins are broken and update plugins later on) 5. Create admin user / or continue with admin and initalpw 6. Open http://localhost:8080/blue/pipelines 7. Generate new Pipeline with Github repository 8. Create Jenkinsfile (e.g. https://github.com/rompic/jenkinspipeline)
  19. 19. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Blue Ocean 1.3.3 released 16.11.2017 • https://wiki.jenkins.io/display/JENKINS/Blue+Ocean+Plugin • Latest additions • JIRA integration • Declarative Pipeline version 1.2 support • Support of different Repositories Public roadmap https://jenkins.io/projects/blueocean/roadmap/
  20. 20. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Overview
  21. 21. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Approval
  22. 22. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Detail
  23. 23. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Multi branch support
  24. 24. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Multi branch (When)
  25. 25. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Pipeline Development and Advanced Tools • Blue Ocean Pipeline Editor • Snippet Generator • IntelliJ IDEA GDSL – Autocomplete • Command-line Pipeline Linter • Replay Feature • Unit Testing Jenkins Pipelines • Shared Libaries https://jenkins.io/doc/book/pipeline/development/ https://jenkins.io/doc/book/pipeline/shared-libraries/
  26. 26. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Pipeline Editor GitLab currently not supported (planned! https://issues.jenkins-ci.org/browse/JENKINS-43148) Workaround: http://localhost:8080/blue/organizations/jenkins/pipeline-editor/ Ctrl-S / Cmd-S to open the load save dialog
  27. 27. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Snippet generator http://localhost:8080/pipeline-syntax https://jenkins.io/doc/book/pipeline/getting-started/#snippet-generator
  28. 28. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com IntelliJ IDEA GDSL - Autocomplete • Autocompletion of steps for scripted pipelines • Install Groovy Plugin • Download http://localhost:8080/pipeline-syntax/gdsl • Add it as e.g. pipeline.gdsl to your projects src path See https://st-g.de/2016/08/jenkins-pipeline-autocompletion-in-intellij https://stackoverflow.com/questions/41062514/use-gdsl-file-in-a-java-project-in-intellij https://stackoverflow.com/a/41149255/3165782 for setting it up.
  29. 29. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Command-line Pipeline Linter Validate Declarative Pipelines from the cli before actually running it/checking it in. See https://jenkins.io/doc/book/pipeline/development/#linter for details. Remember to enable SSH access, expose a port on your docker container and add ssh key to try this!
  30. 30. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Replay Feature • Allows for quick modifications and execution of an existing (valid!) Pipeline without changing the Pipeline configuration or creating a new commit. • Once you are satisfied with the changes, you can use Replay to view them again, copy them back to your Pipeline job or Jenkinsfile, and then commit them using your usual engineering processes https://jenkins.io/doc/book/pipeline/development/
  31. 31. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Shared Libraries Share parts of Pipelines between various projects to reduce redundancies and keep code "DRY“. Functions can than be called from Jenkinsfiles. See https://jenkins.io/doc/book/pipeline/shar ed-libraries/ and https://jenkins.io/blog/2017/10/02/pipe line-templates-with-shared-libraries/ for more details.
  32. 32. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Unit Testing Jenkins Pipelines •Allows to unit test Pipelines and Shared Libraries before running them in full •Provides a mock execution environment that can be used to check for expected behavior •Still quite rough around the edges. (e.g. no support for declarative pipeline yet https://github.com/lesfurets/JenkinsPipelineUnit/pull/13) See: https://github.com/lesfurets/JenkinsPipelineUnit https://github.com/lesfurets/JenkinsPipelineUnit/blob/master/README.md https://issues.jenkins-ci.org/browse/JENKINS-33925
  33. 33. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Things missing / Things to come Missing: • Support for definition of variables in declarative pipelines (see workaround in https://issues.jenkins-ci.org/browse/JENKINS-41335) • Keep build forever (https://issues.jenkins-ci.org/browse/JENKINS-39028 ; workaround via shared lib) • Restart stages for pipelines (checkpoint as commercial feature, https://issues.jenkins- ci.org/browse/JENKINS-33846 ; feature for declarative pipelines planned: https://issues.jenkins-ci.org/browse/JENKINS-45455 ) To Come: • Multiple jenkins files • More editor coverage of declaration syntax • jenkins pipeline support for changelog and changesets with when keyword • GitLab read/write support • ...
  34. 34. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Website: https://jenkins.io Blog: https://jenkins.io/node/ https://www.slideshare.net/legrimpeur/belgium-jenkins-area-meetup-jenkins-blueocean- and-declarative-pipelines Getting Started https://jenkins.io/doc/book/getting-started/ https://jenkins.io/doc/book/pipeline/syntax/ https://jenkins.io/doc/pipeline/steps/ https://jenkins.io/doc/book/blueocean/getting-started/ https://github.com/jenkinsci/pipeline-model-definition-plugin/wiki/getting%20started https://jenkins.io/blog/2017/05/18/pipeline-dev-tools/ Further references & information I
  35. 35. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Docker Files: https://github.com/jenkinsci/docker/blob/master/README.md https://hub.docker.com/r/jenkinsci/blueocean/ Further references & information II
  36. 36. © 2017 Copyright Fluidtime Data Services GmbH | www.fluidtime.com Contact Roman Pickl (@rompic) roman.pickl@fluidtime.com Fluidtime Data Services GmbH Neubaugasse 12-14/25 A–1070 Wien Tel +43 (0)1 5860 180 www.fluidtime.com

×