Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CI and CD Across the Enterprise with Jenkins (devops.com Nov 2014)

9,765 views

Published on

Delivering value to the business faster thanks to Continuous Delivery and DevOps is the new mantra of IT organizations. In this webinar, CloudBees will discuss how Jenkins, the most popular open source Continuous Integration tool, allows DevOps teams to implement Continuous Delivery.

You will learn how to:

* Orchestrate Continuous Delivery pipelines with the new workflow feature,
* Scale Jenkins horizontally in your organization using Jenkins Operations Center by CloudBees,
* Implement end to end traceability with Jenkins and Puppet and Chef.

http://devops.com/news/ci-and-cd-across-enterprise-jenkins/

https://github.com/CloudBees-community/vagrant-puppet-petclinic

Published in: Technology
  • Dating for everyone is here: ♥♥♥ http://bit.ly/2F7hN3u ♥♥♥
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area is here: ❶❶❶ http://bit.ly/2F7hN3u ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

CI and CD Across the Enterprise with Jenkins (devops.com Nov 2014)

  1. 1. CI and CD Across the Enterprise with Jenkins Cyrille Le Clerc, CloudBees ©2014 CloudBees, Inc. All Rights Reserved
  2. 2. ©2014 CloudBees, Inc. All Rights Reserved About Me @cyrilleleclerc Cyrille Le Clerc Open Source CTO Product Management DevOps, Infra as Code, Con9nuous Delivery
  3. 3. ©2014 CloudBees, Inc. All Rights Reserved Agenda • Con9nuous Delivery & Jenkins Workflow • Scaling Jenkins • Con9nuous Delivery with Jenkins and Puppet and Chef
  4. 4. Continuous Delivery ©2014 CloudBees, Inc. All Rights Reserved
  5. 5. Continuous Delivery ©2014 CloudBees, Inc. All Rights Reserved • “Reliable So+ware Releases through Build, Test, and Deployment Automa=on” • DONE = Shippable into Produc9on
  6. 6. Continuous Integration Steps ©2014 CloudBees, Inc. All Rights Reserved Con$nuous Integra$on Source Code Quality Analysis Compile & Unit Tests Binaries
  7. 7. Continuous Delivery Steps QA ©2014 CloudBees, Inc. All Rights Reserved Con$nuous Delivery DEV DEV OPS Source Code Quality Analysis Compile & Unit Tests Binaries Perfs Environment Func Test Environment UAT Environment Production Environment GO LIVE! Release! Staging Environment
  8. 8. Continuous Delivery Pipeline ©2014 CloudBees, Inc. All Rights Reserved Continuous Delivery Pipeline Compile & Unit Tests Perfs Tests Selenium Tests Deploy UAT Sonar Analysis Deploy Staging Deploy Production
  9. 9. Jenkins: Hub of Continuous Delivery ©2014 CloudBees, Inc. All Rights Reserved • Connect the dots • It is the process Source Code Perfs Environment Func Test Environment UAT Environment Binaries Quality Analysis Compile & Unit Tests
  10. 10. CD Pipeline è Workflow ©2014 CloudBees, Inc. All Rights Reserved Continuous Delivery Pipeline Compile & Unit Tests Perfs Tests Selenium Tests Deploy UAT Sonar Analysis Deploy Staging Deploy Production
  11. 11. ©2014 CloudBees, Inc. All Rights Reserved What we need • Complex pipelines involving mul9ple stages • Non-­‐sequen$al logic such as loops, forks … • Long-­‐running builds must survive outages • Interac$ons with humans including pauses, input • Restartable builds in case of transient errors • Reusable defini$ons to avoid duplica9on • Comprehensible scripts with one clear defini9on
  12. 12. Workflow until today ©2014 CloudBees, Inc. All Rights Reserved • Job chaining • Build Flow Plugin • Build Pipeline Plugin • Build Delivery Pipeline
  13. 13. Workflow until today • Many atomic jobs ©2014 CloudBees, Inc. All Rights Reserved • Hard to share variables/state between jobs • Limited logic • Mix build triggers, parameterized build …
  14. 14. Build Flow Plugin • Did have scriptability and extensibility ©2014 CloudBees, Inc. All Rights Reserved • Did not address configura9on “sprawl” • Disjointed view of what really ran • No ability to survive restarts • Almost good enough but could not go further
  15. 15. New Workflow Syntax ©2014 CloudBees, Inc. All Rights Reserved
  16. 16. ©2014 CloudBees, Inc. All Rights Reserved Demo time! https://github.com/cyrille-leclerc/spring-petclinic/blob/petclinic-1.0.0-clc/src/main/jenkins/workflow.groovy
  17. 17. CD Pipeline è Workflow ©2014 CloudBees, Inc. All Rights Reserved Continuous Delivery Pipeline Compile & Unit Tests Perfs Tests Selenium Tests Deploy UAT Sonar Analysis Deploy Staging Deploy Production
  18. 18. ©2014 CloudBees, Inc. All Rights Reserved Key features • En9re flow is one concise Groovy script – For loops, try-­‐finally, fork-­‐join … • Can restart Jenkins while flow is running • Allocate slave nodes and workspaces – As many as you want, when you want • Stages throYle concurrency of builds • Human input/approval integrated into flow • Standard project concepts: SCM, ar9facts, plugins
  19. 19. Groovy DSL vs. Graphical Workflow • Familiar control flow construc9on ©2014 CloudBees, Inc. All Rights Reserved • Familiar “tools” for building abstrac9ons Func9ons, classes, variables, … • Workflow in version control • As opposed to: Graphical workflow designer
  20. 20. ©2014 CloudBees, Inc. All Rights Reserved Project setup • One workflow is defined as a job • Single script for all steps • Build triggers & parameters like regular projects • Each workflow execu9on is a regular Jenkins build displayed in regular Jenkins views • Graphical visualiza9on of actual build possible Not of visualiza9on job defini9on but of build execu9on
  21. 21. Non-sequential logic try {} catch {} ©2014 CloudBees, Inc. All Rights Reserved
  22. 22. Interaction with humans ©2014 CloudBees, Inc. All Rights Reserved
  23. 23. Restartable build / checkpoint ©2014 CloudBees, Inc. All Rights Reserved
  24. 24. ©2014 CloudBees, Inc. All Rights Reserved Visualization
  25. 25. ©2014 CloudBees, Inc. All Rights Reserved Stages • Semaphore • Visualiza9on
  26. 26. Comprehensible scripts ©2014 CloudBees, Inc. All Rights Reserved
  27. 27. Scaling with Jenkins ©2014 CloudBees, Inc. All Rights Reserved
  28. 28. Problem Statement: Scale! • How do I scale Jenkins from 10 to 1000 jobs? Projects? Users? • How do I comply with corporate rules? • How do I keep Jenkins flexible for project teams? ©2014 CloudBees, Inc. All Rights Reserved
  29. 29. Typical Jenkins installations Let’s think about this for a few minutes ©2014 CloudBees, Inc. All Rights Reserved
  30. 30. The top down “structured” and “lets grow vertically” approach ©2014 CloudBees, Inc. All Rights Reserved 5 projects 0 Slaves 25 projects 5 Slaves 75 projects 50 Slaves 150 projects 400 Slaves
  31. 31. Vertical Scaling issues When is the next failure? Single point of failure ©2014 CloudBees, Inc. All Rights Reserved
  32. 32. The “organic” agile growth – “let each group do what they want to” approach Horizontally… virally ©2014 CloudBees, Inc. All Rights Reserved
  33. 33. Horizontal Issues: I Would Like to… • Share executors between masters – Not possible today ©2014 CloudBees, Inc. All Rights Reserved Master-­‐a Master-­‐b Master-­‐c
  34. 34. Horizontal Issues: I Would Like to… • Share security information – Security realm and roles • Push configurations & upgrades ©2014 CloudBees, Inc. All Rights Reserved Master-­‐a Master-­‐b Master-­‐c
  35. 35. oc.cloudbees.com Jenkins Opera9ons Center Sample Topology ©2014 CloudBees, Inc. All Rights Reserved Master-­‐a Master-­‐b Master-­‐c Shared Roles , Security Realm and versions AYached-­‐master client AYached-­‐master client AYached-­‐master client pushed to client master pushed to client master
  36. 36. ©2014 CloudBees, Inc. All Rights Reserved Demo time!
  37. 37. Continuous Delivery traceability with Jenkins and puppet and Chef ©2014 CloudBees, Inc. All Rights Reserved xxx Environment
  38. 38. Puppet environments Prod style environment: puppet apply ©2014 CloudBees, Inc. All Rights Reserved – Master & agents petclinic.war Dev Environment petclinic.war Puppet Agent Application Server war file + puppet manifests & modules “Production style” Environment Puppet Master Puppet Agent war file + puppet manifests & modules Application Server Dev environment: – Standalone agents + “puppet apply”
  39. 39. Test environment with Puppet and Vagrant Dev Environment petclinic.war Puppet Agent Application Server vagrant provision puppet apply war file + puppet manifests ©2014 CloudBees, Inc. All Rights Reserved & modules Vagrant to create “clean” puppet managed servers
  40. 40. CI with Jenkins and Vagrant and Puppet petclinic.war vagrant provision puppet apply Jenkins builds war file and puppet modules and deploys via vagrant ©2014 CloudBees, Inc. All Rights Reserved puppet deployment module java code Dev Environment Puppet Agent Application Server war file + puppet manifests & modules
  41. 41. CI with Jenkins and Vagrant and Puppet petclinic.war notification vagrant provision puppet apply Jenkins builds war file and puppet modules and deploys via vagrant ©2014 CloudBees, Inc. All Rights Reserved puppet deployment module java code Dev Environment Puppet Agent Application Server war file + puppet manifests & modules
  42. 42. Jenkins – Puppet Integration ©2014 CloudBees, Inc. All Rights Reserved
  43. 43. ©2014 CloudBees, Inc. All Rights Reserved Demo time! https://github.com/CloudBees-community/vagrant-puppet-petclinic/tree/vagrant-puppet-petclinic-0.1.0 jenkins-workflow-with-vagrant-and-puppet.groovy: https://gist.github.com/cyrille-leclerc/7f621c341e193622c844
  44. 44. Jenkins & Puppet • Jenkins Deployment No9fica9on Plugin – hYps://wiki.jenkins-­‐ci.org/display/JENKINS/Deployment+No9fica9on+Plugin ©2014 CloudBees, Inc. All Rights Reserved • Puppet module for Jenkins tracking – hYps://github.com/jenkinsci/puppet-­‐jenkinstracking • Puppet Labs plugin for Jenkins – hYps://wiki.jenkins-­‐ci.org/display/JENKINS/Puppet+Plugin • CloudBees DevOps page – hYps://developer.cloudbees.com/bin/view/Main/DevOps • This demo – hYps://github.com/CloudBees-­‐community/vagrant-­‐puppet-­‐petclinic – Jenkins Workflow with Vagrant and Puppet: hYps://gist.github.com/cyrille-­‐leclerc/7f621c341e193622c844
  45. 45. ©2014 CloudBees, Inc. All Rights Reserved Jenkins & Chef? • Similar concepts • Jenkins Deployment No9fica9on Plugin – hYps://wiki.jenkins-­‐ci.org/display/JENKINS/Deployment+No9fica9on+Plugin • Puppet module for Jenkins tracking – hYps://github.com/jenkinsci/puppet-­‐jenkinstracking • Jenkins Chef Tracking – hYps://wiki.jenkins-­‐ci.org/display/JENKINS/Chef+Tracking+Plugin • CloudBees DevOps page – hYps://developer.cloudbees.com/bin/view/Main/DevOps
  46. 46. ©2014 CloudBees, Inc. All Rights Reserved Conclusion • Jenkins workflow to build Con9nuous Delivery pipelines • Jenkins Opera9ons Center by CloudBees to scale Jenkins • Puppet & Chef deployments can be integrated in Jenkins
  47. 47. ©2014 CloudBees, Inc. All Rights Reserved

×