Automated Deployment Pipeline using Jenkins, Puppet, Mcollective and AWS


Published on

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Avoid, cloning of package repository.
  • Automated Deployment Pipeline using Jenkins, Puppet, Mcollective and AWS

    1. 1. Continuous Delivery Pipeline with Jenkins, Puppet and Mcollective From an Engineer’s perspective @diptanu
    2. 2. DevOps ?
    3. 3. Continuous Delivery
    4. 4. Why Continuous Deployment• Get measurable changes fast• Reduce the risk of releases• Loosely coupled but coherent teams• Get rid of the “works-on-my-box” phenomenon• Forces you to think where does state full things live in your system
    5. 5. First Principles• CI is just the orchestrator• Package and version everything• Don’t use the build tool after tests have been run and packages have been made• Divide and Conquer – Use tools that solve one particular problem• Think hard what the pillars of infrastructure are
    6. 6. Jenkins• Distributed Builds• Shard responsibility of build infrastructure – For example, don’t treat the CI server as your artifact repo• Think about what happens when Jenkins goes down• Discard old builds
    7. 7. Jenkins• Use the Jenkins Job Builder to configure jobs• DynaSlave plugin if you need a lot of scale• Build Flow Plugin/Build Pipeline Plugin for visualization• Job DSL Plugin• Configuration Slicing Plugin
    8. 8. Jenkins• Disable Jobs that haven’t built successfully• Set number of builds• Mark build permanent<3 Jenkins everyday
    9. 9. Puppet• Infrastructure as code• Node-less and master less puppet• Make your vizualization tool consume the reports• Fact Driven puppet• Masterless puppet• Decouple data from manifests
    10. 10. NodeLess Puppet• No definitions like node “”• There are only “types” of instances node default { include truth::enforcer }
    11. 11. NodeLess Puppetclass truth::enforcer { if has_role("ci"){ notice("I am a CI Server") include tools::ci } if has_role(”webserver"){ notice(“I am a webserver”) include tools::frontend}}
    12. 12. MCollective Web Servers Application Servers Cacheing Servers Metrics/Loggin g
    13. 13. MCollective• Create Collectives of resources on the network• And operate on collectives rather than single hosts• Understands facts that are exposed by facter mco package –W /dev_server/ install foo-puppet-86 mco service –W /application_servers/ restart iptables
    14. 14. AWS• Ruby/Python SDK• Immutable compute instances• Make instances join the collective Load Compute Nodeless Puppet Balancer Instance Monitori ng Jenkins System
    15. 15. A Pipeline in the wild Team City Build Code Daily ManualCommit Deployment TFS Build DB Manager Quality Assurance Fortnightly Manual Deployment
    16. 16. Software Delivery Pipeline Publish Automated Deploy to Smoke TestsCommit Package Tests DevDeploy to QA Deploy to IntegrationEnvironments Environments
    17. 17. Deployment Pipeline Test andDevelopers SCM Package Ops Package Repositorycurl -i -F package=@foo-puppet-86-1.i386.rpm http://selihost/<repository_name>/upload
    18. 18. Deployment Pipeline Run your functionalDeploy to Dev tests from a different host mco Seli/Jenkins Xtrigger a. yum install –y foobar-puppet-86-1 Dev Instances b. sudo puppet apply … Package Repository
    19. 19. Package Repositories• Upload a package whenever a build happens• Serve Packages• Archive Strategy• Trigger the deployment pipeline whenever• Statistics• API to get status of the metadata
    20. 20. Introducing Seli• A REST API to upload packages• Comes with an embedded HTTP Server• Shows Stats• And specify rules for archiving packages• Tag Packages• Triggers URL when a package is uploaded
    21. 21. In closing• Make your CD Pipeline reflect your release workflow• Automate everything• Measure everything that moves• Think about your infrastructure as collectives
    22. 22. Questions? @diptanu