Continuous Delivery of Puppet Manifests
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Continuous Delivery of Puppet Manifests

on

  • 2,598 views

PuppetCamp Amsterdam 2014 Talk

PuppetCamp Amsterdam 2014 Talk

Statistics

Views

Total Views
2,598
Views on SlideShare
2,573
Embed Views
25

Actions

Likes
4
Downloads
27
Comments
0

3 Embeds 25

https://twitter.com 21
http://www.linkedin.com 2
http://www.slideee.com 2

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Continuous Delivery of Puppet Manifests Presentation Transcript

  • 1. Continuous Delivery of Puppet Code, Lessons Learned Kris Buytaert @krisbuytaert
  • 2. Kris Buytaert ● ● ● ● ● ● ● I used to be a Dev, Then Became an Op Chief Trolling Officer and Open Source Consultant @inuits.eu Everything is an effing DNS Problem Building Clouds since before the bookstore Some books, some papers, some blogs Evangelizing devops
  • 3. Todays Goals ● A reproducable way to deploy and upgrade /etc/puppet ● Automatically ● Fast ● Consistent ● Continuously
  • 4. What's this devops thing anyhow ?
  • 5. devops ● Culture ● (Lean) ● Automation ● Measurement ● Sharing Damon Edwards and John Willis Gene Kim
  • 6. devops (<)> continuous delilvery
  • 7. Nirvana An “ecosystem” that supports continuous delivery, from infrastructure, data and configuration management to business. Through automation of the build, deployment, and testing process, and improved collaboration between developers, testers, and operations, delivery teams can get changes released in a matter of hours — sometimes even minutes–no matter what the size of a project or the complexity of its code base. Continuous Delivery , Jez Humble
  • 8. How many times a day ? ● 10 @ Flickr ● Deployments used to be pain ● Nobody dared to deploy a site ● Practice makes perfect ● Knowing you can vs constantly doing it
  • 9. " Our job as engineers (and ops, dev-ops, QA, support, everyone in the company actually) is to enable the business goals. We strongly feel that in order to do that you must have the ability to deploy code quickly and safely. Even if the business goals are to deploy strongly QA’d code once a month at 3am (it’s not for us, we push all the time), having a reliable and easy deployment should be non-negotiable." Etsy Blog upon releasing Deployinator http://codeascraft.etsy.com/2010/05/20/quantum-of-deployment/
  • 10. For years we've tolerated humans to make structural manual changes to the infrastructure our critical applications are running on. Whilst at the same time demanding those critical applications to go through rigid test scenarios. Who let this happen ?
  • 11. I hate maturity model
  • 12. So, what did we try already ?
  • 13. Level -1:Hacking in production • Cd /etc/puppet/manifests • Vi site.pp
  • 14. Level -1: more production hacking • cd /etc/puppet/manifests • vi site.pp • • • rsync -av * /etc/puppet • WFM Syndrome • Does not work in teams
  • 15. Level 0: I use git • ssh puppetmaster • cd /etc/puppet • git init • git add *.pp • git commit -m “Initial commit of puppet tree”
  • 16. Level 0.1: I almost understand git • • Ssh puppetmaster • Cd /etc/puppet • Git init • Git add *.pp • Git commit “Initial commit of puppet tree” • Git remote add • Git push
  • 17. Level 0.2: But • Ssh puppetmaster • Cd /etc/puppet • Git init • Git add *.pp • Git commit “Initial commit of puppet tree” • Git remote add • Git push • Vi nodes/default.pp
  • 18. Level 0.2:But • I don't always commit or push my changes • When I do commit I do it as root
  • 19. Level 0.3: Development happens locally Puppetmaster runs git pull in a cronjob => code modified on puppet master, not pushed => changes never make it to the platform
  • 20. Level 0.3: Euh modules vi modules/apache/manifest/init.pp Wait, I need to track upstream , How do I isolate my code ?
  • 21. Git subtrees ● Looks nice ● Till you want to track upstream
  • 22. Librarian Puppet ● Hides complexity of submodules ● Easy if you use Forge Modules • • ● ● Does anyone ? Do you trust the internet to be around Librarian = Old English for “can't use submodules” And hmm... which customer uses which patched version again ?
  • 23. Librarian Puppet ● Insert ugly shell script ● ● Even with this in place .. people can still hack on the PuppetMaster
  • 24. We all love branches ● When they are short lived feature branches ● Environment per branch ? • • How many hosts do you connect per branch ? Limited number of branches ?
  • 25. Is R10K faster ? ● R10K ● But what about Testing ? ● You can't do CD without CI
  • 26. Git Submodules ● Basic git, ● No extra tools required Integrates with other (non puppet) projects too
  • 27. Package all the things
  • 28. Release artifacts: ● A module ● A set of manifests ● A set of manifests that work with a strict set of modules
  • 29. Software Release management is not a solved problem
  • 30. Git Submodules ● Release management = main git project cd modules/blah vi manifest/init.pp git add manifests/init.pp git commit -m “Fixed bug #313” cd ../../ git add modules/blah git commit -m “Fixed bug #313” git push
  • 31. Infrastructure as Code ● Treat configuration automation as code ● Development best practices • Model your infrastructure • Version your cookbooks / manifests • Test your cookbooks/ manifests • Dev/ test /uat / prod for your infra ● Model your infrastructure ● A working service = automated ( Application Code + Infrastructure Code + Security + Monitoring )
  • 32. Continuous Integration ● Builds ● Nightly Builds ● Builds with tests ● Nightly Builds with tests ● Frequent integration ● Continuous Integration
  • 33. Jenkins ● Open Source Continuous Integration Server ● A zillion plugins (400) ● Have developers build stable and deployable code ● Test Infra code
  • 34. Jenkins Pipeline
  • 35. What's in your Pipeline ?
  • 36. A pipeline ● Checkout code ● Syntax ● Style ● Code Coverage ● Tests ● Build ● More Tests ● Package
  • 37. Syntax and Style ● Initially , all code, all the time ● Now, only the changed code ● Why not in post Commit Hooks ?
  • 38. Why ops like to package ● Packages give you features •Consistency, security, dependencies ● Uniquely identify where files come from •Package or cfg-mgmt ● Source repo not always available •Firewall / Cloud etc .. ● Weird deployment locations , no easy access ● Little overhead when you automate
  • 39. Jordan Sissel is a Hero !
  • 40. #packaginlove
  • 41. It's not really packaging • It's an immutable branch • It's a tracable release artefact
  • 42. https://github.com/vStone/jenkinspuppet-scripts ● Tests ● Packages full tree in / etc/puppet/environm ents/$environment/
  • 43. A pipeline ● Checkout code ● Syntax ● Style ● Code Coverage ● Tests ● Build ● More Tests ● Package ● Upload to Repo
  • 44. Repository Management ● Pulp • Pro : MirroringLove • Con : Mongo, Stability, .deb ● PRM ? ● https://github.com/ImmobilienScout24/yum-repo-se ?
  • 45. Repository Management
  • 46. A pipeline ● Checkout code ● Upload to Repo ● Syntax ● Deploy on Test ● Style ● Code Coverage ● Tests ● Build ● More Tests ● Package
  • 47. mc-package
  • 48. Repos are SLOW ● Createrepo is slow. ● Pulp is slow ● Bypass repos , upload straight to appropriate PuppetMaster ● Upload to repo for rebootstrapping
  • 49. A pipeline ● Checkout code ● Upload to Repo ● Syntax ● Deploy on Test ● Style ● Check Puppetruns ● Code Coverage ● Check Icinga ● Tests ● Promote to UAT ● Build ● More Tests ● Package
  • 50. Jenkins Promotion
  • 51. Done ? ● Close the feedback loop, ● Send metric on deployment echo "deployed.$package_name 1 `date +%s`" > /dev/tcp/<%= graphite_host %>/2003
  • 52. Contact Kris Buytaert Kris.Buytaert@inuits.be Further Reading @krisbuytaert http://www.krisbuytaert.be/blog/ http://www.inuits.be/ Inuits Duboistraat 50 2060 Antwerpen Belgium 891.514.231 +32 475 961221