Your SlideShare is downloading. ×
Continuous Delivery of (y)our infrastructure.
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Continuous Delivery of (y)our infrastructure.

1,555

Published on

FlossUK 2014 talk by Kris Buytaert about Continuous Delivery off your Puppet Manifests and Infrastructure.

FlossUK 2014 talk by Kris Buytaert about Continuous Delivery off your Puppet Manifests and Infrastructure.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,555
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
40
Comments
0
Likes
4
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Continuous Delivery of yourContinuous Delivery of your infrastructureinfrastructure Kris Buytaert @krisbuytaert
  • 2. Kris BuytaertKris Buytaert ● I used to be a Dev,I used to be a Dev, ● Then Became an OpThen Became an Op ● Chief Trolling Officer and Open SourceChief Trolling Officer and Open Source Consultant @inuits.euConsultant @inuits.eu ● Everything is an effing DNS ProblemEverything is an effing DNS Problem ● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore ● Some books, some papers, some blogsSome books, some papers, some blogs ● Evangelizing devopsEvangelizing devops
  • 3. Todays GoalsTodays Goals ● A reproducable way to deploy and upgradeA reproducable way to deploy and upgrade (y)our infrastructure(y)our infrastructure ● AutomaticallyAutomatically ● FastFast ● ConsistentConsistent ● ContinuouslyContinuously
  • 4. What's this devops thingWhat's this devops thing anyhow ?anyhow ?
  • 5. C(L)AMSC(L)AMS ● CultureCulture ● (Lean)(Lean) ● AutomationAutomation ● MeasurementMeasurement ● SharingSharing Damon Edwards and John WillisDamon Edwards and John Willis Gene KimGene Kim
  • 6. devops (<)> continuous delilverydevops (<)> continuous delilvery
  • 7. NirvanaNirvana An “ecosystem” that supports continuous delivery, fromAn “ecosystem” that supports continuous delivery, from infrastructure, data and configuration management toinfrastructure, data and configuration management to business.business. Through automation of the build, deployment, and testingThrough automation of the build, deployment, and testing process, and improved collaboration between developers,process, and improved collaboration between developers, testers, and operations, delivery teams can get changestesters, and operations, delivery teams can get changes released in a matter of hours — sometimes even minutes–noreleased in a matter of hours — sometimes even minutes–no matter what the size of a project or the complexity of its codematter what the size of a project or the complexity of its code base.base. Continuous Delivery , Jez HumbleContinuous Delivery , Jez Humble
  • 8. How many times a day ?How many times a day ? ● 10 @ Flickr10 @ Flickr ● Deployments used to be painDeployments used to be pain ● Nobody dared to deploy a siteNobody dared to deploy a site ● Practice makes perfectPractice makes perfect ● Knowing you can vs constantly doing itKnowing you can vs constantly doing it
  • 9. " Our job as engineers (and ops, dev-ops, QA," Our job as engineers (and ops, dev-ops, QA, support, everyone in the company actually) is tosupport, everyone in the company actually) is to enable the business goals. We strongly feel thatenable the business goals. We strongly feel that in order to do that you must havein order to do that you must have the ability tothe ability to deploy code quickly and safelydeploy code quickly and safely. Even if the. Even if the business goals are to deploy strongly QA’d codebusiness goals are to deploy strongly QA’d code once a month at 3am (it’s not for us, we push allonce a month at 3am (it’s not for us, we push all the time), having a reliable and easy deploymentthe time), having a reliable and easy deployment should beshould be non-negotiablenon-negotiable."." Etsy Blog upon releasing DeployinatorEtsy Blog upon releasing Deployinator http://codeascraft.etsy.com/2010/05/20/quantum-of-deployment/http://codeascraft.etsy.com/2010/05/20/quantum-of-deployment/
  • 10. For years we've tolerated humans to makeFor years we've tolerated humans to make structural manual changes to the infrastructurestructural manual changes to the infrastructure our critical applications are running on.our critical applications are running on. Whilst at the same time demanding those criticalWhilst at the same time demanding those critical applications to go through rigid test scenarios.applications to go through rigid test scenarios. Who let this happen ?Who let this happen ?
  • 11. Infrastructure as CodeInfrastructure as Code ● Treat configuration automation as codeTreat configuration automation as code ● Development best practicesDevelopment best practices • Model your infrastructureModel your infrastructure • Version your cookbooks / manifestsVersion your cookbooks / manifests • Test your cookbooks/ manifestsTest your cookbooks/ manifests • Dev/ test /uat / prod for your infraDev/ test /uat / prod for your infra ● Model your infrastructureModel your infrastructure ● A working service = automated ( Application Code + InfrastructureA working service = automated ( Application Code + Infrastructure Code + Security + Monitoring )Code + Security + Monitoring )
  • 12. Version all the thingsVersion all the things No more excuses !No more excuses !• Source code ApplicationSource code Application • Source code InfrastructureSource code Infrastructure • BuildsBuilds • TestsTests • PipelinesPipelines • ScriptsScripts • DocumentationDocumentation • Monitoring scriptsMonitoring scripts
  • 13. A random projectA random project [sdog@mine vagrant-graphite]$ ls manifests modules README TODO Vagrantfile [sdog@mine vagrant-graphite]$ tree -dL 2 . ├── manifests │ └── hosts └── modules ├── apache ├── collectd ├── graphite ├── jmxtrans ├── logster ├── statsd └── tattle 10 directories
  • 14. Software ReleaseSoftware Release management is not amanagement is not a solved problemsolved problem
  • 15. Use Git SubmodulesUse Git Submodules ● Basic git,Basic git, ● No extra tools requiredNo extra tools required Integrates with other projects too.Integrates with other projects too. (No need for *-librarian etc ..)(No need for *-librarian etc ..)
  • 16. Continuous IntegrationContinuous Integration Continuous integration (CI) is the practice, in software engineering, of mergingContinuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day. It wasall developer working copies with a shared mainline several times a day. It was first named and proposed as part of extreme programming (XP). Its main aim isfirst named and proposed as part of extreme programming (XP). Its main aim is to prevent integration problems, referred to as "integration hell"to prevent integration problems, referred to as "integration hell" (WikiPedia)(WikiPedia) Does the app you are deploying still work ?Does the app you are deploying still work ? Did you break your puppet / chef code ?Did you break your puppet / chef code ?
  • 17. JenkinsJenkins ● Open Source Continuous Integration ServerOpen Source Continuous Integration Server ● A zillion plugins (400)A zillion plugins (400) ● Have developers build stable and deployableHave developers build stable and deployable codecode ● Test Infra codeTest Infra code
  • 18. Jenkins PipelineJenkins Pipeline
  • 19. What's in your Pipeline ?What's in your Pipeline ?
  • 20. A pipelineA pipeline ● Checkout codeCheckout code ● SyntaxSyntax ● StyleStyle ● Code CoverageCode Coverage ● TestsTests ● BuildBuild ● More TestsMore Tests ● PackagePackage
  • 21. Syntax and StyleSyntax and Style ● Initially ,Initially , all code, all the timeall code, all the time ● Now,Now, only the changed codeonly the changed code ● Why not in post Commit Hooks ?Why not in post Commit Hooks ?
  • 22. Package all the thingsPackage all the things
  • 23. Artifacts:Artifacts: ● Tested artifcts that go trough a pipelineTested artifcts that go trough a pipeline application code,application code, Infra codeInfra code metadatametadata teststests
  • 24. Why ops like to packageWhy ops like to package ● Packages give you featuresPackages give you features •Consistency, security, dependenciesConsistency, security, dependencies ● Uniquely identify where files come fromUniquely identify where files come from •Package or cfg-mgmtPackage or cfg-mgmt ● Source repo not always availableSource repo not always available •Firewall / Cloud etc ..Firewall / Cloud etc .. ● Weird deployment locations , no easy accessWeird deployment locations , no easy access ● Little overhead when you automateLittle overhead when you automate
  • 25. Jordan Sissel is a Hero !Jordan Sissel is a Hero !
  • 26. #packaginlove#packaginlove
  • 27. It's not really packagingIt's not really packaging • It's an immutable branchIt's an immutable branch • It's a tracable release artefactIt's a tracable release artefact
  • 28. https://github.com/vStone/jenkins-https://github.com/vStone/jenkins- puppet-scriptspuppet-scripts ● TestsTests ● Packages full tree inPackages full tree in // etc/puppet/environmetc/puppet/environm ents/$environment/ents/$environment/
  • 29. A pipelineA pipeline ● Checkout codeCheckout code ● SyntaxSyntax ● StyleStyle ● Code CoverageCode Coverage ● TestsTests ● BuildBuild ● More TestsMore Tests ● PackagePackage ● Upload to RepoUpload to Repo
  • 30. Repository ManagementRepository Management ● PulpPulp • Pro : MirroringLovePro : MirroringLove • Con : Mongo, Stability, .debCon : Mongo, Stability, .deb ● PRM ?PRM ? ● https://github.com/ImmobilienScout24/yum-repo-sehttps://github.com/ImmobilienScout24/yum-repo-se ??
  • 31. Repository ManagementRepository Management
  • 32. A pipelineA pipeline ● Checkout codeCheckout code ● SyntaxSyntax ● StyleStyle ● Code CoverageCode Coverage ● TestsTests ● BuildBuild ● More TestsMore Tests ● PackagePackage ● Upload to RepoUpload to Repo ● Deploy on TestDeploy on Test
  • 33. mc-packagemc-package
  • 34. Repos are SLOWRepos are SLOW ● Createrepo is slow.Createrepo is slow. ● Pulp is slowPulp is slow ● Bypass repos , upload straight to appropriateBypass repos , upload straight to appropriate PuppetMasterPuppetMaster ● Upload to repo for rebootstrappingUpload to repo for rebootstrapping
  • 35. A pipelineA pipeline ● Checkout codeCheckout code ● SyntaxSyntax ● StyleStyle ● Code CoverageCode Coverage ● TestsTests ● BuildBuild ● More TestsMore Tests ● PackagePackage ● Upload to RepoUpload to Repo ● Deploy on TestDeploy on Test ● Check PuppetrunsCheck Puppetruns ● Check MonitoringCheck Monitoring ●
  • 36. Testing = MonitoringTesting = Monitoring ● Deploy a host,Deploy a host, ● Add it to the monitoring frameworkAdd it to the monitoring framework ● Add collection toolsAdd collection tools ● Add check definitionsAdd check definitions ● Update the monitoring tool configUpdate the monitoring tool config FULLY AUTOMATEDFULLY AUTOMATED
  • 37. e.g. Stored Configse.g. Stored Configs
  • 38. Collection and ExportCollection and Export Export :Export : @@resource {@@resource { ... }... } Collect:Collect: Resource <<| query |Resource <<| query | >>>> Clean out nodes that dissapearClean out nodes that dissapear puppet node cleanpuppet node clean
  • 39. Exporting and CollectingExporting and Collecting
  • 40. Monitoring a VhostMonitoring a Vhost
  • 41. A pipelineA pipeline ● Checkout codeCheckout code ● SyntaxSyntax ● StyleStyle ● Code CoverageCode Coverage ● TestsTests ● BuildBuild ● More TestsMore Tests ● PackagePackage ● Upload to RepoUpload to Repo ● Deploy on TestDeploy on Test ● Check PuppetrunsCheck Puppetruns ● Check MonitoringCheck Monitoring ● Promote to UATPromote to UAT
  • 42. Jenkins PromotionJenkins Promotion
  • 43. Done ?Done ? ● Close the feedback loop,Close the feedback loop, ● Send metric on deploymentSend metric on deployment echo "deployed.$package_name 1 `date +%s`"echo "deployed.$package_name 1 `date +%s`" > /dev/tcp/<%= graphite_host %>/2003> /dev/tcp/<%= graphite_host %>/2003
  • 44. ContactContact Kris BuytaertKris Buytaert Kris.Buytaert@inuits.beKris.Buytaert@inuits.be Further ReadingFurther Reading @krisbuytaert@krisbuytaert http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/ http://www.inuits.be/http://www.inuits.be/ InuitsInuits Duboistraat 50Duboistraat 50 2060 Antwerpen2060 Antwerpen BelgiumBelgium 891.514.231891.514.231 +32 475 961221+32 475 961221

×