Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

On the Importance of Infrastructure as Code

1,152 views

Published on

IcingaCamp Amsterdam 2016 Talk on the Importance of Infrastructure as Code

Published in: Technology
  • Be the first to comment

On the Importance of Infrastructure as Code

  1. 1. On theimportanceof InfrastructureasCode Kris Buytaert @krisbuytaert
  2. 2. Kris BuytaertKris Buytaert● I used to be a Dev,I used to be a Dev, ● Then Became an OpThen Became an Op ● Chief Trolling Officer and Open SourceChief Trolling Officer and Open Source Consultant @Consultant @inuits.euinuits.eu ● Everything is an effing DNS ProblemEverything is an effing DNS Problem ● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore ● Some books, some papers, some blogsSome books, some papers, some blogs ● Evangelizing devopsEvangelizing devops ● Organiser of #devopsdays, #cfgmgmtcamp,Organiser of #devopsdays, #cfgmgmtcamp, #loadays, ….#loadays, …. ● Part of the travelling geek circusPart of the travelling geek circus
  3. 3. What's this devopsWhat's this devops thing anyhow ?thing anyhow ?
  4. 4. C(L)AMSC(L)AMS ● CultureCulture ● (Lean)(Lean) ● AutomationAutomation ● MeasurementMeasurement ● SharingSharing Damon Edwards and John WillisDamon Edwards and John Willis Gene KimGene Kim
  5. 5. Why automate ?Why automate ?
  6. 6. Common ProblemsCommon Problems ● Many manual changes to systems ● Many undocumented changes ● Emergency Administration only ● Disaster Recovery site is a Disaster ● Time to deliver a box is to slow ● All boxen are different ● Computers don’t work hard enough for us
  7. 7. More ProblemsMore Problems ● How long does it take to reinstall a machine from 0 ● To the exact same point as before ? ● With different Hardware ? In a different cloud ? ● What about your (customer/personal data )
  8. 8. Security ?Security ? ● Monitoring that your platform hasn't changed.Monitoring that your platform hasn't changed. • Why is selinux disabled ?Why is selinux disabled ? • Who added / dropped that firewall ?Who added / dropped that firewall ? • What did this originally look like ?What did this originally look like ? • Is this file really what Bernd meant it to be ?Is this file really what Bernd meant it to be ?
  9. 9. #monitoringsucks#monitoringsucks ● Monitoring is out of sync with realityMonitoring is out of sync with reality ● Managed manuallyManaged manually ● Can't keep up..Can't keep up..
  10. 10. Do you want to ?Do you want to ? ● Install these racks manuallyInstall these racks manually ● Over and over again ?Over and over again ? ● And can you guarantee that installs areAnd can you guarantee that installs are identical ?identical ? ● ““No simple admin taks is fun more thanNo simple admin taks is fun more than twice”twice” ● s/twice/once/g;s/twice/once/g; ● Repeating installs are boring and prone toRepeating installs are boring and prone to errorserrors ● Each installation is unintentionally UniqueEach installation is unintentionally Unique ● Manual installs DO NOT scaleManual installs DO NOT scale
  11. 11. ChallengesChallenges ● ReproducabilityReproducability ● SpeedSpeed ● AuditingAuditing ● Keeping stuff in syncKeeping stuff in sync • MonitoringMonitoring • SecuritySecurity • BackupBackup
  12. 12. The 10The 10thth floor testfloor test ● Grab a random machine (don’t take a backup before) ● Throw it out a 10th floor window ● Can you recover it in 10 minutes ?
  13. 13. Facts!Facts! ● Data Backup is only a part ● Sysadmin backup needs to be done also ● Manual Installations = bad ● Bad installations = unusable infrastructure ● Bad installations = unproductive users ● Bad installations = manual efforts ● Manual efforts = no time ● No time = no updates no patches no security ● Manual work = high costs
  14. 14. Deploying an InfrastructureDeploying an Infrastructure ● 1996 : Manual Installations1996 : Manual Installations ● 2001 : Mondo rescue2001 : Mondo rescue (reproducable single instances)(reproducable single instances) ● 2003 : SystemImager2003 : SystemImager • Reproducable Infrastructure , withReproducable Infrastructure , with “OVERRIDES”“OVERRIDES” • Fast Multicast Image deploymentsFast Multicast Image deployments • Image Sprawl (thank you VMware)Image Sprawl (thank you VMware)
  15. 15. Deploying an InfrastructureDeploying an Infrastructure ● 1996 : Manual Installations1996 : Manual Installations ● 2001 : Mondo rescue2001 : Mondo rescue ● 2003 : SystemImager2003 : SystemImager ● 2005 :2005 : Kickstart / FAIKickstart / FAI • Dreaming of Jeos + IAC (Cfengine)Dreaming of Jeos + IAC (Cfengine)
  16. 16. Deploying an InfrastructureDeploying an Infrastructure ● 1996 : Manual Installations1996 : Manual Installations ● 2001 : Mondo rescue2001 : Mondo rescue ● 2003 : SystemImager2003 : SystemImager ● 2005 : Dreaming of Jeos + IAC2005 : Dreaming of Jeos + IAC ● 2008 : Actual JeOS + IAC2008 : Actual JeOS + IAC ● 2010 : Vagrant for development2010 : Vagrant for development
  17. 17. Imagesprawl ANDImagesprawl AND SnowflakesSnowflakes ● Image Sparwl :Image Sparwl : • Copy vm 3xCopy vm 3x • Modify 2xModify 2x • Copy 21xCopy 21x • How the Heck did we get here ?How the Heck did we get here ? ● SnowFlakes :SnowFlakes : • Don't touch this box it might breakDon't touch this box it might break • Look how nice it is !Look how nice it is !
  18. 18. You never deployYou never deploy something “just” oncesomething “just” once ● Local test … experiment,Local test … experiment, • Vagrant box / local containersVagrant box / local containers ● Integration PlatformIntegration Platform • Same codebase,, different environmentSame codebase,, different environment ● Dev/ UAT/ Prod / DR …Dev/ UAT/ Prod / DR … ● Or your customer just forgot to renew the leaseOr your customer just forgot to renew the lease on his VPS. #toldyousoon his VPS. #toldyouso
  19. 19. What's different in the cloud ?What's different in the cloud ? ● ScaleScale ● VelocityVelocity ● ChangeChange
  20. 20. Your machines as CattleYour machines as Cattle
  21. 21. Treat your people as petsTreat your people as pets
  22. 22. Configuration MgmtConfiguration Mgmt ● Configure 1000 nodes,Configure 1000 nodes, ● Modify 15000 files,Modify 15000 files, ● Think :Think : •Cfengine,Puppet, Chef, SaltCfengine,Puppet, Chef, Salt ● Put configs under version controlPut configs under version control ● Please don't roll your own ...Please don't roll your own ...
  23. 23. Infrastructure as CodeInfrastructure as Code ● Treat configuration automation as codeTreat configuration automation as code ● Development best practicesDevelopment best practices • Model your infrastructureModel your infrastructure • Version your cookbooks / manifestsVersion your cookbooks / manifests • Test your cookbooks/ manifestsTest your cookbooks/ manifests • Dev/ test /uat / prod for your infraDev/ test /uat / prod for your infra ● Model your infrastructureModel your infrastructure ● A working service = automated ( Application Code + InfrastructureA working service = automated ( Application Code + Infrastructure Code + Security + Monitoring )Code + Security + Monitoring ) ● IAC -ne scripting (or translating bash to yaml)IAC -ne scripting (or translating bash to yaml)
  24. 24. IAC Is a TestingIAC Is a Testing RequirementRequirement ● Stable reproducable starting pointStable reproducable starting point
  25. 25. AuditabilityAuditability ● git loggit log ● git blamegit blame ● Review,Review, ● authorizationauthorization
  26. 26. File monitoringFile monitoring
  27. 27. Fixing Monitoring FatigueFixing Monitoring Fatigue
  28. 28. Stored ConfigsStored Configs
  29. 29. Collection and ExportCollection and Export Export :Export : @@resource {@@resource { ... }... } Collect:Collect: Resource <<| query |Resource <<| query | >>>> Clean out nodes that dissapearClean out nodes that dissapear puppet node cleanpuppet node clean
  30. 30. Use Cases:Use Cases: ● Ssh keysSsh keys ● Reverse proxy configsReverse proxy configs ● Monitoring resourcesMonitoring resources ● Measuring resourcesMeasuring resources
  31. 31. Puppetmaster Example:Puppetmaster Example:
  32. 32. Defining a ServiceDefining a Service ● profile that :profile that : • Configures service using a standardConfigures service using a standard module call with hiera based parametersmodule call with hiera based parameters • Configures BackupConfigures Backup • Configures logrotationConfigures logrotation • Configures logshippingConfigures logshipping • Exports Monitoring NeedsExports Monitoring Needs
  33. 33. Chronicle of a failedChronicle of a failed private cloudprivate cloud● Tool X provisions a VMTool X provisions a VM • 3 weeks from the request / can only be done by 1 team3 weeks from the request / can only be done by 1 team ● Tool Y installs patchesTool Y installs patches • 2 weeks2 weeks ● Team Z installs backupTeam Z installs backup • 1 day1 day ● Team A installs monitoringTeam A installs monitoring • 3 weeks3 weeks ● AppApp • Manual deploy on wrong JVM, return to senderManual deploy on wrong JVM, return to sender
  34. 34. Application IncludedApplication Included ● Application =Application = • PackagePackage • ConfigConfig • ServiceService ● No manual scriptingNo manual scripting ● Think about your bootstrapping / scaleoutThink about your bootstrapping / scaleout
  35. 35. Automation ofAutomation of #monitoring#monitoring brought backbrought back thethe #love#love
  36. 36. ConclusionConclusion ● IAC solves a lot of problemsIAC solves a lot of problems • Improves SecurityImproves Security • Creates Monitoring LoveCreates Monitoring Love • Creates SpeedCreates Speed ● But it still is code, and needs to be treated likeBut it still is code, and needs to be treated like code !code !
  37. 37. ContactContact Kris BuytaertKris Buytaert Kris.Buytaert@inuits.beKris.Buytaert@inuits.be Further ReadingFurther Reading @krisbuytaert@krisbuytaert http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/ http://www.inuits.be/http://www.inuits.be/ InuitsInuits Essensteenweg 31Essensteenweg 31 BrasschaatBrasschaat BelgiumBelgium 891.514.231891.514.231 +32 475 961221+32 475 961221

×