Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Infrastructure as Code in your CD pipelines - London Microsoft DevOps 0423

53 views

Published on

London Microsoft DevOps 23 April 2018 Meetup (https://www.meetup.com/London-Microsoft-DevOps/events/249114256/)
Infrastructure as Code in your CD pipelines
from VMs to Containers

He is going to cover the Journey of agile transformation in a non-IT company, bringing in Continuous Delivery, traditional infrastructure and modern cloud DevOps practices.

In this talk, you will hear about the DevOps journey in his company (Glass, Lewis & Co.), from the initial brown-field all-manual state to the current partially automated situation and the strategic destination of a fully automated and monitored process.

In an equilibrium between a high-level view and useful practical tips, he will touch on what informed their decisions, in terms of priorities and technologies, some lessons learned in setting up Infrastructure-as-Code using Terraform for Azure, and how the legacy constraints helped or hindered them on this journey.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Infrastructure as Code in your CD pipelines - London Microsoft DevOps 0423

  1. 1. Infrastructure as Code in your Continuous Delivery pipelines from VMs to Containers Giulio Vian 23rd April 2018 giulio.dev@casavian.eu @giulio_vian
  2. 2. I’m the Doctor. Do everything I tell you, don’t ask stupid questions and don’t wander off. Doctor Who © BBC Story 203 Series 5, Episode 1
  3. 3. but I’m not The Doctor 3 giulio.dev@casavian.eu @giulio_vian http://blog.casavian.eu/ https://tfsaggregator.github.io Hardware spec: 1KB RAM (upg. 16KB) 4KB ROM
  4. 4. What we will talk about? Introduction Company state & Whys Infrastructure-as-Code Mindset & Definitions Layer 1 – Terraform Layer 2 – DSC Layer 3 – Deployment Blurring lines: Docker Wrap-up 4 200-level Visual and practical Deck on SlideShare Bibliography at the end
  5. 5. Each topic can be a talk on its own!
  6. 6. Introduction Company state & Whys
  7. 7. No green-field 3 Active Directory domains + workgroup 3 test environments 2 production environments All manually built Mainly manual deployments TeraByte-size SQL instances VPN connections Centralized version control 4 source code branches
  8. 8. Issues Downtime New releases Windows Update Lack of Scalability Don’t touch it mindset
  9. 9. Technology soup Hosting OS & DB Language
  10. 10. Growth crisis
  11. 11. R e b i u l d
  12. 12. Automation Pillars Infrastructure Application stack Pipelines
  13. 13. What? No ARM? Doctor Who © BBC Story 178 Series 3, Christmas Episode
  14. 14. Three reasons, well, no, … four
  15. 15. JSON is sooo cute
  16. 16. I have an opinion on anything JavaScript… The IT Crowd © Channel 4 Season 3, Episode 1 From Hell
  17. 17. Summary Brownfield apps Brownfield infrastructure Tactical decisions Freight of technical debt Veering course Lust for innovation
  18. 18. Mindset & Definitions Infrastructure-as-Code
  19. 19. Infrastructure-as-Code mindset No manual changes Replace hand built resources with automation Source control
  20. 20. Execution environment Operating System O.S. Modules Local Services, Libraries Application Network & External Services Data 20
  21. 21. Execution environment Operating System O.S. Modules Local Services, Libraries Application Network & External Services Data 21
  22. 22. Investment Takes more time Until you are proficient
  23. 23. Summary Everything automated Nothing comes for free Arrange technologies Spread knowledge
  24. 24. Layer 1 – Terraform Infrastructure-as-Code
  25. 25. A taste of Terraform
  26. 26. Terraform peculiarities State management Code organization Import existing Names are immutable
  27. 27. Terraform state is Metadata about a chunk of your infrastructure
  28. 28. State management Myth: State is map of reality Setup in shared, locked place Azure Storage or AWS S3 Some changes not sensed Learn to use terraform state
  29. 29. Stay organized / repo root modules terraform modules utility general purpose shared common to multiple applications or environments application_name internal or public application non-production can be rebuilt any moment shared common to multiple environments e.g. deploy agents, jumpbox qa Integration test uat User acceptance test perf Load testing production everything here is critical ... details on next slide
  30. 30. Stay organized (cont’d) / repo root production everything here is critical legacy hand made infrastructure e.g. TFS shared common to main and DR e.g. networking live PRODUCTION ENVIRONMENTS network “everlasting” resources data-tier long-lived resources app-tier short-lived resources app_name resources for an app dr Disaster recovery site ... As above
  31. 31. Folders and state Each leaf has a state file Source can refer to existing state files production shared production/shared/terraform.tfstate live network production/live/network/terraform.tfstate app-tier production/live/app-tier/terraform.tfstate
  32. 32. Three steps to import Define as regular resources Add safety clause lifecycle { prevent_destroy = true } Include in state terraform import
  33. 33. Changing names TF deletes and rebuild resource There can be more than one? Security Group Rules Virtual Machine Extensions Disks More is better environment-tier-role-object-instance
  34. 34. Terraform tips Static addresses cidrsubnet cidrhost HCL parser idiosyncrasies Regex might be troublesome is not unusual Study the book
  35. 35. Better luck next time Error: Error applying plan: azurerm_lb_probe.lb_probe_http: Error Creating/Updating LoadBalancer network.LoadBalancersClient#CreateOrUpdate : Failure sending request: StatusCode=0 -- Original Error: Put https://management.azure.com/subscriptions /12345678-9abc-def0-1234- 56789abcdef0/resourceGroups/qa/providers/M icrosoft.Network/loadBalancers/qa- loadbalancer?api-version=2017-09-01: http: ContentLength=1655 with Body length 0
  36. 36. Terraform and Pipelines Command line tool terraform init terraform plan –out temp.json –input=false -detailed-exitcode terraform apply temp.json –input=false –auto-approve Agent must see resource providers State in shared place There are Tasks out there https://marketplace.visualstudio.com/items?itemName=petergroenewegen.Peter Groenewegen-Xpirit-Vsts-Release-Terraform
  37. 37. Terraform & VSTS
  38. 38. Before you do anything rash, like pressing another button, may I make an alternative suggestion? Doctor Who © BBC Story 79 Season 12, Episode 20
  39. 39. IaC Continuous Deployment when Spin-off a timeboxed environment Periodical rebuilding Security patches Eliminate drifting
  40. 40. Summary Learn Terraform way Learn its role Think, don’t hack Easy to drop in pipelines …but will you do it?
  41. 41. Layer 2 – Powershell DSC Infrastructure-as-Code
  42. 42. Powershell Search, search, search Careful with StackOverflow Desired State Configuration Declarative configuration Module management Install-Module is just the first step Testing Pester Limited use
  43. 43. Desired State Configuration (DSC) Configuration MyBox { # Install the IIS role WindowsFeature IIS { Ensure = "Present" Name = "Web-Server" } # Install the ASP .NET 4.5 role WindowsFeature AspNet45 { Ensure = "Present" Name = "Web-Asp-Net45" } #... } Reboots Modes Local / Push Pull ConfigurationMode ApplyOnly ApplyAndMonitor  ApplyAndAutocorrect PowerShell Gallery
  44. 44. Caveat Windows Management Framework (WMF) Powershell Core (≥6.0)
  45. 45. a sip of DSC
  46. 46. Fake it and retry Powershell Remote Powershell DSC
  47. 47. DSC best friend Configuration MyOtherBox { #... cChocoInstaller InstallChoco { DependsOn = '[Script]SetRemoteSignedExecutionPolicy' InstallDir = $chocoDir } cChocoPackageInstaller Chrome { Name = 'googlechrome' DependsOn = '[cChocoInstaller]InstallChoco' Ensure = 'Present' } #... } Chocolatey Package Manager Endorsement? No trust? Own Internal Package Repository Server Study chocolateyInstall.ps1
  48. 48. Working together (upload local scripts) Public repo Orchestrating tool data "external" "uploader_data" { program = ["powershell", "${path.module}/uploader.ps1 -Folder "${var.folder}" -StorageAccount ${var. storage_account} -Container ${var. storage_container} -StorageKey ${var. storage_key}"] } Deploy pipeline step
  49. 49. Working together DSC Pull Server Azure Automation Coordination!!!
  50. 50. DSC & Azure
  51. 51. Managing credentials Terraform -> DSC Extension via protected settings (just like ARM) Automation Credential Script pulls At compile time No UI, Powershell only
  52. 52. Summary Best friend for Windows Robust Easy to centralize Plan for security Plan for modularity
  53. 53. Wrap-up 53
  54. 54. Working software over comprehensive documentation IaC embodies this Agile Principle!
  55. 55. Unexplored paths Database SQL Clusters Feature Toggles
  56. 56. To know more Terraform - Up and Running: Writing Infrastructure as Code — Y.Brikman (O′Reilly) https://www.amazon.co.uk/gp/product/14919770 86/ The DSC Book — Don Jones and Melissa Januszko (O′Reilly) https://leanpub.com/the-dsc-book 1491977086/ The Docker Book — James Turnbull https://www.dockerbook.com/ 56
  57. 57. To know more (cont’d) Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation — J.Humble, D.Farley (Addison-Wesley) https://www.amazon.com/Continuous- Delivery/dp/0321601912/ The DevOps Handbook — G.Kim, P.Debois, J.Willis, J.Humble (IT Revolution Press) https://www.amazon.com/DevOps-Handbook- World-Class-Reliability- Organizations/dp/1942788002/ 57
  58. 58. To know more (cont’d again) Refactoring Databases — Scott J Ambler and Pramod J. Sadalage (Addison-Wesley) https://www.amazon.com/Refactoring-Databases- Evolutionary-paperback-Addison-Wesley/dp/0321774515/ DevOps on the Microsoft Stack — Wouter de Kort (Apress) https://www.amazon.com/DevOps-Microsoft-Stack-Wouter- Kort/dp/1484214471/ Beginning Build and Release Management with TFS 2017 and VSTS — Chandrasekara, Chaminda (Apress) http://www.apress.com/gp/book/9781484228104 58
  59. 59. Links https://continuousdelivery.com/ https://www.terraform.io/ https://github.com/giuliov/terraform-fun https://azure.microsoft.com/en-us/services/key-vault/ https://docs.microsoft.com/en-us/azure/best-practices-availability-paired-regions https://martinfowler.com/articles/evodb.html http://databaserefactoring.com/ http://agiledata.org/essays/databaseRefactoring.html http://martinfowler.com/articles/feature-toggles.html https://launchdarkly.com/ https://blogs.msdn.microsoft.com/buckh/2016/09/30/controlling-exposure-through-feature-flags-in-vs-team-services/ https://azure.microsoft.com/en-us/features/storage-explorer/ http://www.powershellgallery.com/ https://docs.microsoft.com/en-us/powershell/dsc/overview 59
  60. 60. Brick by brick

×