Download as PDF, PPTX
![Example of integrated stacks
Shared
infrastructure
stack
Application
infrastructure
stack
vpc_id = "vpc-abcde"
private_subnets = ["sn-12345", "sn-54321"]](https://image.slidesharecdn.com/evolutionaryinfrastructure-agile2018-kiefmorris-180809121401/85/Evolutionary-infrastructure-agile-2018-kief-morris-89-320.jpg)
![Integration at apply-time
vpc_id = "vpc-abcde"
private_subnets = ["sn-12345", "sn-54321"]](https://image.slidesharecdn.com/evolutionaryinfrastructure-agile2018-kiefmorris-180809121401/85/Evolutionary-infrastructure-agile-2018-kief-morris-90-320.jpg)
![Integration by resource discovery
data "aws_vpc" "vpc" {
filter {
name = "tag:Environment"
values = ["${var.env}"]
}
}
data "aws_subnet" "private_subnet" {
filter {
name = "tag:Environment"
values = ["${var.env}"]
}
}](https://image.slidesharecdn.com/evolutionaryinfrastructure-agile2018-kiefmorris-180809121401/85/Evolutionary-infrastructure-agile-2018-kief-morris-91-320.jpg)
Uploaded byKief Morris
Evolutionary infrastructure agile 2018 - kief morris
The document discusses building evolutionary infrastructures using various tools and methodologies to manage dynamic resources and facilitate continuous delivery. It emphasizes the importance of prioritizing speed and quality through automated testing and effective pipeline designs while reducing complexity through stack management. Key strategies include minimizing configurable surfaces, implementing fitness functions for system integrity, and coordinating changes across independent stacks to enhance flexibility and reduce dependencies.
More Related Content
Similar to Evolutionary infrastructure agile 2018 - kief morris
Evolutionary infrastructure agile 2018 - kief morris
- 1.
- 2. kief@thoughtworks.com Infrastructure Technologist@ Twitter: @kief Book:http://oreil.ly/1JKIBVe Site: http://infrastructure-as-code.com Some visuals found from the Vault of the Atomic Space Age http://bit.ly/2KBTbAd
- 3. Why? Ongoing pressure todeliver while requirements change
- 4. PRODTESTDEV Inconsistent path toproduction
- 5.
- 6. PROD Continuous, reliable deliveryof changes to all your stuff TESTDEV
- 7. "Since we can'tavoid change, we need to exploit it" Building Evolutionary Architectures Neal Ford, Rebecca Parsons, Pat Kua
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14. Slow Careful Careless Prioritize correctness over speed ofchange Fragile mess
- 15. Slow Careful Careless Prioritize correctness over speed ofchange Fragile mess Time pressures Over- engineering Technical debt Barriers to improvement
- 16. FastSlow Careful Careless Agile, Lean, DevOpsBuild the simplestthing Build quality in Optimize for change Fast, frequent feedback
- 17.
- 18. Dynamic infrastructure platform Changedelivery services Infrastructure management tools Operational management services Application Packaging and Deployment Development Tooling Application runtime platform
- 19. Dynamic infrastructure platform Changedelivery services Infrastructure management tools Operational management services Application Packaging and Deployment Development Tooling Application runtime platform
- 20. Dynamic infrastructure platform ComputeStorageNetwork A pool of resources
- 21. Provision resources • Ondemand • Programmatically
- 22. Physical Servers Containers Serverless (FaaS) Variationsof compute resources Virtual Machines
- 23. Dynamic infrastructure platform Changedelivery services Infrastructure management tools Operational management services Application Packaging and Deployment Development Tooling Application runtime platform
- 24. Stack management tool Stackinstances Platform API Stack definition (Terraform, CloudFormation, ...)
- 25. Server configuration tool Configuration tool Configuration definitions Serverinstance (Ansible, Chef, Puppet, Saltstack, ...)
- 26. Server image creation Baseserver image Custom image Server configuration definitions Image definition (Packer, ...)
- 27.
- 28. Infrastructure stack A collectionof infrastructure resources provisioned and updated as a unit
- 29. A stack mayinclude the infrastructure for a single application
- 30. Or, multiple applications might runin a single stack
- 31. Or the partsof one application could be spread across several stacks
- 32. Stack definition A stackdefinition is code used by a tool to provision stack instances Stack definition Stack instanceStack management tool
- 33. Stack instances A singlestack definition may be used to manage one or more stack instances Stack definition Stack instances
- 34.
- 35. Starting small –single stack instance One or two people working on it, ... the system is fairly simple, ... and there are few users, with low stakes
- 36. And then ... Morepeople join the team, ... the system becomes more complex, ... and more people rely on the system to be working
- 37.
- 38. PRODUCTION QA TEST Multiple environments definedin one stack instance our_env/ └── test.tf └── qa.tf └── production.tf
- 39. But the blastradius for a change is too wide PRODUCTION QA TEST
- 40. our_env/ └── test/ └── servers.tf our_env/ └──qa/ └── servers.tf our_env/ └── production/ └── servers.tf PRODUCTION QA TEST Separate stack definition project for each stack instance
- 41. QA PRODUCTION TEST Single definition project template, usedto create a separate stack instance for each environment our_env/ └── servers.tf
- 42.
- 43. Automatically test everychange before applying it
- 44. Promote changes toenvironments using a pipeline BUILDLOCAL APPLY TO QA APPLY TO PROD Sandbox QA Production APPLY TO TEST Test
- 45. Stack tool isonly run by pipeline agents BUILDLOCAL APPLY TO QA APPLY TO PROD APPLY TO TEST
- 46. Why not applychanges from our laptops? Our working environments may not be consistent Multiple people may clash We may shortcut our process We may neglect to run all appropriate tests
- 47. "Local" pre-commit sandboxes VCS Nita'sInstance Alex's Instance Freddy's Instance
- 48.
- 49. server "web" { name= "web-${env}" type = "${type}" } env = "prod" type = "t2.large" env = "qa" type = "t2.large" env = "test" type = "t2.tiny" web-test web-qa web-prod Parameters per stack instance
- 50. Parameters in sourceproject folders web-test web-qa web-prod . ├── infra │ ├── web_server.tf │ ├── subnet.tf │ └── vpc.tf └── environments ├── test.tfvars ├── qa.tfvars └── prod.tfvars
- 51. env = "test" type = "t2.tiny" env = "qa" type = "t2.large" env = "prod" type = "t2.large" Parameters in pipelinejob definitions BUILD APPLY TO QA APPLY TO PROD APPLY TO TEST web-test web-qa web-prod
- 52. Parameters from configurationregistry web-test web-qa web-prod envs/test/type t2.tiny envs/qa/type t2.large envs/prod/type t2.large
- 53. Recommendation Minimize the configurablesurface area to reduce variation between environments
- 54.
- 55. An effective testingregime drives a clean, decoupled, evolvable system design
- 56. Fitness functions Define theimportant characteristics of your system, how to measure these, and how to assess progress
- 57.
- 58. QA PRODTEST ALL TESTJDK COOKBOOK TEST TOMCAT COOKBOOK TEST APP SERVER ROLE BUILD APP SERVER AMI VALIDATE INFRA STACK BUILD OUR-APP TEST INFRA STACK TEST OUR- APP Example pipeline for our-app
- 59. QA PRODTEST ALL TESTJDK COOKBOOK TEST TOMCAT COOKBOOK TEST APP SERVER ROLE APP SERVER AMI VALIDATE INFRA STACK BUILD OUR-APP TEST INFRA STACK TEST OUR- APP APPLICATION STAGES: Compile, unit tests, container tests, packaging, etc. (run on build agents) JUnit, Rspec, etc.
- 60. QA PRODTEST ALL TESTJDK COOKBOOK TEST TOMCAT COOKBOOK TEST APP SERVER ROLE BUILD APP SERVER AMI BUILD INFRA STACK BUILD OUR-APP TEST INFRA STACK TEST OUR- APP SERVER PACKAGES: Validate, test individual package configurations in isolation SERVER ROLE: Test aggregated packages Typically run on build agents, perhaps containerized test-kitchen, serverspec, inspec, puppet-rspec, testinfra, etc.
- 61. QA PRODTEST ALL TESTJDK COOKBOOK TEST TOMCAT COOKBOOK TEST APP SERVER ROLE BUILD APP SERVER AMI VALIDATE INFRA STACK BUILD OUR-APP TEST INFRA STACK TEST OUR- APP SERVER IMAGE: Build and test serverspec, inspec, etc.
- 62. QA PRODTEST ALL TESTJDK COOKBOOK TEST TOMCAT COOKBOOK TEST APP SERVER ROLE BUILD APP SERVER AMI VALIDATE INFRA STACK BUILD OUR-APP TEST INFRA STACK TEST OUR- APP INFRASTRUCTURE STACK Validate (e.g. syntax, provision non-application specific infrastructure elements awspec, inspec, etc.
- 63. QA PRODTEST ALL TESTJDK COOKBOOK TEST TOMCAT COOKBOOK TEST APP SERVER ROLE BUILD APP SERVER AMI VALIDATE INFRA STACK BUILD OUR-APP TEST INFRA STACK TEST OUR- APP INTEGRATED STAGES: Test the various components in combination (deployment environments) selenium, etc.
- 64. Recommendations Those who writethe code write the tests Avoid reflexive testing (AKA testing your tools) Test where there is risk, complexity Test contracts
- 65.
- 66.
- 67. Coordinating changes Team Acould make a change that breaks things for team B
- 68. Divide infrastructure intomultiple, independently changeable stacks
- 69. Each stack hasits own pipeline to deliver changes
- 70.
- 71. Draw boundaries tominimize changes that cross stacks
- 72. Infrastructure code boundaries!= network boundaries
- 73. Infrastructure code boundaries!= network boundaries
- 74. Infrastructure code boundaries!= network boundaries
- 75. Group things togetherwhen they tend to be changed together
- 76. Align team boundariesand change boundaries
- 77.
- 78.
- 79. Base infrastructure modulepipeline Application stack pipelines Components shared across stacks
- 80. Challenges with sharedcomponents Increases coupling
- 81.
- 82. Shared stack instance Shared network resourcestack Multiple application infrastructure stacks
- 83. Shared stack pipeline Applicationinfrastructure pipelines
- 84. Shared infrastructure stacks Theprovider stack should not have knowledge of its consumers Becomes a bottleneck as the number of consumers grows Should be kept simple, minimal Split into smaller stacks to avoid monoliths
- 85.
- 86. Each stack isself-contained
- 87. Each stack isindependently releasable
- 88.
- 89. Example of integratedstacks Shared infrastructure stack Application infrastructure stack vpc_id = "vpc-abcde" private_subnets = ["sn-12345", "sn-54321"]
- 90. Integration at apply-time vpc_id= "vpc-abcde" private_subnets = ["sn-12345", "sn-54321"]
- 91. Integration by resourcediscovery data "aws_vpc" "vpc" { filter { name = "tag:Environment" values = ["${var.env}"] } } data "aws_subnet" "private_subnet" { filter { name = "tag:Environment" values = ["${var.env}"] } }
- 92. Integration by configurationregistry envs/qa/vpc_id vpc-abcde envs/qa/private_subnets "sn-12345", "sn-54321"
- 93.
- 94. Recommendations Stack integration pointsare contracts Follow good practice for API design when making changes to contracts Use fitness functions (automated tests) to ensure the integrity of contracts
- 95. kief@thoughtworks.com Infrastructure Technologist@ Twitter: @kief Book:http://oreil.ly/1JKIBVe Site: http://infrastructure-as-code.com