At Amazon Web Services, we think about Infrastructure as Code being able to impact not just your low level infrastructure or operating systems but everything from the virtual cement floor of your Amazon Virtual Private Cloud up through the applications your customers interface with.
Come take a tour of the space as we see it. Learn what layers there are to managing your infrastructure as code and what services and tools AWS and its Partners exist across these.
2. About me:
Chris Munns - munns@amazon.com, @chrismunns
– Business Development Manager – DevOps
– New Yorker
– Previously:
• AWS Solutions Architect 2011-2014
• Lead of Infrastructure/DevOps @hingeapp
• Formerly on operations teams @Etsy and @Meetup
• Little time at a hedge fund, Xerox and others
– Rochester Institute of Technology: Applied Networking and
Systems Administration ’05
– Internet infrastructure geek
3. Infrastructure as Code is a practice
in which infrastructure is provisioned
and managed using code and
software development techniques,
such as version control and
continuous integration and delivery.
5. Why Infrastructure as Code (IaC)?
• We’re managing larger and more sophisticated infrastructures
• With cloud based infrastructure we can easily create, destroy,
modify things that we’d in a physical datacenter infrastructure
touch as infrequently as possible
• Our applications are moving faster, so our infrastructure
needs to move faster
• We’ve realized as a industry that hand-crafted servers belong
best on Etsy and Amazon Handmade, not in our
infrastructures
6. Infrastructure as Code is a practice
in which infrastructure is provisioned
and managed using code and
software development techniques,
such as version control and
continuous integration and delivery.
10. Infrastructure as Code is a practice
in which infrastructure is provisioned
and managed using code and
software development techniques,
such as version control and
continuous integration and delivery.
11. Warning: Best Practice Ahead
We version control our application code so we can track
changes, we should be doing the same with our infrastructure
code:
– Improved visibility to changes over time(who, what, when, hopefully
why)
– Who was it that made a change(in so far as the repository is
concerned)
– Can often tie this back to ticketing and project management
systems
– Easier to then have multiple environments and have a true lifecycle
of infrastructure changes
Today this is a surprising gap for many!
12. “Cause if you liked it,
then you shoulda put it
in a repo
Oh, oh, oh, oh, oh, oh”
- Queen Bey (DevOps remix)
https://www.flickr.com/photos/evarinaldiphotography/8513571047/
13. Infrastructure as Code is a practice
in which infrastructure is provisioned
and managed using code and
software development techniques,
such as version control and
continuous integration and
delivery.
16. Warning: Best Practice Ahead
With agile development practices we are running our application
code through continuous integration and delivery tools and
processes. We should be doing the same for IaC:
– Once code is in a repository, that repository should be hooked up to
a continuous delivery/pipelining tool
– Linting/syntax checks done first
– Functionality at component level checks next
– Integration environment with multiple components checks after that
– Tested at stages such as dev, pre-prod, “princess” for my Etsy
friends, production, etc
– Promote changes between environments after tests past
Emerging trend, currently done by few
17. Infrastructure as Code “tiers”
AWS Resources
Operating System and Host Configuration
Application Configuration
18. Infrastructure as Code “tiers”
AWS Resources
Operating System and Host Configuration
Application Configuration
AWS CloudFormation
Hashicorp Terraform
19. Infrastructure as Code “tiers”
AWS Resources
Operating System and Host Configuration
Application Configuration
Ansible
Chef
Puppet
Salt
20. AWS Resources
Operating System and Host Configuration
Application Configuration
AWS CodeDeploy
Container frameworks
#serverless frameworks
Infrastructure as Code “tiers”
Swagger
21. AWS Resources
Operating System and Host Configuration
Application Configuration
Infrastructure as Code “tiers”
Jfrog Artifactory
PackageCloud
Sonatype Nexus
Language specific public
repositories
Container image registries
Somewhere shared across the top two
tiers are package repositories and
container registries for our applications
and system software
23. AWS Resources
Operating System and Host Configuration
Application Configuration
Infrastructure as Code “tiers”
AWS Elastic Beanstalk
Hashicorp Terraform
Ansible
Chef
Hashicorp Packer
Salt
AWS CloudFormation
AWS OpsWorks
Jfrog Artifactory
PackageCloud
Sonatype Nexus
Amazon CodeDeploy
Container frameworks
#serverless frameworks
The reality is that many of these
tools cross tiers in many ways!
Puppet
24. How do I choose my IaC tools?
• Pick at least one tool specialized in
each tier
• Some multipurpose tools aren’t
great in all areas, for example,
CloudFormation isn’t the best tool
for on-going OS changes, but
OpsWorks is great at that. Use
both!
• Favor tools that make testing them
easier (aka fit in a CI/CD process)
• Most OS configuration management
tools have supporting testing tools
these days
• Find tools that support a balance of
operability and developer flexibility
https://secure.flickr.com/photos/lox/9408028555
25. The red line between Dev & Ops
A big part of DevOps is shifting responsibility, ownership and
accountability between Developers and Operations such that:
• Developers are blocked less by Operations due to automation
and self service tools provided by Ops
• Operations is passing some responsibility up to Developers
requiring them to often be responsible for more application
testing and on-call
This balance can be thought of as a red line that cuts across our
Infrastructure as Code model matching up to a role and
responsibility model.*
*This idea comes from dozens of meetings with large enterprises
26. The “red line”
AWS Resources
Operating System and Host Configuration
Application Configuration
Dev
Ops
What Operations really wants.
27. The “red line”
AWS Resources
Operating System and Host Configuration
Application Configuration
Dev
What Development really wants.
Ops
28. The “red line”
AWS Resources
Operating System and Host Configuration
Application Configuration
Dev
Ops
Traditionally in many organizations Developers are interfacing very high in the
stack due to operations owning most of the stack at/below the application,
being responsible for availability and uptime of almost everything.
29. AWS Resources
Operating System and Host Configuration
Application Configuration
Dev
Ops
The “red line”
With Infrastructure as Code many organizations are looking to move this line
down to here. In this state Developers have full ownership over applications
and Operations is typically providing lower level resources, the OS, and
shared services across the infrastructure
30. The “red line”
AWS Resources
Operating System and Host Configuration
Application Configuration
Dev
Ops
Containers and ”serverless” applications take this lower where the emphasis
is more on the application layer and any orchestration requirements. Below
the line typically represents a highly ”platform as a serviced” infrastructure
operating model for an organization.
31. The “red line”
AWS Resources
Operating System and Host Configuration
Application Configuration
Security, application sensitivity, organization history, staff skills + experience,
adopted technologies, business goals, leadership, and many other factors
play in here for where to determine where the red line will be in your business.
?
32. FIN, ACK
Infrastructure as Code is a practice in which infrastructure
is provisioned and managed using code and software
development techniques, such as version control and
continuous integration and delivery.
– Almost everything about your infrastructure can be treated as
code
– Store all of this code in a revision control system
– Put this code through a continuous delivery pipeline that verifies
code quality and functionality
– “Promote” changes through environments via this pipeline
– Understand where your “red line” needs to be
Continuous Integration
Continuous Integration is the practice of checking in your code to the continuously and verifying each change with an automated build and test process. Over the past 10 years Continuous Integration has gained popularity in the software community. In the past developers were working in isolation for an extended period of time and only attempting to merge their changes into the mainline of their code once their feature was completed. Batching up changes to merge back into the mainline made not only merging the business logic hard, but it also made merging the test logic difficult. Continuous Integration practices have made teams more productive and allowed them to develop new features faster. Continuous Integration requires teams to write automated tests which, as we learned, improve the quality of the software being released and reduce the time it takes to validate that the new version of the software is good.
There are different definitions of Continuous Integration, but the one we hear from our customers is that CI stops at the build stage, so I’m going to use that definition.
Continuous Delivery
Continuous Delivery extends Continuous Integration to include testing out to production-like stages and running verification testing against those deployments. Continuous Delivery may extend all the way to a production deployment, but they have some form of manual intervention between a code check-in and when that code is available for customers to use.
Continuous Delivery is a big step forward over Continuous Integration allowing teams to be gain a greater level of certainty that their software will work in production.
Continuous Deployment
Continuous Deployment extends continuous delivery and is the automated release of software to customers from check in through to production without human intervention. Many of the teams at Amazon have reached a state of continuous deployment. Continuous Deployment reduces the time for your customers to get value from the code your team has just written, with the team getting faster feedback on the changes you’ve made. This fast customer feedback loop allow you to iterate quickly, allowing you to deliver more valuable software to your customers, quicker.
Continuous Integration
Continuous Integration is the practice of checking in your code to the continuously and verifying each change with an automated build and test process. Over the past 10 years Continuous Integration has gained popularity in the software community. In the past developers were working in isolation for an extended period of time and only attempting to merge their changes into the mainline of their code once their feature was completed. Batching up changes to merge back into the mainline made not only merging the business logic hard, but it also made merging the test logic difficult. Continuous Integration practices have made teams more productive and allowed them to develop new features faster. Continuous Integration requires teams to write automated tests which, as we learned, improve the quality of the software being released and reduce the time it takes to validate that the new version of the software is good.
There are different definitions of Continuous Integration, but the one we hear from our customers is that CI stops at the build stage, so I’m going to use that definition.
Continuous Delivery
Continuous Delivery extends Continuous Integration to include testing out to production-like stages and running verification testing against those deployments. Continuous Delivery may extend all the way to a production deployment, but they have some form of manual intervention between a code check-in and when that code is available for customers to use.
Continuous Delivery is a big step forward over Continuous Integration allowing teams to be gain a greater level of certainty that their software will work in production.
Continuous Deployment
Continuous Deployment extends continuous delivery and is the automated release of software to customers from check in through to production without human intervention. Many of the teams at Amazon have reached a state of continuous deployment. Continuous Deployment reduces the time for your customers to get value from the code your team has just written, with the team getting faster feedback on the changes you’ve made. This fast customer feedback loop allow you to iterate quickly, allowing you to deliver more valuable software to your customers, quicker.