Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tear It Down, Build It Back Up: Empowering Developers with Amazon CloudFormation

As a product grows, and the infrastructure becomes more complex, the Operations team traditionally shoulders the burden of maintaining this infrastructure while deploying code from Software Engineers. Code is sometimes given to Operations with little to no information regarding how it should run or what the criteria for successful deployment is. This is not due to lack of caring, Software Engineers often lack the context themselves to provide production deployment instructions. To Software Engineers, production can be like a walled off city, filled with pathways and rooms not to be explored, guarded by Operations.

This presentation aims to provide a solution to this problem. We will address how the traditional separation of Operations and Software Engineers slows innovation, and redefine their relationship -- blending responsibilities. We will examine the transition of two real teams, an Operations team and Engineering team, from complete isolation, to closer environments through virtual machines, to one cloud environment shared by all and managed with CloudFormation.

  • Login to see the comments

Tear It Down, Build It Back Up: Empowering Developers with Amazon CloudFormation

  1. 1. James Andrew Vaughn (Andy) @MindTouch Tear It Down, Build It Back Up: Empowering Developers with Amazon CloudFormation
  2. 2. James Andrew Vaughn (Andy) • Software Architect at MindTouch • @modethirteen on Twitter & GitHub • Interests • Software Build and Testing Automation • Frontend Web Performance • Web Components & Polymer • SSO and Identity Management
  3. 3. @modethirteen Agenda • What is Amazon CloudFormation? Why use it? • Managing your release testing and production infrastructure code • Give developers the power (`cause knowledge is power!)
  4. 4. @modethirteen Why manage infrastructure as code?
  5. 5. @modethirteen
  6. 6. @modethirteen All of our customers host their brand on our common, hosted infrastructure. One mistake and all customer brands look bad #yousuck
  7. 7. @modethirteen Before CloudFormation • Infrastructure had grown organically over years • Hand rolled scripts with to create different EC2 instance types, and manual Puppet runs to configure them • Non EC2 AWS Resources managed by hand • No infrastructure in different zones or fast, programatic disaster recovery for entire infrastructure • Developers were ignorant of production infrastructure
  8. 8. @modethirteen Weekly releases must be simple, repeatable, non events
  9. 9. @modethirteen Developers cannot be isolated from the infrastructure where their code will ultimately run
  10. 10. @modethirteen Code gives context to problems solved and provides audit trail for infrastructure design
  11. 11. @modethirteen Infrastructure code and server configuration code is versioned with application code
  12. 12. @modethirteen CloudFormation: Define creation of AWS resources (EC2 as well as Security Groups, SQS, RDS, etc) Puppet, Chef, SaltStack, Ansible: Define actions that occur within EC2 instances once they’ve been provisioned
  13. 13. @modethirteen CloudFormation vs Terraform • Access to nearly every AWS resource. Better support for VPC, Security Groups, IAM, Cloudfront, SQS • Stable and mature • JSON infrastructure templates can be generated by Troposphere (with Python logic) • Vendor neutrality: AWS, OpenStack, Heroku, etc • Can execute infrastructure plans as a dry run • DSL for generating infrastructure templates (HCL) • If one resource fails to build, subsequent rebuild will only build tainted resource and those dependent on it • Open source so AWS API coverage can be improved by community Google Docs: Terraform AWS Coverage
  14. 14. @modethirteen CloudFormation Stacks Main Stack Sub Stacks A stack is a collection of AWS resources that can be configured
  15. 15. @modethirteen App Server Pool Stack Database Stack ElasticSearch Stack App Server Pool Stack Main Stack
  16. 16. @modethirteen CloudFormation Stacks Resources are things that can be queried, configured in the AWS API (including CloudFormation sub stacks). Examples: Listing S3 buckets, Adding Route 53 DNS entries, Taking DB snapshots
  17. 17. @modethirteen Database Stack ElasticSearch Stack App Server Pool Stack Main Stack • AutoScaling::AutoScalingGroup • AutoScaling::LaunchConfiguration • IAM::InstanceProfile • IAM::User • AutoScaling::AutoScalingGroup • AutoScaling::LaunchConfiguration • CloudFormation::WaitCondition • IAM::InstanceProfile • IAM::User • RDS::DBInstance • IAM::InstanceProfile • IAM::User
  18. 18. @modethirteen Custom Resources • CloudFormation::CustomResource • Sends custom HTTP message (Service Token) to any of your endpoints, and continues stack execution after response • AWS SNS • AWS Lambda • Node.JS • Your choice!
  19. 19. @modethirteen CloudFormation Stacks Stack parameters come from API input, version controlled JSON templates, or from the output of other stacks
  20. 20. @modethirteen • MySQL Storage Engine App Server Pool Stack Database Stack ElasticSearch Stack App Server Pool Stack Main Stack • ElasticSearch Version • App Server Pool EC2 Group Name • ElasticSearch EC2 Group Name • RDS MySQL IP & Port
  21. 21. @modethirteen CloudFormation Stacks Parameters of stack can be outputted to dependent stacks. Example: IP’s, Security Policies, Custom Values, etc.
  22. 22. @modethirteen Template: {…} App Server Pool Stack Database Stack ElasticSearch Stack App Server Pool Stack Main Stack • MySQL Storage Engine • ElasticSearch EC2 Group Name • RDS MySQL IP & Port • ElasticSearch Version • App Server Pool EC2 Group Name Template: {…}
  23. 23. @modethirteen Stack Policy: Stack Update Resource Access Control
  24. 24. @modethirteen Deploying a Stack
  25. 25. @modethirteen Troposphere
  26. 26. @modethirteen Puppet / Chef / SaltStack / Ansible • Stack includes an EC2 Instance or AutoScaling Group Resource • Resource includes a “UserData” metadata section, for bootstrapping an instance or group of instances • Include data that cloud-init uses to install instance configuration tool of choice • curl • Example: • cloud-init installs puppet from UserData commands • cloud-init runs puppet (configures instance and installs cfn-signal) • cfn-signal notifies CloudFormation that puppet was success or failure
  27. 27. @modethirteen Execute Deployment
  28. 28. @modethirteen Lessons Learned • Goal was to put entire existing AWS infrastructure into CloudFormation, no immediate value was attained • Difficult getting buy in for incremental improvements to infrastructure management • Existing resources cannot be migrated to CloudFormation • Know the caveats of deleting AWS Resources, they can fail a stack tear down • AWS Resources missing from CloudFormation API can be mitigated with Custom Resources • Must understand what a resource does when it updates
  29. 29. @modethirteen Send in the Developers
  30. 30. @modethirteen Approach #1 : Build your own web console for launching test and dev stacks
  31. 31. @modethirteen Approach #2 : Every developer has their own AWS account billed to main AWS account
  32. 32. @modethirteen Approach #3 : One developer AWS account billed to main account
  33. 33. @modethirteen The Teams • Are developer teams responsible for their own container / infrastructure templates, are operators part of these teams • Are developers just as responsible for troubleshooting when infrastructure goes down • What are operator obligations to developers • What are developer obligations to operations
  34. 34. @modethirteen TL;DR • Your product is application code, data, services, and servers • CloudFormation deploys your product to production • CloudFormation deploys your product for development and testing • Your developers can make better decisions • Your operators can make better decisions • Your customers / users are happy
  35. 35. The End. Q?