AWS Infrastructure as code - CloudFormation and Cumulus

2,298 views

Published on

CloudFormation is Amazon Web Service's tool for creating and managing "stacks" of other AWS resources as code.

Cumulus is a tool we wrote at catchoftheday.com.au to help you manage your stacks in CloudFormation and build bigger logical stacks from multiple cloudformation stacks. https://github.com/cotdsa/cumulus.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,298
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
17
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • ----- Meeting Notes (26/11/2013 19:36) -----Intro
  • ----- Meeting Notes (26/11/2013 19:36) -----Peter
  • ----- Meeting Notes (26/11/2013 19:36) -----Marketing
  • ----- Meeting Notes (26/11/2013 19:36) -----ProductRepeatedly, predictably ordered fashionInfrastructure as code - Collection/BlueprintOpposedManually/Console* 2 Terms - Resources defined using JSON ‘Templates’ to create ‘stacks’ of resources.
  • ----- Meeting Notes (26/11/2013 19:36) -----Hard to see
  • ----- Meeting Notes (26/11/2013 19:36) -----4 Main elements, other version desc
  • ----- Meeting Notes (26/11/2013 19:36) -----Parameters – Keyname, constraints
  • ----- Meeting Notes (26/11/2013 19:36) -----Mappings – AWSRegionArch2AMIKey attribute/pair
  • ----- Meeting Notes (26/11/2013 19:36) -----Resources – The only required section of the 4.KeyNameNext slide is 2nd Resources.
  • ----- Meeting Notes (26/11/2013 19:36) -----Resources 2:Fn:FindInMap function - key
  • ----- Meeting Notes (26/11/2013 19:36) -----Outputs returned to caller
  • ----- Meeting Notes (26/11/2013 19:36) -----Templates reusable -> Example web server, database server -> Bring up 2 more stacks, unique names.
  • * Hate JSON* No inclusion like other languages
  • Manual parameters – Or in my case, setting the values as defaults, which meant they were no longer re-usable or shareable.
  • AWS Infrastructure as code - CloudFormation and Cumulus

    1. 1. AWS Infrastructure as code CloudFormation and Cumulus
    2. 2. Who we are  Ash Caire WebOps Lead at Catch Group  Peter Hall WebOps Engineer at Catch Group Twitter: @peterkh
    3. 3. Catch Group  catchoftheday.com.au  scoopon.com.au  groceryrun.com.au  eatnow.com.au  mumgo.com.au
    4. 4. AWS CloudFormation “... an easy way to create and manage a collection of related AWS resources ... ” - http://aws.amazon.com/cloudformation/ Uses JSON templates to describe “stacks” of resources
    5. 5. Full CloudFormation Example
    6. 6. Simple CloudFormation Example
    7. 7. Simple CloudFormation Example
    8. 8. Simple CloudFormation Example
    9. 9. Simple CloudFormation Example
    10. 10. Simple CloudFormation Example
    11. 11. Simple CloudFormation Example
    12. 12. So basically…  Create almost any AWS resources with one API call  Once created, stacks can be updated or deleted  Templates are re-usable and can be used to create multiple stacks simultaneously. Resources are assigned dynamic names  Resources are tagged throughout the stack, with stackname, stack-id and logical-id (name of template resource)
    13. 13. CloudFormation Demo
    14. 14. It works great! …except  Templates are JSON files. No comments. Not the easiest format for humans to write  Templates are standalone and complete. No inclusion of other templates  Creation of stacks requires manual specification of template parameters
    15. 15. …which leads to Very large template files with a “data centre” worth of resources in it or Smaller templates with lots of manual parameters to feed each one or…
    16. 16. Stack resources in templates  It is possible to create a CloudFormation stack inside a stack  Once created, cannot update the parameters of an inside stack by updating the outside stack template (update of outside stack will fail)  Template for inside stack needs to be uploaded to an S3 bucket before stack creation We wanted a better way!
    17. 17. Cumulus  Python command line tool to add extra awesome to CloudFormation  Concept of a mega stack, made up of multiple CloudFormation stacks.  Adds an extra mega-stack yaml file on top of Cloudformation JSON templates  Written by CatchOfTheDay WebOps team
    18. 18. What does this give us?  Keep CloudFormation template files small and re-usable  Single file for a mega stack, including all its parameters  Stacks can be dependent on each other and have dynamic parameter values from other stack resources  More time writing YAML, less time writing JSON  Watch stack events in near real-time
    19. 19. Cumulus YAML file example  See: https://github.com/cotdsa/cumulus/blob/master/exam ples/cumulus_example_stack.yaml
    20. 20. Cumulus Demo
    21. 21. Future improvements  Handle larger templates by using S3 upload  Find a way to show meaningful diffs between running stacks and local templates before doing an update  Abstract away JSON templates altogether?  Generally more robust code, better error handling and writing tests
    22. 22. Please use and improve Open to pull requests, issues and ideas or even just let us know you are getting value from it... https://github.com/cotdsa/cumulus
    23. 23. Questions?

    ×