Your SlideShare is downloading. ×
0
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Real world cloud formation feb 2014 final
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Real world cloud formation feb 2014 final

435

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
435
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Real World CloudFormation AWS User Group Amsterdam February 2014 ! Howard Glynn Cloud Leader | Cloudreach howard.glynn@cloudreach.com | @hglynn
  • 2. Objectives • • • • • • • The creation process Template approaches Bootstrapping and managing resources ! The importance of roles ! Other Concepts Protection strategies Recommendations ! Won’t dive into too much code detail,
 but will assume familiarity with AWS!
 
 Please feel free to ask questions! !2
  • 3. Cloudreach • AWS Premier Consulting Partner • London, Amsterdam, Edinburgh, Vancouver… cloudreach.com !3
  • 4. About CloudFormation • • • • • • Describes resources, relationships in a template Rich feature set - most AWS services supported Complements other AWS and 3rd party tools Free to use, but created resources are not “Infrastructure as Code” Can be tricky! More Info: “DMG201” !4
  • 5. Why Use It? • • • • • • • Controllable and predictable Saves time, helps save money Dev, Staging, Prod Forces more discipline, captures knowledge Multi-region provisioning Can aid disaster recovery It’s amazing to watch! !5
  • 6. Creation Process • • • • • • Open your editor, write JSON • Footnote: CloudFormer IAM credentials Use existing fragments/templates Sometimes hard to debug An iterative, additive process works well - edit/run/edit... Beyond a certain point, you cannot practically manage the JSON - too complex !6
  • 7. Real world approaches... • Troposphere (see github) o o Often 50% lines less than JSON o • python code to describe resources Re-usable, testable And/or choose your tools! o Editors: Take time to learn code folding! o Browsers: Use JSON formatters o AWS CLI is essential !7
  • 8. Templates
  • 9. Template Contents • Parameters o • Mappings o • for example, mapping regional AMIs (but other approaches…) Resources o • up to 60 nowadays the bulk of the template - e.g. subnets, security groups, instances, databases, service logic... Outputs o for example, created endpoints, IDs for reference or chaining !9
  • 10. Monolithic vs Modular CF • Monolithic o quick to start, no parameter passing o least easy to maintain when it grows o • most examples take this approach o easy to hit CF console upload limits (-> S3) cf json Modular o smaller, more manageable o cf json or cf json cf json far more parameter passing - hard o cf json think about layers, outputs, chaining !10
  • 11. Summary Monolithic Modular Simple and Quick ✓ All logic in one place ✘ Logic spanning multiple files DRY (Don’t Repeat Yourself) ✘ Repeated components must be written multiple times ✓ Same file can be used many times with different params Maintainable ✘ Hard to find correct elements and relationships ✓ Different logical concepts in different files !11
  • 12. Outline 3-Layer Modular CF Web Stack Layer Name Typical Parameters Typical Outputs 10-base VPC CIDR Public Subnet CIDRs NAT Size, EIP SSH Keynames VPC ID Subnet IDs, SGs Private Route Table VPC ID Private Route Table Private DB Subnet CIDRs DB Name, Size etc DB Endpoint DB Client SG ID VPC ID Private Route Table Private Web Subnet CIDRs DB Endpoint, Client SG Public Subnet SGs for ELB Web Instance Params Orchestration Params …. ELB Endpoints R53 Sets up VPC, NAT etc 20-db Sets up RDS 30-web Autoscaling instances, ELB etc !12
  • 13. Bootstrapping Instances
  • 14. How to achieve the automation goal Our usual goal is a hands-free automated bring up of our resources. Therefore we need to configure instances automatically. A combination of methods can be used: ! 1. user-data 2. cfn-init / cloud-init metadata 3. bootstrap orchestration tools !14
  • 15. Instance User-data • • Simplest Typically o called one-time by EC2 instance o bash or other shell script, 16kb limit o available at the http://169.254.169.254/… metadata endpoint !15
  • 16. cfn-init / metadata • • If your needs are straightforward, use it! Installs o o Installs packages o Write files to disk o Can pull code from your github, o setup users (not windows) o • • Fetch and parse metadata from CloudFormation Enabled/disable and start/stop services Need to call cfn-init once in user-data Canonical’s cloud-init is similar, not as well supported !16
  • 17. Instance Roles • • • • associate a role to an instance roles feature IAM policies policies give permissions you cannot change the IAM role on
 a running instance so best practice
 is to assign one, even if empty! !17
  • 18. Securely accessing S3 • • • • If your instance has a role/policy… • Why…? Combine with metadata Securely access S3 Blog post, Nov 2013
 http://goo.gl/6bVRLP !18
  • 19. Bootstrapping to Chef • • • Combo of metadata, role-based S3 access, user-data • • • Bootstrap the node with the creds (user-data) Securely grab validation keys Install base chef packages or omnibus installer from metadata / user-data Machine is now under hosted chef control Same applies to puppet? !19
  • 20. Other Concepts
  • 21. Wait Cond. and Signals • In the real world, these can no longer be ignored! • Stacks now build in parallel • Examples: o VPC with NAT, no instance builds until working o DB before Web, no instance builds until DB available !21
  • 22. Windows Instances • • • • • as supportable as Linux instances all the same principles apply use cloudformation::init to download S3 files run individual commands or powershell same password retrieval requirement ! ! !22
  • 23. Only time for a quick mention… • Tagging • Notifications • Custom Resources Custom Resources: More Info: “DMG303” !23
  • 24. Wrapping Up
  • 25. Protect Your Stacks • • • • IAM Roles • • DeletionPolicy attribute • For critical actions, stop, check, do it slow! Stack Policies DisableApiTermination Protection Layer dependencies can
 be your friend Spider-Man (c) Marvel Have separate DEV and
 PROD accounts !25
  • 26. Be aware... • • • • No dry runs • • Blue/green deployments are hard Uncontrolled changes (hacks) - actual vs expected? Some AMIs/Licensing don’t like CF Dependencies are sometimes non-obvious (EIPs and IGWs) Cannot absorb pre-existing resources !26
  • 27. Some real world best practices • • Use IAM deny o Setup Elastic IPs externally o o • • make users assume a delete role o for fixed endpoints avoids losing them supply them as parameters Don’t embed creds, passwords etc o o Use instance roles, secured S3 buckets Cloud init to pull them to instances Don’t let layers get too big/complex o You’ll be too scared to update live !27
  • 28. Some real world best practices • Meaningful stack and resource names o • e.g. prefix the environment onto instance names CF allows for tags o o e.g. we can control costs by stopping overnight o • ENVIRONMENT, STACK, OWNER etc leverage the cloud Get your dependencies right! o otherwise hard to diagnose intermittent stack fails o Two types: Implicit “Refs”, Explicit “DependsOn” !28
  • 29. Some real world best practices • Use the AWS CLI ( http://goo.gl/QASmbS )) o • supports passing parameters in files Take the time to: o o • parameterise inputs -> reusability add potentially useful outputs Do your dev/test away from prod o in a different region (validates reusability!) o in a different account !29
  • 30. Dank je wel ! ! Questions, comments, thoughts?! cloudreach.com/careers/ !30
  • 31. !31
  • 32. !32

×