Cloud Formation

1. Henry Huang

3. Quick Solution
To Deploy One for Dev/QA

4. Problems
• Automation Efforts
• Service Learning
• Partial Setup
• Unique Resource
• Resource Cleanup

5. Automation Efforts
Current Status
• ? No bundle solution from Ops
Wish List
• Not to re-invent the wheel
• Configuration-driven Platform

6. Service Learning
Description
• First have to know most service details
• Long Learning Curve
Wish List
• No detail but more general

7. Partial Setup
Current Status
• Not fully support
• Most used in the deployment
Wish List
• De-couple the deployment solution
• Easy to setup a partial environment

8. Unique Resource
Description
• Environment cannot be shared
• Avoid naming conflicts
Wish List
• Most resources can be unique for each
deployment (special cases)
• Dev provide the required resources and their
relationship
• Ops allocate and maintain the real resources

9. Example

10. Resource Cleanup

11. Cloud
Formation

12. It Brings
Problems Cloud Formation
Automation Efforts JSON
Service Learning Configuration Template
Partial Setup AutoGen-Dependencies
Unique Resource Prefix + Suffix
Resource Cleanup Yes

13. Example
• AdminPortal
 Security Group
 ELB
 Launch Auto-Scaling Group
 R53

14. Template Anatomy
• Format Version Declaration
• Description Declaration
• Parameters Declaration
• Mappings Declaration
• Resource Declaration
• Properties Declaration
• Function Declaration
• Outputs Declaration

15. Recently
I finished a prototype

16. One Click
Service almost Ready

18. Inputs
 Cassandra, Mongo, RDS existing
• DNS
• Security Group Name
• Role AMI should be ready first
• Other pre-defined configuration

19. Demo Time

20. Lessons
Case I
For Production, seamless upgrade requires
• Old version and new one co-existing
• Service data should keep consistent
 SQS, S3
 Database: Cassandra, Mongo, RDS
• Static network attribute should be
smoothly switched
 EIP, high-level DNS

21. Lessons
Case II
Configuration Format need to refine
• Same value but with different format
• S3Cfg Tool need to support more
scalable scenarios
 Partial setup
 Different Layers

22. S3CFG Layers
 Layer1 Role
 Layer2 Machine
 Layer3 Auto-Scaling Group
 Layer4 Global

23. Layer 1: Role

24. Layer 2: Machine

25. Layer 3: Auto-scaling Group
 ASG Setting
• Region, Availability zones
• Instance Numbers: Min/Max/Desired
• ELB
• Instance Tags
• Scaling Policy
 Launch Configuration
• Spot or On-demand Instances
• Image Id
• Instance Type
etc.

26. Lessons
Case III
Issues in Cloud Formation
• Cannot clean up the resources in some
special scenarios (submitted and fixing)
 VPC
 RDS Security Group update
• S3 bucket cannot be cleaned due to the
data existing
• Configuration Too Large
 More abstraction layers needed

27. One day,
Drag and Drop
Service just Born!

28. Lessons
Case IV
Instance Bootstrap not scalable/common
• Too many restrictions
 Configuration downloaded from S3
 Bundled with Instance Tags
• Not scalable enough
 Not easy to modify commands to
execute in instance booting
 Should take the experiences from
cloud-init

29. F A Q