1.
Ten CloudDesign Patterns<br />Shlomo SwidlerFounderOrchestratus<br />

2.
Shlomo Swidler<br />Founder, Orchestratus<br />Strategic and technicalIT consulting<br />Customers include:<br />Cloud Developer Tips bloghttp://shlomoswidler.com/<br />Among top community-ranked contributors to Amazon Web Services discussion forums<br />2<br />

3.
Ten CloudDesign Patterns<br />Shlomo SwidlerFounderOrchestratus<br />

4.
Ten Cloud ApplicationDesign Patterns<br />Shlomo SwidlerFounderOrchestratus<br />

5.
Ten Cloud ApplicationDesign Patterns<br />Many<br />Shlomo SwidlerFounderOrchestratus<br />

6.
What is a Design Pattern<br />A reusable recipe for building (software) systems that solve a particular problem.<br />6<br />

7.
What is a Design Pattern<br />A reusable recipe for building (software) systems that solve a particular problem.<br />AKA Architectural Pattern<br />7<br />

8.
What is a Design Pattern<br />A reusable recipe for building (software) systems that solve a particular problem.<br />Goal<br />AvailableResources<br />Constraints<br />8<br />

9.
A Design Pattern<br />A reusable recipe for building (software) systems that solve a particular problem.<br />Meets affirmativerequirements<br />Goal<br />Does not violatenegative requirements<br />AvailableResources<br />Constraints<br />Can be implemented<br />9<br />

10.
Challenges Faced by Apps in the Cloud<br />Application Scalability<br />Cloud promises rapid (de)provisioning of resources.<br />How do you tap into that to create scalable systems?<br />Application Availability<br />Underlying resource failures happen … usually more frequently than in traditional data centers.<br />How do you overcome that to create highly available systems?<br />10<br />

11.
The Scalability Challenge<br />Scalability: Handle more (or fewer) requests<br />It’s not Performance (handle requests faster)<br />It’s not Availability (tolerate failures)<br />But improving Scalability often improves Availability<br />11<br />

12.
The Scalability Challenge<br />Two different components to scale:<br />State (inputs, data store, output)<br />Behavior (business logic)<br />Any non-trivial application has both.<br />Scaling one component means scaling the other, too.<br />12<br />

13.
App Scalability Patterns for State <br />Data Grids<br />Distributed Caching<br />HTTP Caching<br />Reverse Proxy<br />CDN<br />Concurrency<br />Message-Passing<br />Dataflow<br />Software Transactional Memory<br />Shared-State<br />Partitioning<br />CAP theorem: Data Consistency<br />Eventually Consistent<br />Atomic Data<br />DB Strategies<br />RDBMS<br />Denormalization<br />Sharding<br />NOSQL<br />Key-Value store<br />Document store<br />Data Structure store<br />Graph database<br />13<br />

14.
App Scalability Patterns for Behavior<br />Compute Grids<br />Event-Driven Architecture<br />Messaging<br />Actors<br />Enterprise Service Bus<br />Domain Events<br />Event Stream Processing<br />Event Sourcing<br />Command & Query Responsibility Segregation (CQRS)<br />Load Balancing<br />Round-robin<br />Random<br />Weighted<br />Dynamic<br />Parallel Computing<br />Master/Worker<br />Fork/Join<br />MapReduce<br />SPMD<br />Loop Parallelism<br />14<br />

15.
The Availability Challenge<br />Availability: Tolerate failures<br />Traditional IT focuses on increasing MTTF<br />Mean Time to Failure<br />Cloud IT focuses on reducing MTTR<br />Mean Time to Recovery<br />15<br />

16.
The Availability Challenge<br />Availability: Tolerate failures<br />Traditional IT focuses on increasing MTTF<br />Mean Time to Failure<br />Cloud IT focuses on reducing MTTR<br />Mean Time to Recovery<br />What follows is four availability scenarios: [low, high] X [MTTF, MTTR]<br />16<br />

17.
Availability and MTTF, MTTR<br />17<br />

18.
Availability and MTTF, MTTR<br />Uptime<br />53%<br />86%<br />69%<br />30%<br />18<br />

19.
Availability and MTTF, MTTR<br />Traditional IT<br />Uptime<br />53%<br />86%<br />69%<br />30%<br />19<br />

20.
Availability and MTTF, MTTR<br />Traditional IT<br />Uptime<br />53%<br />86%<br />Cloud<br />69%<br />30%<br />20<br />

21.
Availability and MTTF, MTTR<br />Traditional IT<br />Uptime<br />53%<br />86%<br />Cloud<br />69%<br />30%<br />Cloud done wrong<br />21<br />

22.
Design Patterns for Availability<br />Pattern: Replication<br />Pattern: Fail-Over<br />Often used together.<br />22<br />

23.
Availability Pattern: Fail-Over<br />Source: Michael Nygaard<br />23<br />

24.
Availability Pattern: Fail-Over<br />In practice, fail-over is not this simple<br />Source: Michael Nygaard<br />24<br />

25.
Availability Pattern: Fail-Over<br />Source: Michael Nygaard<br />25<br />

26.
Availability Pattern: Fail-Over with Fail-Back<br />Source: Michael Nygaard<br />26<br />

27.
Availability’s Nemesis<br />Single Points of Failure<br />27<br />

28.
SPOT the SPOF*<br />*Single Point of Failure<br />

29.
Spot the SPOF: 1<br />29<br />Internet<br />Cloud<br />App<br />App Instance<br />

30.
Spot the SPOF: 1b<br />30<br />Internet<br />Cloud<br />App<br />App Instance<br />

31.
Spot the SPOF: 1b<br />31<br />Internet<br />Cloud<br />App<br />App Instance<br />

32.
Spot the SPOF:2<br />32<br />Internet<br />Elastic IP Address<br />Cloud<br />App<br />App<br />App Instance<br />App Instance<br />Fail-over<br />

33.
Spot the SPOF:2<br />33<br />Internet<br />Might work…<br />Until you need more App instances<br />Or until another SPOF fails…<br />Elastic IP Address<br />Cloud<br />App<br />App<br />App Instance<br />App Instance<br />Fail-over<br />

34.
Spot the SPOF: 2a<br />34<br />Internet<br />LB<br />Load Balancer Instance<br />Cloud<br />App<br />App<br />

35.
Spot the SPOF: 2a<br />35<br />Internet<br />LB<br />Load Balancer Instance<br />Cloud<br />App<br />App<br />

36.
Spot the SPOF: 3<br />36<br />Internet<br />Elastic IP Address<br />LB<br />LB<br />Availability Zone<br />Replicated configuration<br />Cloud<br />App<br />App<br />Fail-over<br />

37.
Spot the SPOF: 3<br />37<br />Internet<br />Elastic IP Address<br />LB<br />LB<br />Availability Zone<br />Replicated configuration<br />Cloud<br />App<br />App<br />Fail-over<br />

38.
Spot the SPOF: 4<br />38<br />Internet<br />Elastic Load Balancer (Magic)<br />ELB<br />Availability Zone<br />Cloud<br />App<br />App<br />

39.
Spot the SPOF: 4<br />39<br />Internet<br />Elastic Load Balancer (Magic)<br />ELB<br />Availability Zone<br />Cloud<br />App<br />App<br />

40.
Spot the SPOF: 5<br />40<br />Internet<br />Elastic IP Address<br />LB<br />LB<br />Availability Zone<br />Availability Zone<br />Replicated configuration<br />Region<br />App<br />App<br />App<br />App<br />Fail-over<br />

41.
Spot the SPOF: 5<br />41<br />Internet<br />Elastic IP Address<br />LB<br />LB<br />Availability Zone<br />Availability Zone<br />Replicated configuration<br />Region<br />App<br />App<br />App<br />App<br />Fail-over<br />

42.
Spot the SPOF: 6<br />42<br />Internet<br />Elastic Load Balancer (Magic)<br />ELB<br />Availability Zone<br />Availability Zone<br />Region<br />App<br />App<br />App<br />App<br />

43.
Spot the SPOF: 6<br />43<br />Internet<br />Elastic Load Balancer (Magic)<br />ELB<br />Availability Zone<br />Availability Zone<br />Region<br />App<br />App<br />App<br />App<br />

44.
Spot the SPOF: 7<br />44<br />Internet<br />LB<br />LB<br />LB<br />LB<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />Or…<br />

45.
Spot the SPOF: 7a<br />45<br />Internet<br />LB<br />LB<br />LB<br />LB<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />

46.
Spot the SPOF: 7/7a<br />46<br />Internet<br />Elastic IPs are<br />single-region only<br />LB<br />LB<br />LB<br />LB<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />

47.
Spot the SPOF: 7b<br />47<br />Internet<br />ELB<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />

48.
Spot the SPOF: 7b<br />48<br />Internet<br />ELB is single-region only<br />ELB<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />

49.
Spot the SPOF: 7c<br />49<br />Internet<br />DNS<br />ELB<br />ELB<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />

50.
Spot the SPOF: 7c<br />50<br />Internet<br />ELB Can’t Do That<br />Multiple CNAMEs Violate RFC 2181<br />DNS<br />ELB<br />ELB<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />

51.
Spot the SPOF: 7d<br />51<br />Internet<br />DNS<br />LB<br />LB<br />LB<br />LB<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />

52.
Spot the SPOF: 7d<br />52<br />Internet<br />DNS<br />Cloud Provider<br />LB<br />LB<br />LB<br />LB<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />

53.
Spot the SPOF: 8<br />Internet<br />DNS<br />AWS<br />LB<br />LB<br />LB<br />LB<br />LB<br />Rackspace<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />53<br />

54.
Spot the SPOF: 8<br />Internet<br />DNS<br />AWS<br />LB<br />LB<br />LB<br />LB<br />LB<br />Rackspace<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />54<br />

55.
Spot the SPOF: 8<br />Internet<br />DNS<br />and...<br />AWS<br />LB<br />LB<br />LB<br />LB<br />LB<br />Rackspace<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />55<br />

56.
Spot the SPOF: 8<br />Internet<br />Fail-over<br />mechanism<br />DNS<br />and...<br />AWS<br />LB<br />LB<br />LB<br />LB<br />LB<br />Rackspace<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />56<br />

57.
Spot the SPOF: 8<br />Internet<br />Fail-over<br />mechanism<br />DNS<br />and...<br />Ops staff<br />and<br />AWS<br />LB<br />LB<br />LB<br />LB<br />LB<br />Rackspace<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />Availability Zone<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />App<br />Region<br />Region<br />57<br />

58.
Availability: Ensure Redundancies<br />Physical<br />Virtual resource (instance, disk, etc.)<br />Availability zone<br />Region<br />Provider<br />Human (ops staff)<br />58<br />

59.
Availability Best Practice:Chaos Monkey<br />AKA Error Injection Testing<br />Forcibly create fault conditions in your cloud components.<br />Kill instances, detach disks, screw up DNS, etc.<br />Automate recovery from the errors.<br />The team gets really good at reducing MTTR, increasing availability!<br />Popularized by Netflix, who run it on their live environment.<br />59<br />

60.
For more on Designing forAvailability, Scalability<br />Jonas BonérScalability, Availability, Stability Patterns http://slidesha.re/cK3NJv<br />George ReeseThe AWS Outage: The Cloud’s Shining Momenthttp://oreil.ly/eKCGG9<br />John Ciancutti of Netflix5 Lessons We’ve Learned Using AWShttp://bit.ly/h8rU8b<br />60<br />

61.
Ten Cloud ApplicationDesign Patterns<br />Thank you!<br />Many<br />Shlomo SwidlerFounderOrchestratus<br />shlomo@orchestratus.com<br />@ShlomoSwidler<br />