SlideShare a Scribd company logo
1 of 61
Ten CloudDesign Patterns Shlomo SwidlerFounderOrchestratus
Shlomo Swidler Founder, Orchestratus Strategic and technicalIT consulting Customers include: Cloud Developer Tips bloghttp://shlomoswidler.com/ Among top community-ranked contributors to Amazon Web Services discussion forums 2
Ten CloudDesign Patterns Shlomo SwidlerFounderOrchestratus
Ten Cloud ApplicationDesign Patterns Shlomo SwidlerFounderOrchestratus
Ten Cloud ApplicationDesign Patterns Many Shlomo SwidlerFounderOrchestratus
What is a Design Pattern A reusable recipe for building (software) systems that solve a particular problem. 6
What is a Design Pattern A reusable recipe for building (software) systems that solve a particular problem. AKA Architectural Pattern 7
What is a Design Pattern A reusable recipe for building (software) systems that solve a particular problem. Goal AvailableResources Constraints 8
A Design Pattern A reusable recipe for building (software) systems that solve a particular problem. Meets affirmativerequirements Goal Does not violatenegative requirements AvailableResources Constraints Can be implemented 9
Challenges Faced by Apps in the Cloud Application Scalability Cloud promises rapid (de)provisioning of resources. How do you tap into that to create scalable systems? Application Availability Underlying resource failures happen		… usually more frequently than in		traditional data centers. How do you overcome that to create highly available systems? 10
The Scalability Challenge Scalability: Handle more (or fewer) requests It’s not Performance (handle requests faster) It’s not Availability (tolerate failures) But improving Scalability often improves Availability 11
The Scalability Challenge Two different components to scale: State (inputs, data store, output) Behavior (business logic) Any non-trivial application has both. Scaling one component means scaling the other, too. 12
App Scalability Patterns for State  Data Grids Distributed Caching HTTP Caching Reverse Proxy CDN Concurrency Message-Passing Dataflow Software Transactional Memory Shared-State Partitioning CAP theorem: Data Consistency Eventually Consistent Atomic Data DB Strategies RDBMS Denormalization Sharding NOSQL Key-Value store Document store Data Structure store Graph database 13
App Scalability Patterns for Behavior Compute Grids Event-Driven Architecture Messaging Actors Enterprise Service Bus Domain Events Event Stream Processing Event Sourcing Command & Query Responsibility Segregation (CQRS) Load Balancing Round-robin Random Weighted Dynamic Parallel Computing Master/Worker Fork/Join MapReduce SPMD Loop Parallelism 14
The Availability Challenge Availability: Tolerate failures Traditional IT focuses on increasing MTTF Mean Time to Failure Cloud IT focuses on reducing MTTR Mean Time to Recovery 15
The Availability Challenge Availability: Tolerate failures Traditional IT focuses on increasing MTTF Mean Time to Failure Cloud IT focuses on reducing MTTR Mean Time to Recovery What follows is four availability scenarios:			[low, high] X [MTTF, MTTR] 16
Availability and MTTF, MTTR 17
Availability and MTTF, MTTR Uptime 53% 86% 69% 30% 18
Availability and MTTF, MTTR Traditional IT Uptime 53% 86% 69% 30% 19
Availability and MTTF, MTTR Traditional IT Uptime 53% 86% Cloud 69% 30% 20
Availability and MTTF, MTTR Traditional IT Uptime 53% 86% Cloud 69% 30% Cloud done wrong 21
Design Patterns for Availability Pattern: Replication Pattern: Fail-Over Often used together. 22
Availability Pattern: Fail-Over Source: Michael Nygaard 23
Availability Pattern: Fail-Over In practice, fail-over is not this simple Source: Michael Nygaard 24
Availability Pattern: Fail-Over Source: Michael Nygaard 25
Availability Pattern: Fail-Over with Fail-Back Source: Michael Nygaard 26
Availability’s Nemesis Single Points of Failure 27
SPOT the SPOF* *Single Point of Failure
Spot the SPOF: 1 29 Internet Cloud App App Instance
Spot the SPOF: 1b 30 Internet Cloud App App Instance
Spot the SPOF: 1b 31 Internet Cloud App App Instance
Spot the SPOF:2 32 Internet Elastic IP Address Cloud App App App Instance App Instance Fail-over
Spot the SPOF:2 33 Internet Might work… Until you need more App instances Or until another SPOF fails… Elastic IP Address Cloud App App App Instance App Instance Fail-over
Spot the SPOF: 2a 34 Internet LB Load Balancer Instance Cloud App App
Spot the SPOF: 2a 35 Internet LB Load Balancer Instance Cloud App App
Spot the SPOF: 3 36 Internet Elastic IP Address LB LB Availability Zone Replicated configuration Cloud App App Fail-over
Spot the SPOF: 3 37 Internet Elastic IP Address LB LB Availability Zone Replicated configuration Cloud App App Fail-over
Spot the SPOF: 4 38 Internet Elastic Load Balancer (Magic) ELB Availability Zone Cloud App App
Spot the SPOF: 4 39 Internet Elastic Load Balancer (Magic) ELB Availability Zone Cloud App App
Spot the SPOF: 5 40 Internet Elastic IP Address LB LB Availability Zone Availability Zone Replicated configuration Region App App App App Fail-over
Spot the SPOF: 5 41 Internet Elastic IP Address LB LB Availability Zone Availability Zone Replicated configuration Region App App App App Fail-over
Spot the SPOF: 6 42 Internet Elastic Load Balancer (Magic) ELB Availability Zone Availability Zone Region App App App App
Spot the SPOF: 6 43 Internet Elastic Load Balancer (Magic) ELB Availability Zone Availability Zone Region App App App App
Spot the SPOF: 7 44 Internet LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region Or…
Spot the SPOF: 7a 45 Internet LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
Spot the SPOF: 7/7a 46 Internet Elastic IPs are single-region only LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
Spot the SPOF: 7b 47 Internet ELB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
Spot the SPOF: 7b 48 Internet ELB is single-region only ELB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
Spot the SPOF: 7c 49 Internet DNS ELB ELB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
Spot the SPOF: 7c 50 Internet ELB Can’t Do That Multiple CNAMEs Violate RFC 2181 DNS ELB ELB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
Spot the SPOF: 7d 51 Internet DNS LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
Spot the SPOF: 7d 52 Internet DNS Cloud Provider LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
Spot the SPOF: 8 Internet DNS AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 53
Spot the SPOF: 8 Internet DNS AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 54
Spot the SPOF: 8 Internet DNS and... AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 55
Spot the SPOF: 8 Internet Fail-over mechanism DNS and... AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 56
Spot the SPOF: 8 Internet Fail-over mechanism DNS and... Ops staff and AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 57
Availability: Ensure Redundancies Physical Virtual resource (instance, disk, etc.) Availability zone Region Provider Human (ops staff) 58
Availability Best Practice:Chaos Monkey AKA Error Injection Testing Forcibly create fault conditions in your cloud components. Kill instances, detach disks, screw up DNS, etc. Automate recovery from the errors. The team gets really good at reducing MTTR, increasing availability! Popularized by Netflix, who run it on their live environment. 59
For more on Designing forAvailability, Scalability Jonas BonérScalability, Availability, Stability Patterns  http://slidesha.re/cK3NJv George ReeseThe AWS Outage: The Cloud’s Shining Momenthttp://oreil.ly/eKCGG9 John Ciancutti of Netflix5 Lessons We’ve Learned Using AWShttp://bit.ly/h8rU8b 60
Ten Cloud ApplicationDesign Patterns Thank you! Many Shlomo SwidlerFounderOrchestratus shlomo@orchestratus.com @ShlomoSwidler

More Related Content

What's hot

Adopting Java for the Serverless world at JUG Hamburg
Adopting Java for the Serverless world at  JUG HamburgAdopting Java for the Serverless world at  JUG Hamburg
Adopting Java for the Serverless world at JUG HamburgVadym Kazulkin
 
Can I Contain This?
Can I Contain This?Can I Contain This?
Can I Contain This?Eficode
 
(APP307) Leverage the Cloud with a Blue/Green Deployment Architecture | AWS r...
(APP307) Leverage the Cloud with a Blue/Green Deployment Architecture | AWS r...(APP307) Leverage the Cloud with a Blue/Green Deployment Architecture | AWS r...
(APP307) Leverage the Cloud with a Blue/Green Deployment Architecture | AWS r...Amazon Web Services
 
Adapting Java for the Serverless World at JUG Barcelona
Adapting Java for the Serverless World at JUG BarcelonaAdapting Java for the Serverless World at JUG Barcelona
Adapting Java for the Serverless World at JUG BarcelonaVadym Kazulkin
 
Going Big with Containers: Customer Case Studies of Large-Scale Deployments -...
Going Big with Containers: Customer Case Studies of Large-Scale Deployments -...Going Big with Containers: Customer Case Studies of Large-Scale Deployments -...
Going Big with Containers: Customer Case Studies of Large-Scale Deployments -...Amazon Web Services
 
Kubo (Cloud Foundry Container Platform): Your Gateway Drug to Cloud-native
Kubo (Cloud Foundry Container Platform): Your Gateway Drug to Cloud-nativeKubo (Cloud Foundry Container Platform): Your Gateway Drug to Cloud-native
Kubo (Cloud Foundry Container Platform): Your Gateway Drug to Cloud-nativecornelia davis
 
Netflix Cloud Architecture and Open Source
Netflix Cloud Architecture and Open SourceNetflix Cloud Architecture and Open Source
Netflix Cloud Architecture and Open Sourceaspyker
 
Cloud Foundry | How it works
Cloud Foundry | How it worksCloud Foundry | How it works
Cloud Foundry | How it worksKazuto Kusama
 
Cloudify workshop at CCCEU 2014
Cloudify workshop at CCCEU 2014 Cloudify workshop at CCCEU 2014
Cloudify workshop at CCCEU 2014 Uri Cohen
 
Leveraging elastic web scale computing with AWS
 Leveraging elastic web scale computing with AWS Leveraging elastic web scale computing with AWS
Leveraging elastic web scale computing with AWSShiva Narayanaswamy
 
AWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAmazon Web Services
 
Serverless security: defence against the dark arts
Serverless security: defence against the dark artsServerless security: defence against the dark arts
Serverless security: defence against the dark artsYan Cui
 
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...Amazon Web Services
 
(WEB305) Migrating Your Website to AWS | AWS re:Invent 2014
(WEB305) Migrating Your Website to AWS | AWS re:Invent 2014(WEB305) Migrating Your Website to AWS | AWS re:Invent 2014
(WEB305) Migrating Your Website to AWS | AWS re:Invent 2014Amazon Web Services
 
Implementing Infrastructure as Code ConfigMgtCamp 2017
Implementing Infrastructure as Code ConfigMgtCamp 2017Implementing Infrastructure as Code ConfigMgtCamp 2017
Implementing Infrastructure as Code ConfigMgtCamp 2017Kief Morris
 
Security in serverless world
Security in serverless worldSecurity in serverless world
Security in serverless worldYan Cui
 
AWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAmazon Web Services
 

What's hot (20)

Adopting Java for the Serverless world at JUG Hamburg
Adopting Java for the Serverless world at  JUG HamburgAdopting Java for the Serverless world at  JUG Hamburg
Adopting Java for the Serverless world at JUG Hamburg
 
Can I Contain This?
Can I Contain This?Can I Contain This?
Can I Contain This?
 
(APP307) Leverage the Cloud with a Blue/Green Deployment Architecture | AWS r...
(APP307) Leverage the Cloud with a Blue/Green Deployment Architecture | AWS r...(APP307) Leverage the Cloud with a Blue/Green Deployment Architecture | AWS r...
(APP307) Leverage the Cloud with a Blue/Green Deployment Architecture | AWS r...
 
Mini-Training: Netflix Simian Army
Mini-Training: Netflix Simian ArmyMini-Training: Netflix Simian Army
Mini-Training: Netflix Simian Army
 
Adapting Java for the Serverless World at JUG Barcelona
Adapting Java for the Serverless World at JUG BarcelonaAdapting Java for the Serverless World at JUG Barcelona
Adapting Java for the Serverless World at JUG Barcelona
 
Going Big with Containers: Customer Case Studies of Large-Scale Deployments -...
Going Big with Containers: Customer Case Studies of Large-Scale Deployments -...Going Big with Containers: Customer Case Studies of Large-Scale Deployments -...
Going Big with Containers: Customer Case Studies of Large-Scale Deployments -...
 
PaaS with Java
PaaS with JavaPaaS with Java
PaaS with Java
 
Kubo (Cloud Foundry Container Platform): Your Gateway Drug to Cloud-native
Kubo (Cloud Foundry Container Platform): Your Gateway Drug to Cloud-nativeKubo (Cloud Foundry Container Platform): Your Gateway Drug to Cloud-native
Kubo (Cloud Foundry Container Platform): Your Gateway Drug to Cloud-native
 
Netflix and Open Source
Netflix and Open SourceNetflix and Open Source
Netflix and Open Source
 
Netflix Cloud Architecture and Open Source
Netflix Cloud Architecture and Open SourceNetflix Cloud Architecture and Open Source
Netflix Cloud Architecture and Open Source
 
Cloud Foundry | How it works
Cloud Foundry | How it worksCloud Foundry | How it works
Cloud Foundry | How it works
 
Cloudify workshop at CCCEU 2014
Cloudify workshop at CCCEU 2014 Cloudify workshop at CCCEU 2014
Cloudify workshop at CCCEU 2014
 
Leveraging elastic web scale computing with AWS
 Leveraging elastic web scale computing with AWS Leveraging elastic web scale computing with AWS
Leveraging elastic web scale computing with AWS
 
AWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for Government
 
Serverless security: defence against the dark arts
Serverless security: defence against the dark artsServerless security: defence against the dark arts
Serverless security: defence against the dark arts
 
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...
 
(WEB305) Migrating Your Website to AWS | AWS re:Invent 2014
(WEB305) Migrating Your Website to AWS | AWS re:Invent 2014(WEB305) Migrating Your Website to AWS | AWS re:Invent 2014
(WEB305) Migrating Your Website to AWS | AWS re:Invent 2014
 
Implementing Infrastructure as Code ConfigMgtCamp 2017
Implementing Infrastructure as Code ConfigMgtCamp 2017Implementing Infrastructure as Code ConfigMgtCamp 2017
Implementing Infrastructure as Code ConfigMgtCamp 2017
 
Security in serverless world
Security in serverless worldSecurity in serverless world
Security in serverless world
 
AWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for Government
 

Similar to Ten^H^H^H Many Cloud App Design Patterns

SolarWinds Scalability for the Enterprise
SolarWinds Scalability for the EnterpriseSolarWinds Scalability for the Enterprise
SolarWinds Scalability for the EnterpriseSolarWinds
 
From Zero to still Zero: The most beautiful mistakes going into the cloud.
From Zero to still Zero: The most beautiful mistakes going into the cloud. From Zero to still Zero: The most beautiful mistakes going into the cloud.
From Zero to still Zero: The most beautiful mistakes going into the cloud. OPEN KNOWLEDGE GmbH
 
Oracle Ravello Presentation 7Dec16 v1
Oracle Ravello Presentation 7Dec16 v1Oracle Ravello Presentation 7Dec16 v1
Oracle Ravello Presentation 7Dec16 v1Kurt Liu
 
QConSF2016-JoshEvans-MasteringChaosANetflixGuidetoMicroservices-compressed.pdf
QConSF2016-JoshEvans-MasteringChaosANetflixGuidetoMicroservices-compressed.pdfQConSF2016-JoshEvans-MasteringChaosANetflixGuidetoMicroservices-compressed.pdf
QConSF2016-JoshEvans-MasteringChaosANetflixGuidetoMicroservices-compressed.pdfSimranjyotSuri
 
Mastering Chaos - A Netflix Guide to Microservices
Mastering Chaos - A Netflix Guide to MicroservicesMastering Chaos - A Netflix Guide to Microservices
Mastering Chaos - A Netflix Guide to MicroservicesJosh Evans
 
Bring Service Mesh To Cloud Native-apps
Bring Service Mesh To Cloud Native-appsBring Service Mesh To Cloud Native-apps
Bring Service Mesh To Cloud Native-appsThang Chung
 
Accelerate your digital transformation
Accelerate your digital transformationAccelerate your digital transformation
Accelerate your digital transformationMichael Dawson
 
Oracle ravello overview
Oracle ravello overviewOracle ravello overview
Oracle ravello overviewwk c
 
todd-ncts-2011-110828224616-phpapp02 (1).pptx
todd-ncts-2011-110828224616-phpapp02 (1).pptxtodd-ncts-2011-110828224616-phpapp02 (1).pptx
todd-ncts-2011-110828224616-phpapp02 (1).pptxkumarkaushal17
 
Observability Best Practices for Your Cloud DBaaS
Observability Best Practices for Your Cloud DBaaSObservability Best Practices for Your Cloud DBaaS
Observability Best Practices for Your Cloud DBaaSScyllaDB
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Cloud Computing Design Best Practices
Cloud Computing Design Best PracticesCloud Computing Design Best Practices
Cloud Computing Design Best PracticesMoshe Kaplan
 
Clipper: A Low-Latency Online Prediction Serving System
Clipper: A Low-Latency Online Prediction Serving SystemClipper: A Low-Latency Online Prediction Serving System
Clipper: A Low-Latency Online Prediction Serving SystemDatabricks
 
Why Cloud Management Makes Sense
Why Cloud Management Makes SenseWhy Cloud Management Makes Sense
Why Cloud Management Makes SenseRightScale
 
Driving Down Costs of z Systems™ Storage
Driving Down Costs of z Systems™ StorageDriving Down Costs of z Systems™ Storage
Driving Down Costs of z Systems™ StorageCA Technologies
 
Level up your SQL and Azure, by using Rubrik
Level up your SQL and Azure, by using RubrikLevel up your SQL and Azure, by using Rubrik
Level up your SQL and Azure, by using RubrikJaap Brasser
 
Availability in a cloud native world v1.6 (Feb 2019)
Availability in a cloud native world v1.6 (Feb 2019)Availability in a cloud native world v1.6 (Feb 2019)
Availability in a cloud native world v1.6 (Feb 2019)Haytham Elkhoja
 

Similar to Ten^H^H^H Many Cloud App Design Patterns (20)

SolarWinds Scalability for the Enterprise
SolarWinds Scalability for the EnterpriseSolarWinds Scalability for the Enterprise
SolarWinds Scalability for the Enterprise
 
From Zero to still Zero: The most beautiful mistakes going into the cloud.
From Zero to still Zero: The most beautiful mistakes going into the cloud. From Zero to still Zero: The most beautiful mistakes going into the cloud.
From Zero to still Zero: The most beautiful mistakes going into the cloud.
 
The Future of Cloud Innovation, featuring Adrian Cockcroft
The Future of Cloud Innovation, featuring Adrian CockcroftThe Future of Cloud Innovation, featuring Adrian Cockcroft
The Future of Cloud Innovation, featuring Adrian Cockcroft
 
Oracle Ravello Presentation 7Dec16 v1
Oracle Ravello Presentation 7Dec16 v1Oracle Ravello Presentation 7Dec16 v1
Oracle Ravello Presentation 7Dec16 v1
 
QConSF2016-JoshEvans-MasteringChaosANetflixGuidetoMicroservices-compressed.pdf
QConSF2016-JoshEvans-MasteringChaosANetflixGuidetoMicroservices-compressed.pdfQConSF2016-JoshEvans-MasteringChaosANetflixGuidetoMicroservices-compressed.pdf
QConSF2016-JoshEvans-MasteringChaosANetflixGuidetoMicroservices-compressed.pdf
 
Mastering Chaos - A Netflix Guide to Microservices
Mastering Chaos - A Netflix Guide to MicroservicesMastering Chaos - A Netflix Guide to Microservices
Mastering Chaos - A Netflix Guide to Microservices
 
Building the Case for System z Linux
Building the Case for System z LinuxBuilding the Case for System z Linux
Building the Case for System z Linux
 
Bring Service Mesh To Cloud Native-apps
Bring Service Mesh To Cloud Native-appsBring Service Mesh To Cloud Native-apps
Bring Service Mesh To Cloud Native-apps
 
Accelerate your digital transformation
Accelerate your digital transformationAccelerate your digital transformation
Accelerate your digital transformation
 
Oracle ravello overview
Oracle ravello overviewOracle ravello overview
Oracle ravello overview
 
Oracle Ravello Overview
Oracle Ravello OverviewOracle Ravello Overview
Oracle Ravello Overview
 
todd-ncts-2011-110828224616-phpapp02 (1).pptx
todd-ncts-2011-110828224616-phpapp02 (1).pptxtodd-ncts-2011-110828224616-phpapp02 (1).pptx
todd-ncts-2011-110828224616-phpapp02 (1).pptx
 
Observability Best Practices for Your Cloud DBaaS
Observability Best Practices for Your Cloud DBaaSObservability Best Practices for Your Cloud DBaaS
Observability Best Practices for Your Cloud DBaaS
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Cloud Computing Design Best Practices
Cloud Computing Design Best PracticesCloud Computing Design Best Practices
Cloud Computing Design Best Practices
 
Clipper: A Low-Latency Online Prediction Serving System
Clipper: A Low-Latency Online Prediction Serving SystemClipper: A Low-Latency Online Prediction Serving System
Clipper: A Low-Latency Online Prediction Serving System
 
Why Cloud Management Makes Sense
Why Cloud Management Makes SenseWhy Cloud Management Makes Sense
Why Cloud Management Makes Sense
 
Driving Down Costs of z Systems™ Storage
Driving Down Costs of z Systems™ StorageDriving Down Costs of z Systems™ Storage
Driving Down Costs of z Systems™ Storage
 
Level up your SQL and Azure, by using Rubrik
Level up your SQL and Azure, by using RubrikLevel up your SQL and Azure, by using Rubrik
Level up your SQL and Azure, by using Rubrik
 
Availability in a cloud native world v1.6 (Feb 2019)
Availability in a cloud native world v1.6 (Feb 2019)Availability in a cloud native world v1.6 (Feb 2019)
Availability in a cloud native world v1.6 (Feb 2019)
 

Recently uploaded

Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 

Recently uploaded (20)

Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 

Ten^H^H^H Many Cloud App Design Patterns

  • 1. Ten CloudDesign Patterns Shlomo SwidlerFounderOrchestratus
  • 2. Shlomo Swidler Founder, Orchestratus Strategic and technicalIT consulting Customers include: Cloud Developer Tips bloghttp://shlomoswidler.com/ Among top community-ranked contributors to Amazon Web Services discussion forums 2
  • 3. Ten CloudDesign Patterns Shlomo SwidlerFounderOrchestratus
  • 4. Ten Cloud ApplicationDesign Patterns Shlomo SwidlerFounderOrchestratus
  • 5. Ten Cloud ApplicationDesign Patterns Many Shlomo SwidlerFounderOrchestratus
  • 6. What is a Design Pattern A reusable recipe for building (software) systems that solve a particular problem. 6
  • 7. What is a Design Pattern A reusable recipe for building (software) systems that solve a particular problem. AKA Architectural Pattern 7
  • 8. What is a Design Pattern A reusable recipe for building (software) systems that solve a particular problem. Goal AvailableResources Constraints 8
  • 9. A Design Pattern A reusable recipe for building (software) systems that solve a particular problem. Meets affirmativerequirements Goal Does not violatenegative requirements AvailableResources Constraints Can be implemented 9
  • 10. Challenges Faced by Apps in the Cloud Application Scalability Cloud promises rapid (de)provisioning of resources. How do you tap into that to create scalable systems? Application Availability Underlying resource failures happen … usually more frequently than in traditional data centers. How do you overcome that to create highly available systems? 10
  • 11. The Scalability Challenge Scalability: Handle more (or fewer) requests It’s not Performance (handle requests faster) It’s not Availability (tolerate failures) But improving Scalability often improves Availability 11
  • 12. The Scalability Challenge Two different components to scale: State (inputs, data store, output) Behavior (business logic) Any non-trivial application has both. Scaling one component means scaling the other, too. 12
  • 13. App Scalability Patterns for State Data Grids Distributed Caching HTTP Caching Reverse Proxy CDN Concurrency Message-Passing Dataflow Software Transactional Memory Shared-State Partitioning CAP theorem: Data Consistency Eventually Consistent Atomic Data DB Strategies RDBMS Denormalization Sharding NOSQL Key-Value store Document store Data Structure store Graph database 13
  • 14. App Scalability Patterns for Behavior Compute Grids Event-Driven Architecture Messaging Actors Enterprise Service Bus Domain Events Event Stream Processing Event Sourcing Command & Query Responsibility Segregation (CQRS) Load Balancing Round-robin Random Weighted Dynamic Parallel Computing Master/Worker Fork/Join MapReduce SPMD Loop Parallelism 14
  • 15. The Availability Challenge Availability: Tolerate failures Traditional IT focuses on increasing MTTF Mean Time to Failure Cloud IT focuses on reducing MTTR Mean Time to Recovery 15
  • 16. The Availability Challenge Availability: Tolerate failures Traditional IT focuses on increasing MTTF Mean Time to Failure Cloud IT focuses on reducing MTTR Mean Time to Recovery What follows is four availability scenarios: [low, high] X [MTTF, MTTR] 16
  • 18. Availability and MTTF, MTTR Uptime 53% 86% 69% 30% 18
  • 19. Availability and MTTF, MTTR Traditional IT Uptime 53% 86% 69% 30% 19
  • 20. Availability and MTTF, MTTR Traditional IT Uptime 53% 86% Cloud 69% 30% 20
  • 21. Availability and MTTF, MTTR Traditional IT Uptime 53% 86% Cloud 69% 30% Cloud done wrong 21
  • 22. Design Patterns for Availability Pattern: Replication Pattern: Fail-Over Often used together. 22
  • 23. Availability Pattern: Fail-Over Source: Michael Nygaard 23
  • 24. Availability Pattern: Fail-Over In practice, fail-over is not this simple Source: Michael Nygaard 24
  • 25. Availability Pattern: Fail-Over Source: Michael Nygaard 25
  • 26. Availability Pattern: Fail-Over with Fail-Back Source: Michael Nygaard 26
  • 27. Availability’s Nemesis Single Points of Failure 27
  • 28. SPOT the SPOF* *Single Point of Failure
  • 29. Spot the SPOF: 1 29 Internet Cloud App App Instance
  • 30. Spot the SPOF: 1b 30 Internet Cloud App App Instance
  • 31. Spot the SPOF: 1b 31 Internet Cloud App App Instance
  • 32. Spot the SPOF:2 32 Internet Elastic IP Address Cloud App App App Instance App Instance Fail-over
  • 33. Spot the SPOF:2 33 Internet Might work… Until you need more App instances Or until another SPOF fails… Elastic IP Address Cloud App App App Instance App Instance Fail-over
  • 34. Spot the SPOF: 2a 34 Internet LB Load Balancer Instance Cloud App App
  • 35. Spot the SPOF: 2a 35 Internet LB Load Balancer Instance Cloud App App
  • 36. Spot the SPOF: 3 36 Internet Elastic IP Address LB LB Availability Zone Replicated configuration Cloud App App Fail-over
  • 37. Spot the SPOF: 3 37 Internet Elastic IP Address LB LB Availability Zone Replicated configuration Cloud App App Fail-over
  • 38. Spot the SPOF: 4 38 Internet Elastic Load Balancer (Magic) ELB Availability Zone Cloud App App
  • 39. Spot the SPOF: 4 39 Internet Elastic Load Balancer (Magic) ELB Availability Zone Cloud App App
  • 40. Spot the SPOF: 5 40 Internet Elastic IP Address LB LB Availability Zone Availability Zone Replicated configuration Region App App App App Fail-over
  • 41. Spot the SPOF: 5 41 Internet Elastic IP Address LB LB Availability Zone Availability Zone Replicated configuration Region App App App App Fail-over
  • 42. Spot the SPOF: 6 42 Internet Elastic Load Balancer (Magic) ELB Availability Zone Availability Zone Region App App App App
  • 43. Spot the SPOF: 6 43 Internet Elastic Load Balancer (Magic) ELB Availability Zone Availability Zone Region App App App App
  • 44. Spot the SPOF: 7 44 Internet LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region Or…
  • 45. Spot the SPOF: 7a 45 Internet LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
  • 46. Spot the SPOF: 7/7a 46 Internet Elastic IPs are single-region only LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
  • 47. Spot the SPOF: 7b 47 Internet ELB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
  • 48. Spot the SPOF: 7b 48 Internet ELB is single-region only ELB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
  • 49. Spot the SPOF: 7c 49 Internet DNS ELB ELB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
  • 50. Spot the SPOF: 7c 50 Internet ELB Can’t Do That Multiple CNAMEs Violate RFC 2181 DNS ELB ELB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
  • 51. Spot the SPOF: 7d 51 Internet DNS LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
  • 52. Spot the SPOF: 7d 52 Internet DNS Cloud Provider LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region
  • 53. Spot the SPOF: 8 Internet DNS AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 53
  • 54. Spot the SPOF: 8 Internet DNS AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 54
  • 55. Spot the SPOF: 8 Internet DNS and... AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 55
  • 56. Spot the SPOF: 8 Internet Fail-over mechanism DNS and... AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 56
  • 57. Spot the SPOF: 8 Internet Fail-over mechanism DNS and... Ops staff and AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 57
  • 58. Availability: Ensure Redundancies Physical Virtual resource (instance, disk, etc.) Availability zone Region Provider Human (ops staff) 58
  • 59. Availability Best Practice:Chaos Monkey AKA Error Injection Testing Forcibly create fault conditions in your cloud components. Kill instances, detach disks, screw up DNS, etc. Automate recovery from the errors. The team gets really good at reducing MTTR, increasing availability! Popularized by Netflix, who run it on their live environment. 59
  • 60. For more on Designing forAvailability, Scalability Jonas BonérScalability, Availability, Stability Patterns http://slidesha.re/cK3NJv George ReeseThe AWS Outage: The Cloud’s Shining Momenthttp://oreil.ly/eKCGG9 John Ciancutti of Netflix5 Lessons We’ve Learned Using AWShttp://bit.ly/h8rU8b 60
  • 61. Ten Cloud ApplicationDesign Patterns Thank you! Many Shlomo SwidlerFounderOrchestratus shlomo@orchestratus.com @ShlomoSwidler

Editor's Notes

  1. Nemesis by Alfred Rethel, 1837.
  2. More on other SPOFs here in a minute. Now, let’s see what you do if you want that scalability