Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Finding Patterns in the Clouds - Cloud Design Patterns

920 views

Published on

Cloud computing is quickly becoming the new normal for enterprise software developers. Whether it's more traditional Infrastructure-as-a-Service, container-based deployments, or fully serverless deployments, moving to the cloud offers something for almost every organization. But with it come new challenges for ensuring your applications are robust, scalable, fast, and don't overuse utilization-billed resources. Design patterns offer solutions to known challenges that can help you quickly recognize and address problems as you encounter them, saving you and your organization time and money. Come learn a few practical patterns that will help you avoid common problems with cloud-based systems.

Published in: Technology
  • Be the first to comment

Finding Patterns in the Clouds - Cloud Design Patterns

  1. 1. Finding Patterns in the Clouds Steve “ardalis” Smith @ardalis | steve@ardalis.com ardalis.com | weeklydevtips.com Design Patterns for Cloud- Native Applications
  2. 2. Please Rate in the App • AttendeeHub @ardalis | Finding Patterns in the Clouds
  3. 3. More Resources • Podcast WeeklyDevTips.com • Group Mentoring Program DevBetter.com • Free Microsoft eBooks ardalis.com/architecture-ebook ardalis.com/cloud-native-book @ardalis | Finding Patterns in the Clouds
  4. 4. Let’s take a trip back to the beginning of today’s web… @ardalis | Finding Patterns in the Clouds
  5. 5. (not that far)
  6. 6. A Simpler Time “Just” ~20 years ago… Compaq AlphaServer DS20 circa 1999 @ardalis | Finding Patterns in the Clouds
  7. 7. A Simpler Time “Just” 20 years ago…
  8. 8. One Web Server To Rule Them All Client Machine Server Request Response App Data (and one Webmaster to run it all – the original Full Stack Developer™) @ardalis | Finding Patterns in the Clouds NCSA Mosaic
  9. 9. One Web Server To Rule Them All Client Machine Server Request Response
  10. 10. Report Card: One Server To Rule Them All @ardalis | Finding Patterns in the Clouds Availability Data Management Design and Implementation Messaging Security Management and Monitoring Resiliency Performance and Scalability
  11. 11. Web App Considerations and Challenges @ardalis | Finding Patterns in the Clouds
  12. 12. Cloud-Hosted Web App Considerations and Challenges @ardalis | Finding Patterns in the Clouds
  13. 13. Availability How often is the system or service up? Often expressed as a percentage. 99.99% uptime = 1 minute of downtime per week 99.999% uptime = 26 seconds of downtime per month @ardalis | Finding Patterns in the Clouds
  14. 14. Data Management Many more options than in traditional-hosted single-database apps Distributed data Consistency Synchronization @ardalis | Finding Patterns in the Clouds
  15. 15. Design and Implementation Consistency is key Consider factors like • Maintenance ease • Administration • Development • Diagnostics • Cost @ardalis | Finding Patterns in the Clouds
  16. 16. Messaging How do subsystems communicate? Direct, synchronous calls? Asynchronous messaging? Each option presents challenges. @ardalis | Finding Patterns in the Clouds
  17. 17. Management and Monitoring No direct server access to PaaS resources means other tools are critical. Cloud resources are more like cattle herds than pets. @ardalis | Finding Patterns in the Clouds
  18. 18. Performance and Scalability How responsive is the system to requests? How does this responsiveness change with increased load? Scaling up Scaling out @ardalis | Finding Patterns in the Clouds
  19. 19. Resiliency Can the system gracefully (and automatically) recover from errors or failures? Detect failures and replace resources automatically @ardalis | Finding Patterns in the Clouds
  20. 20. Security Protect from attacks Guard sensitive data Restrict access to approved users @ardalis | Finding Patterns in the Clouds
  21. 21. One Machine To Rule Them All Client Machine Server Request Response App Data @ardalis | Finding Patterns in the Clouds
  22. 22. Report Card: One Server To Rule Them All 🙁 🙂 🙂 🙂 🙂 🙂 but also 🙁 😐 😐 @ardalis | Finding Patterns in the Clouds
  23. 23. Demand Grows
  24. 24. 🙁 Vertical Scaling is Maxed Out 🙁 Performance is suffering at times Remedy: Move Database to a separate server 🙂 Performance improves Current Assessment: @ardalis | Finding Patterns in the Clouds
  25. 25. 1 Web, 1 DB Server Client Machine Web Server Request Response App Data DB Server @ardalis | Finding Patterns in the Clouds
  26. 26. Parts of the App are SLOW
  27. 27. 🙁 Vertical Scaling is Maxed Out (both servers) – or at least there’s no budget for more right now 🙂 Data has been optimized with indexes, etc. No more gains to be had here. 🙁 Some queries just hammer the database, take time, and impact other queries. Current Assessment: @ardalis | Finding Patterns in the Clouds
  28. 28. Cache Aside Pattern @ardalis | Finding Patterns in the Clouds
  29. 29. Read-Through Strategy @ardalis | Finding Patterns in the Clouds
  30. 30. Write-Through Strategy Data 1. Update the data store 2. Invalidate (or update) its cache entry $123 @ardalis | Finding Patterns in the Clouds Another option is to use a very short cache duration, something I refer to as micro-caching
  31. 31. Add Simple Memory Caching Client Machine Web Server Request Response App Data DB Server Cache @ardalis | Finding Patterns in the Clouds
  32. 32. There are only 2 hard things in Computer Science 0. Cache Invalidation 1. Naming Things 2. Off-by-One Errors @ardalis | Finding Patterns in the Clouds
  33. 33. Speaking of Naming Things… • Use a standard way to generate a cache keys for given scenarios • Avoid hard-coding keys, especially as local method literals • Many caching patterns require access to keys from different parts of your applications (including read vs. write operations) @ardalis | Finding Patterns in the Clouds
  34. 34. 🙂 Performance is usually (much) better 🙁 Some users still complain of delays due to cache misses 🙁 Keeping Cache up to date is a new challenge 🙁 Customers may now see stale data New Problems… 🙁 New monitoring required for cache 🙁 New tools to clear or update cache required Hmm, that’s a lot of 🙁
  35. 35. Demand Grows
  36. 36. Time to Scale Out!
  37. 37. Simple Web Farm Client Machine Web Server Request Response App Data DB Server Cache Load Balancer Web Server App Cache @ardalis | Finding Patterns in the Clouds
  38. 38. 🙂Performance and Scalability improved 🙁 Some users still complain due to cache misses 😧 Keeping Multiple Caches up to date is a big challenge (in this model) New Behavior… 🙁New monitoring and tools required for load balancer 🙁 More servers to manage 🙂 Web servers can be updated without taking down the whole system @ardalis | Finding Patterns in the Clouds
  39. 39. Embrace the Cloud! @ardalis | Finding Patterns in the Clouds
  40. 40. The Cloud (xkcd.com/908) @ardalis | Finding Patterns in the Clouds
  41. 41. Simple Cloud Architecture Client Machine Request Response Load Balancer App Service Instances Azure Cache / Redis Instance Azure SQL Database @ardalis | Finding Patterns in the Clouds
  42. 42. 🙂 Scalability improved 🙁 Some users still complain due to cache misses (consider priming the cache) 🙂 Cache synchronization easier New Behavior… 🙂 No more servers to manage 🙂 Monitoring tools built-in to platform 🙂 Web instances easily managed without downtime That ratio of 🙂 to 🙁 is a lot better…@ardalis | Finding Patterns in the Clouds
  43. 43. “Let’s build more of these apps” @ardalis | Finding Patterns in the Clouds
  44. 44. More Apps Client Machine Request Response Load Balancer App Service Instances Azure Cache / Redis Instance Azure SQL Database @ardalis | Finding Patterns in the Clouds
  45. 45. More Apps App Service Instances Azure Cache / Redis Instance Azure SQL Database App A App Service Instances App B App Service Instances App C Shared Resources
  46. 46. Everything’s Great! Except… Vendor lock-in Shared database hurts Shared resources (e.g. data schema) limit app developer agility We’ll address these in a moment but first… @ardalis | Finding Patterns in the Clouds
  47. 47. “Don’t forget auth!” @ardalis | Finding Patterns in the Clouds
  48. 48. Authentication and Identity These apps require: • Single Sign-on • Security – protection from unauthorized use • This was surely built into the apps before this point, but the pain becomes apparent now @ardalis | Finding Patterns in the Clouds
  49. 49. Simple Database Managed Identity App Service Instances Azure Cache / Redis Instance Azure SQL Database App A App Service Instances App B App Service Instances App C Identity Tables/Data Username Password Login Username Password Login Username Password Login YOU get a login screen, and YOU get a login screen
  50. 50. Federated Identity Pattern @ardalis | Finding Patterns in the Clouds
  51. 51. Redis Cache Azure SQL App A Redis cache Azure SQL App B Redis cache Azure SQL App C Identity Microservice Leveraging Containers and Federated Identity Azure SQL Client Machine 1. Authenticate 2. Get Secure Token 3. Present Token Note: Vendor lock-in mitigated by containers
  52. 52. “What about data sync?” @ardalis | Finding Patterns in the Clouds
  53. 53. Redis Cache Azure SQL App A Redis cache Azure SQL App B Redis cache Azure SQL App C Identity Microservice Implementing a Message/Event Bus Azure SQL @ardalis | Finding Patterns in the Clouds
  54. 54. Moving from Apps to Microservices • Apps are very coarse-grained to deploy, scale • Common functionality duplicated between apps • Stable parts of apps disrupted by deployment of unstable bits • Decompose apps into small, independent, cohesive microservices @ardalis | Finding Patterns in the Clouds
  55. 55. App A – An eCommerce Site Redis Cache Azure SQL App A Client apps Mobile app Web app @ardalis | Finding Patterns in the Clouds
  56. 56. Microservice 2 Microservice 1 container container Web API Web API Microservice 3 container Web API Client apps Microservices Split into Microservices Call each as appropriate from clients Mobile app Web app
  57. 57. Security Concerns • Client apps may not need every microservice feature • Microservices may have multiple clients; shouldn’t need to know security rules of every one • Should limit feature surface area specific to client needs @ardalis | Finding Patterns in the Clouds
  58. 58. API Gateway Pattern @ardalis | Finding Patterns in the Clouds
  59. 59. Using a custom API Gateway Service Microservice 2 Microservice 1 Client WebApp MVC container container Web API Web API ASP.NET Core MVC container Microservice 3 container Web API Client SPA Web app JavaScript Client mobile app API Gateway ASP.NET Core Web API container Back end Traditional Web app Browser HTML HTML JSON JSON
  60. 60. API Gateway with Azure API Management Architecture Client WebApp MVC ASP.NET Core MVC container Client SPA Web app JavaScript Client mobile app Developer portal API Gateway Publisher portal Azure API Management Microservice 2 Microservice 1 container container Web API Web API Microservice 3 container Web API Back end
  61. 61. Accessing Secure Files @ardalis | Finding Patterns in the Clouds
  62. 62. New Problem – Secure File Access • Apps control access to media files based on authorized user • Simple approach of a dumb CDN doesn’t protect actual media URLs from being accessed by anyone • Current solution: Web App authenticates user, accesses the file, and streams it to the end user @ardalis | Finding Patterns in the Clouds
  63. 63. Secure Media File Access Client Machine Request Response Web App Request Response File/BLOB Store (not publicly accessible) @ardalis | Finding Patterns in the Clouds
  64. 64. Concerns 🙁 Load on web app higher than necessary 🙁 Cost! 💰 May be paying extra to move files in/out of web app 🙁 Greater chance of downtime with web app and file store both required to stream file @ardalis | Finding Patterns in the Clouds
  65. 65. Valet Key • Provide direct access to media files using an access token • Azure supports Shared Access Signatures (SAS) for this purpose • File transfers occur directly between file store and client @ardalis | Finding Patterns in the Clouds
  66. 66. Remote Resources @ardalis | Finding Patterns in the Clouds
  67. 67. “I want fast, reliable, always-on services!” @ardalis | Finding Patterns in the Clouds
  68. 68. Retry Pattern First attempt failed • Is it likely to be a transient problem? If not, give up. • Immediately try again (maybe it was just a fluke) • (wait) • Try again • (wait longer) • Try again • Give up. @ardalis | Finding Patterns in the Clouds
  69. 69. Retries can overload downstream service • Imagine typical load is 10 requests per second. • With a typical “try 3 times then fail” strategy, when the service comes up it’s immediately seeing 30 requests per second of load! • Can result in longer time to respond and creation of more resources than necessary (more hosting $$$) • Don’t DOS (denial of service) yourself! @ardalis | Finding Patterns in the Clouds
  70. 70. Circuit-Breaker Pattern 3 States • Closed (working) • Open (not working) • Half-Open (throttled)
  71. 71. Circuit Breaker States @ardalis | Finding Patterns in the Clouds Closed Open When failCount > threshold, Open Half-Open After [timeout], go to Half-Open Still not responding… Reset failCount to zero and Close Retry on failure; increment failCount
  72. 72. Consider a tool like Polly @ardalis | Finding Patterns in the Clouds
  73. 73. Bonus Microservice Anti-Pattern: Reach-In Reporting Approach 1 • Microservices access reporting data directly • Introduces coupling • Reduces microservice independence • Bypasses microservice logic Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x
  74. 74. Bonus Microservice Anti-Pattern: Reach-In Reporting Approach 2 • Reporting app hits microservices directly • Poor performance • Data may be too large for HTTP • Difficult to perform complex queries Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x
  75. 75. Bonus Microservice Anti-Pattern: Reach-In Reporting Approach 3 • Batch job updates reporting db from microservice dbs • Same coupling as approach 1. Changes to microservice db schemas break batch data job. Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x
  76. 76. Bonus Microservice Anti-Pattern: Reach-In Reporting Solution • Async event publication • Encapsulation and independence of microservices is preserved • Performance is usually acceptable Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x
  77. 77. Key Takeaways • Cloud architecture abstracts away servers • Cache Aside pattern is great for performance improvements • Containers offer improved deployment and scaling options with less vendor lock-in • Microservices offer finer-grained control over app functionality • Federated Identity improves security and user experience • API Gateways help secure collections of services • Valet key provides cheaper, faster access to secure media • Consider Retry but you may need a Circuit-Breaker • Avoid the Reach In Reporting anti-pattern for your microservices @ardalis | Finding Patterns in the Clouds
  78. 78. More Cloud Design Patterns https://bit.ly/1T8q2w8 @ardalis | Finding Patterns in the Clouds
  79. 79. Thank You! • Contact me! twitter.com/ardalis steve@ardalis.com • Podcast WeeklyDevTips.com • Group Mentoring Program DevBetter.com • Free Microsoft eBooks ardalis.com/architecture-ebook ardalis.com/cloud-native-book

×