Architecting for Massive Scalability - St. Louis Day of .NET 2011 - Aug 6, 2011


Published on

The Cloud now makes seemingly infinite amounts of computing power accessible to everyone. However, to maximize this power, your applications need to scale. In this session, we will explore patterns that enable massive scalability. We will examine Brewer’s CAP Theorem and contrast it to the ACID principles that guide traditional LOB applications. And finally, we will explore how to apply these patterns when building applications for the Cloud using Windows Azure.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Architecting for Massive Scalability - St. Louis Day of .NET 2011 - Aug 6, 2011

  1. 1. Architecting forMassive ScalabilityEric D. BoydDirector, Chicago + Cloud Practice
  2. 2. Introduction Eric D. Boyd 15 Years in Technology
  3. 3. I’m From Here
  4. 4. I Moved Here
  5. 5. I Work Here
  6. 6. I Work On
  7. 7. I Blog @
  8. 8. I Tweet @EricDBoyd
  9. 9. Agenda What is Scalability? ACID, BASE & CAP Compute Patterns Data Patterns Q&A
  10. 10. What is Scalability? A service is said to be scalable if when we increase the resources in a system, it results in increased performance in a manner proportional to resources added. --Werner Vogels, CTO,
  11. 11. What is Scalability? Measures of Performance Response Time Throughput Measures of Scalability Transaction Volume Concurrent Users Size of Data and Growth
  12. 12. What can be Scaled? Servers CPU Memory Hard Disk Licensing Network Ports Bandwidth Cooling Power Racks Floor Space
  13. 13. Why is Scalability Important? Predictable Growth Unpredictable Growth Design for Scale Benefits of Scalability Dynamic Capacity Planning Cost is Linear
  14. 14. Workload Patterns On and Off Growing Fast Compute Compute Inactivity On off Period Average Usage Average Usage Time Time • On & off workloads (e.g. batch job) • Successful services needs to grow/scale • Over provisioned capacity is wasted • Keeping up w/ growth is big IT challenge • Time to market can be cumbersome • Complex lead time for deployment Unpredictable Bursting Predictable Bursting“ Compute Compute Average Usage Average Usage Time Time • Unexpected/unplanned peak in demand • Services with micro seasonality trends • Sudden spike impacts performance • Peaks due to periodic increased demand • Can’t over provision for extreme cases • IT complexity and wasted capacity
  15. 15. Traditional Line-of-Business (LOB) Apps A - Atomic C - Consistent I - Isolated D - Durable
  16. 16. CAP Theorem Consistency. The client perceives that a set of operations has occurred all at once. Availability. Every operation must terminate in an intended response. Partition tolerance. Operations will complete, even if individual components are unavailable.
  17. 17. Brewer’s CAP Theorem Consistency Partition Availability Tolerance
  18. 18. BASE Basically Available Soft State Eventually Consistent
  19. 19. Eventual Consistency Given a sufficiently long period of time over which no updates are sent, we can expect that during this period, all updates will eventually propagate through the system and all the replicas will be consistent. --Wikipedia,
  20. 20. Real-World twitter Facebook eBay Amazon Foursquare
  21. 21. Compute Patterns
  22. 22. Scale Up
  23. 23. Scale Out
  24. 24. Why Isn’t this the Norm? Data is typically Bottleneck Designed for and expects ACID Not designed for Scale Out Load is fixed and constant Mind shift and additional work
  25. 25. I Need Scalability Loose Coupling Stateless Messaging & SOA Async & Background Processes Queuing Monitoring and Diagnostics
  26. 26. Front-End (FE) + Back-End (BE) Model FE BE Persistent Storage
  27. 27. Front-End (FE) + Back-End (BE) Architecture (FE) requests (BE) to perform work (BE) completes work (BE) reports results back to (FE)
  28. 28. Worker-Queue Model BE Work Queue FE Persistent Storage
  29. 29. Worker-Queue Architecture Front-End (FE) loads work in a Queue Back-End (BE) gets work off of Queue Back-End (BE) completes work
  30. 30. Job Manager-Worker-Queue Model Request Queue BEFE Work Queue JM Result Queue Persistent Storage
  31. 31. Job Manager-Worker-Queue Architecture Front-End (FE) loads work in a Queue Job Manager (JM) breaks up work Job Manager (JM) requests (BE) to do work Back-End (BE) completes work (BE) reports results back to (JM) (JM) aggregates results (JM) reports results to (FE) (JM) Monitors and Manages Resources
  32. 32. Data Patterns
  33. 33. Local Storage Use Cautiously Stateless Node Affinity
  34. 34. Caching Data will be stale Reduce load on Data Store Compute nodes Volatile stores Good use of Local Storage Distributed cache Transformed and denormalized Persistent stores Volatile stores
  35. 35. Geographic Distribution Minimize Travel Costs Localize Data Centers CDNs
  36. 36. CQRS Command Query Responsibility Segregation More Reads than Writes
  37. 37. CQRS Cache Cache Cache BEFE BE Persistent Storage
  38. 38. Partitioning Balance load across servers Avoid transactions across partitions Might need to rebalance
  39. 39. Vertical Partitioning Functional Partitioning Systems Catalog Customers Vendors Orders Reference Data
  40. 40. Vertical Partitioning FE Customers Catalog eCommerce Orders Vendors
  41. 41. Horizontal Partitioning (Sharding) Slices of the same functional data Not Relational, No Joins No referential integrity across partitions Constraints are moved to the application Partitioning Strategy Name, Geography, Date, etc. Application needs to be partition-aware One logical DB, multiple physical partitions Rebalance Scales for both reads and writes
  42. 42. Horizontal Partitioning (Sharding) FE Eastern Central Customers Mountain Pacific
  43. 43. Relational vs. Non-Relational (NoSQL) Relational Non-RelationalNormalization Normalized DenormalizedDuplication Avoided EmbracedTransactions Distributed Scoped to PartitionStructure Schema Schema-less, JaggedResponsibility Database ApplicationScalability Vertical or Sharded Horizontal
  44. 44. Takeaways Design for Scale Scale Compute Out Decouple Processing Async Scale Data CAP & BASE over ACID Partition
  45. 45. Resources Brewers Keynote on CAP CAP articles: BASE Articles High Scalability Werner Vogels, CTO of Amazon
  46. 46. Other Sessions Exploring Scalability and Roles Insight, Next Session, Mike Benkovich Making $$$ with Windows Phone Imagination C/D, 3:00PM-4:00PM
  47. 47. Q&A
  48. 48.
  49. 49. Thank You!Eric D. BoydDirector, Chicago + Cloud PracticeCentareEmail: eric.boyd@centare.comTwitter: