The Cloud now makes seemingly infinite amounts of computing power accessible to everyone. However, to maximize this power, your applications need to scale. In this session, we will explore patterns that enable massive scalability. We will examine Brewer’s CAP Theorem and contrast it to the ACID principles that guide traditional LOB applications. And finally, we will explore how to apply these patterns when building applications for the Cloud using Windows Azure.

1. Architecting for
Massive Scalability
Eric D. Boyd
Director, Chicago + Cloud Practice

2. Introduction
Eric D. Boyd
15 Years in Technology

3. I’m From Here

4. I Moved Here

5. I Work Here

6. I Work On

7. I Blog @ ericdboyd.com

8. I Tweet @EricDBoyd

13. Agenda
What is Scalability?
ACID, BASE & CAP
Compute Patterns
Data Patterns
Q&A

14. What is Scalability?
A service is said to be scalable if when we
increase the resources in a system, it results
in increased performance in a manner
proportional to resources added.
--Werner Vogels, CTO, Amazon.com

15. What is Scalability?
Measures of Performance
Response Time
Throughput
Measures of Scalability
Transaction Volume
Concurrent Users
Size of Data and Growth

16. What can be Scaled?
Servers
CPU
Memory
Hard Disk
Licensing
Network Ports
Bandwidth
Cooling
Power
Racks
Floor Space

18. Why is Scalability Important?
Predictable Growth
Unpredictable Growth
Design for Scale
Benefits of Scalability
Dynamic Capacity Planning
Cost is Linear

19. Workload Patterns
On and Off Growing Fast
Compute
Compute
Inactivity On off
Period
Average Usage
Average Usage
Time Time
• On & off workloads (e.g. batch job) • Successful services needs to grow/scale
• Over provisioned capacity is wasted • Keeping up w/ growth is big IT challenge
• Time to market can be cumbersome • Complex lead time for deployment
Unpredictable Bursting Predictable Bursting“
Compute
Compute
Average Usage Average Usage
Time Time
• Unexpected/unplanned peak in demand • Services with micro seasonality trends
• Sudden spike impacts performance • Peaks due to periodic increased demand
• Can’t over provision for extreme cases • IT complexity and wasted capacity

20. Traditional Line-of-Business (LOB) Apps
A - Atomic
C - Consistent
I - Isolated
D - Durable

21. CAP Theorem
Consistency. The client perceives that a
set of operations has occurred all at
once.
Availability. Every operation must
terminate in an intended response.
Partition tolerance. Operations will
complete, even if individual components
are unavailable.

22. Brewer’s CAP Theorem
Consistency
Partition
Availability
Tolerance

23. BASE
Basically Available
Soft State
Eventually Consistent

24. Eventual Consistency
Given a sufficiently long period of time over
which no updates are sent, we can expect that
during this period, all updates will eventually
propagate through the system and all the
replicas will be consistent.
--Wikipedia,
http://en.wikipedia.org/wiki/Eventual_consistency

25. Real-World
twitter
Facebook
eBay
Amazon
Foursquare

26. Compute Patterns

27. Scale Up

28. Scale Out

29. Why Isn’t this the Norm?
Data is typically Bottleneck
Designed for and expects ACID
Not designed for Scale Out
Load is fixed and constant
Mind shift and additional work

30. I Need Scalability
Loose Coupling
Stateless
Messaging & SOA
Async & Background Processes
Queuing
Monitoring and Diagnostics

31. Front-End (FE) + Back-End (BE) Model
FE BE
Persistent Storage

32. Front-End (FE) + Back-End (BE) Architecture
(FE) requests (BE) to perform work
(BE) completes work
(BE) reports results back to (FE)

33. Worker-Queue Model
BE
Work Queue
FE
Persistent Storage

34. Worker-Queue Architecture
Front-End (FE) loads work in a Queue
Back-End (BE) gets work off of Queue
Back-End (BE) completes work

35. Job Manager-Worker-Queue Model
Request Queue BE
FE
Work Queue
JM
Result Queue
Persistent Storage

36. Job Manager-Worker-Queue Architecture
Front-End (FE) loads work in a Queue
Job Manager (JM) breaks up work
Job Manager (JM) requests (BE) to do work
Back-End (BE) completes work
(BE) reports results back to (JM)
(JM) aggregates results
(JM) reports results to (FE)
(JM) Monitors and Manages Resources

37. Data Patterns

38. Local Storage
Use Cautiously
Stateless
Node Affinity

39. Caching
Data will be stale
Reduce load on Data Store
Compute nodes
Volatile stores
Good use of Local Storage
Distributed cache
Transformed and denormalized
Persistent stores
Volatile stores

40. Geographic Distribution
Minimize Travel Costs
Localize Data Centers
CDNs

41. CQRS
Command Query Responsibility Segregation
More Reads than Writes

42. CQRS
Cache
Cache
Cache
BE
FE
BE
Persistent
Storage

43. Partitioning
Balance load across servers
Avoid transactions across partitions
Might need to rebalance

44. Vertical Partitioning
Functional Partitioning
Systems
Catalog
Customers
Vendors
Orders
Reference Data

45. Vertical Partitioning
FE
Customers Catalog eCommerce
Orders Vendors

46. Horizontal Partitioning (Sharding)
Slices of the same functional data
Not Relational, No Joins
No referential integrity across partitions
Constraints are moved to the application
Partitioning Strategy
Name, Geography, Date, etc.
Application needs to be partition-aware
One logical DB, multiple physical partitions
Rebalance
Scales for both reads and writes

47. Horizontal Partitioning (Sharding)
FE
Eastern Central Customers
Mountain Pacific

48. Relational vs. Non-Relational (NoSQL)
Relational Non-Relational
Normalization Normalized Denormalized
Duplication Avoided Embraced
Transactions Distributed Scoped to Partition
Structure Schema Schema-less, Jagged
Responsibility Database Application
Scalability Vertical or Sharded Horizontal

49. Takeaways
Design for Scale
Scale Compute Out
Decouple Processing Async
Scale Data
CAP & BASE over ACID
Partition

50. Resources
Brewer's Keynote on CAP
http://www.cs.berkeley.edu/~brewer/cs262b-2004/PODC-keynote.pdf
CAP articles:
http://www.julianbrowne.com/article/viewer/brewers-cap-theorem
http://www.allthingsdistributed.com/2008/12/eventually_consistent.html
BASE Articles
http://queue.acm.org/detail.cfm?id=1394128
High Scalability
http://highscalability.com
Werner Vogels, CTO of Amazon
http://www.allthingsdistributed.com/

51. Other Sessions
Exploring Scalability and Roles
Insight, Next Session, Mike Benkovich
Making $$$ with Windows Phone
Imagination C/D, 3:00PM-4:00PM

52. Q&A

53. http://www.AzureStudyGroup.com

54. Thank You!
Eric D. Boyd
Director, Chicago + Cloud Practice
Centare
Email: eric.boyd@centare.com
Twitter: www.twitter.com/EricDBoyd
Blog: www.ericdboyd.com