Scalable and Available, Patterns for Success

  • 17,420 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Great presentation. Would love to hear the commentary behind this as well. Was the video captured as well ?
    Are you sure you want to
    Your message goes here
  • Its a stock theme in Keynote on Mac..
    Are you sure you want to
    Your message goes here
  • Hi there, could you share the backbround mesh on ppt?
    Thank you.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
17,420
On Slideshare
0
From Embeds
0
Number of Embeds
8

Actions

Shares
Downloads
682
Comments
3
Likes
74

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Scalable & Available Patterns for SuccessTuesday, March 8, 2011 1
  • 2. Derek Collison @derekcollison dcollison@vmware.com derek.collison@gmail.comTuesday, March 8, 2011 2
  • 3. Special Thanks Jonas Bonér twitter: @jboner http://jonasboner.com/ http://www.slideshare.net/jboner/scalability- availability-stability-patternsTuesday, March 8, 2011 3
  • 4. Background • Scalable Apps maintain performance under load • More requests, More users, More data • Available Apps maintain the experience during failures • Hardware failures, Network splits/partitioning • Simple Designs tend to scale betterTuesday, March 8, 2011 4
  • 5. Background Good Performance is goodTuesday, March 8, 2011 5
  • 6. Background Predictable Performance is king!Tuesday, March 8, 2011 6
  • 7. Background Understand your data!Tuesday, March 8, 2011 7
  • 8. Background Understand the user experience!Tuesday, March 8, 2011 8
  • 9. Background Measure everything (can’t fix what you don’t know)Tuesday, March 8, 2011 9
  • 10. Background Don’t be a failure of your own successTuesday, March 8, 2011 10
  • 11. Background Master the • Good Performance is good • Predictably Good Performance is king! Tradeoffs • Measure everything (can’t fix what you don’t know) • Understand app and your data!) (For your your data • Understand your user experience • Don’t be a failure of your own successTuesday, March 8, 2011 11
  • 12. Master the Tradeoffs Performance vs ScalabilityTuesday, March 8, 2011 12
  • 13. Master the Tradeoffs Latency vs ThroughputTuesday, March 8, 2011 13
  • 14. Master the Tradeoffs Availability vs ConsistencyTuesday, March 8, 2011 14
  • 15. Lots of ways to skin a cat!Tuesday, March 8, 2011 15
  • 16. Scalability PatternsTuesday, March 8, 2011 16
  • 17. Performance vs ScalabilityTuesday, March 8, 2011 17
  • 18. How do I know if I have a performance problem? If your system is slow for a single request/userTuesday, March 8, 2011 18
  • 19. How do I know if I have a scalability problem? If your system is fast for a single request/user but slow for many usersTuesday, March 8, 2011 19
  • 20. Latency vs ThroughputTuesday, March 8, 2011 20
  • 21. You should strive for maximal throughput with acceptable latencyTuesday, March 8, 2011 21
  • 22. Performance vs Scalability Response Time Concurrent RequestsTuesday, March 8, 2011 22
  • 23. Know what to scale! • CPU or IO Bound? • Scale up or Scale out? • Waiting on IO? What? Disk/Net/Other System? • How many components are used per request? • Know who and what the slowest will be!Tuesday, March 8, 2011 23
  • 24. Scalability Patterns BehaviorTuesday, March 8, 2011 24
  • 25. Scalability Patterns: Behavior ✓Event-Driven Architectures ✓Load-Balancing ✓Parallel ComputingTuesday, March 8, 2011 25
  • 26. Event-Driven Architecture ✓Events ✓Messaging ✓Asynchronous ✓Non-blockingTuesday, March 8, 2011 26
  • 27. Messaging ✓Publish-Subscribe ✓Queuing ✓Request-Reply ✓Store and ForwardTuesday, March 8, 2011 27
  • 28. Messaging - Publish Subscribe 1:N Subscriber Publisher Subject Subscriber SubscriberTuesday, March 8, 2011 28
  • 29. Messaging - Queuing 1:1 Subscriber Publisher Queue Subscriber Message #1 SubscriberTuesday, March 8, 2011 29
  • 30. Messaging - Queuing 1:1 Subscriber Publisher Queue Subscriber Message #2 SubscriberTuesday, March 8, 2011 30
  • 31. Messaging - Queuing 1:1 Subscriber Publisher Queue Subscriber Message #3 SubscriberTuesday, March 8, 2011 31
  • 32. Messaging - Request Reply 1:1 Reply Subscriber Publisher Subject Subscriber SubscriberTuesday, March 8, 2011 32
  • 33. Messaging Patterns ✓Addressing, discovery ✓Command and control ✓Load-balancing ✓N-way scalabilityTuesday, March 8, 2011 33
  • 34. Messaging ✓Standards ✓ AMQP (wire) ✓ JMS (api) ✓Products ✓ RabbitMQ ✓ ZeroMQ ✓ ActiveMQ ✓ TIBCO ✓ MQSeriesTuesday, March 8, 2011 34
  • 35. Asynchronous and Non-Blocking ✓Don’t wait, go doing something else ✓Never block ✓All callbacks all the time can get messy! ✓Good language/framework support ✓functional closures ✓co-routinesTuesday, March 8, 2011 35
  • 36. Load Balancing ✓Multiple endpoints to perform work ✓Can be semantically aware ✓Chainable: DNS, hardware, software ✓Endpoints can be Hardware, VM, process, thread, co-routine, fiber, etc.Tuesday, March 8, 2011 36
  • 37. Load Balancing Selection ✓Random ✓Round Robin ✓Weighted ✓Dynamically “aware” ✓Least connections ✓Least loadedTuesday, March 8, 2011 37
  • 38. Load Balancing Technologies ✓DNS Round Robin ✓Anycast ✓Reverse Proxies ✓Clustering ✓Hardware Load BalancersTuesday, March 8, 2011 38
  • 39. Load Balancing Reverse Proxies ✓Nginx ✓HAProxy ✓Apache (mod_proxy) ✓SquidTuesday, March 8, 2011 39
  • 40. Parallel Computing ✓Divide and Conquer ✓Worker queues ✓Map Reduce ✓UE = Unit of Execution ✓VM, process, thread, co-routine, fiber, callbackTuesday, March 8, 2011 40
  • 41. Parallel Computing Worker Queues ✓Good for offloading tasks ✓Need bounded time check in master ✓Async result processing ✓Fork/Join patternTuesday, March 8, 2011 41
  • 42. Parallel Computing MapReduce ✓Used internally at Google ✓Variation of Fork and Join ✓Distributed ✓Originally used for logs processingTuesday, March 8, 2011 42
  • 43. Parallel Computing MapReduce ✓Google’s MapReduce ✓Hadoop ✓Amazon’s Elastic MapReduce ✓RIAK uses it internally for queriesTuesday, March 8, 2011 43
  • 44. Scalability Patterns StateTuesday, March 8, 2011 44
  • 45. Scalability Patterns: State Harder than scaling behaviorTuesday, March 8, 2011 45
  • 46. Scalability Patterns: State ✓Master Record ✓Replication ✓Sharding ✓Caching ✓NoSQL ✓ConcurrencyTuesday, March 8, 2011 46
  • 47. Master Record ✓Normally Relational Databases (RDBMS) ✓NoSQL Databases emerging ✓Can’t lose this data ✓Scaling can be a challengeTuesday, March 8, 2011 47
  • 48. Master Record: Scaling ✓Traditonally Scale Up ✓Technology will help here ✓SSD (50k-100k IOPs) ✓More memory/cores per box ✓Faster network connectivity ✓Clustering AppliancesTuesday, March 8, 2011 48
  • 49. Clustering Appliances 64 bit Infiniband SSDTuesday, March 8, 2011 49
  • 50. Master Record: Scaling ✓Scaling Reads vs Writes? ✓Scaling Reads with Slaves ✓Synchronous (Speed of Light) ✓AsynchronousTuesday, March 8, 2011 50
  • 51. Master Record: Scaling How do we scale OUT?Tuesday, March 8, 2011 51
  • 52. Master Record: Replication ✓Synchronous vs Asynchronous ✓Master / Slave Replication ✓Master / Master Replication ✓Tree Replication ✓Buddy ReplicationTuesday, March 8, 2011 52
  • 53. Replication: Master / SlaveTuesday, March 8, 2011 53
  • 54. Replication: Master / MasterTuesday, March 8, 2011 54
  • 55. Replication: TreeTuesday, March 8, 2011 55
  • 56. Replication: BuddyTuesday, March 8, 2011 56
  • 57. Sharding ✓Partitioning state ✓Requests need to know where to go ✓Distributed Hash ✓Load Balancer ✓MessagingTuesday, March 8, 2011 57
  • 58. Sharding: ParitioningTuesday, March 8, 2011 58
  • 59. Sharding: ReplicationTuesday, March 8, 2011 59
  • 60. Sharding: Over-provision ✓Use N partitions ✓Use Y replicas ✓Use message based requests ✓First back wins ✓Therefore user wins (Google Search)Tuesday, March 8, 2011 60
  • 61. Master Record: RDBMS Do we really need an RDBMS?Tuesday, March 8, 2011 61
  • 62. Master Record: RDBMS Don’t underestimate RDBMS or the ability of a single machineTuesday, March 8, 2011 62
  • 63. Master Record: RDBMS What about alternatives?Tuesday, March 8, 2011 63
  • 64. NoSQL ✓Key-Value ✓Column Databases ✓Document Databases ✓Graph Databases ✓Datastructure DatabasesTuesday, March 8, 2011 64
  • 65. NoSQL ✓Key-Value: (Memcache, Redis, Riak) ✓Column Databases: (Cassandra, Vertica) ✓Document Databases: (MongoDB, CouchDB) ✓Graph Databases: (Neo4J, AllegroGraph) ✓Datastructure Databases: (Redis, Hazelcast)Tuesday, March 8, 2011 65
  • 66. NoSQL in the wild ✓Google: Bigtable, Colossus ✓Twitter: Redis ✓Amazon: Dynamo, SimpleDB ✓Yahoo: HBase (Hadoop) ✓Facebook: Cassandra, HBaseTuesday, March 8, 2011 66
  • 67. Caching ✓Cache early and often ✓Usually biggest bang for the buck ✓Referential Transparency ✓Polyglot APIs coming ✓NoSQL stores ✓Cache invalidation is still hard!Tuesday, March 8, 2011 67
  • 68. Caching ✓HTTP (HTML, JS, CSS, Images, Media) ✓Key/Value Data ✓Semantic Data structuresTuesday, March 8, 2011 68
  • 69. HTTP Caching ✓Varnish ✓Squid ✓Pound ✓Nginx ✓Rack-cacheTuesday, March 8, 2011 69
  • 70. HTTP Caching CDN ✓Akamai ✓Limelight ✓Level3 ✓Digital Fountain (Qualcomm) ✓aiCacheTuesday, March 8, 2011 70
  • 71. HTTP Caching CDNTuesday, March 8, 2011 71
  • 72. HTTP Caching ✓Lives in browsers, proxies, CDNs, apps ✓Hard to control, so do it right! ✓Master page controls other resources ✓master page not cached (at least too far) ✓read-only resources ✓change link in master pageTuesday, March 8, 2011 72
  • 73. Key/Value Caching ✓Memcache ✓Redis ✓Riak ✓VoldemortTuesday, March 8, 2011 73
  • 74. Data Structure CachingTuesday, March 8, 2011 74
  • 75. Data Structure Caching ✓Standalone ✓Augment RDBMS ✓In Memory or on DiskTuesday, March 8, 2011 75
  • 76. Data Structure Caching ✓Data Types ✓Strings, Hashes, Lists, Sets, Sorted Sets ✓Atomic Operations ✓Push, pop, ranges, set operations (intersect, union)Tuesday, March 8, 2011 76
  • 77. Caching Patterns ✓Write Through ✓Write Behind ✓Replicated ✓P2PTuesday, March 8, 2011 77
  • 78. Cache Invalidation ✓TTL (Time to Live) ✓Bounded FIFO or LIFO ✓Explicit cache invalidation ✓Explicit non-use of read-only resource ✓Harder problem the more master items usedTuesday, March 8, 2011 78
  • 79. Scalability Key Points ✓The problem is not where you think ;) ✓Autoscaling is a myth ✓Can’t fix what you can’t measure ✓Scaling master record writes is hard ✓Scaling reads is more tractable ✓What is the opex cost of your choices?Tuesday, March 8, 2011 79
  • 80. Availability PatternsTuesday, March 8, 2011 80
  • 81. What do you do when things go bad?Tuesday, March 8, 2011 81
  • 82. Availability Patterns Available vs ConsistentTuesday, March 8, 2011 82
  • 83. Availability Patterns AvailableTuesday, March 8, 2011 83
  • 84. We have been here before, right?Tuesday, March 8, 2011 84
  • 85. Yes, we have been here before!?Tuesday, March 8, 2011 85
  • 86. Scalability Patterns BehaviorTuesday, March 8, 2011 86
  • 87. Scalability Patterns: Behavior ✓Event-Driven Architectures ✓Load-Balancing ✓Parallel ComputingTuesday, March 8, 2011 87
  • 88. Scalability Patterns StateTuesday, March 8, 2011 88
  • 89. Scalability Patterns: State ✓Master Record ✓Replication ✓Sharding ✓Caching ✓NoSQL ✓ConcurrencyTuesday, March 8, 2011 89
  • 90. But let’s talk more about your dataTuesday, March 8, 2011 90
  • 91. Availability Patterns Available vs ConsistentTuesday, March 8, 2011 91
  • 92. Brewster’s CAP TheoremTuesday, March 8, 2011 92
  • 93. Brewster’s CAP TheoremTuesday, March 8, 2011 93
  • 94. You can only pick 2 Consistency Availability Partition ToleranceTuesday, March 8, 2011 94
  • 95. Centralized Systems ✓If the system is centralized ✓no P (network partitions) ✓So you get both: ✓Availability ✓ConsistencyTuesday, March 8, 2011 95
  • 96. Distributed Systems ✓If the system is distributed ✓you will have P! (network partitions) ✓So you get pick one: ✓Availability ✓ConsistencyTuesday, March 8, 2011 96
  • 97. CAP in reality ✓There is only once choice to make: ✓When there is a network partition, which do you sacrifice? ✓Availability ✓ConsistencyTuesday, March 8, 2011 97
  • 98. BASETuesday, March 8, 2011 98
  • 99. What is BASE?Tuesday, March 8, 2011 99
  • 100. BASE Basically Available Soft State Eventually ConsistentTuesday, March 8, 2011 100
  • 101. Eventually Consistent ✓Great tradeoff for the right kind of data ✓Can’t be used everywhere ✓Works in more places than you think ✓Solved speed of light problemTuesday, March 8, 2011 101
  • 102. Availability Patterns FailoverTuesday, March 8, 2011 102
  • 103. Availability Patterns: Failover ✓Failover is complex ✓Switch time is critical ✓Failback is equally as complexTuesday, March 8, 2011 103
  • 104. Availability Patterns: Failover Copyright Michael NygaardTuesday, March 8, 2011 104
  • 105. Availability Patterns: Failback Copyright Michael NygaardTuesday, March 8, 2011 105
  • 106. Availability Patterns: Replication ✓Synchronous vs Asynchronous ✓Master / Slave Replication ✓Master / Master ReplicationTuesday, March 8, 2011 106
  • 107. Availability Patterns: Redirection ✓DNS ✓Load Balancers ✓Secondary SitesTuesday, March 8, 2011 107
  • 108. Availability Key Points ✓Always have a dial tone ✓Syntactically correct is good ✓Semantically correct is better ✓Be transparentTuesday, March 8, 2011 108
  • 109. Background Beating the • Good Performance is good • Predictably Good Performance is king! dead horse • Measure everything (can’t fix what you don’t know) • Understand your data • Understand your user experience • Don’t be a failure of your own successTuesday, March 8, 2011 109
  • 110. Background Understand • Good Performance is good • Predictably Good Performance is king! your data! • Measure everything (can’t fix what you don’t know) • Understand your data • Understand your user experience • Don’t be a failure of your own successTuesday, March 8, 2011 110
  • 111. Background Understand • Good Performance is good • Predictably Good Performance is king! your user! • Measure everything (can’t fix what you don’t know) • Understand your data • Understand your user experience • Don’t be a failure of your own successTuesday, March 8, 2011 111
  • 112. Background Understand • Good Performance is good the • Predictably Good Performance is king! • Measure everything (can’t fix what you don’t know) experience! • Understand your data • Understand your user experience • Don’t be a failure of your own successTuesday, March 8, 2011 112
  • 113. Background Master the • Good Performance is good • Predictably Good Performance is king! Tradeoffs • Measure everything (can’t fix what you don’t know) • Understand app and your data!) (For your your data • Understand your user experience • Don’t be a failure of your own successTuesday, March 8, 2011 113
  • 114. Thank YouTuesday, March 8, 2011 114
  • 115. Thank YouTuesday, March 8, 2011 115
  • 116. Questions?Tuesday, March 8, 2011 116
  • 117. Derek Collison @derekcollison dcollison@vmware.com derek.collison@gmail.comTuesday, March 8, 2011 117