• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Scalable and Available, Patterns for Success
 

Scalable and Available, Patterns for Success

on

  • 17,859 views

 

Statistics

Views

Total Views
17,859
Views on SlideShare
17,032
Embed Views
827

Actions

Likes
72
Downloads
667
Comments
3

20 Embeds 827

http://5net.com 397
http://www.scoop.it 216
https://twitter.com 112
http://blog.cloudpack.jp 27
http://paper.li 15
http://www.techgig.com 13
http://thinkery.me 11
http://webcache.googleusercontent.com 6
http://a0.twimg.com 5
http://twitter.com 5
http://webclusterpre.si.unav.es 5
http://flavors.me 3
http://localhost 3
http://feeds.feedburner.com 3
http://grumpy.junta.com.au 1
http://www.onlydoo.com 1
http://trunk.ly 1
http://www.slideshare.net 1
http://www.linkedin.com 1
http://us-w1.rockmelt.com 1
More...

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

13 of 3 previous next Post a comment

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Great presentation. Would love to hear the commentary behind this as well. Was the video captured as well ?
    Are you sure you want to
    Your message goes here
    Processing…
  • Its a stock theme in Keynote on Mac..
    Are you sure you want to
    Your message goes here
    Processing…
  • Hi there, could you share the backbround mesh on ppt?
    Thank you.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Scalable and Available, Patterns for Success Scalable and Available, Patterns for Success Presentation Transcript

    • Scalable & Available Patterns for SuccessTuesday, March 8, 2011 1
    • Derek Collison @derekcollison dcollison@vmware.com derek.collison@gmail.comTuesday, March 8, 2011 2
    • Special Thanks Jonas Bonér twitter: @jboner http://jonasboner.com/ http://www.slideshare.net/jboner/scalability- availability-stability-patternsTuesday, March 8, 2011 3
    • Background • Scalable Apps maintain performance under load • More requests, More users, More data • Available Apps maintain the experience during failures • Hardware failures, Network splits/partitioning • Simple Designs tend to scale betterTuesday, March 8, 2011 4
    • Background Good Performance is goodTuesday, March 8, 2011 5
    • Background Predictable Performance is king!Tuesday, March 8, 2011 6
    • Background Understand your data!Tuesday, March 8, 2011 7
    • Background Understand the user experience!Tuesday, March 8, 2011 8
    • Background Measure everything (can’t fix what you don’t know)Tuesday, March 8, 2011 9
    • Background Don’t be a failure of your own successTuesday, March 8, 2011 10
    • Background Master the • Good Performance is good • Predictably Good Performance is king! Tradeoffs • Measure everything (can’t fix what you don’t know) • Understand app and your data!) (For your your data • Understand your user experience • Don’t be a failure of your own successTuesday, March 8, 2011 11
    • Master the Tradeoffs Performance vs ScalabilityTuesday, March 8, 2011 12
    • Master the Tradeoffs Latency vs ThroughputTuesday, March 8, 2011 13
    • Master the Tradeoffs Availability vs ConsistencyTuesday, March 8, 2011 14
    • Lots of ways to skin a cat!Tuesday, March 8, 2011 15
    • Scalability PatternsTuesday, March 8, 2011 16
    • Performance vs ScalabilityTuesday, March 8, 2011 17
    • How do I know if I have a performance problem? If your system is slow for a single request/userTuesday, March 8, 2011 18
    • How do I know if I have a scalability problem? If your system is fast for a single request/user but slow for many usersTuesday, March 8, 2011 19
    • Latency vs ThroughputTuesday, March 8, 2011 20
    • You should strive for maximal throughput with acceptable latencyTuesday, March 8, 2011 21
    • Performance vs Scalability Response Time Concurrent RequestsTuesday, March 8, 2011 22
    • Know what to scale! • CPU or IO Bound? • Scale up or Scale out? • Waiting on IO? What? Disk/Net/Other System? • How many components are used per request? • Know who and what the slowest will be!Tuesday, March 8, 2011 23
    • Scalability Patterns BehaviorTuesday, March 8, 2011 24
    • Scalability Patterns: Behavior ✓Event-Driven Architectures ✓Load-Balancing ✓Parallel ComputingTuesday, March 8, 2011 25
    • Event-Driven Architecture ✓Events ✓Messaging ✓Asynchronous ✓Non-blockingTuesday, March 8, 2011 26
    • Messaging ✓Publish-Subscribe ✓Queuing ✓Request-Reply ✓Store and ForwardTuesday, March 8, 2011 27
    • Messaging - Publish Subscribe 1:N Subscriber Publisher Subject Subscriber SubscriberTuesday, March 8, 2011 28
    • Messaging - Queuing 1:1 Subscriber Publisher Queue Subscriber Message #1 SubscriberTuesday, March 8, 2011 29
    • Messaging - Queuing 1:1 Subscriber Publisher Queue Subscriber Message #2 SubscriberTuesday, March 8, 2011 30
    • Messaging - Queuing 1:1 Subscriber Publisher Queue Subscriber Message #3 SubscriberTuesday, March 8, 2011 31
    • Messaging - Request Reply 1:1 Reply Subscriber Publisher Subject Subscriber SubscriberTuesday, March 8, 2011 32
    • Messaging Patterns ✓Addressing, discovery ✓Command and control ✓Load-balancing ✓N-way scalabilityTuesday, March 8, 2011 33
    • Messaging ✓Standards ✓ AMQP (wire) ✓ JMS (api) ✓Products ✓ RabbitMQ ✓ ZeroMQ ✓ ActiveMQ ✓ TIBCO ✓ MQSeriesTuesday, March 8, 2011 34
    • Asynchronous and Non-Blocking ✓Don’t wait, go doing something else ✓Never block ✓All callbacks all the time can get messy! ✓Good language/framework support ✓functional closures ✓co-routinesTuesday, March 8, 2011 35
    • Load Balancing ✓Multiple endpoints to perform work ✓Can be semantically aware ✓Chainable: DNS, hardware, software ✓Endpoints can be Hardware, VM, process, thread, co-routine, fiber, etc.Tuesday, March 8, 2011 36
    • Load Balancing Selection ✓Random ✓Round Robin ✓Weighted ✓Dynamically “aware” ✓Least connections ✓Least loadedTuesday, March 8, 2011 37
    • Load Balancing Technologies ✓DNS Round Robin ✓Anycast ✓Reverse Proxies ✓Clustering ✓Hardware Load BalancersTuesday, March 8, 2011 38
    • Load Balancing Reverse Proxies ✓Nginx ✓HAProxy ✓Apache (mod_proxy) ✓SquidTuesday, March 8, 2011 39
    • Parallel Computing ✓Divide and Conquer ✓Worker queues ✓Map Reduce ✓UE = Unit of Execution ✓VM, process, thread, co-routine, fiber, callbackTuesday, March 8, 2011 40
    • Parallel Computing Worker Queues ✓Good for offloading tasks ✓Need bounded time check in master ✓Async result processing ✓Fork/Join patternTuesday, March 8, 2011 41
    • Parallel Computing MapReduce ✓Used internally at Google ✓Variation of Fork and Join ✓Distributed ✓Originally used for logs processingTuesday, March 8, 2011 42
    • Parallel Computing MapReduce ✓Google’s MapReduce ✓Hadoop ✓Amazon’s Elastic MapReduce ✓RIAK uses it internally for queriesTuesday, March 8, 2011 43
    • Scalability Patterns StateTuesday, March 8, 2011 44
    • Scalability Patterns: State Harder than scaling behaviorTuesday, March 8, 2011 45
    • Scalability Patterns: State ✓Master Record ✓Replication ✓Sharding ✓Caching ✓NoSQL ✓ConcurrencyTuesday, March 8, 2011 46
    • Master Record ✓Normally Relational Databases (RDBMS) ✓NoSQL Databases emerging ✓Can’t lose this data ✓Scaling can be a challengeTuesday, March 8, 2011 47
    • Master Record: Scaling ✓Traditonally Scale Up ✓Technology will help here ✓SSD (50k-100k IOPs) ✓More memory/cores per box ✓Faster network connectivity ✓Clustering AppliancesTuesday, March 8, 2011 48
    • Clustering Appliances 64 bit Infiniband SSDTuesday, March 8, 2011 49
    • Master Record: Scaling ✓Scaling Reads vs Writes? ✓Scaling Reads with Slaves ✓Synchronous (Speed of Light) ✓AsynchronousTuesday, March 8, 2011 50
    • Master Record: Scaling How do we scale OUT?Tuesday, March 8, 2011 51
    • Master Record: Replication ✓Synchronous vs Asynchronous ✓Master / Slave Replication ✓Master / Master Replication ✓Tree Replication ✓Buddy ReplicationTuesday, March 8, 2011 52
    • Replication: Master / SlaveTuesday, March 8, 2011 53
    • Replication: Master / MasterTuesday, March 8, 2011 54
    • Replication: TreeTuesday, March 8, 2011 55
    • Replication: BuddyTuesday, March 8, 2011 56
    • Sharding ✓Partitioning state ✓Requests need to know where to go ✓Distributed Hash ✓Load Balancer ✓MessagingTuesday, March 8, 2011 57
    • Sharding: ParitioningTuesday, March 8, 2011 58
    • Sharding: ReplicationTuesday, March 8, 2011 59
    • Sharding: Over-provision ✓Use N partitions ✓Use Y replicas ✓Use message based requests ✓First back wins ✓Therefore user wins (Google Search)Tuesday, March 8, 2011 60
    • Master Record: RDBMS Do we really need an RDBMS?Tuesday, March 8, 2011 61
    • Master Record: RDBMS Don’t underestimate RDBMS or the ability of a single machineTuesday, March 8, 2011 62
    • Master Record: RDBMS What about alternatives?Tuesday, March 8, 2011 63
    • NoSQL ✓Key-Value ✓Column Databases ✓Document Databases ✓Graph Databases ✓Datastructure DatabasesTuesday, March 8, 2011 64
    • NoSQL ✓Key-Value: (Memcache, Redis, Riak) ✓Column Databases: (Cassandra, Vertica) ✓Document Databases: (MongoDB, CouchDB) ✓Graph Databases: (Neo4J, AllegroGraph) ✓Datastructure Databases: (Redis, Hazelcast)Tuesday, March 8, 2011 65
    • NoSQL in the wild ✓Google: Bigtable, Colossus ✓Twitter: Redis ✓Amazon: Dynamo, SimpleDB ✓Yahoo: HBase (Hadoop) ✓Facebook: Cassandra, HBaseTuesday, March 8, 2011 66
    • Caching ✓Cache early and often ✓Usually biggest bang for the buck ✓Referential Transparency ✓Polyglot APIs coming ✓NoSQL stores ✓Cache invalidation is still hard!Tuesday, March 8, 2011 67
    • Caching ✓HTTP (HTML, JS, CSS, Images, Media) ✓Key/Value Data ✓Semantic Data structuresTuesday, March 8, 2011 68
    • HTTP Caching ✓Varnish ✓Squid ✓Pound ✓Nginx ✓Rack-cacheTuesday, March 8, 2011 69
    • HTTP Caching CDN ✓Akamai ✓Limelight ✓Level3 ✓Digital Fountain (Qualcomm) ✓aiCacheTuesday, March 8, 2011 70
    • HTTP Caching CDNTuesday, March 8, 2011 71
    • HTTP Caching ✓Lives in browsers, proxies, CDNs, apps ✓Hard to control, so do it right! ✓Master page controls other resources ✓master page not cached (at least too far) ✓read-only resources ✓change link in master pageTuesday, March 8, 2011 72
    • Key/Value Caching ✓Memcache ✓Redis ✓Riak ✓VoldemortTuesday, March 8, 2011 73
    • Data Structure CachingTuesday, March 8, 2011 74
    • Data Structure Caching ✓Standalone ✓Augment RDBMS ✓In Memory or on DiskTuesday, March 8, 2011 75
    • Data Structure Caching ✓Data Types ✓Strings, Hashes, Lists, Sets, Sorted Sets ✓Atomic Operations ✓Push, pop, ranges, set operations (intersect, union)Tuesday, March 8, 2011 76
    • Caching Patterns ✓Write Through ✓Write Behind ✓Replicated ✓P2PTuesday, March 8, 2011 77
    • Cache Invalidation ✓TTL (Time to Live) ✓Bounded FIFO or LIFO ✓Explicit cache invalidation ✓Explicit non-use of read-only resource ✓Harder problem the more master items usedTuesday, March 8, 2011 78
    • Scalability Key Points ✓The problem is not where you think ;) ✓Autoscaling is a myth ✓Can’t fix what you can’t measure ✓Scaling master record writes is hard ✓Scaling reads is more tractable ✓What is the opex cost of your choices?Tuesday, March 8, 2011 79
    • Availability PatternsTuesday, March 8, 2011 80
    • What do you do when things go bad?Tuesday, March 8, 2011 81
    • Availability Patterns Available vs ConsistentTuesday, March 8, 2011 82
    • Availability Patterns AvailableTuesday, March 8, 2011 83
    • We have been here before, right?Tuesday, March 8, 2011 84
    • Yes, we have been here before!?Tuesday, March 8, 2011 85
    • Scalability Patterns BehaviorTuesday, March 8, 2011 86
    • Scalability Patterns: Behavior ✓Event-Driven Architectures ✓Load-Balancing ✓Parallel ComputingTuesday, March 8, 2011 87
    • Scalability Patterns StateTuesday, March 8, 2011 88
    • Scalability Patterns: State ✓Master Record ✓Replication ✓Sharding ✓Caching ✓NoSQL ✓ConcurrencyTuesday, March 8, 2011 89
    • But let’s talk more about your dataTuesday, March 8, 2011 90
    • Availability Patterns Available vs ConsistentTuesday, March 8, 2011 91
    • Brewster’s CAP TheoremTuesday, March 8, 2011 92
    • Brewster’s CAP TheoremTuesday, March 8, 2011 93
    • You can only pick 2 Consistency Availability Partition ToleranceTuesday, March 8, 2011 94
    • Centralized Systems ✓If the system is centralized ✓no P (network partitions) ✓So you get both: ✓Availability ✓ConsistencyTuesday, March 8, 2011 95
    • Distributed Systems ✓If the system is distributed ✓you will have P! (network partitions) ✓So you get pick one: ✓Availability ✓ConsistencyTuesday, March 8, 2011 96
    • CAP in reality ✓There is only once choice to make: ✓When there is a network partition, which do you sacrifice? ✓Availability ✓ConsistencyTuesday, March 8, 2011 97
    • BASETuesday, March 8, 2011 98
    • What is BASE?Tuesday, March 8, 2011 99
    • BASE Basically Available Soft State Eventually ConsistentTuesday, March 8, 2011 100
    • Eventually Consistent ✓Great tradeoff for the right kind of data ✓Can’t be used everywhere ✓Works in more places than you think ✓Solved speed of light problemTuesday, March 8, 2011 101
    • Availability Patterns FailoverTuesday, March 8, 2011 102
    • Availability Patterns: Failover ✓Failover is complex ✓Switch time is critical ✓Failback is equally as complexTuesday, March 8, 2011 103
    • Availability Patterns: Failover Copyright Michael NygaardTuesday, March 8, 2011 104
    • Availability Patterns: Failback Copyright Michael NygaardTuesday, March 8, 2011 105
    • Availability Patterns: Replication ✓Synchronous vs Asynchronous ✓Master / Slave Replication ✓Master / Master ReplicationTuesday, March 8, 2011 106
    • Availability Patterns: Redirection ✓DNS ✓Load Balancers ✓Secondary SitesTuesday, March 8, 2011 107
    • Availability Key Points ✓Always have a dial tone ✓Syntactically correct is good ✓Semantically correct is better ✓Be transparentTuesday, March 8, 2011 108
    • Background Beating the • Good Performance is good • Predictably Good Performance is king! dead horse • Measure everything (can’t fix what you don’t know) • Understand your data • Understand your user experience • Don’t be a failure of your own successTuesday, March 8, 2011 109
    • Background Understand • Good Performance is good • Predictably Good Performance is king! your data! • Measure everything (can’t fix what you don’t know) • Understand your data • Understand your user experience • Don’t be a failure of your own successTuesday, March 8, 2011 110
    • Background Understand • Good Performance is good • Predictably Good Performance is king! your user! • Measure everything (can’t fix what you don’t know) • Understand your data • Understand your user experience • Don’t be a failure of your own successTuesday, March 8, 2011 111
    • Background Understand • Good Performance is good the • Predictably Good Performance is king! • Measure everything (can’t fix what you don’t know) experience! • Understand your data • Understand your user experience • Don’t be a failure of your own successTuesday, March 8, 2011 112
    • Background Master the • Good Performance is good • Predictably Good Performance is king! Tradeoffs • Measure everything (can’t fix what you don’t know) • Understand app and your data!) (For your your data • Understand your user experience • Don’t be a failure of your own successTuesday, March 8, 2011 113
    • Thank YouTuesday, March 8, 2011 114
    • Thank YouTuesday, March 8, 2011 115
    • Questions?Tuesday, March 8, 2011 116
    • Derek Collison @derekcollison dcollison@vmware.com derek.collison@gmail.comTuesday, March 8, 2011 117