Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Scaling Systems: Architectures          that GrowFundamental Patterns for scaling you can      implement incrementally
Who Am I?• Kendall Miller• One of the Founders of Gibraltar Software   – Small Independent Software Vendor Founded in 2008...
Fair Warning
What is Scale?Scaling is the ability to cope   and perform under an    increasing workload.
What is Scale?Scaling to a load = available    sustaining that load
What is Scale? Being available is really about a request beingcompleted in a period of         time.
What’s your Target?0.00E+00   1.00E+07    2.00E+07    3.00E+07   4.00E+07   5.00E+07   6.00E+07   7.00E+07                ...
What’s your Target?1.00E+00 1.00E+01 1.00E+02 1.00E+03 1.00E+04 1.00E+05 1.00E+06 1.00E+07 1.00E+08                       ...
What’s your Target?  25,000 Visitors/Day = 125,000 Pages/Day11 High Traffic Hours/Day = 12,000 Pages/Hour   12,000 Pages/H...
Specific Architectures•   Gossip                   • Load Balancers + Shared•   Map Reduce                 Nothing Units• ...
ACD/C• Async – Do the work whenever• Caching – Don’t do any work you don’t have  to• Distribution – Get as many people to ...
Async• Decouple operations so you do the minimum  amount of work in performance critical paths• Queue work that can be com...
Caching• Save results of earlier work nearby where they  are handy to use again later• Apply in front of anything that’s t...
Why Caching?• Loading the world is impractical• Apps ask a lot of repeating questions.  – Stateless applications even more...
Distribution• Distribute requests across multiple systems• Classic web “Scale Out” approach• The less state held, the easi...
Consistency• The degree to which all parties observe the  same state of the system at the same time• Scaling inevitably re...
Consistency Challenges• Singleton Data Structures (Order numbers..)• State held between the endpoints of a process• Consis...
Typical Application                   Session State          Transaction Isolation                    SSL Session         ...
Caching100%       50%     10%      1% Client           Server  (Web             (Web            StorageBrowser)          S...
Distribution                            Session State and Identity                             need to be factored out    ...
Partitioned Storage Zones                   Server Client             (Web                   Server  (Web             Serv...
Partitioned Storage Intra-Zone Client                 Server                                   Orders  (Web     Customer B...
Asynchronous ProcessingServer                   Orders (Web      OrderServerServer)    Queue (WebServerServer) (WebServerS...
Fallacies of Distributed Computing•   The network is reliable•   Latency is zero•   Bandwidth is infinite•   The network i...
Fresh Problems: Partial Failures Client           Server  (Web             (Web ClientBrowser)          Server)  (Web Clie...
Fresh Problems: Partial Failures1. Break system into individual failure zones2. Monitor each instance of each zone for   p...
Withoutmonitoring, redundancy is       worthless
Fresh Problems: Upgrades                  Server Client            (Web                  Server  (Web            Server)  ...
Fresh Problems: Upgrades1. Break system into individual upgrade zones2. Upgrade each zone – Drain &   Stop, Upgrade, Verif...
Design for Software Update      From the Start    • Don’t forget Data Schemas
Bringing Home the Bacon       Testing       Testing       Testing
Critical Lessons Learned• ACD/C• Clear Consistency  Strategy• Build in monitoring and  management
Additional Information:   Websites      – www.GibraltarSoftware.com      – www.eSymmetrix.com   Follow Up      – Kendall.M...
Upcoming SlideShare
Loading in …5
×

Scaling Systems: Architectures that Grow

2,121 views

Published on

It's harder than ever to predict the load your application will need to handle in advance, so how do you design your architecture so you can afford to implement as you go and be ready for whatever comes your way.

It's easy to focus on optimizing each part of your application but your application architecture determines the options you have to make big leaps in scalability.

In this talk we'll cover practical patterns you can build today to meet the needs of rapid development while still creating systems that can scale up and out. Specific code examples will focus on .NET but the principles apply across many technologies. Real world systems will be discussed based on our experience helping customers around the world optimize their enterprise applications.

Published in: Technology
  • Be the first to comment

Scaling Systems: Architectures that Grow

  1. 1. Scaling Systems: Architectures that GrowFundamental Patterns for scaling you can implement incrementally
  2. 2. Who Am I?• Kendall Miller• One of the Founders of Gibraltar Software – Small Independent Software Vendor Founded in 2008 – Developers of VistaDB and Gibraltar – Engineers, not Sales People• Enterprise Systems Architect & Developer since 1995• BSE in Computer Engineering, University of Illinois Urbana-Champaign (UIUC)• Twitter: @KendallMiller
  3. 3. Fair Warning
  4. 4. What is Scale?Scaling is the ability to cope and perform under an increasing workload.
  5. 5. What is Scale?Scaling to a load = available sustaining that load
  6. 6. What is Scale? Being available is really about a request beingcompleted in a period of time.
  7. 7. What’s your Target?0.00E+00 1.00E+07 2.00E+07 3.00E+07 4.00E+07 5.00E+07 6.00E+07 7.00E+07 Microsoft.com Twitter.com Amazon.com Target.com Slashdot.org DevExpress.com Hanselman.com Gibraltar Software Average daily traffic in Visitors / Day
  8. 8. What’s your Target?1.00E+00 1.00E+01 1.00E+02 1.00E+03 1.00E+04 1.00E+05 1.00E+06 1.00E+07 1.00E+08 Microsoft.com Twitter.com Amazon.com Target.com Slashdot.org DevExpress.com Hanselman.com Gibraltar Software Average daily traffic in Visitors / Day
  9. 9. What’s your Target? 25,000 Visitors/Day = 125,000 Pages/Day11 High Traffic Hours/Day = 12,000 Pages/Hour 12,000 Pages/Hour = 3.3 Pages/Second
  10. 10. Specific Architectures• Gossip • Load Balancers + Shared• Map Reduce Nothing Units• Tree of Responsibility • Load Balancers +• Stream Processing Stateless Nodes + Scalable Storage• Scalable Storage • Content Addressable• Publish/Subscribe Networks• Distributed Queues • General Peer to Peer
  11. 11. ACD/C• Async – Do the work whenever• Caching – Don’t do any work you don’t have to• Distribution – Get as many people to do the work as you can• Consistency – We all agree on these key things
  12. 12. Async• Decouple operations so you do the minimum amount of work in performance critical paths• Queue work that can be completed later to smooth out load• Speculative Execution• Scheduled Requests (Nightly processes)
  13. 13. Caching• Save results of earlier work nearby where they are handy to use again later• Apply in front of anything that’s time consuming• Easiest to apply from the left to the right• Simple strategies can be really effective (EF Dump all on update)
  14. 14. Why Caching?• Loading the world is impractical• Apps ask a lot of repeating questions. – Stateless applications even more so• Answers don’t change often• Authoritative information is expensive
  15. 15. Distribution• Distribute requests across multiple systems• Classic web “Scale Out” approach• The less state held, the easier to distribute work. – Distributed database = hard – Distributed static content server = easy• Request routing for distribution can serve other availability purposes
  16. 16. Consistency• The degree to which all parties observe the same state of the system at the same time• Scaling inevitably requires compromise – Forces one source of the truth for absolute consistency and requires extensive locking to ensure parties agree – The real world doesn’t require the consistency we tend to demand of our systems
  17. 17. Consistency Challenges• Singleton Data Structures (Order numbers..)• State held between the endpoints of a process• Consistent results of queries across partitioned datasets
  18. 18. Typical Application Session State Transaction Isolation SSL Session Reader/Writer Locks Log Contention Singleton Data Structures Memory Allocation/GC Network Sockets Request Queue Client Server (Web (Web StorageBrowser) Server) (Database)
  19. 19. Caching100% 50% 10% 1% Client Server (Web (Web StorageBrowser) Server) (Database)
  20. 20. Distribution Session State and Identity need to be factored out Partition (Sticky Session) First, then stateless nodes Client Server (Web (Web ClientBrowser) Server) (Web Client StorageBrowser) (Web (Database) Client ServerBrowser) (Web (WebBrowser) Server)
  21. 21. Partitioned Storage Zones Server Client (Web Server (Web Server) (Web Storage Client (Database)Browser) Server) (Web ClientBrowser) (Web Client ServerBrowser) (Web (WebBrowser) Server Server) Storage (Web Server) (Database)
  22. 22. Partitioned Storage Intra-Zone Client Server Orders (Web Customer B (Web Server ClientBrowser) Server) (Web (Web Server Client Server)Browser) (Web (Web Server Client Server)Browser) (Web Products (Web Server)Browser) Inventory
  23. 23. Asynchronous ProcessingServer Orders (Web OrderServerServer) Queue (WebServerServer) (WebServerServer) (Web ProductsServer) Order Processing Server Inventory
  24. 24. Fallacies of Distributed Computing• The network is reliable• Latency is zero• Bandwidth is infinite• The network is secure• Topology doesn’t change• There is one administrator• Transport cost is zero• The network is homogeneous
  25. 25. Fresh Problems: Partial Failures Client Server (Web (Web ClientBrowser) Server) (Web Client StorageBrowser) (Web (Database) Client ServerBrowser) (Web (WebBrowser) Server)
  26. 26. Fresh Problems: Partial Failures1. Break system into individual failure zones2. Monitor each instance of each zone for problems3. Route around bad instances
  27. 27. Withoutmonitoring, redundancy is worthless
  28. 28. Fresh Problems: Upgrades Server Client (Web Server (Web Server) (Web Storage Client (Database)Browser) Server) (Web ClientBrowser) (Web Client ServerBrowser) (Web (WebBrowser) Server Server) Storage (Web Server) (Database)
  29. 29. Fresh Problems: Upgrades1. Break system into individual upgrade zones2. Upgrade each zone – Drain & Stop, Upgrade, Verify.3. Cut traffic over to updated zones
  30. 30. Design for Software Update From the Start • Don’t forget Data Schemas
  31. 31. Bringing Home the Bacon Testing Testing Testing
  32. 32. Critical Lessons Learned• ACD/C• Clear Consistency Strategy• Build in monitoring and management
  33. 33. Additional Information: Websites – www.GibraltarSoftware.com – www.eSymmetrix.com Follow Up – Kendall.Miller@eSymmetrix.com – Twitter: kendallmiller

×