Designing Scalable Applications

2. WHAT IS SCALABILITY? The concept

3. “Scalability is the capability of a system, network, or process to handle a growing amount of work, or its potential to be enlarged in order to accommodate that growth.” Bondi, André B. (2000). Characteristics of scalability and their impact on performance. 3

4. Let's go deeper… 4

5. 5Scalability for sandwiches How hungry are you? A bigger sandwich may ﬁt your need (vertical scaling)

7. 7 How many people are hungry? Think about it…

8. 8 Fast-food restaurant CHAINS You need to scale the entire solution (horizontal scaling)

9. Scaling Computing Resources 9 • Processing (CPU) • Memory • Storage • Network Scale-up vs Scale-out

10. Vertical Scaling (Scale-up) 10 Generally refers to adding more processors and RAM, buying a more expensive and robust server. • Pros •Generally less challenging to implement •Less power consumption than running multiple servers •Less licensing costs • Less network hardware than scaling horizontally. • Cons • Hardware costs •Greater risk of hardware failure causing bigger outages •Generally severe vendor lock-in and limited upgradeability in the future

11. Horizontal Scaling (Scale-out): 11 • Generally refers to adding more servers with less processors and RAM. This is usually cheaper overall and can possibly scale inﬁnitely. • Pros •Much cheaper than scaling vertically •Easier to run fault-tolerance •Easy to upgrade • Cons •More licensing fees •Bigger footprint in the Data Center •Higher utility cost (Electricity and cooling) •Possible need for more networking equipment (switches/routers)

12. Related concepts 12 • Performance •Average Response Time (ART) •Latency (usually network and disk I/O) •Resource consumption • High Availability •Single Point of Failure (SPOF) •Fault tolerance

13. USE CASE Weather Forecast Designing the software

14. 14 • Mobile App to show world- wide weather forecasts • You planned to reach 1k users at ﬁrst week. • Important notes: •How many users will access the system at same time? •How to model workload? •How to predict the future? Weather Forecast App

15. 15 • Use performance tests based on scenarios. •Test types: • Performance •Validates the ART on a scenario (e.g.: systems response takes 3secs with 100k users) • Load • Aims to understand required resources for running the system (planning infrastructure). • Stress •Look for system failures under stress conditions (DoS, outages, etc.) Notes

16. 16Solution #1: Two Tier Architecture (O ) Client Server Internet 4Gb Mem 1 vCPU* * This virtual CPU unit represents the processing power of a CPU with 2 cores

17. 17Solution #1: Two Tier Architecture • Pros • Easy setup • Cons •Single point of failure (1 SPOF) • Vertical scaling costs (buy better hardware, CPU, etc.)

18. 18 • Your app is growing fast and now it reached 10k users after one month. Weather Forecast App

19. 19Solution #2: Three Tier Architecture O ) Client Web Server Data Server (  4Gb Mem 1 vCPU 4Gb Mem 1 vCPU Network Latency Internet Local/Private Network

20. 20 • Pros • Doubled the computing power • Dedicated and optimized infrastructure (disk, memory, etc.) • IT operations specialists (database, server management). • Cons • More points of failure on server side (2 SPOFs) •Vertical scaling costs • More network latency Solution #2: Three Tier Architecture

21. 21 • Forecast app is being used in most important country cities and it has at about 500k users after two months. •Some users are noticing failures and outages while trying to get the latest weather information. Weather Forecast App

22. 22Solution #3: Clustering O )Client Web Cluster Data Server (  4Gb Mem 1 vCPU 4Gb Mem 1 vCPU Internet Local/Private Network ( Load Balancer +

23. 23 • Pros •High availability and fault tolerance •Grid computing •Out-of-box setup scaling for stateless applications • Cons •IT management more complex, e.g.: Deployment and rollback procedures. •Need to use a data sharing service for stateful applications or use sticky sessions which may not scale properly •If Data Server is the bottleneck, you may also need a cluster for it. •More network latency Solution #3: Clustering

24. 24Solution #3: Stateful vs Stateless O ) User2 Web Cluster Data Server ( Internet Local/Private Network ( Load Balancer A B OUser1 Session 1 Session 2 Web session stickiness example +

25. 25 • After a global market campaign, some users started to complain about a slow UI loading in mobile application, when opening the latest weather information from the internet. Weather Forecast App

26. 26 • Example of browser caching Solution #4: Caching [1] Reference: https://developers.google.com/web/fundamentals/performance/optimizing-content-efﬁciency/http-caching#cache-control

27. 27Solution #4: Caching O ) User2 Web Cluster Data Server (  Internet + ( Load Balancer OUser1 ( Cache Server In Memory Caching Local Caching Local Caching Client Caching Client Caching Proxy Caching In Memory Caching Example: CDN for Static Web Content: html, css, images, etc.

28. 28Solution #4: Caching Content Delivery Network(CDN)

29. 29 • Pros •Reduce wait time caused by latency problems •Avoid waste of computing resources by caching long- running operations like remote calls, disk I/O, etc. •Cons • Need to deﬁne metadata for control the cache storage. •May require to minimal hardware for getting better results (SSD, RAM, etc.), some devices such as mobile have critical limitations. • You may see out-of-date results while the old cache was not invalidated yet. Solution #4: Caching

30. 30 • Your app is growing too fast, it is known throughout the world with more than 1 million users in two month only. •It is time to have your product running in a cloud solution. Weather Forecast App

31. 31Solution #5: Full Distributed Cloud Service

32. Comments 32 ,

Designing Scalable Applications

You are reading a preview.

Check these out next

Designing Scalable Applications

More Related Content

Slideshows for you (20)

Viewers also liked (20)

Similar to Designing Scalable Applications (20)

Recently uploaded (20)