This document discusses various patterns for scaling web applications. It begins by describing single machine, two-tier, and multi-tier web application architectures. It then covers message bus and service-oriented architectures. The document discusses scaling up approaches like identifying bottlenecks, utilizing bottlenecks, adjusting non-bottlenecks, elevating bottlenecks, and reviewing processes. Specific scaling techniques covered include caching, threading and queues, and using content delivery networks.

2. Design patterns for
scaling web applications
SysAdmin Bulgaria meetup #01

3. Ivan Dimitrov
Manager Technical Operations
Xogito Group, Inc
ivan.dimitrov@xogito.com

4. tl;dr
Web application architecture
Life’s too short for bad software
- Lew Cirne

5. tl;dr
Scalability patterns
The only real problem is scaling.
Everything else is a sub-problem.
- O’Dell’s Axiom

6. Let’s start with hardware

7. Web application architecture
● Single machine web server
● Two-tier web application
● Multi-Tier application
● Message Bus architecture
● Service-Oriented architecture

8. Single machine web server

9. Single machine web server
● All in one server
o Real IP
o Web server
o Application server
o Database
o Storage
o Compute
● All in one server
o Security problem
o Shared CPU
o Shared memory
o SPOF
o Maintenance =
Downtime

10. Single machine web server

11. Two-tier web application

12. Two-tier web application
● Dedicated
resources
● Multiple POF
● Internal network
● Maintenance =
downtime
● Different hardware
generations
● Expensive

13. Multi Tier web application

14. Multi-tier web application
● Separate responsibility to teams
● Most of security at first tier
● Independant scaling
● Front-end can talk with APIs internaly
● Every tier can have it’s own hardware
requirements
● Global Load Balancer

15. Global Load Balancer

16. Message Bus application

17. Message Bus application
● Many to Many communication between
servers
● Replaces constantly polling the database
● Asyncronious, not sorted
● Loosly coupled systems
● Support different hardware

18. Service Oriented Architecture

20. Service Oriented Architecture
● Loosly coupled systems
● Service oriented company - small teams
● Can provide services to third parties
● Can subscribe to services from third parties

21. Scaling Up
● Bottleneck
● Measure Results
● Horizontal
duplication
● Functional/Service
split
● Lookup-oriented
split
● Caching
● Data sharding
● Threading
● Queueing
● Content Delivery
Networks

22. Theory of constraints
● Identify the Bottleneck
● Utilize the Bottleneck
● Adjust Non-Bottlenecks
● Elevate the Bottleneck
● Review the Process

23. Finding the bottleneck
● Accumulation
● Throughput
● Full Capacity
● Wait Times

24. Measure results

26. Caching
cache hit ratio = cache lookups / cache hits
● Store most recent data
● Store most used data
● Store only repeatedly used data
Cache invalidation - TTL

28. Threads and Queues
● Single thread process vs Multi thread
process
● Queues FIFO, LIFO

29. Content delivery network

30. Additional reading