A quick reference guide for system design/architecture and scalability

1. System Design & Scalability
A quick reference guide
By John DiFini

2. Overview
• Types of Scaling & Load Balancing
• Data Storage Design
• Message-oriented Middleware (MOM)
• Fault Handling
• Networking
• MapReduce

3. Types of Scaling & Load Balancing

4. Horizontal vs. Vertical Scaling
Type Scaling
Process
Example Complexity Limited
Vertical Add more
resources to
a single node
Add more
memory to a
single server
Easy Yes (e.g. can
only add so
much memory)
Horizontal Add more
nodes
Add more
servers
Hard Practically
unlimited

5. Load Balancing
• Round Robin
• Source IP Hash - a given client IP address will always go
to the same server
• Request Hash - a given request type will always go to the
same server/cache; avoids cache duplication
• Least Connections
• Least Traffic
• Least Latency
Reference

6. Data Storage Design

7. Database - Read vs. Write Performance
• Normalize vs. Denormalize
 Normalize - ↓duplicate data ⇨ ↑write perf but ↓read perf
 Denormalize - ↑duplicate data ⇨ ↑read perf but ↓write perf
• Have your cake and… - Use an append-only structure for
writes; then asynchronously restructure data into a read-
optimized format[*]

8. Database - Structure
• Relational - general purpose for tabular/table-
based data
• Specialized - for data structures that don't easily fit
the tabular format (e.g. multi-level nesting &
hierarchies)
 NoSQL
 Others

9. Not to be confused with...
Cache
• DB reads are expensive; i.e. hold as much
of it in memory as possible
• Cache Hit - data were found in cache;
Cache Miss - data not found, so retrieve
it from DB[*]
• Local vs. Distributed Rule of Thumb - use local cache for small
data sets, with predictable number of immutable records[*]
• Cache Warming - anticipate queries and "prime" the cache
not only on startup but also in real-time (e.g. load surrounding
tiles of a recently-requested map)

10. Cache - Replacement Policy
• Replacement Policy - algorithm used to maximize
cache performance by choosing which data to
eject & which data to add in its place[*]
 LRU - ejects the most Least Recently Used data
 advanced - considers access frequency, size of items,
latency & throughput
LRUMRUCache:

11. Data Store Sharding
Sharding - partition data
across multiple nodes
Not to be confused with...
Type Scaling Process Drawback
Table-based Put Table A on Node 1, Table B on Node 2,
etc.
What if a table gets too large for its node?
Hash-based Primary key is hashed, and every node is
responsible for a range of hashed keys
What happens if the # of nodes changes? ->
need to reallocate all the data
Directory-based lookup service keeps track of which data
are stored in which shard
What if directory service is down (i.e. single point
of failure)?
What if directory service has to process to many
requests (i.e. a bottleneck)?

12. Message-oriented Middleware
(MOM)

13. MOM Considerations
• Used by distributed systems to communicate
amongst nodes[*]
• Abstracts OS & network intricacies (e.g. endian
format, sockets, etc.)

14. MOM Types
Type Use Case Examples Underlying
Protocol
Cast
Request/
Response
1 sender; 1 receiver
(point-to-point)
e.g. Stock Trade
Order[*]
Synchronous - JSON
Web Services
Asynchronous -
message queues
like ActiveMQ, IBM
MQ
TCP
("guaranteed"
delivery)
Unicast
Publish/
Subscribe
1 sender; many
receivers/listeners
e.g. Stock Tick
Kafka[*], TIBCO
Rendezvous/RV
UDP Broadcast (all
nodes) or Multicast
(node groups)

15. Fault Handling

16. Fault Handling
• High Availability (HA) - delayed recovery to
secondary
• Fault Tolerant - immediate recovery
 Active/Passive - primary fails over to secondary
 Active/Active - no primary vs. secondary; when
1 fails, the other(s) takes the additional load
What is dead
should never die
• Great YouTube video on the subject!
• @todo - explain no-special-node, ring topologies

17. Networking

18. Network Metrics
• Bandwidth - The maximum amount of data that can
be transferred in a unit of time (e.g. 100Mbps)[*]
• Throughput - The actual amount of data that is
transferred in a unit of time (e.g. 88MBps)
• Latency - The time it takes to send & receive
(round-trip) a packet of data (e.g. 20ms)[*]

19. Network Metrics - Analogy
Given a water pipe, its diameter determines its
throughput, and its length determines its latency.
Therefore, to improve:
• Throughput - Get a fatter pipe
• Latency - Colocate to reduce distance or reduce
network hops (point-to-point), which also reduces
distance that data have to travel

20. MapReduce

21. MapReduce
Uses parallel & distributed systems to process large
data sets[*]
• Implementations - Spark, Hadoop, etc.[*]
• YouTube presentation

22. MapReduce - Steps
Fundamentally, consists of two steps, Map & Reduce, but
Shuffle step is also prevalent:
• Map - Organizes/filters/sorts. Think of putting elements
into a typical Map Interface with key-value pairs (e.g. <key,
value>)
• Shuffle - Redistributes data so that all data pertaining to a
given key reside on reside on the same node
• Reduce - Summary/aggregation (e.g. sum all values for a
given key)

23. Coming Soon

24. Coming Soon
Define P9s

25. templates

26. color palette

27. section template

28. bullet template
asdf
asdf
asdf
asdf
asdf
asdf