Scalability and Performance

Copyright © 2014 HashedIn Technologies Pvt Ltd. All rights reserved. – CONFIDENTIAL
Scalability &
Performance
Principles & Techniques

What is a Performance Problem?
System is Slow
for a Single User

What is a Scalability Problem?
Fast for Single User
but
Slow under Heavy
Load

How do you measure Performance?
Response Time for 1 User
i.e. how long the user waits

Number of Users that
can work simultaneously
with acceptable performance
How do you measure Scalability?

95% of
time is
spent in
fronten

But, we won’t talk about
frontend performance
improvements today.

Let’s
talk
about
backend

Relationship between Performance & Scalability

Performance & Scalability Mantra
Strive for maximum throughput with
acceptable response times

Copyright © 2014-2017 HashedIn Technologies Pvt Ltd. All rights reserved. – CONFIDENTIAL
To Improve Scalability...
Improve
Performance
Add
Capacity
OR

Response Time : 1s
Workers : 4
Machines : 1
Poor Performance & Scalability
4 requests/second
Slowest response : 1s

Better Performance & Scalability
Response Time :
500ms
Workers : 4
Machines : 1
8 requests/second

Even Better Performance ...
Response Time :
100ms
Workers : 4
Machines : 1
40 requests/second

Great Performance & Scalability
Response Time : 10ms
Workers : 4
Machines : 1
400 requests/second

You cannot make response time 0ms
To support more than
400 requests….
Increase
Number of
Workers

With double the workers...
Workers : 8
Machines : 2
800 requests/second

and with even more workers...
Workers : 16
Machines : 4
1600 requests/second

Coming Back To Reality
Increasing
Capacity Should
Increase
Throughput
It Does
Not.
Not Until You
Design Your
Application
Correctly.

You have to
increase capacity at
each layer.
Web & App Server
Cache Server, Database
CPU, Network, Disk
Why?
And there are
locks.
Database Locks
Synchronized Code Blocks.
Mutexes
File System Locks

What should our goal be?
1. Reduce Response Times
2. Make it possible to add
more Capacity

Scaling Data Storage

Browser
Edge
Server
Load
Balancer
Web
Server
App Server REST API DatabaseUser
Browser
Edge
Server
Load
Balancer
Web
Server
App Server REST API DatabaseUser
Browser
Cache
CDN
Web
Accelerator
Nginx /
Apache
Object
Cache
Akamai
AWS Cloudfront
VarnishAll Browsers
All Mobiles
mod_proxy Static Variables
Redis, Memcached
EHCache
Less Granularity, More Effective More Granularity, Less Effective

HTTP Caching Strategy
Cache Forever
Cache-Control : maxage
Expires :
Do Not Cache
Cache-Control : maxage=-1
Expires : 1970-01-01
Cache Temporarily
Cache-Control : maxage=3600
Expires : <now plus 1 hour>
➔ Use for HTML pages with dynamic content
➔ Avoid for static resources
➔ Use for high traffic public html pages - i.e. homepage
➔ Specify etag or expires header to use conditional GET
➔ Use javascript to load user specific data
➔ Avoid for static resources
➔ Use for static resources - css, images, js
➔ Change URL in HTML when resource is modified
➔ Use a pre-processor to simplify management

Cache Temporarily
Render this using JS
after page load
Cache for 2h.
Don’t overdo. You
cannot change the URL
of your homepage if the
content has to change.

Cache Static Files Permanently
Static files are loaded
from browser cache
Cache Forever
If base.css changes, serve it
from base.css?v=2
New URL. Fresh Download.

Cache-Control: private
Logged in user-
specific pages or APIs.
Only Cached By
Browser
CDN, Web Accelerator, Proxies
and Web Servers will not cache.

Ask These Questions
1.How often does the data change?
2.Can you tolerate stale data? For
how long?
3.How critical is the data? Can you
lose some of it?

Caching Strategy : Split Objects
Split objects based on
frequency of change.
Freshers create their profile once,
but apply to jobs very often.
Fresher
Personal Details
Education & Work Ex
Job Application Status
Fresher Profile
Personal Details
Education & Work Ex
Job Applications
Job Application Status
Create Two Objects With Different TTLs

Cache In-Memory
Put them in
settings file
➔Frequently Accessed Data
➔Infrequently Changing Data
➔Configuration & Settings
Make It Easy To Deploy Just The Settings

Manually Clear Cache
Cache Forever
Delete as Needed
➔Dynamic Settings
➔Throttles & Blacklists

Delete on Modification, Rebuild on Read
When data changes, delete it from the cache.
Next read will automatically fill up the cache.

DB Design For Scalability

First, Know your Data
1. Sizing
How many records in 6 months / 1 year?
2. Query Volumes
How many reads / writes?
3. Hot Tables
Most frequently accessed tables?
4. Criticality of Data
How important is it to not lose data?
5. Availability v/s Consistency
How important is it to not lose data?

What is Availability?
Ensuring your system can be
used anytime

What is Consistency?
Data is in same state
across all the copies

You can’t get both...
You can choose only one
Consistency or Availability

Two Salesmen Selling Apartments
➔ Each has a diary of sold flats
➔ Call and confirm before selling
What happens when
one salesman is offline,
and a customer calls?
1. He takes the order. But there is
a chance the other salesman
also sold the same flat…
Not Consistent
1. He does not take the order…
Not Available.

Different Data, Different Guarantees
Sales Transactions must be Consistent
Product Catalog must be Available

Start with a Normalized Schema...
…which essentially means no
redundant data.

Optimize Heavy Read Operations
Selectively de-
normalize to
eliminate joins.
1. Counts of objects
2. Summary Statistics
3. Events / Activities

Denormalize - Number of people watching this Issue

Not everything needs to be Accurate
Choose
wisely
between
Accuracy
and
Performance

Create a Separate Reporting Schema
1. Use a Star Schema
2. Aggregate Data
a. by time
hour, day, week, month, quarter, year
b. by region
north, south, east, west, central

Use a Search Engine
1. Relational DB as Source of Truth
2. SOLR or ElasticSearch as Index
3. Cron Job to update Search Engine

Things to Avoid in a RDBMS
1. Don’t store files in DB
2. Don’t create task queues
3. Don’t maintain counters

Use Read Replicas
Use Master-Slave Replication,
and use Slaves for Reads.
Only use for non-transactional reads.

Shard your Data
1. Choose shard key wisely
Location is usually a poor choice
2. Sharding later is painful
If you think you may need it, shard
upfront.

Async & Availability - Best Friends
Pre-computation
Slow Jobs
Offload Work to
a Job Queue

Database Optimization

Quick Recap
For New Systems - See
Database Design
For Existing Systems - See
Database Optimization

Asking Deepak for a Notepad
How does he get
you a notepad?
The stationery shop is the
database
1. Waits for the elevator
2. Walks down the street
3. Waits for the pedestrian traffic light
4. Reaches Store
5. Waits for the previous customer
6. Requests for a Notepad
7. Waits for the attendant to search
8. Bonus : Attendant misplaces
notepads

Three General Techniques
1. Minimize Queries
2. Do More Work in One Trip
3. Make the Query Efficient

The Fastest Query...
is Never Executed
Cache Aggressively to Minimize Queries
Don’t use ORM for Reports

DB Optimization : More in One Query

N + 1 Problem
doc_ids = db.query(“select id from documents where user = ?“, user)
docs = []
for docid in doc_ids:
doc = db.query(“select … from documents where id = ?”, docid)
docs.append(doc)
Query in a Loop = Disaster

Homogeneous Queries - Union All
Tables
monthly_income
monthly_expenditure
Select ‘income’ as heading, month, income from monthly_income
UNION ALL
Select ‘expenditure’ as heading, month, expenses from monthly_expenditure

Anti-Pattern : Fetch and Update

Bulk Operations & Batch Inserts

DB Optimization : Efficient Queries

Find matching lines from a book

Finding a word in a novel
Full Table Scan

Finding a Topic in a Tech Book
Index Seek

Lookup Meaning of a Word
Find by Primary Key
Clustered Index
Data is Sorted

Find Words like ab* and ac*
Clustered Indexes are
Great for
Range Queries

Query Planner Algorithm
For each table in a query :
Find constraining columns (where, join)
For each Index on the table :
Find if the index can be used
If multiple indexes :
Find Best Index

Find if Index can be used
1. Index column must be in where clause
2. For multi-column indexes, the starting columns must
be in where clause

Best Index?
Index Cardinality
Table Statistics

When is an Index NOT Used?

Slow Queries / Profiler
MySQL Slow Queries Log
MS SQL Profiler

Anti Pattern : in clause with subquery
Select …. from table1 where id in
(select id from table2 where…)

Database Locks & Isolation Levels

What is a Lock?
Mechanism to prevent data corruption
when multiple people access the
database concurrently.

Scalability and Performance

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (14)

Similar to Scalability and Performance

Similar to Scalability and Performance (20)

Recently uploaded

Recently uploaded (20)

Scalability and Performance