If you’re building a SaaS application, you probably already have the notion of tenancy built in your data model. Typically, most information relates to tenants / customers / accounts and your database tables capture this natural relation. With smaller amounts of data, it’s easy to throw more hardware at the problem and scale up your database. As these tables grow however, you need to think about ways to scale your multi-tenant (B2B) database across dozens or hundreds of machines. In this talk, we're first going to talk about motivations behind scaling your SaaS (multi-tenant) database and several heuristics we found helpful on deciding when to scale. We'll then describe three design patterns that are common in scaling SaaS databases: (1) Create one database per tenant, (2) Create one schema per tenant, and (3) Have all tenants share the same table(s). Next, we'll highlight the tradeoffs involved with each design pattern and focus on one pattern that scales to hundreds of thousands of tenants. We'll also share an example architecture from the industry that describes this pattern in more detail. Last, we'll talk about key PostgreSQL properties, such as semi-structured data types, that make building multi-tenant applications easy. We'll also mention Citus as a method to scale out your multi-tenant database. We'll conclude by answering frequently asked questions on multi-tenant databases and Q&A.

1. Designing
your
SaaS
Database
for
Scale
with
Postgres
Lukas
Fi9l
Ozgun
Erdogan

2. What
is
Citus?
• Citus
extends
PostgreSQL
(not
a
fork)
to
provide
it
with
distributed
funcGonality.
• Citus
scales-­‐out
Postgres
across
servers
using
sharding
and
replicaGon.
Its
query
engine
parallelizes
SQL
queries
across
many
servers.
• Citus
is
open
source:
h9ps://github.com/citusdata/citus

3. When
is
Citus
a
good
fit?
Three
common
use-­‐cases:
1. MulG-­‐tenant
database:
Citus
allows
you
to
scale
out
your
mulG-­‐tenant
(B2B)
database
to
100K+
tenants.
2. Real-­‐Gme
analyGcs:
Citus
enables
ingesGng
large
volumes
of
data
and
running
analyGcal
queries
on
that
in
human
real-­‐Gme.
3. NoSQL++:
If
you
have
high
ingest
requirements
of
500K/sec,
Citus
can
help
you
combine
the
power
of
structured
and
semi-­‐structured
data.

4. Talk
Outline
1. Scaling
Databases
2. MulG-­‐tenant
(SaaS)
databases
3. Data
Modeling
4. How
to
Scale
MulG-­‐tenant
databases
5. ApplicaGon
IntegraGon
6. Demo
7. Q
&
A

5. What
does
it
mean
to
“scale”?
• Scaling:
AllocaGng
more
resources
to
your
applicaGon
or
database
to
improve
performance.
• Scaling
databases
is
harder
than
scaling
apps.
• Types
of
resources
you
can
scale:
1. Soeware
resources:
ConnecGons,
number
of
processes
2. Hardware
resources:
CPU,
memory,
and
storage

6. Scaling
Up
Hardware
PostgreSQL
Database
r3.xlarge
PostgreSQL
Database
r3.4xlarge
4
vCPUs
30
GB
RAM
80
GB
SSD
16
vCPUs
120
GB
RAM
320
GB
SSD

7. Scaling
Out
Hardware
Node
#1
PostgreSQL
ApplicaGon
(Ruby,
Pyhton,
Java,
…)
Node
#8
PostgreSQL
…….
Node
#7
PostgreSQL

8. When
is
the
right
Gme
to
scale
out
• Scaling
up
is
easier
than
scaling
out.
If
you
can
throw
more
hardware
at
the
problem,
that’s
the
easiest
way
to
scale.
• Also
tune
your
database:
h9p://pgconfsv.com/postgresql-­‐when-­‐its-­‐not-­‐
your-­‐day-­‐job
• When
is
the
right
Gme
to
start
thinking
about
scaling
out?

9. HeurisGc
#1
on
when
to
scale
• Your
SaaS
business
is
growing,
you’re
on
the
second
largest
instance
type
available
on
your
cloud
/
infrastructure
provider
• Example
Gpping
points
– We
signed
a
big
customer,
and
now
all
our
customers
are
hurGng
– One-­‐off
operaGonal
queries
are
bringing
the
database
to
a
halt
– We
expect
to
grow
by
10x
next
year

10. HeurisGc
#2
• Even
aeer
tuning,
PostgreSQL’s
autovacuum
daemon
can’t
catch
up
with
our
write
traffic

11. HeurisGc
#3
• Databases
will
cache
recent
and
frequently
accessed
data
in
memory
for
you
• The
database
will
track
how
oeen
you
use
the
cache
and
hit
disk
• For
OLTP
applicaGons,
most
of
your
working
set
should
be
fulfilled
from
the
cache
– Look
to
serve
99%
from
the
cache

12. HeurisGc
#3
–
cache
hit
raGo
query
Source:
Heroku
-­‐-­‐
Determining
Cache
Size

13. Plan
ahead
• Plan
ahead,
opGmize
queries,
and
don’t
wait
unGl
there
isn’t
another
opGon
• When
it’s
Gme
to
scale
out,
you
need
to
be9er
understand
your
workload.
1. B2B
(mulG-­‐tenant
databases)
or
B2C
applicaGons
2. TransacGonal
(OLTP)
or
analyGcal
(OLAP)

14. What
is
a
mulG-­‐tenant
database
• If
you’re
building
a
B2B
applicaGon,
you
already
have
the
noGon
of
tenancy
built
into
your
data
model
• B2B
applicaGons
that
serve
other
tenants
/
accounts
/
organizaGons
use
mulG-­‐tenant
dbs
– Physical
service
providers.
For
example,
food
services
to
other
businesses
– Digital
service
providers:
AdverGsing,
markeGng,
and
sales
automaGon

15. Trends
in
scaling
mulG-­‐tenant
apps
• MulG-­‐tenant
databases
were
commonplace
in
on-­‐premises
• SaaS
applicaGons
introduced
the
moGvaGon
to
scale
further
– Cloud
enables
serving
many
smaller
tenants
– Instead
of
dozens
of
tenants,
new
SaaS
apps
reach
to
and
handle
1K-­‐100K
tenants
– Storage
is
cheap:
You
can
store
events
or
track
a
field’s
history

16. Google
F1
–
An
Example
• Google
F1
is
an
example
that
demonstrates
a
mulG-­‐tenant
database.
• AdWords
serves
more
than
1M
tenants.
• F1
leverages
key
relaGonal
database
features:
– TransacGons
– Joins
–
avoid
data
duplicaGon
– Primary
and
foreign
key
constraints

17. Data
modeling
for
mulG-­‐tenant

18. Key
Insight
• If
you
shard
your
tables
on
their
primary
key
(in
the
relaGonal
model),
then
distributed
transacGons,
joins,
and
foreign
key
constraints
become
expensive.
• Model
your
tables
using
the
hierarchical
database
model
by
adding
tenant_id.
This
colocates
data
for
the
same
tenant
together
and
dramaGcally
reduces
cost.

19. Concept
of
co-­‐locaGon

20. Does
everything
fit
into
hierarchical?
• What
happens
if
I
have
a
table
that
doesn’t
fit
into
the
hierarchical
database
model?
1. Large
table
outside
the
hierarchy:
Orgs
and
users
that
are
shared
across
orgs
– Shard
on
different
column
and
don’t
join
2. Small
table
that
is
common
to
hierarchy
– Create
reference
table
replicated
across
all
nodes

21. Scaling
MulG-­‐tenant
Databases
• How
to
do
you
scale
your
mulG-­‐tenant
database?
• Three
high
level
opGons:
1. Create
one
database
per
tenant
2. Create
one
schema
per
tenant
3. Have
all
tenants
share
the
same
tables
(and
parGGon
/
shard
tables)

22. Create
one
database
per
tenant
Tenant
5
Tenant
1251
Tenant
1252

23. Create
one
database
per
tenant
• Create
a
separate
database
for
each
tenant
• IsolaGon
of
tenants
and
more
predictable
compliance
story
• DBA
responsible
for
managing
separate
databases
and
resource
allocaGon
between
them

24. Create
one
schema
per
tenant
Database
Tenant
5
Tenant
1251
Tenant
1252

25. Create
one
schema
per
tenant
• Create
a
separate
namespace
(schema)
for
each
tenant
• Isolate
data
/
queries
for
one
tenant
in
a
schema.
Make
be9er
use
of
resources
than
the
“one
database
per
tenant”
model

26. Have
all
tenants
share
the
same
tables
Accounts
Campaigns
Leads
TenantId
5
5
5
1251
Database

27. Have
all
tenants
share
the
same
tables
• Have
all
tenants
share
the
same
tables
by
adding
a
tenant_id
column
(and
shard)
• Requires
the
applicaGon
to
control
access
to
database,
or
row
based
access
controls
• Scales
to
1K-­‐100K
tenants
through
be9er
resource
sharing
and
simplifies
operaGons
and
maintenance

28. Rule
of
thumb
(simplified)
• Each
design
opGon
can
address
quesGons
around
scale
and
isolaGon
with
enough
effort.
What’s
the
primary
criteria
for
you
app?
• If
you’re
building
for
scale:
Have
all
tenants
share
the
same
table(s)
• If
you’re
building
for
isolaGon:
Create
one
database
per
tenant

29. Scaling:
Resources
• If
you
create
a
separate
database
/
schema
for
each
tenant,
you
need
to
allocate
resources
to
that
database.
• Hardware:
disk,
memory,
cpu,
and
network
management
• Database
soeware:
shared_buffers,
connecGon
counts,
backend
processes
• ORM
soeware:
Cached
informaGon
about
databases
/
schemas

30. Scaling:
OperaGonal
Simplicity
• Your
database
grows
with
your
SaaS
applicaGon.
• Schema
changes
(Alter
Table
…
Add
Column)
and
index
creaGons
(Create
Index)
are
common
operaGons.
• What
happens
when
you
have
10K
tenants
and
you
changed
the
schema
for
5,000
of
those
tenants
and
observed
a
failure?

31. FAQ:
How
does
largest
tenant
impact
scale?
• MulG-­‐tenant
databases
usually
follow
a
Zipf
/
Power
law
distribuGon

32. FAQ:
How
does
largest
tenant
impact
scale?
• What
percentage
of
the
total
data
size
belongs
to
the
largest
tenant?
• Guidelines
around
a
standard
Zipf
distribuGon
and
different
tenant
counts:
– 10
tenants:
Largest
tenant
holds
60%
of
data
(*)
– 10K
tenants:
Largest
tenant
holds
2%
of
data
(*)
• Look
at
your
data’s
distribuGon
to
make
informed
scaling
decisions

33. FAQ:
Data
that
varies
across
tenants
• What
about
data
that
varies
across
tenants?
• Different
tenants
/
organizaGons
may
have
their
own
needs
that
a
rigid
data
model
won’t
be
able
to
address.
• One
organizaGon
may
need
to
track
their
stores
in
the
US
through
their
zip
codes.
Another
customer
in
Europe
may
only
want
to
keep
tax
raGos
for
each
store.

34. FAQ:
Salesforce
Architecture
• If
your
tenants
share
the
same
table(s),
one
approach
is
creaGng
a
huge
table
with
many
string
columns
(Value0,
Value1,
…,
Value500).
(*)
Salesforce’s
mulG-­‐tenant
arch:
www.developerforce.com/media/
ForcedotcomBookLibrary/Force.com_MulGtenancy_WP_101508.pdf

35. FAQ:
Semi-­‐structured
data
types
• PostgreSQL
has
powerful
semi-­‐structured
data
types:
hstore,
json,
and
jsonb.
These
data
types
can
express
scalar,
array,
and
nested
fields.

36. ApplicaGon
IntegraGon
• Citus
follows
the
“shared
tables”
approach
that
easily
scales
to
100K+
tenants
with
minimal
applicaGon
changes.
• For
example
applicaGon
integraGon:
github.com/citusdata/acGverecord-­‐mulG-­‐
tenant

37. MulG-­‐tenant
Database
Demo
• <<
MulG-­‐tenant
(B2B)
Database
Demo
>>
– Tenant
isolaGon
– Reference
tables
– Online
shard
rebalancing

38. Summary
• You
can
verGcally
or
horizontally
scale
your
database.
Several
heurisGcs
help
in
deciding
the
right
Gme
to
horizontally
scale.
• To
scale
out
a
mulG-­‐tenant
(B2B)
database,
picking
the
right
distribuGon
column
and
table
colocaGon
are
key.
• There
are
three
design
pa9erns
to
scaling
out
a
mulG-­‐tenant
database.
The
“shared
tables”
approach
offers
the
best
scaling
characterisGcs.

39. Q
&
A
QuesGons
www.citusdata.com/get_started