How to isolate business domain in DB with minimum costs.

1. Business domain isolation
in DB (PostgreSQL)
Andrey Koleshko
@ka8725

2. When does the problem arise?
● Monolithic application break down to micro-services
● Logical isolation parts of a monolithic application, i.e. split
to modules (e.g. Rails engines)
Note, we will talk about solution for PostgreSQL

3. Initial state of the system
● Monolithic application
● Production setup
● One DB (PostgreSQL)
● Additional application (e.g. analytics) that only reads data
from the DB
○ This application relies on the current tables structure
○ It relies on some static data, such as enum values

4. Initial state of the system
● The consumer app relies
on the current tables
structure
● It relies on some static
data, such as enum
values

5. ● Isolate a specific business domain in the DB
● Zero-downtime migration
● As less changes as possible to reduce costs and risks
The goal

6. ● Set of tables that are “hidden” from the main app
Example:
○ Initially DB has tables: users and locations
○ Assume locations is a table from a business domain
○ After:
■ users remains in the main DB
■ locations is “hidden” from users
What is isolated business domain in DB?

7. The business
domain is a set of
tables that should be
isolated, i.e. “hidden”
from the main DB
The target

8. Natural solution
Setup a separate DB
and create the
domain tables there,
point all apps to use
this new DB
additionally to the
main DB

9. Pros
● Full isolation
○ while the business domain
DB fails, the main DB
remains functioning
● Separate configuration
● Separate data
Natural solution: pros and cons
Cons
● Implementation cost
● Maintenance cost
● Complex solution

10. Natural solution: cons - complex solution

12. Idea
Isolate the business
domain tables
logically inside DB

13. ● Use schemas
○ https://www.postgresql.org/docs/12/ddl-schemas.html
○ many users use one DB without interfering each other
○ DB objects are organized into logical groups to make
them more manageable
○ third-party applications can be put into separate
schemas not to collide
How to implement this in PostgreSQL?

14. “Schemas are analogous to directories at the operating
system level, except that schemas cannot be nested.”
Metaphor

15. Result
● All tables remain in the same DB
● Domains are isolated:
○ public.users
○ business.locations

16. ● Remove foreign keys between the tables
● Create the business schema with necessary tables,
indexes, constraints, foreign keys, etc. inside this schema
● Start writing data into these new tables
● But don’t stop writing into the old tables yet
● Switch all apps to read from the new tables
● Drop the old tables along with all related DB objects
Zero-downtime approach

17. ● Remove foreign keys between the tables
● Create the business schema with necessary tables,
indexes, constraints, foreign keys, etc. inside this schema
● Start writing data into these new tables
● But don’t stop writing into the old tables yet
● Switch all apps to read from the new tables
● Drop the old tables along with all related DB objects
Zero-downtime approach

18. ● Use triggers
○ https://www.postgresql.org/docs/12/trigger-definition.html
○ It’s like callbacks in ActiveRecord, but on DB level
Start writing data into the new tables

19. Switch to read from the new tables: step 1
Define an
interface
(contract/API)
takes data from
the old tables
and is used by
readers

20. Write to the
new and old
tables in
parallel
Switch to read from the new tables: step 2

21. Change the
API to read
from the the
new tables
and drop the
old tables
Switch all apps to read from the new tables: step 3
Note, the new
tables are
namespaced
in a view

22. ● Use views
○ http://www.postgresqltutorial.com/postgresql-views/
○ Consider view as a virtual table reads data from one or
many other tables
○ If the tables used by views change, views fail
automatically - API tests out of box!
○ Enum values should be specified explicitly to avoid
broken API readers
○ Namespace the views in a specific schema
How to implement the API?

23. create schema api;
create view api.locations as
select
id, lat, lng, status
from public.locations
where public.locations.status =
any(array['completed', 'pending']);
View example

24. View example caveats
● Expose only data with enums that are known by the Consumer app
(array['completed', 'pending'])
○ If new enum value appears, the Consumer app will have to prepare code
for this and then the row with the new enum value can be exposed
○ This way it guarantees the Consumer app is not broken due to the new
enum value

25. ● Rails was not prepared for that plan
● We’ve patched it:
https://gist.github.com/ka8725/a45214a0e9d93fb7b7072206d7f2d0b4
● Usage example:
dumper = SchemaDumper.new(ActiveRecord::Base.connection)
dumper.dump('api', Rails.root.join('db/api/schema.rb'))
● Created a rake task dumps these custom schemas and put it as a prerequisite
for the standard db:schema:dump
Generate separate schemas for Rails

26. ● Easy to implement and maintain solution
● Rails is not friendly for non-trivial approaches and needs patches
● The final solution works for:
○ monolithic modular applications
○ micro-services
● The API idea is applicable for any app exposes data to internal services
● Follow your thoughts and learn using tools
Summary

27. Andrey Koleshko
@ka8725
ka8725@gmail.com
https://railsguides.net/
Questions