While the physical replication in PostgreSQL is quite robust, however, it doesn’t fit well in the picture when: - You need partial replication only - You want to replicate between different major versions of PostgreSQL - You need to replicate multiple databases to the same target - Transformation of the data is needed - You want to replicate in order to upgrade without downtime The answer to these use cases is logical replication This talk will discuss and cover these use cases followed by a logical replication demo.

1. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Muhammad Haroon
Replication goes logical

2. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
$psql~#: d haroon
● Working in PostgreSQL space @ 2ndQuadrant
● Part of PostgreSQL family for nearly a decade and a half
● Work in almost all areas around PostgreSQL from development to professional services
● Past stints with PostgreSQL family include
○ EnterpriseDB
○ OpenSCG
● Managed Engineering & IT efforts @ IBEX Group
● Served as Principal Architect/Product Owner @ TRG
Email: muhammad.haroon@2ndQuadrant.com

3. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Disclaimer:
- All images and content used in the slide deck is in line with the
copyrights of the respective content.
- Neither sponsors nor anyone else is responsible for anything listed
in the presentation.

4. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
- Because data is critical ?
- It can cost businesses THE BUSINESS
- Heavy duty reporting, business critical decision making
- Disaster recovery
- Etc etc
Why do we need to replicate data ??

5. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Replication in PostgreSQL until 9.6
- Physical/Streaming replication
- Built in replication system
- First introduced in 9.0
- A lot of improvements over the years
- Third part replication solutions
- Slony
- Londiste
- .
- .
- … so on
- NOT GOING TO DISCUSS THESE

6. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Replication replication over the years
- 7.x
- Introduced WAL
- 8.x
- Point-In-Time-Recovery
- pg standby i.e. pg_standby for warm standby database server (8.2)
- 9.x
- Streaming replication & hot_standby
- Synchronous replication and pg_basebackup (9.1)
- Cascading replication (9.2)
- Standbys now get to switch timeline to follow change in primary (9.3)
- Logical decoding a.k.a changeset extraction (9.4)
- replication slots (9.4)
- pg_rewind (9.5)

7. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
App server
Primary/Master Standby1
Standby2
Cascading Standby1
Reporting server
Physical/Streaming Replication

8. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018

9. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Enter Logical Replication ...
Over to green screen

10. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Potential uses
- Major version upgrades

11. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Potential uses (Cont.)
- Aggregation

12. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Potential uses (Cont.)
- Scaling out

13. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Potential uses
- Partial replication
- Anything other scenario where physical/streaming replication might not be
an option

14. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Moving parts - Publications
CREATE
- CREATE PUBLICATION mypublication FOR TABLE users, departments;
- CREATE PUBLICATION alltables FOR ALL TABLES;
- CREATE PUBLICATION insert_update FOR TABLE mydata
WITH (publish = 'insert,update');

15. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Moving parts - Publications
ALTER
- ALTER PUBLICATION noinsert SET (publish = 'update, delete');
- ALTER PUBLICATION mypublication ADD TABLE users, departments;
- ALTER PUBLICATION mypublication DROP TABLE users;

16. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Moving parts - Publications
DROP
- DROP PUBLICATION noinsert;
- DROP PUBLICATION mypublication, noinsert;

17. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Moving parts - Subscription
CREATE
- CREATE SUBSCRIPTION mysub
CONNECTION 'host=host port=5432 user=test
dbname=databasename'
PUBLICATION mypublication, insert_only;
- CREATE SUBSCRIPTION mysub
CONNECTION 'host=host port=5432 user=test
dbname=databasename'
PUBLICATION insert_only
WITH (enabled = false
create_slot = false
slot_name = ‘slotname’, ...);

18. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Moving parts - Subscriptions
ALTER
- ALTER SUBSCRIPTION mysub ENABLE;
- ALTER SUBSCRIPTION mysub DISABLE;
- ALTER SUBSCRIPTION mysub SET PUBLICATION insert_only;
- ALTER SUBSCRIPTION mysub
CONNECTION '...';
- ALTER SUBSCRIPTION mysub REFRESH PUBLICATION;

19. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Moving parts - Subscriptions
DROP
- DROP SUBSCRIPTION mysub;

20. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Some tidbits
- Tables and columns on both ends are mapped by name
- You can change table definition e.g. add more columns to the receiving
end
- Data gets transferred in text format
- It is possible to have different column order

21. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Publisher: Configuration of interest
- wal_level = logical
- max_wal_senders
- max_replication_slots
Remote subscriber should be able to connect to the IP address/database
using the specified configuration in pg_hba.conf

22. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Subscriber: Configuration parameters of interest
- max_replication_slots
- max_logical_replication_workers
- max_worker_processes
- max_sync_workers_per_subscription

23. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Monitoring interfaces
- pg_stat_replication
- pg_replication_slots
- pg_stat_subscription
- pg_stat_activity

24. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Who can setup logical replication
- Publisher:
- CREATE privilege on database
- Table owner
- Subscriber:
- Super user for creating subscription
- Remote replication user for replication
- Pg_hba.conf entry
- host dbname replication_user IP/netmask authentication-method

25. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Logical Replication in PG10 vs pglogical ?
They are not the same

26. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Areas of Improvement
- Schema not replicated
- DDL not replicated
- SEQUENCES are not replicated
- TRUNCATE not replicated
- Only base table to base table replication supported
- Conflict management and monitoring
- Failover ?

27. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Further readings
- https://www.postgresql.org/docs/10/static/logical-replication.html
- https://www.2ndquadrant.com/en/resources/pglogical/
-

28. https://www.2ndQuadrant.com
PGConf APAC
Singapore, March 22, 2018
Thank you!!