The document discusses PostgreSQL replication and high availability (HA) methods, including types like synchronous and asynchronous replication, as well as physical and logical replication. It outlines the importance of replication in maintaining database availability and performance, detailing various architectures and tools such as Patroni and repmgr for effective management. Additionally, it covers essential terminologies and practices related to replication and HA frameworks.

1. PostgreSQL Replication/HA Methods
Aakash M
DBRE, Mydbops
January 30, 2021
Mydbops Database Meetup -8

2. Database Reliability Engineer
Interest in PostgreSQL and MySQL
Active Blogger
Aakash M
About Me

3. Database
Consulting
Services
Managed
Database
Services
Focuses on Top Opensource database MySQL,
MongoDB and PostgreSQL ON Premises and Cloud
Mydbops Services

4. End of Life on February 11, 2021
Current Major Release is 13
AWS - March 16, 2021
PostgreSQL 9.5

5. Modes of Replication
Replication
Agenda
Replication Types
How to achieve HA

6. Replication

7. Continuous copying of data changes from one database server to another.
What is Replication?
Application
Changes
Replicate
Database Servers

8. High Availabilty
Why do we need it?
Application
S
Q
L
Database Servers

9. Load Balancing
Why do we need it?
Application
S
Q
L
Database Servers
S
Q
L
50%
50%

10. Terminologies

11. Terminologies
Write Ahead Log (WAL)
Every transactions -> Transaction Log
Each statement creates a new WAL record
Each WAL record contains a bunch of records
16MB by default
pg_xlog/pg_wal
WAL Records
WAL File

12. Terminologies
Write Ahead Log (WAL)
Crash Recovery
Point-In-Time-Recovery(PITR)
Replication

13. Terminologies
Primary Server / Master
Direct Writes
Application Primary Server
Writes

14. Terminologies
Standby / Secondary Server
Following up the primary and applying the changes
Application
Primary Server
Writes
Secondary Server

15. Terminologies
Warm / Hot Standby Server
Standby
Warm Standby Hot Standby

16. Failover
Switching to the backup server(standby) when the primary server is not available
Terminologies
Application
S
Q
L
Primary Server Standby Server

17. Replication Modes

18. Replication Modes
Replication Modes
Synchronous
Replication
Asynchronous
Replication

19. Asynchronous Replication
Transaction complete - considering only the primary server
Never consider its replicas
Can be out of sync

20. Asynchronous Replication
Time
Secondary - II
Secondary - I
Primary
Replication Lag
Replication Lag
Ins/Upd/Del
X Y Z

21. Asynchronous Replication
No Additional Overhead
Chances for dataloss

22. Synchronous Replication
Transaction Complete - when changes have been replicated to the replicas.
Replicas should be available all the time
Replicas will be in sync

23. Synchronous Replication
Time
Secondary - II
Secondary - I
Primary
Ins/Upd/Del
X

24. Synchronous Replication
No data loss
Additional Overhead

25. Types of Replication

26. Types of Replication
Replication
Physical Logical
Log Shipping
Streaming
Replication
Publisher
Subscriber

27. Physical Vs Logical Replication
Physical Replication
Primary - at offset 11 of disk page 5 of relation 14312, add a row with hex value
0x234...
Secondaries - at offset 11 of disk page 5 of relation 14312, add a row with hex
value 0x234...
Disk Block Level changes made by PostgreSQL
An exact copy of the Primary
Example:

28. Physical Vs Logical Replication
Logical Replication
Primary - INSERT INTO test1 VALUES (now());
Secondaries - INSERT INTO test1 VALUES (2021-01-31);
Data getting in and is replicated
Example:

29. Physical Replication
Log Shipping
Streaming Replication

30. Physical Replication - Log Shipping
PostgreSQL 8.2
Replication based on the archival
archive_command on the primary
restore_command on the standby
File Based

31. Physical Replication - Log Shipping
pgxlog/
pg_wal
2920514960194996
NFS / Archive Server
Primary Server Standby Server
Archive Command
Restore Command
WAL File

32. Physical Replication - Log Shipping
No direct connection
No replication user
Replica always be in lag
No support for synchronous mode

33. Physical Replication - Streaming Replication
PostgreSQL 9.0
Stream WAL files to the standbys
primary_conninfo in recovery.conf
Block Based

34. Physical Replication - Streaming Replication
WAL Receiver - on Standbys
Startup - on Standbys
WAL Sender - on Primary

35. Physical Replication - Streaming Replication
Start Connection
primary_conninfo
Authentication
Password Response
Authentication Done
Identify System
and start to
stream WAL
System Identified and
send WAL files
Standby
WAL Receiver
Primary
WAL Sender

36. Physical Replication - Streaming Replication
Primary Server Secondary Server
WAL Records
WAL Records
WAL Records
WAL Receiver
WAL Sender

37. Physical Replication - Streaming Replication
Replication Lag can be reduced
Supports Synchronous Replication
Used for Failover
Requires direct connection
Requires DB user

38. Physical Replication - Streaming Replication / Log Shipping
Same PostgreSQL Version
Same Architecture
Entire Dataset
No writes are possible

39. Logical Replication - Publisher/Subscriber Model
table - t1 and t2 on database test
Connection Details
PostgreSQL 10
Publication - Source Server
Publisher
Subscription - Destination Server
Subsciber

40. Logical Replication - Publisher/Subscriber Model
WAL Sender - Publisher
Logical Replication Launcher - Subscriber

41. Logical Replication - Publisher/Subscriber Model
Source/Publisher Destination/Subscriber
WAL Records WAL Sender Logical Replication
Launcher
Publication
Decoding
Logical
Messages
Subscriber

42. Logical Replication - Publisher/Subscriber Model
Granularity in replicating data
can write on the destination
Replicate across different versions
DDL is not supported
Not for failover

43. Physical / Logical

44. How to achieve HA

45. How to achieve HA
Application
Primary Server
Writes
Secondary Servers
Reads

46. How to achieve HA
Application
Primary Server
Writes
Secondary Servers
Reads

47. High Availability
Frameworks

48. High Availability Frameworks
Patroni
Repmgr
PAF
pgpool -II

49. High Availability Frameworks - Patroni
ETCD Server
Secondary Servers
Primary Server
Patroni
Patroni
Patroni

50. High Availability Frameworks - Patroni
Continuous Monitoring
Automatic Failover
Easy to add or remove the node
Join back the node
Support manual switchover too
Scheduling maintenance

51. Points to Note
Test the tool for auto-failover
Load Balancing
Frequent DR activity
Maintain Backups
Proper Monitoring

52. Reach Us : Info@mydbops.com
Thank You