The Magic of Hot                      Streaming Replication                                  BRUCE MOMJIAN,               ...
Introduction    How does WAL combined with a disk image enable standby servers?     (review)    How do you configure contin...
Write-Ahead Logging (xlog)                         Postgres               Postgres                 Postgres               ...
Pre-9.0 Continuous Archiving /                         Point-In-Time Recovery (PITR)           00                         ...
PITR Backup Procedures1. archive_command = ’cp -i %p /mnt/server/pgsql/%f < /dev/null’2. SELECT pg_start_backup(’label’);3...
PITR Recovery           00                                                   30                                           ...
PITR Recovery Procedures1. Stop postmaster2. Restore file system-level backup3. Make adjustments as outlined in the documen...
Disadvantages    Only complete 16MB files can be shipped     archive_timeout can be used to force more frequent shipping (t...
9.0 Streaming Replication / Hot Standby    Changes are streamed to the standby, greatly reducing log shipping     delays  ...
Streaming Replication Differs from PITR    File system backup is restored immediately on the standby server     WAL files a...
How Does Streaming Replication Work?             00                                                    00                 ...
Live Streaming Replication                          Primary                     Standby                                   ...
Enable Streaming to the StandbyEnable the proper WAL contents:    wal_level = hot_standbyRetain WAL files needed by the sta...
Enable Standby Connection PermissionsAdd permission for replication to pg_hba.conf:    host         replication           ...
Perform a WAL-Supported File System BackupStart psql and issue:    SELECT pg_start_backup(’testing’);Copy the database /u/...
Configure the StandbyRemove /data2/postmaster.pid so the standby server does not see theprimary server’s pid as its own:   ...
Configure the Standby For Streaming ReplicationCreate recovery.conf:    cp /u/pg/share/recovery.conf.sample /u/pg/data2/rec...
Test Streaming Replication and Hot Standby    $ psql -p 5432 -c ’CREATE TABLE streamtest(x int)’ postgres    $ psql -p 543...
Additional Complexities    Multi-server permissions     Stream from /pg_xlog and the continuous archive directory if     a...
Primary/Standby Synchronization IssuesThe primary server can take actions that cause long-running queries onthe standby to...
Conclusionhttp://momjian.us/presentations                       Manet, Bar at Folies BergèreThe Magic of Hot Streaming Rep...
Upcoming SlideShare
Loading in...5
×

The Magic of Hot Streaming Replication (Bruce Momjian)

614
-1

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
614
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The Magic of Hot Streaming Replication (Bruce Momjian)

  1. 1. The Magic of Hot Streaming Replication BRUCE MOMJIAN, ENTERPRISEDB October, 2010 Abstract POSTGRESQL 9.0 offers new facilities for maintaining a current standby server and for issuing read-only queries on the standby server. This tutorial covers these new facilities.Creative Commons Attribution License http://momjian.us/presentations
  2. 2. Introduction How does WAL combined with a disk image enable standby servers?  (review) How do you configure continuous archiving?  How do you configure a streaming, read-only server?  Multi-Server complexities  Primary/Standby synchronization complexities The Magic of Hot Streaming Replication 2
  3. 3. Write-Ahead Logging (xlog) Postgres Postgres Postgres Backend Backend Backend PostgreSQL Shared Buffer Cache Write−Ahead Log fsync Kernel Disk Buffer Cache fsync Disk BlocksThe Magic of Hot Streaming Replication 3
  4. 4. Pre-9.0 Continuous Archiving / Point-In-Time Recovery (PITR) 00 00 00 00 02 09 11 13 WAL AL AL W W File System− Continuous Level Backup Archive (WAL)The Magic of Hot Streaming Replication 4
  5. 5. PITR Backup Procedures1. archive_command = ’cp -i %p /mnt/server/pgsql/%f < /dev/null’2. SELECT pg_start_backup(’label’);3. Perform file system-level backup (can be inconsistent)4. SELECT pg_stop_backup();The Magic of Hot Streaming Replication 5
  6. 6. PITR Recovery 00 30 40 55 17 17 17 17 WAL AL AL W W File System− Continuous Level Backup Archive (WAL)The Magic of Hot Streaming Replication 6
  7. 7. PITR Recovery Procedures1. Stop postmaster2. Restore file system-level backup3. Make adjustments as outlined in the documentation4. Create recovery.conf5. restore_command = ’cp /mnt/server/pgsql/%f %p’6. Start the postmasterThe Magic of Hot Streaming Replication 7
  8. 8. Disadvantages Only complete 16MB files can be shipped  archive_timeout can be used to force more frequent shipping (this  increases archive storage requirements) No queries on the standby The Magic of Hot Streaming Replication 8
  9. 9. 9.0 Streaming Replication / Hot Standby Changes are streamed to the standby, greatly reducing log shipping  delays Standby can accept read-only queries The Magic of Hot Streaming Replication 9
  10. 10. Streaming Replication Differs from PITR File system backup is restored immediately on the standby server  WAL files are streamed to the slave  WAL files can also be archived if point-in-time recovery (PITR) is  desiredThe Magic of Hot Streaming Replication 10
  11. 11. How Does Streaming Replication Work? 00 00 00 00 02 09 11 13 WAL AL AL W W File System− Standby Level Backup ServerThe Magic of Hot Streaming Replication 11
  12. 12. Live Streaming Replication Primary Standby Network /pg_xlog /pg_xlog archive WAL archive command Archive command DirectoryThe Magic of Hot Streaming Replication 12
  13. 13. Enable Streaming to the StandbyEnable the proper WAL contents: wal_level = hot_standbyRetain WAL files needed by the standby: wal_keep_segments = 50Enable the ability to stream WAL to the standby: max_wal_senders = 1The Magic of Hot Streaming Replication 13
  14. 14. Enable Standby Connection PermissionsAdd permission for replication to pg_hba.conf: host replication all 127.0.0.1/32 trustStart the primary server: pg_ctl -l /u/pg/data/server.log startThe Magic of Hot Streaming Replication 14
  15. 15. Perform a WAL-Supported File System BackupStart psql and issue: SELECT pg_start_backup(’testing’);Copy the database /u/pg/data to a new directory, /u/pg/data2: cp -p -R /u/pg/data /u/pg/data2Dash-p preserves ownership. The copy is inconsistent, but that is okay(WAL replay will correct that).Signal the backup is complete from psql: SELECT pg_stop_backup();The Magic of Hot Streaming Replication 15
  16. 16. Configure the StandbyRemove /data2/postmaster.pid so the standby server does not see theprimary server’s pid as its own: rm /u/pg/data2/postmaster.pid(This is only necessary because we are testing with the primary and slaveon the same computer.)Edit postgresql.conf on the standby and change the port to 5433 port = 5433Enable hot standby in postgresql.conf: hot_standby = onThe Magic of Hot Streaming Replication 16
  17. 17. Configure the Standby For Streaming ReplicationCreate recovery.conf: cp /u/pg/share/recovery.conf.sample /u/pg/data2/recovery.confEnable streaming in recovery.conf: standby_mode = ’on’ primary_conninfo = ’host=localhost port=5432’Start the standby server: PGDATA=/u/pg/data2 pg_ctl -l /u/pg/data2/server.log startThe Magic of Hot Streaming Replication 17
  18. 18. Test Streaming Replication and Hot Standby $ psql -p 5432 -c ’CREATE TABLE streamtest(x int)’ postgres $ psql -p 5433 -c ’d’ postgres List of relations Schema | Name | Type | Owner --------+------------+-------+---------- public | streamtest | table | postgres (1 row) $ psql -p 5432 -c ’INSERT INTO streamtest VALUES (1)’ postgres INSERT 0 1 $ psql -p 5433 -c ’INSERT INTO streamtest VALUES (1)’ postgres ERROR: cannot execute INSERT in a read-only transactionThe Magic of Hot Streaming Replication 18
  19. 19. Additional Complexities Multi-server permissions  Stream from /pg_xlog and the continuous archive directory if  archive_mode is enabled on the primaryThe Magic of Hot Streaming Replication 19
  20. 20. Primary/Standby Synchronization IssuesThe primary server can take actions that cause long-running queries onthe standby to be cancelled. Specifically, the cleanup of unnecessaryrows that are still of interest to long-running queries on the standby cancause long-running queries to be cancelled on the standby. Standbyquery cancellation can be minimized in two ways:1. Delay cleanup of old records on the primary with vacuum_defer_cleanup_age in postgresql.conf.2. Delay application of WAL logs on the standby with max_standby_delay in postgresql.conf. The default is 30 seconds; -1 causes application to delay indefinitely to prevent query cancellation. This also delays changes from appearing on the standby and can lengthen the time required for failover to the slave.The Magic of Hot Streaming Replication 20
  21. 21. Conclusionhttp://momjian.us/presentations Manet, Bar at Folies BergèreThe Magic of Hot Streaming Replication 21
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×