PostgreSQL Replication in 10 Minutes - SCALE

18,339 views

Published on

A very brief review of how to do PostgreSQL 9.3 Streaming Replication. Meant to accompany a demo.

Published in: Technology
0 Comments
25 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
18,339
On SlideShare
0
From Embeds
0
Number of Embeds
27
Actions
Shares
0
Downloads
514
Comments
0
Likes
25
Embeds 0
No embeds

No notes for slide

PostgreSQL Replication in 10 Minutes - SCALE

  1. 1. PostgreSQL Replication in 10 Minutes Josh Berkus PostgreSQL Experts, Inc. SCALE 2014
  2. 2. 1. See a multistage replication demo 2. Hear how to set up and configure replication 3. Understand different setups and replication issues
  3. 3. replication terms master / slave master / standby master / replica primary / secondary primary / replica
  4. 4. replication mechanisms 1. statement 2. row 3. binary
  5. 5. replication mechanisms 1. queries 2. rows 3. data pages
  6. 6. also called ... ● streaming replication – ● refers to the ability to stream new data pages over a network connection hot standby – refers to the ability of standbys to run readonly queries while in standby mode
  7. 7. more terms ● recovery – ● snapshot, clone, basebackup – ● binary replication came from binary backup, i.e. Point In Time Recovery taking a moment-in-time copy of a running database server standalone – a lone read-write server, neither master nor replica
  8. 8. administering replication
  9. 9. configuration files ● postgresql.conf – ● same settings for master, replica recovery.conf – – presence turns on replication must be in $PGDATA
  10. 10. views & functions process list ● pg_stat_replication ● pg_is_in_recovery() ● pg_xlog* functions ●
  11. 11. permissions & security A. B. C. D. replication permission pg_hba.conf max_wal_senders firewall/network
  12. 12. archiving replication
  13. 13. archiving replication 1. 2. 3. 4. 5. 6. set up archiving start archiving pg_start_backup('label') rsync all files pg_stop_backup() bring up replica
  14. 14. reasons to archive replica out-of-sync ● combine with PITR or DR ● very erratic connection to master ● need remastering before 9.3 ●
  15. 15. archiving tips use a script which handles copy failure ● use a shared drive ● put archive on a partition ● monitor for archive growth ● compression ●
  16. 16. failover, failback & remastering
  17. 17. more terms ● failover, promotion – ● failback – ● making a replica into a master/standalone returning the original master to master status remastering – designating a new master in a group of servers
  18. 18. replica promotion pg_ctl promote ● trigger file ● rm recovery.conf & restart ●
  19. 19. failover has 3 parts 1. failing over the database 2. failing over the connections 3. STONITH
  20. 20. manual failover ● advantages: – – ● easy to set up fewer accidental failovers disadvantages: – – downtime being woken up at 3am
  21. 21. automated failover ● advantages: – – ● low downtime sleep through the night disadvantages: – – – hard to set up correctly need broker server accidentally triggered failovers
  22. 22. automated failover logic
  23. 23. www.handyrep.org
  24. 24. STONITH use corosync/VIP ● use connection failover ● use peer broker server ●
  25. 25. remastering need the replica which is “furthest ahead” ● measure both receive point and replay point ● need 9.3 for “streaming-only” remastering ●
  26. 26. replication lag & query cancel
  27. 27. reasons for lag ● network delay – speed of light replica too busy ● file operations block ● – – VACUUM DROP TABLE
  28. 28. replication lag issues inconsistency (if load-balancing) ● query cancel ● – applications need to retry queries catch-up speed ● burden on master ●
  29. 29. synchronous replication
  30. 30. what synch rep does guarantee against data loss
  31. 31. what it doesn't ● enforce global consistency – – ● master can be behind replica snapshot can be behind help availability
  32. 32. “I would rather be down than potentially lose data.”
  33. 33. how to synch rep 1. pick one (or a pool) of servers to be your synch replicas 2. change application_name 3. change master's postgresql.conf 4. reload
  34. 34. Postgres specialities implements only 1-redundant model ● synch is per-transaction ● – – not per replica synch only important transactions
  35. 35. synch rep design 1 replica is synch replica ● several asych replicas ● load-balance to asynch only ● always failover to synch replica ●
  36. 36. cascading replication
  37. 37. how to cascade 1. 2. 3. 4. have master & replica clone the master or the replica point primary_conninfo to the replica bring up the new replica
  38. 38. why to cascade limit connections to master ● don't clone master ● know which replica is ahead ●
  39. 39. why to cascade Los Angeles walsender Shanghai walsender walreciever walreciever walreciever
  40. 40. why not cascade? complexity ● cycles ● increases SPOFs ●
  41. 41. replication in the cloud
  42. 42. use a shared archive WAL-E
  43. 43. ephemeral replicas no sync to disk ● do not recover from crash ● – ● spin up a replacement instead turn off all logging/disk – fsync off, bgwriter off, full_page_writes off
  44. 44. sharding and replication server 1 server 2
  45. 45. sharding and replication server 1 server 2
  46. 46. changes coming up ● in dev: pg_rewind – failback without full clone 9.4: merge recovery.conf & postgresql.conf ● 9.4+: “logical” streaming replication ●
  47. 47. questions? ● HandyRep: www.handyrep.org ● Josh Berkus: josh@pgexperts.com – – ● PGX: www.pgexperts.com Blog: www.databasesoup.com Upcoming Events – – NYC pgDay: April 2-3 pgCon, Ottawa, May 21-24 Copyright 2013 PostgreSQL Experts Inc. Released under the Creative Commons Share-Alike 3.0 License. All images are the property of their respective owners. The WAL-E image is the property of Disney Inc. and is use here as parody.

×