Streaming ReplicationMasao FujiiNTT OSS CenterCopyright(c)2009 NTT, Inc. All Rights Reserved.
What’s Streaming Replication?• Postgres’ built-in replication   – Now under development in the community   – Will be avail...
Streaming Replication vs Hot Standby• Streaming Replication (SR)   – Capability to send changes to the slave, to keep it c...
History• Historical policy   – Avoid putting replication into core Postgres   – "one size fits all" replication solution• ...
Road to core• Growing frustration   – Too complex to install and use for simple cases   – vs. other dbms• Proposal of buil...
Master - Slaves• One master and one or more slaves   – Only master accepts write queries   – One-way replication: master  ...
Log-shipping• WAL is shipped as data changes   – Slave can be kept current by replaying WAL   – WAL records in partially-f...
Log-shipping• No migration required   – Provides the same interface as the stand-alone     Postgres does   ⇔Tables must ha...
Per database cluster granularity• Replicates all database objects   – No need to specify which object is replicated   – NO...
Shared nothing architecture• WAL is shipped via network    – No single point of failure    – No special H/W like shared di...
Management• Fail Over   – Anytime, slave can be brought up   – Automatic failover is not supported   ⇒ Need to use cluster...
Management• Online-Resync   – Anytime, new slave can be added to replication     without downtime               Client    ...
Synchronization mode        Master         Slavemodes        fsync    write fsync apply                                   ...
Synchronization mode• async   – Asynchronous replication   – Possible risk of data loss   – No master overhead• write, fsy...
Synchronization mode• The mode can be specified per transaction   – Mission critical trasanction like banking would     re...
Built-in• Easy to install and use   – Just install postgres   – Replication connection can be treat the same as normal    ...
Road to v8.5• Needs your help   – Comment about interface, design and architecture   – Test and review the code   – Correc...
Upcoming SlideShare
Loading in...5
×

Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)

1,520
-1

Published on

PostgreSQL Conference 2009 Japan
Keynote Lecture
http://www.postgresql.jp/events/pgcon09j/e/program_1#2a
http://www.postgresql.jp/events/pgcon09j/j/program_1#2a

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,520
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
38
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)

  1. 1. Streaming ReplicationMasao FujiiNTT OSS CenterCopyright(c)2009 NTT, Inc. All Rights Reserved.
  2. 2. What’s Streaming Replication?• Postgres’ built-in replication – Now under development in the community – Will be available in v8.5• Why replication? – Fail Over – Load Balancing Client query query Master Slave changes
  3. 3. Streaming Replication vs Hot Standby• Streaming Replication (SR) – Capability to send changes to the slave, to keep it current• Hot Standby (HS) – Capability to run queries on the slave Client Hot Standby query query Master Slave changes Streaming Replication
  4. 4. History• Historical policy – Avoid putting replication into core Postgres – "one size fits all" replication solution• Replication war! rubyrep PL/Proxy warm-standby DBmirror PG on DRBD Slony-I Cybercluster syncreplicator PGCluster-II Londiste Postgres-R Monmoth Bucardo pgpool Sequoia RepDB twin pgpool-II PyReplica GridSQL PostgresForest PG on Shared Disk PGCluster
  5. 5. Road to core• Growing frustration – Too complex to install and use for simple cases – vs. other dbms• Proposal of built-in replication – by NTT OSSC @ PGCon 2008 Ottawa• Core team statement – It is time to include a simple, reliable basic replication feature in the core system – NOT replace the existing projects
  6. 6. Master - Slaves• One master and one or more slaves – Only master accepts write queries – One-way replication: master slaves Client write query Slaves Master changes
  7. 7. Log-shipping• WAL is shipped as data changes – Slave can be kept current by replaying WAL – WAL records in partially-filled WAL file can be shipped ⇔Cannot in warm-standby Client write query H/W architecture and major postgres release Master Slave level must be the same WAL Recovery WAL WAL Database
  8. 8. Log-shipping• No migration required – Provides the same interface as the stand-alone Postgres does ⇔Tables must have a primary key for Slony-I ⇔Some queries (e.g., sequences) would have different values on different servers in pgpool Client Client No need to change the existing application Master Slave code for replication Stand-alone
  9. 9. Per database cluster granularity• Replicates all database objects – No need to specify which object is replicated – NOT replicate: server’s log file, statistics information, ...etc ⇔Per-table in Slony-IPer database cluster Per table Master Slave Master Slave
  10. 10. Shared nothing architecture• WAL is shipped via network – No single point of failure – No special H/W like shared disk requiredShared nothing Shared disk Master Slave Master Slave
  11. 11. Management• Fail Over – Anytime, slave can be brought up – Automatic failover is not supported ⇒ Need to use clusterware like heartbeat for that• Split – Anytime, master can become stand-alone Fail Over Split Client Client Master Slave Master Slave
  12. 12. Management• Online-Resync – Anytime, new slave can be added to replication without downtime Client Master Slave
  13. 13. Synchronization mode Master Slavemodes fsync write fsync apply fastasync ✔write ✔ ✔fsync ✔ ✔ ✔apply ✔ ✔ ✔ durable
  14. 14. Synchronization mode• async – Asynchronous replication – Possible risk of data loss – No master overhead• write, fsync – Semi-synchronous – No data loss• apply – Synchronous – READ COMMITTED on slave is guaranteed – Large master overhead
  15. 15. Synchronization mode• The mode can be specified per transaction – Mission critical trasanction like banking would requires write, fsync or apply mode – async would be sufficient for transaction like web crawling Needs durability Needs speed SET mode TO fsync SET mode TO async
  16. 16. Built-in• Easy to install and use – Just install postgres – Replication connection can be treat the same as normal backend connection: pg_hba.conf, keepalive, SSL, ...• Highly active community – Bug fix soon – Active development and collaboration – Many users
  17. 17. Road to v8.5• Needs your help – Comment about interface, design and architecture – Test and review the code – Correct my English on the doc• Project site – http://wiki.postgresql.org/wiki/Streaming_Replication• Where to comment and report bug ... – pgsql-hackers@postgresql.org – pgsql-jp@ml.postgresql.jp
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×