Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Streaming ReplicationMasao FujiiNTT OSS CenterCopyright(c)2009 NTT, Inc. All Rights Reserved.
What’s Streaming Replication?• Postgres’ built-in replication   – Now under development in the community   – Will be avail...
Streaming Replication vs Hot Standby• Streaming Replication (SR)   – Capability to send changes to the slave, to keep it c...
History• Historical policy   – Avoid putting replication into core Postgres   – "one size fits all" replication solution• ...
Road to core• Growing frustration   – Too complex to install and use for simple cases   – vs. other dbms• Proposal of buil...
Master - Slaves• One master and one or more slaves   – Only master accepts write queries   – One-way replication: master  ...
Log-shipping• WAL is shipped as data changes   – Slave can be kept current by replaying WAL   – WAL records in partially-f...
Log-shipping• No migration required   – Provides the same interface as the stand-alone     Postgres does   ⇔Tables must ha...
Per database cluster granularity• Replicates all database objects   – No need to specify which object is replicated   – NO...
Shared nothing architecture• WAL is shipped via network    – No single point of failure    – No special H/W like shared di...
Management• Fail Over   – Anytime, slave can be brought up   – Automatic failover is not supported   ⇒ Need to use cluster...
Management• Online-Resync   – Anytime, new slave can be added to replication     without downtime               Client    ...
Synchronization mode        Master         Slavemodes        fsync    write fsync apply                                   ...
Synchronization mode• async   – Asynchronous replication   – Possible risk of data loss   – No master overhead• write, fsy...
Synchronization mode• The mode can be specified per transaction   – Mission critical trasanction like banking would     re...
Built-in• Easy to install and use   – Just install postgres   – Replication connection can be treat the same as normal    ...
Road to v8.5• Needs your help   – Comment about interface, design and architecture   – Test and review the code   – Correc...
Upcoming SlideShare
Loading in …5
×

Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)

1,957 views

Published on

PostgreSQL Conference 2009 Japan
Keynote Lecture
http://www.postgresql.jp/events/pgcon09j/e/program_1#2a
http://www.postgresql.jp/events/pgcon09j/j/program_1#2a

Published in: Technology
  • Be the first to comment

Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)

  1. 1. Streaming ReplicationMasao FujiiNTT OSS CenterCopyright(c)2009 NTT, Inc. All Rights Reserved.
  2. 2. What’s Streaming Replication?• Postgres’ built-in replication – Now under development in the community – Will be available in v8.5• Why replication? – Fail Over – Load Balancing Client query query Master Slave changes
  3. 3. Streaming Replication vs Hot Standby• Streaming Replication (SR) – Capability to send changes to the slave, to keep it current• Hot Standby (HS) – Capability to run queries on the slave Client Hot Standby query query Master Slave changes Streaming Replication
  4. 4. History• Historical policy – Avoid putting replication into core Postgres – "one size fits all" replication solution• Replication war! rubyrep PL/Proxy warm-standby DBmirror PG on DRBD Slony-I Cybercluster syncreplicator PGCluster-II Londiste Postgres-R Monmoth Bucardo pgpool Sequoia RepDB twin pgpool-II PyReplica GridSQL PostgresForest PG on Shared Disk PGCluster
  5. 5. Road to core• Growing frustration – Too complex to install and use for simple cases – vs. other dbms• Proposal of built-in replication – by NTT OSSC @ PGCon 2008 Ottawa• Core team statement – It is time to include a simple, reliable basic replication feature in the core system – NOT replace the existing projects
  6. 6. Master - Slaves• One master and one or more slaves – Only master accepts write queries – One-way replication: master slaves Client write query Slaves Master changes
  7. 7. Log-shipping• WAL is shipped as data changes – Slave can be kept current by replaying WAL – WAL records in partially-filled WAL file can be shipped ⇔Cannot in warm-standby Client write query H/W architecture and major postgres release Master Slave level must be the same WAL Recovery WAL WAL Database
  8. 8. Log-shipping• No migration required – Provides the same interface as the stand-alone Postgres does ⇔Tables must have a primary key for Slony-I ⇔Some queries (e.g., sequences) would have different values on different servers in pgpool Client Client No need to change the existing application Master Slave code for replication Stand-alone
  9. 9. Per database cluster granularity• Replicates all database objects – No need to specify which object is replicated – NOT replicate: server’s log file, statistics information, ...etc ⇔Per-table in Slony-IPer database cluster Per table Master Slave Master Slave
  10. 10. Shared nothing architecture• WAL is shipped via network – No single point of failure – No special H/W like shared disk requiredShared nothing Shared disk Master Slave Master Slave
  11. 11. Management• Fail Over – Anytime, slave can be brought up – Automatic failover is not supported ⇒ Need to use clusterware like heartbeat for that• Split – Anytime, master can become stand-alone Fail Over Split Client Client Master Slave Master Slave
  12. 12. Management• Online-Resync – Anytime, new slave can be added to replication without downtime Client Master Slave
  13. 13. Synchronization mode Master Slavemodes fsync write fsync apply fastasync ✔write ✔ ✔fsync ✔ ✔ ✔apply ✔ ✔ ✔ durable
  14. 14. Synchronization mode• async – Asynchronous replication – Possible risk of data loss – No master overhead• write, fsync – Semi-synchronous – No data loss• apply – Synchronous – READ COMMITTED on slave is guaranteed – Large master overhead
  15. 15. Synchronization mode• The mode can be specified per transaction – Mission critical trasanction like banking would requires write, fsync or apply mode – async would be sufficient for transaction like web crawling Needs durability Needs speed SET mode TO fsync SET mode TO async
  16. 16. Built-in• Easy to install and use – Just install postgres – Replication connection can be treat the same as normal backend connection: pg_hba.conf, keepalive, SSL, ...• Highly active community – Bug fix soon – Active development and collaboration – Many users
  17. 17. Road to v8.5• Needs your help – Comment about interface, design and architecture – Test and review the code – Correct my English on the doc• Project site – http://wiki.postgresql.org/wiki/Streaming_Replication• Where to comment and report bug ... – pgsql-hackers@postgresql.org – pgsql-jp@ml.postgresql.jp

×