SlideShare a Scribd company logo
1 of 32
Download to read offline
Replication Solutions for
            PostgreSQL
             Peter Eisentraut
          petere@postgresql.org
What's in a Term?
•   Replication?
•   Clustering?
•   High availability?
•   Failover?
•   Standby?

• Putting data on more than one computer

                                           2
Space of Possibilities
• Goals
  • What do you want to achieve?
• Techniques
  • How can this be implemented?
• Solutions
  • What software is available to do this?




                                             3
Goals
• High availability
• Performance
  • Read
  • Write
• Wide-area networks
• Offline peers



                              4
Goal: High Availability
• No one wants “low availability”!
• Provisions for system failures
  • Software faults
  • Hardware faults
  • External interference




                                     5
Goal: Read Performance
• Applications with:
  • many readers (e.g., library information system)
  • resource-intensive readers (e.g., data
    warehousing)
• Distribute readers on more hardware.
• Most often, one physical machine is
  enough.



                                                  6
Goal: Write Performance
• Applications with:
  • Many writers
• Distribute writers on more hardware?
  • Constraint checking, conflict resolution?!?
• Faster writing contradicts replication.
  • Partition, don't replicate!
  • RAID 0/striping is not replication – it makes
    things “worse”.
  • RAID 10 is a good idea, but not the topic here.
                                                  7
Goal: Wide-Area Networks
• Faster access across WANs
• Reading?
  • Local copies
• Writing?
  • Synchronization?




                               8
Goal: Offline Peers
• Synchronize data with laptops,
  handhelds, ...
• “Road warriors”
• May be considered very-low-latency WANs




                                        9
Techniques
• Replication
  • Master/Slave
     • Asynchronous
     • Synchronous
  • Multi-Master
     • Asynchronous
     • Synchronous
• Proxy
• Standby system
                             10
Technique: Replication
   Master/Slave Asynchronous
• High(er) availability(?)
• Read performance
  • Load spreading,
    load balancing
• Offline peers              M   asy c
                                    n    S
  (unidirectional sync.)




                                             11
Technique: Replication
    Master/Slave Synchronous
• High availibility
• Better read
  performance
• Worse write
  performance         M   sy c
                            n    S




                                     12
Technique: Replication
   Multi-Master Asynchronous
• Read performance
• Faster access across
  WANs
• Manage offline peers
                         M           M
• Requires conflict          asy c
                                n


  resolution mechanism



                                         13
Technique: Replication
       Multi-Master Synchronous
•   “Holy grail of replication”
•   High availability
•   Read performance
•   Difficult to get good
                                  M          M
    write performance                 sy c
                                        n




                                                 14
Technique: Proxy
• High availability
• Read performance
• Proxy instance should       Proxy
  be redundant
• Transparent to the
  application
                          C           C



                                          15
Technique: Standby System
• High availability




                      M   sy c
                            n    S




                                     16
Constraints
• Hardware
• Operating system
• Application




                           17
No Built-In Solution?
• FIXME




                              18
Solutions
•   Slony-I, -II
•   PGCluster
•   DBMirror
•   pgpool
•   WAL replication
•   Sequoia
•   DRBD
•   Shared storage
                             19
Solution: Slony-I
(Slony ← слоны ← elephants)
• Asynchronous master/slave replication
• Multiple slaves, cascading possible
• Particularly useful for:
  • Read performance (load balancing with pgpool)
  • Limited form of high availability
  • Offline slaves via file-based log shipping
http://www.slony.info/

                                               20
Solution: Slony-II
• Synchronous master/master
  replication?
• See Gavin Sherry's session for details




                                           21
Solution: PGCluster
• Synchronous master/master replication
• Replicates the query string
• Particularly useful for:
  • Load balancing
  • High availability



http://pgcluster.projects.postgresql.org/
                                            22
Solution: DBMirror
• Asynchronous master/slave replication
• Very simple (compared to Slony-I)
• Particularly useful for:
  • Read performance
  • Offline peers



contrib/dbmirror/ in PostgreSQL source tree
                                              23
Solution: pgpool
• Connection pool daemon for PostgreSQL
• Supports simple proxying
• Useful as frontend for Slony-I




http://pgpool.projects.postgresql.org/
                                          24
Solution: WAL Replication
• Use the “archived” WAL logs for “recovery”
  on a standby system
• Disadvantages:
  • Only full database cluster replication
  • Master and slave must be binary-compatible
  • Rather slow across network
• Useful for:
  • High availability

                                                 25
Solution: Sequoia
• Formerly C[lustered]-JDBC
• Proxy offering clustering, load
  balancing and failover services
• Particularly useful for:
  • High availability
  • Read performance
• Currently only for Java/JDBC applications
http://sequoia.continuent.org/

                                              26
Solution: DRBD
•   File system (block device) replication
•   Linux kernel module
•   Standby system
•   Useful for:
    • High availability
    • Secure any service, not just a database system


http://www.drbd.org/
                                                  27
Solution: Shared Storage
•   NAS, iSCSI, Fiberchannel, ...
•   Available from many vendors
•   Standby system
•   Useful for:
    • High availability
    • Secure any service, not just a database system
• Single storage system is a possible point of
  failure

                                                  28
Summary
• Plenty of solutions for diverse applications
• Make a (project) plan.




                                                 29
Suggestions
• Minimum for any production installation:
  • Sensible disk clustering
     • RAID 10
     • Tablespace management
     • Separate disk(s) for WAL
  • DRBD or shared storage
• Slony-I for load balancing or warehousing
• Java developers consider Sequoia


                                              30
Outlook
•   Slony-II
•   WAL replication management
•   XA support
•   More packaging efforts




                                 31
The End
Replication Solutions for PostgreSQL




                                       32

More Related Content

What's hot

Tuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris EnvironmentTuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris EnvironmentJignesh Shah
 
Built-in Replication in PostgreSQL
Built-in Replication in PostgreSQLBuilt-in Replication in PostgreSQL
Built-in Replication in PostgreSQLMasao Fujii
 
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)Masao Fujii
 
PostgreSQL and Linux Containers
PostgreSQL and Linux ContainersPostgreSQL and Linux Containers
PostgreSQL and Linux ContainersJignesh Shah
 
Postgres on OpenStack
Postgres on OpenStackPostgres on OpenStack
Postgres on OpenStackEDB
 
Architecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres ClusterArchitecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres ClusterAshnikbiz
 
Linux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performanceLinux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performancePostgreSQL-Consulting
 
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte DataProblems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte DataJignesh Shah
 
Best Practices with PostgreSQL on Solaris
Best Practices with PostgreSQL on SolarisBest Practices with PostgreSQL on Solaris
Best Practices with PostgreSQL on SolarisJignesh Shah
 
PostgreSQL9.3 Switchover/Switchback
PostgreSQL9.3 Switchover/SwitchbackPostgreSQL9.3 Switchover/Switchback
PostgreSQL9.3 Switchover/SwitchbackVibhor Kumar
 
Deploying Maximum HA Architecture With PostgreSQL
Deploying Maximum HA Architecture With PostgreSQLDeploying Maximum HA Architecture With PostgreSQL
Deploying Maximum HA Architecture With PostgreSQLDenish Patel
 
Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016PostgreSQL-Consulting
 
My experience with embedding PostgreSQL
 My experience with embedding PostgreSQL My experience with embedding PostgreSQL
My experience with embedding PostgreSQLJignesh Shah
 
Geographically Distributed PostgreSQL
Geographically Distributed PostgreSQLGeographically Distributed PostgreSQL
Geographically Distributed PostgreSQLmason_s
 
Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4Denish Patel
 
Deep Dive into RDS PostgreSQL Universe
Deep Dive into RDS PostgreSQL UniverseDeep Dive into RDS PostgreSQL Universe
Deep Dive into RDS PostgreSQL UniverseJignesh Shah
 
Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2 Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2 Denish Patel
 
PostgreSQL Hangout Parameter Tuning
PostgreSQL Hangout Parameter TuningPostgreSQL Hangout Parameter Tuning
PostgreSQL Hangout Parameter TuningAshnikbiz
 
PostgreSQL Disaster Recovery with Barman
PostgreSQL Disaster Recovery with BarmanPostgreSQL Disaster Recovery with Barman
PostgreSQL Disaster Recovery with BarmanGabriele Bartolini
 

What's hot (20)

Tuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris EnvironmentTuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris Environment
 
Built-in Replication in PostgreSQL
Built-in Replication in PostgreSQLBuilt-in Replication in PostgreSQL
Built-in Replication in PostgreSQL
 
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
 
PostgreSQL and Linux Containers
PostgreSQL and Linux ContainersPostgreSQL and Linux Containers
PostgreSQL and Linux Containers
 
Postgres on OpenStack
Postgres on OpenStackPostgres on OpenStack
Postgres on OpenStack
 
Architecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres ClusterArchitecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres Cluster
 
Linux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performanceLinux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performance
 
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte DataProblems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
 
Best Practices with PostgreSQL on Solaris
Best Practices with PostgreSQL on SolarisBest Practices with PostgreSQL on Solaris
Best Practices with PostgreSQL on Solaris
 
PostgreSQL9.3 Switchover/Switchback
PostgreSQL9.3 Switchover/SwitchbackPostgreSQL9.3 Switchover/Switchback
PostgreSQL9.3 Switchover/Switchback
 
Deploying Maximum HA Architecture With PostgreSQL
Deploying Maximum HA Architecture With PostgreSQLDeploying Maximum HA Architecture With PostgreSQL
Deploying Maximum HA Architecture With PostgreSQL
 
Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016
 
My experience with embedding PostgreSQL
 My experience with embedding PostgreSQL My experience with embedding PostgreSQL
My experience with embedding PostgreSQL
 
Geographically Distributed PostgreSQL
Geographically Distributed PostgreSQLGeographically Distributed PostgreSQL
Geographically Distributed PostgreSQL
 
Gluster 3.3 deep dive
Gluster 3.3 deep diveGluster 3.3 deep dive
Gluster 3.3 deep dive
 
Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4
 
Deep Dive into RDS PostgreSQL Universe
Deep Dive into RDS PostgreSQL UniverseDeep Dive into RDS PostgreSQL Universe
Deep Dive into RDS PostgreSQL Universe
 
Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2 Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2
 
PostgreSQL Hangout Parameter Tuning
PostgreSQL Hangout Parameter TuningPostgreSQL Hangout Parameter Tuning
PostgreSQL Hangout Parameter Tuning
 
PostgreSQL Disaster Recovery with Barman
PostgreSQL Disaster Recovery with BarmanPostgreSQL Disaster Recovery with Barman
PostgreSQL Disaster Recovery with Barman
 

Viewers also liked

Introduction to pg_cheat_funcs
Introduction to pg_cheat_funcsIntroduction to pg_cheat_funcs
Introduction to pg_cheat_funcsMasao Fujii
 
PostgreSQL HA
PostgreSQL   HAPostgreSQL   HA
PostgreSQL HAharoonm
 
Storage Replication in High-Performance High-Availability Environments
Storage Replication in High-Performance High-Availability EnvironmentsStorage Replication in High-Performance High-Availability Environments
Storage Replication in High-Performance High-Availability Environmentshastexo
 
High Availability Storage (susecon2016)
High Availability Storage (susecon2016)High Availability Storage (susecon2016)
High Availability Storage (susecon2016)Roger Zhou 周志强
 
HA+DRBD+Postgres - PostgresWest '08
HA+DRBD+Postgres - PostgresWest '08HA+DRBD+Postgres - PostgresWest '08
HA+DRBD+Postgres - PostgresWest '08Jesse Young
 
PostgreSQL9.0アップデート レプリケーションがやってきた!
PostgreSQL9.0アップデート レプリケーションがやってきた!PostgreSQL9.0アップデート レプリケーションがやってきた!
PostgreSQL9.0アップデート レプリケーションがやってきた!Masao Fujii
 
]project-open[ Screenshots
]project-open[ Screenshots ]project-open[ Screenshots
]project-open[ Screenshots Klaus Hofeditz
 
Drbd9 and drbdmanage_june_2016
Drbd9 and drbdmanage_june_2016Drbd9 and drbdmanage_june_2016
Drbd9 and drbdmanage_june_2016Philipp Reisner
 

Viewers also liked (9)

Introduction to pg_cheat_funcs
Introduction to pg_cheat_funcsIntroduction to pg_cheat_funcs
Introduction to pg_cheat_funcs
 
PostgreSQL HA
PostgreSQL   HAPostgreSQL   HA
PostgreSQL HA
 
Storage Replication in High-Performance High-Availability Environments
Storage Replication in High-Performance High-Availability EnvironmentsStorage Replication in High-Performance High-Availability Environments
Storage Replication in High-Performance High-Availability Environments
 
High Availability Storage (susecon2016)
High Availability Storage (susecon2016)High Availability Storage (susecon2016)
High Availability Storage (susecon2016)
 
HA+DRBD+Postgres - PostgresWest '08
HA+DRBD+Postgres - PostgresWest '08HA+DRBD+Postgres - PostgresWest '08
HA+DRBD+Postgres - PostgresWest '08
 
Development of 8.3 In India
Development of 8.3 In IndiaDevelopment of 8.3 In India
Development of 8.3 In India
 
PostgreSQL9.0アップデート レプリケーションがやってきた!
PostgreSQL9.0アップデート レプリケーションがやってきた!PostgreSQL9.0アップデート レプリケーションがやってきた!
PostgreSQL9.0アップデート レプリケーションがやってきた!
 
]project-open[ Screenshots
]project-open[ Screenshots ]project-open[ Screenshots
]project-open[ Screenshots
 
Drbd9 and drbdmanage_june_2016
Drbd9 and drbdmanage_june_2016Drbd9 and drbdmanage_june_2016
Drbd9 and drbdmanage_june_2016
 

Similar to Replication Solutions for PostgreSQL

High Scalability Toronto: Meetup #2
High Scalability Toronto: Meetup #2High Scalability Toronto: Meetup #2
High Scalability Toronto: Meetup #2ScribbleLive
 
Intro to Big Data and NoSQL
Intro to Big Data and NoSQLIntro to Big Data and NoSQL
Intro to Big Data and NoSQLDon Demcsak
 
Writing Scalable Software in Java
Writing Scalable Software in JavaWriting Scalable Software in Java
Writing Scalable Software in JavaRuben Badaró
 
Leveraging MongoDB: An Introductory Case Study
Leveraging MongoDB: An Introductory Case StudyLeveraging MongoDB: An Introductory Case Study
Leveraging MongoDB: An Introductory Case StudySean Laurent
 
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013Christopher Curtin
 
End of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph ReplicationEnd of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph ReplicationCeph Community
 
Petabyte scale on commodity infrastructure
Petabyte scale on commodity infrastructurePetabyte scale on commodity infrastructure
Petabyte scale on commodity infrastructureelliando dias
 
Java scalability considerations yogesh deshpande
Java scalability considerations   yogesh deshpandeJava scalability considerations   yogesh deshpande
Java scalability considerations yogesh deshpandeIndicThreads
 
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInJay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInLinkedIn
 
Lessons learned from running Spark on Docker
Lessons learned from running Spark on DockerLessons learned from running Spark on Docker
Lessons learned from running Spark on DockerDataWorks Summit
 
Scalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeScalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeAmir Ghaffari
 
MongoDB Case Study at NoSQL Now 2012
MongoDB Case Study at NoSQL Now 2012MongoDB Case Study at NoSQL Now 2012
MongoDB Case Study at NoSQL Now 2012Sean Laurent
 
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache KafkaBuilding Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache KafkaGuozhang Wang
 
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...Qian Lin
 
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...Nagios
 
What CloudStackers Need To Know About LINSTOR/DRBD
What CloudStackers Need To Know About LINSTOR/DRBDWhat CloudStackers Need To Know About LINSTOR/DRBD
What CloudStackers Need To Know About LINSTOR/DRBDShapeBlue
 
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)Lars Marowsky-Brée
 
Polyglot and functional (Devoxx Nov/2011)
Polyglot and functional (Devoxx Nov/2011)Polyglot and functional (Devoxx Nov/2011)
Polyglot and functional (Devoxx Nov/2011)Martijn Verburg
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsJonas Bonér
 
MongoBoston - MongoHQ
MongoBoston - MongoHQMongoBoston - MongoHQ
MongoBoston - MongoHQbenwyrosdick
 

Similar to Replication Solutions for PostgreSQL (20)

High Scalability Toronto: Meetup #2
High Scalability Toronto: Meetup #2High Scalability Toronto: Meetup #2
High Scalability Toronto: Meetup #2
 
Intro to Big Data and NoSQL
Intro to Big Data and NoSQLIntro to Big Data and NoSQL
Intro to Big Data and NoSQL
 
Writing Scalable Software in Java
Writing Scalable Software in JavaWriting Scalable Software in Java
Writing Scalable Software in Java
 
Leveraging MongoDB: An Introductory Case Study
Leveraging MongoDB: An Introductory Case StudyLeveraging MongoDB: An Introductory Case Study
Leveraging MongoDB: An Introductory Case Study
 
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
 
End of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph ReplicationEnd of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph Replication
 
Petabyte scale on commodity infrastructure
Petabyte scale on commodity infrastructurePetabyte scale on commodity infrastructure
Petabyte scale on commodity infrastructure
 
Java scalability considerations yogesh deshpande
Java scalability considerations   yogesh deshpandeJava scalability considerations   yogesh deshpande
Java scalability considerations yogesh deshpande
 
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInJay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
 
Lessons learned from running Spark on Docker
Lessons learned from running Spark on DockerLessons learned from running Spark on Docker
Lessons learned from running Spark on Docker
 
Scalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeScalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and Practice
 
MongoDB Case Study at NoSQL Now 2012
MongoDB Case Study at NoSQL Now 2012MongoDB Case Study at NoSQL Now 2012
MongoDB Case Study at NoSQL Now 2012
 
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache KafkaBuilding Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
 
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
 
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
 
What CloudStackers Need To Know About LINSTOR/DRBD
What CloudStackers Need To Know About LINSTOR/DRBDWhat CloudStackers Need To Know About LINSTOR/DRBD
What CloudStackers Need To Know About LINSTOR/DRBD
 
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
 
Polyglot and functional (Devoxx Nov/2011)
Polyglot and functional (Devoxx Nov/2011)Polyglot and functional (Devoxx Nov/2011)
Polyglot and functional (Devoxx Nov/2011)
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 
MongoBoston - MongoHQ
MongoBoston - MongoHQMongoBoston - MongoHQ
MongoBoston - MongoHQ
 

More from Peter Eisentraut

Programming with Python and PostgreSQL
Programming with Python and PostgreSQLProgramming with Python and PostgreSQL
Programming with Python and PostgreSQLPeter Eisentraut
 
Getting Started with PL/Proxy
Getting Started with PL/ProxyGetting Started with PL/Proxy
Getting Started with PL/ProxyPeter Eisentraut
 
Linux distribution for the cloud
Linux distribution for the cloudLinux distribution for the cloud
Linux distribution for the cloudPeter Eisentraut
 
Most Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL FeaturesMost Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL FeaturesPeter Eisentraut
 
Porting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQLPorting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQLPeter Eisentraut
 
Porting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQLPorting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQLPeter Eisentraut
 
XML Support: Specifications and Development
XML Support: Specifications and DevelopmentXML Support: Specifications and Development
XML Support: Specifications and DevelopmentPeter Eisentraut
 
PostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie DatenbankalternativePostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie DatenbankalternativePeter Eisentraut
 
The Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future DevelopmentsThe Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future DevelopmentsPeter Eisentraut
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsPeter Eisentraut
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsPeter Eisentraut
 
Docbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XMLDocbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XMLPeter Eisentraut
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...Peter Eisentraut
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...Peter Eisentraut
 
The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)Peter Eisentraut
 

More from Peter Eisentraut (20)

Programming with Python and PostgreSQL
Programming with Python and PostgreSQLProgramming with Python and PostgreSQL
Programming with Python and PostgreSQL
 
Getting Started with PL/Proxy
Getting Started with PL/ProxyGetting Started with PL/Proxy
Getting Started with PL/Proxy
 
Linux distribution for the cloud
Linux distribution for the cloudLinux distribution for the cloud
Linux distribution for the cloud
 
Most Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL FeaturesMost Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL Features
 
Porting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQLPorting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQL
 
Porting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQLPorting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQL
 
PostgreSQL and XML
PostgreSQL and XMLPostgreSQL and XML
PostgreSQL and XML
 
XML Support: Specifications and Development
XML Support: Specifications and DevelopmentXML Support: Specifications and Development
XML Support: Specifications and Development
 
PostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie DatenbankalternativePostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie Datenbankalternative
 
The Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future DevelopmentsThe Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future Developments
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-Frontends
 
PostgreSQL and PL/Java
PostgreSQL and PL/JavaPostgreSQL and PL/Java
PostgreSQL and PL/Java
 
PostgreSQL News
PostgreSQL NewsPostgreSQL News
PostgreSQL News
 
PostgreSQL News
PostgreSQL NewsPostgreSQL News
PostgreSQL News
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-Frontends
 
Docbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XMLDocbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XML
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...
 
Spaß mit PostgreSQL
Spaß mit PostgreSQLSpaß mit PostgreSQL
Spaß mit PostgreSQL
 
The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)
 

Recently uploaded

My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptxLBM Solutions
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhisoniya singh
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsSnow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsHyundai Motor Group
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 

Recently uploaded (20)

My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptx
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsSnow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
 
The transition to renewables in India.pdf
The transition to renewables in India.pdfThe transition to renewables in India.pdf
The transition to renewables in India.pdf
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 

Replication Solutions for PostgreSQL

  • 1. Replication Solutions for PostgreSQL Peter Eisentraut petere@postgresql.org
  • 2. What's in a Term? • Replication? • Clustering? • High availability? • Failover? • Standby? • Putting data on more than one computer 2
  • 3. Space of Possibilities • Goals • What do you want to achieve? • Techniques • How can this be implemented? • Solutions • What software is available to do this? 3
  • 4. Goals • High availability • Performance • Read • Write • Wide-area networks • Offline peers 4
  • 5. Goal: High Availability • No one wants “low availability”! • Provisions for system failures • Software faults • Hardware faults • External interference 5
  • 6. Goal: Read Performance • Applications with: • many readers (e.g., library information system) • resource-intensive readers (e.g., data warehousing) • Distribute readers on more hardware. • Most often, one physical machine is enough. 6
  • 7. Goal: Write Performance • Applications with: • Many writers • Distribute writers on more hardware? • Constraint checking, conflict resolution?!? • Faster writing contradicts replication. • Partition, don't replicate! • RAID 0/striping is not replication – it makes things “worse”. • RAID 10 is a good idea, but not the topic here. 7
  • 8. Goal: Wide-Area Networks • Faster access across WANs • Reading? • Local copies • Writing? • Synchronization? 8
  • 9. Goal: Offline Peers • Synchronize data with laptops, handhelds, ... • “Road warriors” • May be considered very-low-latency WANs 9
  • 10. Techniques • Replication • Master/Slave • Asynchronous • Synchronous • Multi-Master • Asynchronous • Synchronous • Proxy • Standby system 10
  • 11. Technique: Replication Master/Slave Asynchronous • High(er) availability(?) • Read performance • Load spreading, load balancing • Offline peers M asy c n S (unidirectional sync.) 11
  • 12. Technique: Replication Master/Slave Synchronous • High availibility • Better read performance • Worse write performance M sy c n S 12
  • 13. Technique: Replication Multi-Master Asynchronous • Read performance • Faster access across WANs • Manage offline peers M M • Requires conflict asy c n resolution mechanism 13
  • 14. Technique: Replication Multi-Master Synchronous • “Holy grail of replication” • High availability • Read performance • Difficult to get good M M write performance sy c n 14
  • 15. Technique: Proxy • High availability • Read performance • Proxy instance should Proxy be redundant • Transparent to the application C C 15
  • 16. Technique: Standby System • High availability M sy c n S 16
  • 17. Constraints • Hardware • Operating system • Application 17
  • 19. Solutions • Slony-I, -II • PGCluster • DBMirror • pgpool • WAL replication • Sequoia • DRBD • Shared storage 19
  • 20. Solution: Slony-I (Slony ← слоны ← elephants) • Asynchronous master/slave replication • Multiple slaves, cascading possible • Particularly useful for: • Read performance (load balancing with pgpool) • Limited form of high availability • Offline slaves via file-based log shipping http://www.slony.info/ 20
  • 21. Solution: Slony-II • Synchronous master/master replication? • See Gavin Sherry's session for details 21
  • 22. Solution: PGCluster • Synchronous master/master replication • Replicates the query string • Particularly useful for: • Load balancing • High availability http://pgcluster.projects.postgresql.org/ 22
  • 23. Solution: DBMirror • Asynchronous master/slave replication • Very simple (compared to Slony-I) • Particularly useful for: • Read performance • Offline peers contrib/dbmirror/ in PostgreSQL source tree 23
  • 24. Solution: pgpool • Connection pool daemon for PostgreSQL • Supports simple proxying • Useful as frontend for Slony-I http://pgpool.projects.postgresql.org/ 24
  • 25. Solution: WAL Replication • Use the “archived” WAL logs for “recovery” on a standby system • Disadvantages: • Only full database cluster replication • Master and slave must be binary-compatible • Rather slow across network • Useful for: • High availability 25
  • 26. Solution: Sequoia • Formerly C[lustered]-JDBC • Proxy offering clustering, load balancing and failover services • Particularly useful for: • High availability • Read performance • Currently only for Java/JDBC applications http://sequoia.continuent.org/ 26
  • 27. Solution: DRBD • File system (block device) replication • Linux kernel module • Standby system • Useful for: • High availability • Secure any service, not just a database system http://www.drbd.org/ 27
  • 28. Solution: Shared Storage • NAS, iSCSI, Fiberchannel, ... • Available from many vendors • Standby system • Useful for: • High availability • Secure any service, not just a database system • Single storage system is a possible point of failure 28
  • 29. Summary • Plenty of solutions for diverse applications • Make a (project) plan. 29
  • 30. Suggestions • Minimum for any production installation: • Sensible disk clustering • RAID 10 • Tablespace management • Separate disk(s) for WAL • DRBD or shared storage • Slony-I for load balancing or warehousing • Java developers consider Sequoia 30
  • 31. Outlook • Slony-II • WAL replication management • XA support • More packaging efforts 31
  • 32. The End Replication Solutions for PostgreSQL 32