SlideShare a Scribd company logo
1 of 32
Download to read offline
Replication Solutions for
            PostgreSQL
             Peter Eisentraut
          petere@postgresql.org
What's in a Term?
•   Replication?
•   Clustering?
•   High availability?
•   Failover?
•   Standby?

• Putting data on more than one computer

                                           2
Space of Possibilities
• Goals
  • What do you want to achieve?
• Techniques
  • How can this be implemented?
• Solutions
  • What software is available to do this?




                                             3
Goals
• High availability
• Performance
  • Read
  • Write
• Wide-area networks
• Offline peers



                              4
Goal: High Availability
• No one wants “low availability”!
• Provisions for system failures
  • Software faults
  • Hardware faults
  • External interference




                                     5
Goal: Read Performance
• Applications with:
  • many readers (e.g., library information system)
  • resource-intensive readers (e.g., data
    warehousing)
• Distribute readers on more hardware.
• Most often, one physical machine is
  enough.



                                                  6
Goal: Write Performance
• Applications with:
  • Many writers
• Distribute writers on more hardware?
  • Constraint checking, conflict resolution?!?
• Faster writing contradicts replication.
  • Partition, don't replicate!
  • RAID 0/striping is not replication – it makes
    things “worse”.
  • RAID 10 is a good idea, but not the topic here.
                                                  7
Goal: Wide-Area Networks
• Faster access across WANs
• Reading?
  • Local copies
• Writing?
  • Synchronization?




                               8
Goal: Offline Peers
• Synchronize data with laptops,
  handhelds, ...
• “Road warriors”
• May be considered very-low-latency WANs




                                        9
Techniques
• Replication
  • Master/Slave
     • Asynchronous
     • Synchronous
  • Multi-Master
     • Asynchronous
     • Synchronous
• Proxy
• Standby system
                             10
Technique: Replication
   Master/Slave Asynchronous
• High(er) availability(?)
• Read performance
  • Load spreading,
    load balancing
• Offline peers              M   asy c
                                    n    S
  (unidirectional sync.)




                                             11
Technique: Replication
    Master/Slave Synchronous
• High availibility
• Better read
  performance
• Worse write
  performance         M   sy c
                            n    S




                                     12
Technique: Replication
   Multi-Master Asynchronous
• Read performance
• Faster access across
  WANs
• Manage offline peers
                         M           M
• Requires conflict          asy c
                                n


  resolution mechanism



                                         13
Technique: Replication
       Multi-Master Synchronous
•   “Holy grail of replication”
•   High availability
•   Read performance
•   Difficult to get good
                                  M          M
    write performance                 sy c
                                        n




                                                 14
Technique: Proxy
• High availability
• Read performance
• Proxy instance should       Proxy
  be redundant
• Transparent to the
  application
                          C           C



                                          15
Technique: Standby System
• High availability




                      M   sy c
                            n    S




                                     16
Constraints
• Hardware
• Operating system
• Application




                           17
No Built-In Solution?
• FIXME




                              18
Solutions
•   Slony-I, -II
•   PGCluster
•   DBMirror
•   pgpool
•   WAL replication
•   Sequoia
•   DRBD
•   Shared storage
                             19
Solution: Slony-I
(Slony ← слоны ← elephants)
• Asynchronous master/slave replication
• Multiple slaves, cascading possible
• Particularly useful for:
  • Read performance (load balancing with pgpool)
  • Limited form of high availability
  • Offline slaves via file-based log shipping
http://www.slony.info/

                                               20
Solution: Slony-II
• Synchronous master/master
  replication?
• See Gavin Sherry's session for details




                                           21
Solution: PGCluster
• Synchronous master/master replication
• Replicates the query string
• Particularly useful for:
  • Load balancing
  • High availability



http://pgcluster.projects.postgresql.org/
                                            22
Solution: DBMirror
• Asynchronous master/slave replication
• Very simple (compared to Slony-I)
• Particularly useful for:
  • Read performance
  • Offline peers



contrib/dbmirror/ in PostgreSQL source tree
                                              23
Solution: pgpool
• Connection pool daemon for PostgreSQL
• Supports simple proxying
• Useful as frontend for Slony-I




http://pgpool.projects.postgresql.org/
                                          24
Solution: WAL Replication
• Use the “archived” WAL logs for “recovery”
  on a standby system
• Disadvantages:
  • Only full database cluster replication
  • Master and slave must be binary-compatible
  • Rather slow across network
• Useful for:
  • High availability

                                                 25
Solution: Sequoia
• Formerly C[lustered]-JDBC
• Proxy offering clustering, load
  balancing and failover services
• Particularly useful for:
  • High availability
  • Read performance
• Currently only for Java/JDBC applications
http://sequoia.continuent.org/

                                              26
Solution: DRBD
•   File system (block device) replication
•   Linux kernel module
•   Standby system
•   Useful for:
    • High availability
    • Secure any service, not just a database system


http://www.drbd.org/
                                                  27
Solution: Shared Storage
•   NAS, iSCSI, Fiberchannel, ...
•   Available from many vendors
•   Standby system
•   Useful for:
    • High availability
    • Secure any service, not just a database system
• Single storage system is a possible point of
  failure

                                                  28
Summary
• Plenty of solutions for diverse applications
• Make a (project) plan.




                                                 29
Suggestions
• Minimum for any production installation:
  • Sensible disk clustering
     • RAID 10
     • Tablespace management
     • Separate disk(s) for WAL
  • DRBD or shared storage
• Slony-I for load balancing or warehousing
• Java developers consider Sequoia


                                              30
Outlook
•   Slony-II
•   WAL replication management
•   XA support
•   More packaging efforts




                                 31
The End
Replication Solutions for PostgreSQL




                                       32

More Related Content

What's hot

Tuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris EnvironmentTuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris EnvironmentJignesh Shah
 
Built-in Replication in PostgreSQL
Built-in Replication in PostgreSQLBuilt-in Replication in PostgreSQL
Built-in Replication in PostgreSQLMasao Fujii
 
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)Masao Fujii
 
PostgreSQL and Linux Containers
PostgreSQL and Linux ContainersPostgreSQL and Linux Containers
PostgreSQL and Linux ContainersJignesh Shah
 
Postgres on OpenStack
Postgres on OpenStackPostgres on OpenStack
Postgres on OpenStackEDB
 
Architecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres ClusterArchitecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres ClusterAshnikbiz
 
Linux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performanceLinux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performancePostgreSQL-Consulting
 
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte DataProblems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte DataJignesh Shah
 
Best Practices with PostgreSQL on Solaris
Best Practices with PostgreSQL on SolarisBest Practices with PostgreSQL on Solaris
Best Practices with PostgreSQL on SolarisJignesh Shah
 
PostgreSQL9.3 Switchover/Switchback
PostgreSQL9.3 Switchover/SwitchbackPostgreSQL9.3 Switchover/Switchback
PostgreSQL9.3 Switchover/SwitchbackVibhor Kumar
 
Deploying Maximum HA Architecture With PostgreSQL
Deploying Maximum HA Architecture With PostgreSQLDeploying Maximum HA Architecture With PostgreSQL
Deploying Maximum HA Architecture With PostgreSQLDenish Patel
 
Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016PostgreSQL-Consulting
 
My experience with embedding PostgreSQL
 My experience with embedding PostgreSQL My experience with embedding PostgreSQL
My experience with embedding PostgreSQLJignesh Shah
 
Geographically Distributed PostgreSQL
Geographically Distributed PostgreSQLGeographically Distributed PostgreSQL
Geographically Distributed PostgreSQLmason_s
 
Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4Denish Patel
 
Deep Dive into RDS PostgreSQL Universe
Deep Dive into RDS PostgreSQL UniverseDeep Dive into RDS PostgreSQL Universe
Deep Dive into RDS PostgreSQL UniverseJignesh Shah
 
Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2 Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2 Denish Patel
 
PostgreSQL Hangout Parameter Tuning
PostgreSQL Hangout Parameter TuningPostgreSQL Hangout Parameter Tuning
PostgreSQL Hangout Parameter TuningAshnikbiz
 
PostgreSQL Disaster Recovery with Barman
PostgreSQL Disaster Recovery with BarmanPostgreSQL Disaster Recovery with Barman
PostgreSQL Disaster Recovery with BarmanGabriele Bartolini
 

What's hot (20)

Tuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris EnvironmentTuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris Environment
 
Built-in Replication in PostgreSQL
Built-in Replication in PostgreSQLBuilt-in Replication in PostgreSQL
Built-in Replication in PostgreSQL
 
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
 
PostgreSQL and Linux Containers
PostgreSQL and Linux ContainersPostgreSQL and Linux Containers
PostgreSQL and Linux Containers
 
Postgres on OpenStack
Postgres on OpenStackPostgres on OpenStack
Postgres on OpenStack
 
Architecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres ClusterArchitecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres Cluster
 
Linux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performanceLinux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performance
 
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte DataProblems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
 
Best Practices with PostgreSQL on Solaris
Best Practices with PostgreSQL on SolarisBest Practices with PostgreSQL on Solaris
Best Practices with PostgreSQL on Solaris
 
PostgreSQL9.3 Switchover/Switchback
PostgreSQL9.3 Switchover/SwitchbackPostgreSQL9.3 Switchover/Switchback
PostgreSQL9.3 Switchover/Switchback
 
Deploying Maximum HA Architecture With PostgreSQL
Deploying Maximum HA Architecture With PostgreSQLDeploying Maximum HA Architecture With PostgreSQL
Deploying Maximum HA Architecture With PostgreSQL
 
Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016
 
My experience with embedding PostgreSQL
 My experience with embedding PostgreSQL My experience with embedding PostgreSQL
My experience with embedding PostgreSQL
 
Geographically Distributed PostgreSQL
Geographically Distributed PostgreSQLGeographically Distributed PostgreSQL
Geographically Distributed PostgreSQL
 
Gluster 3.3 deep dive
Gluster 3.3 deep diveGluster 3.3 deep dive
Gluster 3.3 deep dive
 
Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4
 
Deep Dive into RDS PostgreSQL Universe
Deep Dive into RDS PostgreSQL UniverseDeep Dive into RDS PostgreSQL Universe
Deep Dive into RDS PostgreSQL Universe
 
Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2 Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2
 
PostgreSQL Hangout Parameter Tuning
PostgreSQL Hangout Parameter TuningPostgreSQL Hangout Parameter Tuning
PostgreSQL Hangout Parameter Tuning
 
PostgreSQL Disaster Recovery with Barman
PostgreSQL Disaster Recovery with BarmanPostgreSQL Disaster Recovery with Barman
PostgreSQL Disaster Recovery with Barman
 

Viewers also liked

Introduction to pg_cheat_funcs
Introduction to pg_cheat_funcsIntroduction to pg_cheat_funcs
Introduction to pg_cheat_funcsMasao Fujii
 
PostgreSQL HA
PostgreSQL   HAPostgreSQL   HA
PostgreSQL HAharoonm
 
Storage Replication in High-Performance High-Availability Environments
Storage Replication in High-Performance High-Availability EnvironmentsStorage Replication in High-Performance High-Availability Environments
Storage Replication in High-Performance High-Availability Environmentshastexo
 
High Availability Storage (susecon2016)
High Availability Storage (susecon2016)High Availability Storage (susecon2016)
High Availability Storage (susecon2016)Roger Zhou 周志强
 
HA+DRBD+Postgres - PostgresWest '08
HA+DRBD+Postgres - PostgresWest '08HA+DRBD+Postgres - PostgresWest '08
HA+DRBD+Postgres - PostgresWest '08Jesse Young
 
PostgreSQL9.0アップデート レプリケーションがやってきた!
PostgreSQL9.0アップデート レプリケーションがやってきた!PostgreSQL9.0アップデート レプリケーションがやってきた!
PostgreSQL9.0アップデート レプリケーションがやってきた!Masao Fujii
 
]project-open[ Screenshots
]project-open[ Screenshots ]project-open[ Screenshots
]project-open[ Screenshots Klaus Hofeditz
 
Drbd9 and drbdmanage_june_2016
Drbd9 and drbdmanage_june_2016Drbd9 and drbdmanage_june_2016
Drbd9 and drbdmanage_june_2016Philipp Reisner
 

Viewers also liked (9)

Introduction to pg_cheat_funcs
Introduction to pg_cheat_funcsIntroduction to pg_cheat_funcs
Introduction to pg_cheat_funcs
 
PostgreSQL HA
PostgreSQL   HAPostgreSQL   HA
PostgreSQL HA
 
Storage Replication in High-Performance High-Availability Environments
Storage Replication in High-Performance High-Availability EnvironmentsStorage Replication in High-Performance High-Availability Environments
Storage Replication in High-Performance High-Availability Environments
 
High Availability Storage (susecon2016)
High Availability Storage (susecon2016)High Availability Storage (susecon2016)
High Availability Storage (susecon2016)
 
HA+DRBD+Postgres - PostgresWest '08
HA+DRBD+Postgres - PostgresWest '08HA+DRBD+Postgres - PostgresWest '08
HA+DRBD+Postgres - PostgresWest '08
 
Development of 8.3 In India
Development of 8.3 In IndiaDevelopment of 8.3 In India
Development of 8.3 In India
 
PostgreSQL9.0アップデート レプリケーションがやってきた!
PostgreSQL9.0アップデート レプリケーションがやってきた!PostgreSQL9.0アップデート レプリケーションがやってきた!
PostgreSQL9.0アップデート レプリケーションがやってきた!
 
]project-open[ Screenshots
]project-open[ Screenshots ]project-open[ Screenshots
]project-open[ Screenshots
 
Drbd9 and drbdmanage_june_2016
Drbd9 and drbdmanage_june_2016Drbd9 and drbdmanage_june_2016
Drbd9 and drbdmanage_june_2016
 

Similar to Replication Solutions for PostgreSQL

High Scalability Toronto: Meetup #2
High Scalability Toronto: Meetup #2High Scalability Toronto: Meetup #2
High Scalability Toronto: Meetup #2ScribbleLive
 
Intro to Big Data and NoSQL
Intro to Big Data and NoSQLIntro to Big Data and NoSQL
Intro to Big Data and NoSQLDon Demcsak
 
Writing Scalable Software in Java
Writing Scalable Software in JavaWriting Scalable Software in Java
Writing Scalable Software in JavaRuben Badaró
 
Leveraging MongoDB: An Introductory Case Study
Leveraging MongoDB: An Introductory Case StudyLeveraging MongoDB: An Introductory Case Study
Leveraging MongoDB: An Introductory Case StudySean Laurent
 
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013Christopher Curtin
 
End of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph ReplicationEnd of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph ReplicationCeph Community
 
Petabyte scale on commodity infrastructure
Petabyte scale on commodity infrastructurePetabyte scale on commodity infrastructure
Petabyte scale on commodity infrastructureelliando dias
 
Java scalability considerations yogesh deshpande
Java scalability considerations   yogesh deshpandeJava scalability considerations   yogesh deshpande
Java scalability considerations yogesh deshpandeIndicThreads
 
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInJay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInLinkedIn
 
Lessons learned from running Spark on Docker
Lessons learned from running Spark on DockerLessons learned from running Spark on Docker
Lessons learned from running Spark on DockerDataWorks Summit
 
Scalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeScalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeAmir Ghaffari
 
MongoDB Case Study at NoSQL Now 2012
MongoDB Case Study at NoSQL Now 2012MongoDB Case Study at NoSQL Now 2012
MongoDB Case Study at NoSQL Now 2012Sean Laurent
 
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache KafkaBuilding Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache KafkaGuozhang Wang
 
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...Qian Lin
 
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...Nagios
 
What CloudStackers Need To Know About LINSTOR/DRBD
What CloudStackers Need To Know About LINSTOR/DRBDWhat CloudStackers Need To Know About LINSTOR/DRBD
What CloudStackers Need To Know About LINSTOR/DRBDShapeBlue
 
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)Lars Marowsky-Brée
 
Polyglot and functional (Devoxx Nov/2011)
Polyglot and functional (Devoxx Nov/2011)Polyglot and functional (Devoxx Nov/2011)
Polyglot and functional (Devoxx Nov/2011)Martijn Verburg
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsJonas Bonér
 
MongoBoston - MongoHQ
MongoBoston - MongoHQMongoBoston - MongoHQ
MongoBoston - MongoHQbenwyrosdick
 

Similar to Replication Solutions for PostgreSQL (20)

High Scalability Toronto: Meetup #2
High Scalability Toronto: Meetup #2High Scalability Toronto: Meetup #2
High Scalability Toronto: Meetup #2
 
Intro to Big Data and NoSQL
Intro to Big Data and NoSQLIntro to Big Data and NoSQL
Intro to Big Data and NoSQL
 
Writing Scalable Software in Java
Writing Scalable Software in JavaWriting Scalable Software in Java
Writing Scalable Software in Java
 
Leveraging MongoDB: An Introductory Case Study
Leveraging MongoDB: An Introductory Case StudyLeveraging MongoDB: An Introductory Case Study
Leveraging MongoDB: An Introductory Case Study
 
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
 
End of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph ReplicationEnd of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph Replication
 
Petabyte scale on commodity infrastructure
Petabyte scale on commodity infrastructurePetabyte scale on commodity infrastructure
Petabyte scale on commodity infrastructure
 
Java scalability considerations yogesh deshpande
Java scalability considerations   yogesh deshpandeJava scalability considerations   yogesh deshpande
Java scalability considerations yogesh deshpande
 
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInJay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
 
Lessons learned from running Spark on Docker
Lessons learned from running Spark on DockerLessons learned from running Spark on Docker
Lessons learned from running Spark on Docker
 
Scalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeScalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and Practice
 
MongoDB Case Study at NoSQL Now 2012
MongoDB Case Study at NoSQL Now 2012MongoDB Case Study at NoSQL Now 2012
MongoDB Case Study at NoSQL Now 2012
 
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache KafkaBuilding Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
 
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
 
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
 
What CloudStackers Need To Know About LINSTOR/DRBD
What CloudStackers Need To Know About LINSTOR/DRBDWhat CloudStackers Need To Know About LINSTOR/DRBD
What CloudStackers Need To Know About LINSTOR/DRBD
 
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
 
Polyglot and functional (Devoxx Nov/2011)
Polyglot and functional (Devoxx Nov/2011)Polyglot and functional (Devoxx Nov/2011)
Polyglot and functional (Devoxx Nov/2011)
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 
MongoBoston - MongoHQ
MongoBoston - MongoHQMongoBoston - MongoHQ
MongoBoston - MongoHQ
 

More from Peter Eisentraut

Programming with Python and PostgreSQL
Programming with Python and PostgreSQLProgramming with Python and PostgreSQL
Programming with Python and PostgreSQLPeter Eisentraut
 
Getting Started with PL/Proxy
Getting Started with PL/ProxyGetting Started with PL/Proxy
Getting Started with PL/ProxyPeter Eisentraut
 
Linux distribution for the cloud
Linux distribution for the cloudLinux distribution for the cloud
Linux distribution for the cloudPeter Eisentraut
 
Most Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL FeaturesMost Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL FeaturesPeter Eisentraut
 
Porting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQLPorting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQLPeter Eisentraut
 
Porting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQLPorting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQLPeter Eisentraut
 
XML Support: Specifications and Development
XML Support: Specifications and DevelopmentXML Support: Specifications and Development
XML Support: Specifications and DevelopmentPeter Eisentraut
 
PostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie DatenbankalternativePostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie DatenbankalternativePeter Eisentraut
 
The Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future DevelopmentsThe Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future DevelopmentsPeter Eisentraut
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsPeter Eisentraut
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsPeter Eisentraut
 
Docbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XMLDocbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XMLPeter Eisentraut
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...Peter Eisentraut
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...Peter Eisentraut
 
The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)Peter Eisentraut
 

More from Peter Eisentraut (20)

Programming with Python and PostgreSQL
Programming with Python and PostgreSQLProgramming with Python and PostgreSQL
Programming with Python and PostgreSQL
 
Getting Started with PL/Proxy
Getting Started with PL/ProxyGetting Started with PL/Proxy
Getting Started with PL/Proxy
 
Linux distribution for the cloud
Linux distribution for the cloudLinux distribution for the cloud
Linux distribution for the cloud
 
Most Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL FeaturesMost Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL Features
 
Porting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQLPorting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQL
 
Porting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQLPorting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQL
 
PostgreSQL and XML
PostgreSQL and XMLPostgreSQL and XML
PostgreSQL and XML
 
XML Support: Specifications and Development
XML Support: Specifications and DevelopmentXML Support: Specifications and Development
XML Support: Specifications and Development
 
PostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie DatenbankalternativePostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie Datenbankalternative
 
The Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future DevelopmentsThe Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future Developments
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-Frontends
 
PostgreSQL and PL/Java
PostgreSQL and PL/JavaPostgreSQL and PL/Java
PostgreSQL and PL/Java
 
PostgreSQL News
PostgreSQL NewsPostgreSQL News
PostgreSQL News
 
PostgreSQL News
PostgreSQL NewsPostgreSQL News
PostgreSQL News
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-Frontends
 
Docbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XMLDocbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XML
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...
 
Spaß mit PostgreSQL
Spaß mit PostgreSQLSpaß mit PostgreSQL
Spaß mit PostgreSQL
 
The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)
 

Recently uploaded

Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 

Recently uploaded (20)

Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 

Replication Solutions for PostgreSQL

  • 1. Replication Solutions for PostgreSQL Peter Eisentraut petere@postgresql.org
  • 2. What's in a Term? • Replication? • Clustering? • High availability? • Failover? • Standby? • Putting data on more than one computer 2
  • 3. Space of Possibilities • Goals • What do you want to achieve? • Techniques • How can this be implemented? • Solutions • What software is available to do this? 3
  • 4. Goals • High availability • Performance • Read • Write • Wide-area networks • Offline peers 4
  • 5. Goal: High Availability • No one wants “low availability”! • Provisions for system failures • Software faults • Hardware faults • External interference 5
  • 6. Goal: Read Performance • Applications with: • many readers (e.g., library information system) • resource-intensive readers (e.g., data warehousing) • Distribute readers on more hardware. • Most often, one physical machine is enough. 6
  • 7. Goal: Write Performance • Applications with: • Many writers • Distribute writers on more hardware? • Constraint checking, conflict resolution?!? • Faster writing contradicts replication. • Partition, don't replicate! • RAID 0/striping is not replication – it makes things “worse”. • RAID 10 is a good idea, but not the topic here. 7
  • 8. Goal: Wide-Area Networks • Faster access across WANs • Reading? • Local copies • Writing? • Synchronization? 8
  • 9. Goal: Offline Peers • Synchronize data with laptops, handhelds, ... • “Road warriors” • May be considered very-low-latency WANs 9
  • 10. Techniques • Replication • Master/Slave • Asynchronous • Synchronous • Multi-Master • Asynchronous • Synchronous • Proxy • Standby system 10
  • 11. Technique: Replication Master/Slave Asynchronous • High(er) availability(?) • Read performance • Load spreading, load balancing • Offline peers M asy c n S (unidirectional sync.) 11
  • 12. Technique: Replication Master/Slave Synchronous • High availibility • Better read performance • Worse write performance M sy c n S 12
  • 13. Technique: Replication Multi-Master Asynchronous • Read performance • Faster access across WANs • Manage offline peers M M • Requires conflict asy c n resolution mechanism 13
  • 14. Technique: Replication Multi-Master Synchronous • “Holy grail of replication” • High availability • Read performance • Difficult to get good M M write performance sy c n 14
  • 15. Technique: Proxy • High availability • Read performance • Proxy instance should Proxy be redundant • Transparent to the application C C 15
  • 16. Technique: Standby System • High availability M sy c n S 16
  • 17. Constraints • Hardware • Operating system • Application 17
  • 19. Solutions • Slony-I, -II • PGCluster • DBMirror • pgpool • WAL replication • Sequoia • DRBD • Shared storage 19
  • 20. Solution: Slony-I (Slony ← слоны ← elephants) • Asynchronous master/slave replication • Multiple slaves, cascading possible • Particularly useful for: • Read performance (load balancing with pgpool) • Limited form of high availability • Offline slaves via file-based log shipping http://www.slony.info/ 20
  • 21. Solution: Slony-II • Synchronous master/master replication? • See Gavin Sherry's session for details 21
  • 22. Solution: PGCluster • Synchronous master/master replication • Replicates the query string • Particularly useful for: • Load balancing • High availability http://pgcluster.projects.postgresql.org/ 22
  • 23. Solution: DBMirror • Asynchronous master/slave replication • Very simple (compared to Slony-I) • Particularly useful for: • Read performance • Offline peers contrib/dbmirror/ in PostgreSQL source tree 23
  • 24. Solution: pgpool • Connection pool daemon for PostgreSQL • Supports simple proxying • Useful as frontend for Slony-I http://pgpool.projects.postgresql.org/ 24
  • 25. Solution: WAL Replication • Use the “archived” WAL logs for “recovery” on a standby system • Disadvantages: • Only full database cluster replication • Master and slave must be binary-compatible • Rather slow across network • Useful for: • High availability 25
  • 26. Solution: Sequoia • Formerly C[lustered]-JDBC • Proxy offering clustering, load balancing and failover services • Particularly useful for: • High availability • Read performance • Currently only for Java/JDBC applications http://sequoia.continuent.org/ 26
  • 27. Solution: DRBD • File system (block device) replication • Linux kernel module • Standby system • Useful for: • High availability • Secure any service, not just a database system http://www.drbd.org/ 27
  • 28. Solution: Shared Storage • NAS, iSCSI, Fiberchannel, ... • Available from many vendors • Standby system • Useful for: • High availability • Secure any service, not just a database system • Single storage system is a possible point of failure 28
  • 29. Summary • Plenty of solutions for diverse applications • Make a (project) plan. 29
  • 30. Suggestions • Minimum for any production installation: • Sensible disk clustering • RAID 10 • Tablespace management • Separate disk(s) for WAL • DRBD or shared storage • Slony-I for load balancing or warehousing • Java developers consider Sequoia 30
  • 31. Outlook • Slony-II • WAL replication management • XA support • More packaging efforts 31
  • 32. The End Replication Solutions for PostgreSQL 32