Building Tungsten Clusters with PostgreSQL Hot Standby and Streaming Replication

3,369 views

Published on

Hot standby and streaming replication move the needle forward for high availability and scaling for a wide number of applications. Continuent Tungsten has earlier supported PostgreSQL clustering using warm standby. In this talk we will describe how to build clusters using the PostgreSQL 9 features and give our report from the trenches.

This talk will cover how PostgreSQL 9 hot standby and streaming replication work from a user perspective, then dive into a description of how Tungsten really makes them to shine.

We'll cover the following issues:

* Configuration of warm standby and streaming replication
* Provisioning new standby instances
* Strategies for balancing reads across primary and standby database
* Managing failover

Please join us for an enlightening presentation a set of PostgreSQL 9 features that are interesting to a wide range of PostgreSQL users.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,369
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
54
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Building Tungsten Clusters with PostgreSQL Hot Standby and Streaming Replication

  1. 1. Linas Virbalas Continuent, Inc.© Continuent 2010
  2. 2. /  Introductions /  What is Tungsten? /  PG 9: Moving from Warm to Hot Standby /  Tungsten + PostgreSQL Hot Standby and Streaming Replication /  Tungsten’s Feature: MySQL to PostgreSQL Replication /  Demo! /  Questions and Answers© Continuent 2010
  3. 3. © Continuent 2010
  4. 4. /  Our Value: Data reliability for data-driven businesses •  Enterprise capability •  Open source cost •  Integrated solution /  Our Solution: Tungsten Database Clustering /  Our Company: •  Founded in Finland, HQ in USA •  Venture-backed •  US/European presence •  Deep experience in databases and distributed systems© Continuent 2010
  5. 5. Data-Driven Businesses and Their Discontents© Continuent 2010
  6. 6. Challenge 1: Business Growth /  Growing business = growing transaction and data volumes /  Non-linear labor cost increases /  Limits in the amount of data and machines that you can practically manage© Continuent 2010
  7. 7. Challenge 2: True High Availability Fault tolerance alone is no longer sufficient for high availability The number one cause of downtime in many applications is scheduled database maintenance and upgrades© Continuent 2010
  8. 8. Challenge 3: Complex Data Management /  Data-driven businesses have “big league” topologies /  Multi-master and cross-site flows are standard Shared Data Shared Data Tenant and Tenant Tenant Shared Data Apps Apps Tenant Tenant Data Data Tenant Tenant Analytics Analytics© Continuent 2010
  9. 9. What is Tungsten and How Does It Work?© Continuent 2010
  10. 10. What Is Tungsten? /  Tungsten creates integrated data services using of-the-shelf PostgreSQL/MySQL databases to: • Protect data • Keep applications constantly available • Improve resource utilization • Raise performance© Continuent 2010
  11. 11. What’s Inside Tungsten? /  Replication - Making copies •  Tungsten Replicator -- Database-neutral, platform independent master/slave replication /  Connectivity -- Finding databases •  Tungsten Connector -- Fast MySQL/PostgreSQL client to JDBC proxying •  Tungsten SQL Router --JDBC wrapper for high-performance and transparent failover, load-balancing, and partitioning (no proxy required) /  Management -- Administering the database •  Tungsten Manager -- Distributed administration with autonomic, rule-based configuration and no single point of failure •  Tungsten Monitor -- Track resource status and© Continuent 2010
  12. 12. Apache/Mod_PHP Apache/Mod_PHP Manager Connector Connector Manager Replicator Replicator Replicator Manager Manager Manager© Continuent 2010
  13. 13. Technology: Replication Pipelines© Continuent 2010
  14. 14. Technology: SQL Routing Java App Server PHP Application Tungsten SQL Router libmysqlclient.a MySQL JDBC Driver Tungsten Connector Clustered Databases© Continuent 2010
  15. 15. Technology: Distributed Management Admin Client Manager Manager (Coordinator) Admin Client Manager Admin Client© Continuent 2010
  16. 16. © Continuent 2010
  17. 17. PostgreSQL 8.4 Warm Standby© Continuent 2010
  18. 18. Limitations of Warm Standby 1.  Utilization -- Cannot open the standby •  To bring up the standby for queries you must end recovery •  Standby hardware is idle •  Difficult to track state of recovery since you cannot query log position 2.  Data Loss -- Warm standby transfers only full WAL files •  Can bound loss using archive_timeout •  Low values create large numbers of WAL files; complicate point- in-time recovery •  Workarounds using DRBD, etc. are complex© Continuent 2010
  19. 19. Introducing Hot Standby /  Allows users to connect to standby in read-only mode •  Allowed: SELECT, SET, LOAD, COMMIT/ROLLBACK •  Disallowed: INSERT, UPDATE, DELETE, CREATE, 2PC, SELECT … FOR SHARE/UPDATE, nextval(), LISTEN, LOCK, •  No admin commands: ANALYZE, VACUUM, REINDEX, GRANT /  Can come out of recovery while queries are running© Continuent 2010
  20. 20. Introducing Log Streaming© Continuent 2010
  21. 21. Configuration and Usage /  Log streaming layers on top of existing warm standby log shipping /  Multiple standby servers allowed /  Failure of one standby does not affect others /  Manual management is not simple - must coordinate provisioning & WAL shipping to set up/restart© Continuent 2010
  22. 22. © Continuent 2010
  23. 23. Moving Tungsten to PostgreSQL /  Problem: We can’t read PostgreSQL logs (yet) Tungsten solution is to manage: /  Warm Standby + WAL Shipping (for PostgreSQL <9) •  Good basic availability/fast failover •  Slaves open up for reads only after failover •  No load balancing/scaling /  Hot Standby + Streaming Replication (for PostgreSQL >=9) •  Slaves opened up for reads = Tungsten scaling facilities work •  Add Streaming Replication = minimal delay in replicating data •  With these features available, Tungsten brings true scale-out solution for PostgreSQL 9. With Tungsten PostgreSQL users gain high availability and scalability MySQL users have been able to enjoy over the years© Continuent 2010
  24. 24. What is Tungsten’s Added Value To PostgreSQL? /  15 minute cluster installation /  Single commands to: •  View cluster status •  Provision a new standby •  Confirm liveness of replication •  Switch servers safely for maintenance •  Failover a dead server to most current replica /  Automatic discovery of new database replicas /  Automatic failover when databases fail /  Simple procedures for provisioning /  Transparent application routing /  Easy scaling© Continuent 2010
  25. 25. How Tungsten covers all this? Tungsten Manager Replicator JMX Interface Monitor Replication State Model Open Script Plugin© Continuent 2010
  26. 26. © Continuent 2010
  27. 27. Master Slave Replicator Replicator Transaction Transaction History Log History Log Filters Filters MySQL PostgreSQL Extractor Applier© Continuent 2010
  28. 28. © Continuent 2010
  29. 29. © Continuent 2010
  30. 30. HQ and Americas EMEA and APAC 560 S. Winchester Blvd., Suite 500 Lars Sonckin kaari 16 San Jose, CA 95128 02600 Espoo, Finland Tel (866) 998-3642 Tel +358 50 517 9059 Fax (408) 668-1009 Fax +358 9 863 0060 e-mail: sales@continuent.com Continuent Web Site: http://www.continuent.com© Continuent 2010

×