Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Linas Virbalas                    Continuent, Inc.© Continuent 2010
/  Introductions  /  What is Tungsten?  /  PG 9: Moving from Warm to Hot Standby  /  Tungsten + PostgreSQL Hot Standby    ...
© Continuent 2010
/  Our Value: Data reliability for data-driven businesses           •  Enterprise capability           •  Open source cost...
Data-Driven Businesses                    and Their Discontents© Continuent 2010
Challenge 1: Business Growth  /  Growing business = growing transaction and data     volumes  /  Non-linear labor cost inc...
Challenge 2: True High Availability               Fault tolerance alone is no longer                 sufficient for high a...
Challenge 3: Complex Data Management  /  Data-driven businesses have “big league” topologies  /  Multi-master and cross-si...
What is Tungsten and How                    Does It Work?© Continuent 2010
What Is Tungsten?  /  Tungsten creates integrated data services     using of-the-shelf PostgreSQL/MySQL     databases to: ...
What’s Inside Tungsten?  /  Replication - Making copies           •  Tungsten Replicator -- Database-neutral, platform ind...
Apache/Mod_PHP                      Apache/Mod_PHP                      Manager     Connector                Connector    ...
Technology: Replication Pipelines© Continuent 2010
Technology: SQL Routing                    Java App Server       PHP Application                    Tungsten SQL Router   ...
Technology: Distributed Management                       Admin Client                                                     ...
© Continuent 2010
PostgreSQL 8.4 Warm Standby© Continuent 2010
Limitations of Warm Standby  1.  Utilization -- Cannot open the standby           •        To bring up the standby for que...
Introducing Hot Standby  /  Allows users to connect to standby in read-only mode           •  Allowed: SELECT, SET, LOAD, ...
Introducing Log Streaming© Continuent 2010
Configuration and Usage  /  Log streaming layers on top of existing warm standby     log shipping  /  Multiple standby ser...
© Continuent 2010
Moving Tungsten to PostgreSQL  /  Problem: We can’t read PostgreSQL logs (yet)                      Tungsten solution is t...
What is Tungsten’s Added Value To  PostgreSQL?  /  15 minute cluster installation  /  Single commands to:           •  Vie...
How Tungsten covers all this?                                 Tungsten Manager                     Replicator JMX Interfac...
© Continuent 2010
Master         Slave                    Replicator    Replicator                    Transaction   Transaction             ...
© Continuent 2010
© Continuent 2010
HQ and Americas                      EMEA and APAC   560 S. Winchester Blvd., Suite 500   Lars Sonckin kaari 16   San Jose...
Upcoming SlideShare
Loading in …5
×

Building Tungsten Clusters with PostgreSQL Hot Standby and Streaming Replication

3,628 views

Published on

Hot standby and streaming replication move the needle forward for high availability and scaling for a wide number of applications. Continuent Tungsten has earlier supported PostgreSQL clustering using warm standby. In this talk we will describe how to build clusters using the PostgreSQL 9 features and give our report from the trenches.

This talk will cover how PostgreSQL 9 hot standby and streaming replication work from a user perspective, then dive into a description of how Tungsten really makes them to shine.

We'll cover the following issues:

* Configuration of warm standby and streaming replication
* Provisioning new standby instances
* Strategies for balancing reads across primary and standby database
* Managing failover

Please join us for an enlightening presentation a set of PostgreSQL 9 features that are interesting to a wide range of PostgreSQL users.

Published in: Technology
  • Be the first to comment

Building Tungsten Clusters with PostgreSQL Hot Standby and Streaming Replication

  1. 1. Linas Virbalas Continuent, Inc.© Continuent 2010
  2. 2. /  Introductions /  What is Tungsten? /  PG 9: Moving from Warm to Hot Standby /  Tungsten + PostgreSQL Hot Standby and Streaming Replication /  Tungsten’s Feature: MySQL to PostgreSQL Replication /  Demo! /  Questions and Answers© Continuent 2010
  3. 3. © Continuent 2010
  4. 4. /  Our Value: Data reliability for data-driven businesses •  Enterprise capability •  Open source cost •  Integrated solution /  Our Solution: Tungsten Database Clustering /  Our Company: •  Founded in Finland, HQ in USA •  Venture-backed •  US/European presence •  Deep experience in databases and distributed systems© Continuent 2010
  5. 5. Data-Driven Businesses and Their Discontents© Continuent 2010
  6. 6. Challenge 1: Business Growth /  Growing business = growing transaction and data volumes /  Non-linear labor cost increases /  Limits in the amount of data and machines that you can practically manage© Continuent 2010
  7. 7. Challenge 2: True High Availability Fault tolerance alone is no longer sufficient for high availability The number one cause of downtime in many applications is scheduled database maintenance and upgrades© Continuent 2010
  8. 8. Challenge 3: Complex Data Management /  Data-driven businesses have “big league” topologies /  Multi-master and cross-site flows are standard Shared Data Shared Data Tenant and Tenant Tenant Shared Data Apps Apps Tenant Tenant Data Data Tenant Tenant Analytics Analytics© Continuent 2010
  9. 9. What is Tungsten and How Does It Work?© Continuent 2010
  10. 10. What Is Tungsten? /  Tungsten creates integrated data services using of-the-shelf PostgreSQL/MySQL databases to: • Protect data • Keep applications constantly available • Improve resource utilization • Raise performance© Continuent 2010
  11. 11. What’s Inside Tungsten? /  Replication - Making copies •  Tungsten Replicator -- Database-neutral, platform independent master/slave replication /  Connectivity -- Finding databases •  Tungsten Connector -- Fast MySQL/PostgreSQL client to JDBC proxying •  Tungsten SQL Router --JDBC wrapper for high-performance and transparent failover, load-balancing, and partitioning (no proxy required) /  Management -- Administering the database •  Tungsten Manager -- Distributed administration with autonomic, rule-based configuration and no single point of failure •  Tungsten Monitor -- Track resource status and© Continuent 2010
  12. 12. Apache/Mod_PHP Apache/Mod_PHP Manager Connector Connector Manager Replicator Replicator Replicator Manager Manager Manager© Continuent 2010
  13. 13. Technology: Replication Pipelines© Continuent 2010
  14. 14. Technology: SQL Routing Java App Server PHP Application Tungsten SQL Router libmysqlclient.a MySQL JDBC Driver Tungsten Connector Clustered Databases© Continuent 2010
  15. 15. Technology: Distributed Management Admin Client Manager Manager (Coordinator) Admin Client Manager Admin Client© Continuent 2010
  16. 16. © Continuent 2010
  17. 17. PostgreSQL 8.4 Warm Standby© Continuent 2010
  18. 18. Limitations of Warm Standby 1.  Utilization -- Cannot open the standby •  To bring up the standby for queries you must end recovery •  Standby hardware is idle •  Difficult to track state of recovery since you cannot query log position 2.  Data Loss -- Warm standby transfers only full WAL files •  Can bound loss using archive_timeout •  Low values create large numbers of WAL files; complicate point- in-time recovery •  Workarounds using DRBD, etc. are complex© Continuent 2010
  19. 19. Introducing Hot Standby /  Allows users to connect to standby in read-only mode •  Allowed: SELECT, SET, LOAD, COMMIT/ROLLBACK •  Disallowed: INSERT, UPDATE, DELETE, CREATE, 2PC, SELECT … FOR SHARE/UPDATE, nextval(), LISTEN, LOCK, •  No admin commands: ANALYZE, VACUUM, REINDEX, GRANT /  Can come out of recovery while queries are running© Continuent 2010
  20. 20. Introducing Log Streaming© Continuent 2010
  21. 21. Configuration and Usage /  Log streaming layers on top of existing warm standby log shipping /  Multiple standby servers allowed /  Failure of one standby does not affect others /  Manual management is not simple - must coordinate provisioning & WAL shipping to set up/restart© Continuent 2010
  22. 22. © Continuent 2010
  23. 23. Moving Tungsten to PostgreSQL /  Problem: We can’t read PostgreSQL logs (yet) Tungsten solution is to manage: /  Warm Standby + WAL Shipping (for PostgreSQL <9) •  Good basic availability/fast failover •  Slaves open up for reads only after failover •  No load balancing/scaling /  Hot Standby + Streaming Replication (for PostgreSQL >=9) •  Slaves opened up for reads = Tungsten scaling facilities work •  Add Streaming Replication = minimal delay in replicating data •  With these features available, Tungsten brings true scale-out solution for PostgreSQL 9. With Tungsten PostgreSQL users gain high availability and scalability MySQL users have been able to enjoy over the years© Continuent 2010
  24. 24. What is Tungsten’s Added Value To PostgreSQL? /  15 minute cluster installation /  Single commands to: •  View cluster status •  Provision a new standby •  Confirm liveness of replication •  Switch servers safely for maintenance •  Failover a dead server to most current replica /  Automatic discovery of new database replicas /  Automatic failover when databases fail /  Simple procedures for provisioning /  Transparent application routing /  Easy scaling© Continuent 2010
  25. 25. How Tungsten covers all this? Tungsten Manager Replicator JMX Interface Monitor Replication State Model Open Script Plugin© Continuent 2010
  26. 26. © Continuent 2010
  27. 27. Master Slave Replicator Replicator Transaction Transaction History Log History Log Filters Filters MySQL PostgreSQL Extractor Applier© Continuent 2010
  28. 28. © Continuent 2010
  29. 29. © Continuent 2010
  30. 30. HQ and Americas EMEA and APAC 560 S. Winchester Blvd., Suite 500 Lars Sonckin kaari 16 San Jose, CA 95128 02600 Espoo, Finland Tel (866) 998-3642 Tel +358 50 517 9059 Fax (408) 668-1009 Fax +358 9 863 0060 e-mail: sales@continuent.com Continuent Web Site: http://www.continuent.com© Continuent 2010

×