Your SlideShare is downloading. ×
0
Linas Virbalas                    Continuent, Inc.© Continuent 2010
/  Introductions  /  What is Tungsten?  /  PG 9: Moving from Warm to Hot Standby  /  Tungsten + PostgreSQL Hot Standby    ...
© Continuent 2010
/  Our Value: Data reliability for data-driven businesses           •  Enterprise capability           •  Open source cost...
Data-Driven Businesses                    and Their Discontents© Continuent 2010
Challenge 1: Business Growth  /  Growing business = growing transaction and data     volumes  /  Non-linear labor cost inc...
Challenge 2: True High Availability               Fault tolerance alone is no longer                 sufficient for high a...
Challenge 3: Complex Data Management  /  Data-driven businesses have “big league” topologies  /  Multi-master and cross-si...
What is Tungsten and How                    Does It Work?© Continuent 2010
What Is Tungsten?  /  Tungsten creates integrated data services     using of-the-shelf PostgreSQL/MySQL     databases to: ...
What’s Inside Tungsten?  /  Replication - Making copies           •  Tungsten Replicator -- Database-neutral, platform ind...
Apache/Mod_PHP                      Apache/Mod_PHP                      Manager     Connector                Connector    ...
Technology: Replication Pipelines© Continuent 2010
Technology: SQL Routing                    Java App Server       PHP Application                    Tungsten SQL Router   ...
Technology: Distributed Management                       Admin Client                                                     ...
© Continuent 2010
PostgreSQL 8.4 Warm Standby© Continuent 2010
Limitations of Warm Standby  1.  Utilization -- Cannot open the standby           •        To bring up the standby for que...
Introducing Hot Standby  /  Allows users to connect to standby in read-only mode           •  Allowed: SELECT, SET, LOAD, ...
Introducing Log Streaming© Continuent 2010
Configuration and Usage  /  Log streaming layers on top of existing warm standby     log shipping  /  Multiple standby ser...
© Continuent 2010
Moving Tungsten to PostgreSQL  /  Problem: We can’t read PostgreSQL logs (yet)                      Tungsten solution is t...
What is Tungsten’s Added Value To  PostgreSQL?  /  15 minute cluster installation  /  Single commands to:           •  Vie...
How Tungsten covers all this?                                 Tungsten Manager                     Replicator JMX Interfac...
© Continuent 2010
Master         Slave                    Replicator    Replicator                    Transaction   Transaction             ...
© Continuent 2010
© Continuent 2010
HQ and Americas                      EMEA and APAC   560 S. Winchester Blvd., Suite 500   Lars Sonckin kaari 16   San Jose...
Upcoming SlideShare
Loading in...5
×

Building Tungsten Clusters with PostgreSQL Hot Standby and Streaming Replication

2,877

Published on

Hot standby and streaming replication move the needle forward for high availability and scaling for a wide number of applications. Continuent Tungsten has earlier supported PostgreSQL clustering using warm standby. In this talk we will describe how to build clusters using the PostgreSQL 9 features and give our report from the trenches.

This talk will cover how PostgreSQL 9 hot standby and streaming replication work from a user perspective, then dive into a description of how Tungsten really makes them to shine.

We'll cover the following issues:

* Configuration of warm standby and streaming replication
* Provisioning new standby instances
* Strategies for balancing reads across primary and standby database
* Managing failover

Please join us for an enlightening presentation a set of PostgreSQL 9 features that are interesting to a wide range of PostgreSQL users.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,877
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
46
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Building Tungsten Clusters with PostgreSQL Hot Standby and Streaming Replication"

  1. 1. Linas Virbalas Continuent, Inc.© Continuent 2010
  2. 2. /  Introductions /  What is Tungsten? /  PG 9: Moving from Warm to Hot Standby /  Tungsten + PostgreSQL Hot Standby and Streaming Replication /  Tungsten’s Feature: MySQL to PostgreSQL Replication /  Demo! /  Questions and Answers© Continuent 2010
  3. 3. © Continuent 2010
  4. 4. /  Our Value: Data reliability for data-driven businesses •  Enterprise capability •  Open source cost •  Integrated solution /  Our Solution: Tungsten Database Clustering /  Our Company: •  Founded in Finland, HQ in USA •  Venture-backed •  US/European presence •  Deep experience in databases and distributed systems© Continuent 2010
  5. 5. Data-Driven Businesses and Their Discontents© Continuent 2010
  6. 6. Challenge 1: Business Growth /  Growing business = growing transaction and data volumes /  Non-linear labor cost increases /  Limits in the amount of data and machines that you can practically manage© Continuent 2010
  7. 7. Challenge 2: True High Availability Fault tolerance alone is no longer sufficient for high availability The number one cause of downtime in many applications is scheduled database maintenance and upgrades© Continuent 2010
  8. 8. Challenge 3: Complex Data Management /  Data-driven businesses have “big league” topologies /  Multi-master and cross-site flows are standard Shared Data Shared Data Tenant and Tenant Tenant Shared Data Apps Apps Tenant Tenant Data Data Tenant Tenant Analytics Analytics© Continuent 2010
  9. 9. What is Tungsten and How Does It Work?© Continuent 2010
  10. 10. What Is Tungsten? /  Tungsten creates integrated data services using of-the-shelf PostgreSQL/MySQL databases to: • Protect data • Keep applications constantly available • Improve resource utilization • Raise performance© Continuent 2010
  11. 11. What’s Inside Tungsten? /  Replication - Making copies •  Tungsten Replicator -- Database-neutral, platform independent master/slave replication /  Connectivity -- Finding databases •  Tungsten Connector -- Fast MySQL/PostgreSQL client to JDBC proxying •  Tungsten SQL Router --JDBC wrapper for high-performance and transparent failover, load-balancing, and partitioning (no proxy required) /  Management -- Administering the database •  Tungsten Manager -- Distributed administration with autonomic, rule-based configuration and no single point of failure •  Tungsten Monitor -- Track resource status and© Continuent 2010
  12. 12. Apache/Mod_PHP Apache/Mod_PHP Manager Connector Connector Manager Replicator Replicator Replicator Manager Manager Manager© Continuent 2010
  13. 13. Technology: Replication Pipelines© Continuent 2010
  14. 14. Technology: SQL Routing Java App Server PHP Application Tungsten SQL Router libmysqlclient.a MySQL JDBC Driver Tungsten Connector Clustered Databases© Continuent 2010
  15. 15. Technology: Distributed Management Admin Client Manager Manager (Coordinator) Admin Client Manager Admin Client© Continuent 2010
  16. 16. © Continuent 2010
  17. 17. PostgreSQL 8.4 Warm Standby© Continuent 2010
  18. 18. Limitations of Warm Standby 1.  Utilization -- Cannot open the standby •  To bring up the standby for queries you must end recovery •  Standby hardware is idle •  Difficult to track state of recovery since you cannot query log position 2.  Data Loss -- Warm standby transfers only full WAL files •  Can bound loss using archive_timeout •  Low values create large numbers of WAL files; complicate point- in-time recovery •  Workarounds using DRBD, etc. are complex© Continuent 2010
  19. 19. Introducing Hot Standby /  Allows users to connect to standby in read-only mode •  Allowed: SELECT, SET, LOAD, COMMIT/ROLLBACK •  Disallowed: INSERT, UPDATE, DELETE, CREATE, 2PC, SELECT … FOR SHARE/UPDATE, nextval(), LISTEN, LOCK, •  No admin commands: ANALYZE, VACUUM, REINDEX, GRANT /  Can come out of recovery while queries are running© Continuent 2010
  20. 20. Introducing Log Streaming© Continuent 2010
  21. 21. Configuration and Usage /  Log streaming layers on top of existing warm standby log shipping /  Multiple standby servers allowed /  Failure of one standby does not affect others /  Manual management is not simple - must coordinate provisioning & WAL shipping to set up/restart© Continuent 2010
  22. 22. © Continuent 2010
  23. 23. Moving Tungsten to PostgreSQL /  Problem: We can’t read PostgreSQL logs (yet) Tungsten solution is to manage: /  Warm Standby + WAL Shipping (for PostgreSQL <9) •  Good basic availability/fast failover •  Slaves open up for reads only after failover •  No load balancing/scaling /  Hot Standby + Streaming Replication (for PostgreSQL >=9) •  Slaves opened up for reads = Tungsten scaling facilities work •  Add Streaming Replication = minimal delay in replicating data •  With these features available, Tungsten brings true scale-out solution for PostgreSQL 9. With Tungsten PostgreSQL users gain high availability and scalability MySQL users have been able to enjoy over the years© Continuent 2010
  24. 24. What is Tungsten’s Added Value To PostgreSQL? /  15 minute cluster installation /  Single commands to: •  View cluster status •  Provision a new standby •  Confirm liveness of replication •  Switch servers safely for maintenance •  Failover a dead server to most current replica /  Automatic discovery of new database replicas /  Automatic failover when databases fail /  Simple procedures for provisioning /  Transparent application routing /  Easy scaling© Continuent 2010
  25. 25. How Tungsten covers all this? Tungsten Manager Replicator JMX Interface Monitor Replication State Model Open Script Plugin© Continuent 2010
  26. 26. © Continuent 2010
  27. 27. Master Slave Replicator Replicator Transaction Transaction History Log History Log Filters Filters MySQL PostgreSQL Extractor Applier© Continuent 2010
  28. 28. © Continuent 2010
  29. 29. © Continuent 2010
  30. 30. HQ and Americas EMEA and APAC 560 S. Winchester Blvd., Suite 500 Lars Sonckin kaari 16 San Jose, CA 95128 02600 Espoo, Finland Tel (866) 998-3642 Tel +358 50 517 9059 Fax (408) 668-1009 Fax +358 9 863 0060 e-mail: sales@continuent.com Continuent Web Site: http://www.continuent.com© Continuent 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×