Downloaded 62 times
Building Tungsten Clusters with PostgreSQL Hot Standby and Streaming Replication
- 1. Linas Virbalas Continuent, Inc. © Continuent 2010
- 2. / Introductions / What is Tungsten? / PG 9: Moving from Warm to Hot Standby / Tungsten + PostgreSQL Hot Standby and Streaming Replication / Tungsten’s Feature: MySQL to PostgreSQL Replication / Demo! / Questions and Answers © Continuent 2010
- 3.
- 4. / Our Value:Data reliability for data-driven businesses • Enterprise capability • Open source cost • Integrated solution / Our Solution: Tungsten Database Clustering / Our Company: • Founded in Finland, HQ in USA • Venture-backed • US/European presence • Deep experience in databases and distributed systems © Continuent 2010
- 5. Data-Driven Businesses and Their Discontents © Continuent 2010
- 6. Challenge 1: BusinessGrowth / Growing business = growing transaction and data volumes / Non-linear labor cost increases / Limits in the amount of data and machines that you can practically manage © Continuent 2010
- 7. Challenge 2: TrueHigh Availability Fault tolerance alone is no longer sufficient for high availability The number one cause of downtime in many applications is scheduled database maintenance and upgrades © Continuent 2010
- 8. Challenge 3: ComplexData Management / Data-driven businesses have “big league” topologies / Multi-master and cross-site flows are standard Shared Data Shared Data Tenant and Tenant Tenant Shared Data Apps Apps Tenant Tenant Data Data Tenant Tenant Analytics Analytics © Continuent 2010
- 9. What is Tungstenand How Does It Work? © Continuent 2010
- 10. What Is Tungsten? / Tungsten creates integrated data services using of-the-shelf PostgreSQL/MySQL databases to: • Protect data • Keep applications constantly available • Improve resource utilization • Raise performance © Continuent 2010
- 11. What’s Inside Tungsten? / Replication - Making copies • Tungsten Replicator -- Database-neutral, platform independent master/slave replication / Connectivity -- Finding databases • Tungsten Connector -- Fast MySQL/PostgreSQL client to JDBC proxying • Tungsten SQL Router --JDBC wrapper for high-performance and transparent failover, load-balancing, and partitioning (no proxy required) / Management -- Administering the database • Tungsten Manager -- Distributed administration with autonomic, rule-based configuration and no single point of failure • Tungsten Monitor -- Track resource status and © Continuent 2010
- 12. Apache/Mod_PHP Apache/Mod_PHP Manager Connector Connector Manager Replicator Replicator Replicator Manager Manager Manager © Continuent 2010
- 13. Technology: Replication Pipelines ©Continuent 2010
- 14. Technology: SQL Routing Java App Server PHP Application Tungsten SQL Router libmysqlclient.a MySQL JDBC Driver Tungsten Connector Clustered Databases © Continuent 2010
- 15. Technology: Distributed Management Admin Client Manager Manager (Coordinator) Admin Client Manager Admin Client © Continuent 2010
- 16.
- 17. PostgreSQL 8.4 WarmStandby © Continuent 2010
- 18. Limitations of WarmStandby 1. Utilization -- Cannot open the standby • To bring up the standby for queries you must end recovery • Standby hardware is idle • Difficult to track state of recovery since you cannot query log position 2. Data Loss -- Warm standby transfers only full WAL files • Can bound loss using archive_timeout • Low values create large numbers of WAL files; complicate point- in-time recovery • Workarounds using DRBD, etc. are complex © Continuent 2010
- 19. Introducing Hot Standby / Allows users to connect to standby in read-only mode • Allowed: SELECT, SET, LOAD, COMMIT/ROLLBACK • Disallowed: INSERT, UPDATE, DELETE, CREATE, 2PC, SELECT … FOR SHARE/UPDATE, nextval(), LISTEN, LOCK, • No admin commands: ANALYZE, VACUUM, REINDEX, GRANT / Can come out of recovery while queries are running © Continuent 2010
- 20. Introducing Log Streaming ©Continuent 2010
- 21. Configuration and Usage / Log streaming layers on top of existing warm standby log shipping / Multiple standby servers allowed / Failure of one standby does not affect others / Manual management is not simple - must coordinate provisioning & WAL shipping to set up/restart © Continuent 2010
- 22.
- 23. Moving Tungsten toPostgreSQL / Problem: We can’t read PostgreSQL logs (yet) Tungsten solution is to manage: / Warm Standby + WAL Shipping (for PostgreSQL <9) • Good basic availability/fast failover • Slaves open up for reads only after failover • No load balancing/scaling / Hot Standby + Streaming Replication (for PostgreSQL >=9) • Slaves opened up for reads = Tungsten scaling facilities work • Add Streaming Replication = minimal delay in replicating data • With these features available, Tungsten brings true scale-out solution for PostgreSQL 9. With Tungsten PostgreSQL users gain high availability and scalability MySQL users have been able to enjoy over the years © Continuent 2010
- 24. What is Tungsten’sAdded Value To PostgreSQL? / 15 minute cluster installation / Single commands to: • View cluster status • Provision a new standby • Confirm liveness of replication • Switch servers safely for maintenance • Failover a dead server to most current replica / Automatic discovery of new database replicas / Automatic failover when databases fail / Simple procedures for provisioning / Transparent application routing / Easy scaling © Continuent 2010
- 25. How Tungsten coversall this? Tungsten Manager Replicator JMX Interface Monitor Replication State Model Open Script Plugin © Continuent 2010
- 26.
- 27. Master Slave Replicator Replicator Transaction Transaction History Log History Log Filters Filters MySQL PostgreSQL Extractor Applier © Continuent 2010
- 28.
- 29.
- 30. HQ and Americas EMEA and APAC 560 S. Winchester Blvd., Suite 500 Lars Sonckin kaari 16 San Jose, CA 95128 02600 Espoo, Finland Tel (866) 998-3642 Tel +358 50 517 9059 Fax (408) 668-1009 Fax +358 9 863 0060 e-mail: sales@continuent.com Continuent Web Site: http://www.continuent.com © Continuent 2010