Linas Virbalas                    Continuent, Inc.© Continuent 2010
/  Introductions  /  What is Tungsten?  /  Architecture of a Rule–Based Management     Framework for Database Clusters  / ...
© Continuent 2010
/  Our Mission: Continuent is the data reliability company     that keeps business in business  /  Our Solution:          ...
© Continuent 2010
What Is Tungsten?  /  Tungsten implements master/slave clusters to:           •  Protect data           •  Maintain high a...
What’s Inside Tungsten?  /  Replication - Making copies           •  Tungsten Replicator -- Database-neutral, platform ind...
Application Server           Application Server               SQL Router/                SQL Router/                      ...
Multiple Routes to Databases                    Java App Server       PHP Application                    Tungsten SQL Rout...
Tungsten on Open Source Databases  /  MySQL 5.0/5.1           •  Read and replicate from MySQL binlog           •  Works w...
Tungsten Added Value  /  15 minute cluster installation  /  Single commands to:           •  View cluster status          ...
© Continuent 2010
Distributed Rule-Based Management                       Admin Client                                                      ...
Group Communications in Clustering  /  Tungsten Manager processes connect to each     other via Group Communications proto...
What is a Rule?  /  Trivial Rule:  /  Example (pseudo code):© Continuent 2010
What is a Rule Engine?© Continuent 2010
How does the Rule Engine Work?                                   Application                                   with a Rule...
How does the Rule Engine get its Facts?                        Tungsten       Tungsten                         Monitor    ...
Who is The Coordinator?  /  Only one Coordinator for a group (cluster)           •  Only one Manager must act on important...
© Continuent 2010
© Continuent 2010
The Rules are “Layered”  /  Lower level rules react on events from           •  Database servers           •  Replicators ...
Scenario: Database Crashes  /  Failover is handled by a controlled chain reaction of     rules rather than a single rule: ...
© Continuent 2010
© Continuent 2010
Conclusion  /  Business Rules are a very powerful and flexible     tool for automating otherwise complex tasks  /  They ar...
© Continuent 2010
© Continuent 2010
HQ and Americas                      EMEA and APAC   560 S. Winchester Blvd., Suite 500   Lars Sonckin kaari 16   San Jose...
Upcoming SlideShare
Loading in …5
×

Living the Easy Life with Rules-Based Autonomic Database Clusters

760
-1

Published on

Our business at Continuent is development of database clusters with highly simplified management and the ability to be operated unattended for prolonged periods of time. As part of our Tungsten Clustering product we developed an extensible, low-latency, fault-tolerant management framework for database clusters built around a core of group communications and business rules. We have found that our system is easy to maintain and to extend. For example, an extension to switch virtual IP addresses in the event of a database node failure was implemented in an afternoon as a set of two rules and a single bash script. In our talk we will cover the following:

* Basic architecture of a rules-based management framework for databases
* Introduction to business rules, with code examples showing how they work to repair problems ranging from simple process failures to network partitions
* A quick demo of business rules in operation.
* Finally, some thoughts on the benefits of the approach and our experiences (good and bad) with autonomic management of database clusters.

This is an approach to management that we believe will be of interest to anyone who cares about keeping important data highly available as well anyone interested in learning about rules technology.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
760
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Living the Easy Life with Rules-Based Autonomic Database Clusters

  1. 1. Linas Virbalas Continuent, Inc.© Continuent 2010
  2. 2. /  Introductions /  What is Tungsten? /  Architecture of a Rule–Based Management Framework for Database Clusters /  Demo of Business Rules in Operation /  Business Rules in Source Code /  Questions and Comments© Continuent 2010
  3. 3. © Continuent 2010
  4. 4. /  Our Mission: Continuent is the data reliability company that keeps business in business /  Our Solution: •  Continuent Tungsten (Master/Slave Database Replication) /  Our Value: •  Enterprise class data management •  Open source cost •  Integrated solution /  Our Technical Expertise •  Database replication •  Database cluster management •  Application connectivity •  Software-as-a-Service (SaaS)© Continuent 2010
  5. 5. © Continuent 2010
  6. 6. What Is Tungsten? /  Tungsten implements master/slave clusters to: •  Protect data •  Maintain high availability •  Improve resource utilization •  Raise performance /  Install and set up in a few minutes /  Integrated backup/restore and data integrity checks /  Efficient failover operations /  Distributed, rule-driven management /  No/minimal application changes /  Highly pluggable /  No specialized hardware requirements© Continuent 2010
  7. 7. What’s Inside Tungsten? /  Replication - Making copies •  Tungsten Replicator -- Database-neutral, platform independent master/slave replication /  Connectivity -- Finding databases •  Tungsten Connector -- Fast MySQL/PostgreSQL client to JDBC proxying •  Tungsten SQL Router --JDBC wrapper for high-performance and transparent failover, load-balancing, and partitioning (no proxy required) /  Management -- Administering the database •  Tungsten Manager -- Distributed administration with autonomic, rule-based configuration and no single point of failure •  Tungsten Monitor -- Track resource status and database liveness© Continuent 2010
  8. 8. Application Server Application Server SQL Router/ SQL Router/ Manager Manager Connector Connector Replicator Replicator Monitor Monitor Manager Manager Management Management Client Client© Continuent 2010
  9. 9. Multiple Routes to Databases Java App Server PHP Application Tungsten SQL Router libpq.a PostgreSQL JDBC Driver Tungsten Connector Tungsten Cluster© Continuent 2010
  10. 10. Tungsten on Open Source Databases /  MySQL 5.0/5.1 •  Read and replicate from MySQL binlog •  Works with Oracle, MySQL and look-alikes (Percona, MariaDB) /  PostgreSQL < 9 •  Warm Standby + WAL Shipping •  Good basic availability/fast failover •  Slaves open up for reads only after failover /  PostgreSQL >= 9 •  Hot Standby + Streaming Replication •  Slaves opened up for reads = Tungsten scaling facilities work •  Add Streaming Replication = minimal delay in replicating data /  Sometime in the future: Read PG logs directly!© Continuent 2010
  11. 11. Tungsten Added Value /  15 minute cluster installation /  Single commands to: •  View cluster status •  Provision a new standby •  Confirm liveness of replication •  Switch servers safely for maintenance •  Failover a dead server to most current replica /  Automatic discovery of new replicas /  Automatic failover when databases fail /  Simple procedures for provisioning /  Transparent application routing /  Easy scaling© Continuent 2010
  12. 12. © Continuent 2010
  13. 13. Distributed Rule-Based Management Admin Client Manager Manager (Coordinator) Admin Client Manager Admin Client© Continuent 2010
  14. 14. Group Communications in Clustering /  Tungsten Manager processes connect to each other via Group Communications protocol /  Group Communications provide essential features for running multi-node environments: •  Membership detection •  Notification about joined/left/crashed members •  Reliable message broadcast (lost messages retransmitted) •  Ordered message broadcast (all members receive messages in the same order) /  Different transport protocols can be used •  IP Multicast – for open local networks •  Peer to Peer TCP – for firewalled local networks •  Centralized Gossip Server – for Cloud environments© Continuent 2010
  15. 15. What is a Rule? /  Trivial Rule: /  Example (pseudo code):© Continuent 2010
  16. 16. What is a Rule Engine?© Continuent 2010
  17. 17. How does the Rule Engine Work? Application with a Rule Engine (Tungsten Manager)© Continuent 2010
  18. 18. How does the Rule Engine get its Facts? Tungsten Tungsten Monitor Replicator Tungsten Manager© Continuent 2010
  19. 19. Who is The Coordinator? /  Only one Coordinator for a group (cluster) •  Only one Manager must act on important events like failures •  Having more than one Coordinator would lead to race conditions and eventually chaotic cluster behavior /  Tungsten chooses Coordinator from the Group Communications •  Coordinator is the oldest member of the group •  New members handshake with the Coordinator when joining •  Upon Coordinator death, GC automatically promotes a new one© Continuent 2010
  20. 20. © Continuent 2010
  21. 21. © Continuent 2010
  22. 22. The Rules are “Layered” /  Lower level rules react on events from •  Database servers •  Replicators •  Managers /  Outcome: the state change of a concept “data source” /  Higher level rules react on •  Data source state changes /  Outcome: actual operations to the cluster components (eg. promote slave to a master)© Continuent 2010
  23. 23. Scenario: Database Crashes /  Failover is handled by a controlled chain reaction of rules rather than a single rule: 1.  "DETECT AND FENCE FAILED DATASERVER" rule detects state transition of the master DB from ONLINE to STOPPED   It then fences the failure by setting the data source of the stopped DB server to FAILED 2.  "AUTOMATIC FAILOVER" rule fires, because DataSource(state == ResourceState.FAILED && role == "master”)   It then calls a method in the ClusterManagementHelper to do the actual failover, after which it refreshes the data source states i  If the data source was of a slave, it would stay in the FAILED state and no further action would take place© Continuent 2010
  24. 24. © Continuent 2010
  25. 25. © Continuent 2010
  26. 26. Conclusion /  Business Rules are a very powerful and flexible tool for automating otherwise complex tasks /  They are especially useful in implementing automated DB clusters /  A robust top layer connecting all the components underneath into a seamless experience© Continuent 2010
  27. 27. © Continuent 2010
  28. 28. © Continuent 2010
  29. 29. HQ and Americas EMEA and APAC 560 S. Winchester Blvd., Suite 500 Lars Sonckin kaari 16 San Jose, CA 95128 02600 Espoo, Finland Tel (866) 998-3642 Tel +358 50 517 9059 Fax (408) 668-1009 Fax +358 9 863 0060 e-mail: sales@continuent.com, linas.virbalas@continuent.com Continuent Web Site: http://www.continuent.com© Continuent 2010

×