0
©Continuent 2014
Harnessing the Power of
Master/Slave Clusters for Data-
Driven Business on MySQL
Robert Hodges, CEO
Giuse...
©Continuent 2014 2
Introducing Continuent
©Continuent 2014
Introducing Continuent
3
• The leading provider of clustering and
replication for open source DBMS
• Our ...
©Continuent 2014
Quick Continuent Facts
• Largest Tungsten installation by data volume
processes over 800 million transact...
©Continuent 2014©Continuent 2014
Continuent Tungsten Customers
5
1
©Continuent 2014 6
Too Good to Be True
©Continuent 2014
The Dream: multiple, active DBMS
servers with identical data
7
Server A
Server B
Server C
Transparent	

r...
©Continuent 2014
Multi-Master Clusters in Action: Galera
8
Group
Ordering and
Delivery
Server A
Server B
Server C
foo 	

p...
©Continuent 2014
Synchronous Optimistic Locking
9
Group
Ordering and
Delivery
Server A
Server B
Server C
foo 	

pk=1, v=5
...
©Continuent 2014
Trouble in Paradise
• Transaction failures due to conflicts
• Operations like SELECT FOR UPDATE not
suppo...
©Continuent 2014
Can Master/Slave Offer Same
Benefits?
11
Server A
Server B Server C
Updates	

propagated	

immediately?
H...
©Continuent 2014 12
Tungsten Master/Slave Clusters
©Continuent 2014
Tungsten Database-as-a-Service
13
Tungsten clusters combine off-
the-shelf open source DBMS
servers into ...
©Continuent 2014
Cluster Architecture
14
Application
Tungsten Connector
Application
Tungsten Connector
Replicator
Db2
Repl...
©Continuent 2014
Tungsten Connector Basics
• Any MySQL client can connect
• Initiates connections using JDBC
15
App Native...
©Continuent 2014
Tungsten Connector Basics
• Pass-through operation after connection
• Full transparency & low overhead fo...
©Continuent 2014
SmartScale Session Load Balancing
17
App
Connect/Write
Connector
Transaction	

History Log
Slave	

replay...
©Continuent 2014
SmartScale Session Load Balancing
18
App
Read
Connector
Transaction	

History Log
Slave	

replay	

positi...
©Continuent 2014
SmartScale Session Load Balancing
19
App
Read
Connector
Transaction	

History Log
Slave	

replay	

positi...
©Continuent 2014
Connectivity Example
20
Tungsten Connector Tungsten Connector
Db2 Db3Db1
SlaveMasterSlave
(SmartScale) (S...
©Continuent 2014 21
Demo: Transparent
Connectivity to Replicas
©Continuent 2014 22
Transaction Scaling with
Master/Slave Clusters
©Continuent 2014
Understanding Replication
23
Master
(Transactions + Metadata)
Slave
THL
DBMS	

Logs
Replicator
(Transacti...
©Continuent 2014
Master Replication Service
24
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Pipeline
MySQL	

Mast...
©Continuent 2014
Slave Service with Parallel Apply
25
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Stage
Pipeline...
©Continuent 2014
Understanding Transaction Loads
26
Large	

TransactionsSlow 	

queries
Reports
Db2 Db3Db1
SlaveMasterSlav...
©Continuent 2014 27
Demo: Scalable
Transaction Processing
©Continuent 2014 28
Master/Slave Clustering
Across Regions
©Continuent 2014
Primary/Disaster Recovery Clustering
29
App Server
+ Connector
App Server
+ Connector
Master Slave
San Jo...
©Continuent 2014
Further Cross-Site Topologies
30
Data Warehouse
All	

Masters
Fan-In
©Continuent 2014 31
Failover and Maintenance
©Continuent 2014
Automatic Failover
32
Application
Tungsten Connector
©Continuent 2014
Automatic Failover
33
Application
Tungsten Connector
1. Detect non-
responsive
master
2. Halt master conn...
©Continuent 2014
Automatic Failover
34
Application
Tungsten Connector
3. Select most advanced
slave and wait for pending
t...
©Continuent 2014
Automatic Failover
35
Application
Tungsten Connector
4. Promote to master,
reconfigure slave(s), and
re-e...
©Continuent 2014
Automatic Failover
36
Application
Tungsten Connector
5. Recover failed master
©Continuent 2014
Rolling MySQL maintenance
• Task: change the innodb log file size
• Problem: it requires a database serve...
©Continuent 2014
Rolling MySQL Maintenance
38
Final UpgradeSwitchSlave Upgrade Slave Upgrade
• Shun slave 1,
run upgrade,
...
©Continuent 2014 39
Demo: Zero-Downtime
Upgrade and Failover
©Continuent 2014 40
Wrap-Up
©Continuent 2014
Master/Slave Performs Very Well!
41
Server A
Server B Server C
Updates	

propagated	

without cost 	

to ...
©Continuent 2014
In Conclusion: Tungsten Offers...
• Transparent access to a highly available cluster of
off-the-shelf MyS...
©Continuent 2014
www.continuent.com	

Follow us on Twitter @continuent	

!
Tungsten Replicator:	

http://code.google.com/p...
Upcoming SlideShare
Loading in...5
×

Harnessing the Power of Master/Slave Clusters to Operate Data-Driven Businesses on MySQL

162

Published on

Users seeking high availability, disaster recovery and zero downtime maintenance operation for business-critical MySQL applications face confusing choices. Is multi-master or master/slave clustering better? What about synchronous versus asynchronous replication? Using a plain vanilla, stock MySQL or a modified version of it? Which of these choices are right for data-driven businesses that depend on fast, reliable data access?

This no-BS webinar cuts through the FUD to explore the real trade-offs between the different clustering and replication methods, then show you how Continuent Tungsten asynchronous master/slave clusters support these important capabilities for business-critical applications:

- High application write rates
- Mixed workloads consisting of large and small transactions
- Data across multiple geographically distributed locations
- Failures and more importantly recovery from them
- Zero downtime maintenance and software upgrades
- Use of off-the-shelf MySQL/MariaDB to avoid application changes and allow clusters to improve as MySQL itself does.

We illustrate key points with demonstrations and case studies from deployed systems. Join us for an educational session on the cutting edge of MySQL data management!

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
162
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Harnessing the Power of Master/Slave Clusters to Operate Data-Driven Businesses on MySQL"

  1. 1. ©Continuent 2014 Harnessing the Power of Master/Slave Clusters for Data- Driven Business on MySQL Robert Hodges, CEO Giuseppe Maxia, Director of QA
  2. 2. ©Continuent 2014 2 Introducing Continuent
  3. 3. ©Continuent 2014 Introducing Continuent 3 • The leading provider of clustering and replication for open source DBMS • Our Product: Continuent Tungsten • Clustering - Commercial-grade HA, performance scaling and data management for MySQL • Replication - Flexible, high-performance data movement
  4. 4. ©Continuent 2014 Quick Continuent Facts • Largest Tungsten installation by data volume processes over 800 million transactions per day on 225 terabytes of relational data • Largest installation by transaction volume handles up to 8 billion transactions daily • Wide variety of topologies including MySQL, Oracle, Vertica, and Hadoop in production • Cross-site topologies widely deployed 4
  5. 5. ©Continuent 2014©Continuent 2014 Continuent Tungsten Customers 5 1
  6. 6. ©Continuent 2014 6 Too Good to Be True
  7. 7. ©Continuent 2014 The Dream: multiple, active DBMS servers with identical data 7 Server A Server B Server C Transparent read/write to any server Updates propagated immediately High Performance High Availability
  8. 8. ©Continuent 2014 Multi-Master Clusters in Action: Galera 8 Group Ordering and Delivery Server A Server B Server C foo pk=1, v=5 foo pk=6, v=25 foo pk=1, v=6 [1] pk=1, v=6 [2] pk=1, v=5 [3] pk=6, v=25 [1] pk=1, v=6 [2] pk=1, v=5 [3] pk=6, v=25 [1] pk=1, v=6 [2] pk=1, v=5 [3] pk=6, v=25
  9. 9. ©Continuent 2014 Synchronous Optimistic Locking 9 Group Ordering and Delivery Server A Server B Server C foo pk=1, v=5 foo pk=6, v=25 foo pk=1, v=6 [1] pk=1, v=6 [2] pk=1, v=5 [3] pk=6, v=25 [1] pk=1, v=6 [2] pk=1, v=5 [3] pk=6, v=25 [1] pk=1, v=6 [2] pk=1, v=5 [3] pk=6, v=25 COMMIT COMMIT DEADLOCK
  10. 10. ©Continuent 2014 Trouble in Paradise • Transaction failures due to conflicts • Operations like SELECT FOR UPDATE not supported • Slow writes due to synchronous messaging • Large transactions lock system/cause failures • Unstable cross-site replication 10
  11. 11. ©Continuent 2014 Can Master/Slave Offer Same Benefits? 11 Server A Server B Server C Updates propagated immediately? High Performance? High Availability? Transparent read/write to any server?
  12. 12. ©Continuent 2014 12 Tungsten Master/Slave Clusters
  13. 13. ©Continuent 2014 Tungsten Database-as-a-Service 13 Tungsten clusters combine off- the-shelf open source DBMS servers into data services with: ! • 24x7 data access • Scaling of load on replicas • Simple management commands ! ...without app changes or data migration Amazon US West apache /php GonzoPortal.com Connector Connector
  14. 14. ©Continuent 2014 Cluster Architecture 14 Application Tungsten Connector Application Tungsten Connector Replicator Db2 Replicator Db3Db1 Replicator Slave SlaveMaster Manager Manager Manager MonitoringandControl MonitoringandControl Data Service: sjc
  15. 15. ©Continuent 2014 Tungsten Connector Basics • Any MySQL client can connect • Initiates connections using JDBC 15 App Native protocol ! COM_QUERY COM_INIT_DB COM_DROP_DB Connector
  16. 16. ©Continuent 2014 Tungsten Connector Basics • Pass-through operation after connection • Full transparency & low overhead for clients 16 App Protocol packet COM_QUERY select * from t Connector Protocol packet OK ResultSet Rows:1
  17. 17. ©Continuent 2014 SmartScale Session Load Balancing 17 App Connect/Write Connector Transaction History Log Slave replay position Session x Session y Slave DBMS Master DBMS Slave DBMS Initial Write
  18. 18. ©Continuent 2014 SmartScale Session Load Balancing 18 App Read Connector Transaction History Log Slave replay position Session x Session y Slave DBMS Master DBMS Slave DBMS Read from master NO read from slave!
  19. 19. ©Continuent 2014 SmartScale Session Load Balancing 19 App Read Connector Transaction History Log Slave replay position Session x Slave DBMS Master DBMS Slave DBMS Read from master Read from slave
  20. 20. ©Continuent 2014 Connectivity Example 20 Tungsten Connector Tungsten Connector Db2 Db3Db1 SlaveMasterSlave (SmartScale) (Strict Consistency)
  21. 21. ©Continuent 2014 21 Demo: Transparent Connectivity to Replicas
  22. 22. ©Continuent 2014 22 Transaction Scaling with Master/Slave Clusters
  23. 23. ©Continuent 2014 Understanding Replication 23 Master (Transactions + Metadata) Slave THL DBMS Logs Replicator (Transactions + Metadata) THLReplicator Download transactions via network Apply using JDBC
  24. 24. ©Continuent 2014 Master Replication Service 24 Extract Filter Apply Stage Extract Filter Apply Stage Pipeline MySQL Master Transaction History Log In-Memory Queue Slave ReplicatorsBinlog tcp/ip
  25. 25. ©Continuent 2014 Slave Service with Parallel Apply 25 Extract Filter Apply Stage Extract Filter Apply Stage Stage Pipeline Transaction History Log Parallel Queue Slave DBMS Extract Filter Apply Extract Filter Apply Extract Filter Apply Master Replicator tcp/ip
  26. 26. ©Continuent 2014 Understanding Transaction Loads 26 Large TransactionsSlow queries Reports Db2 Db3Db1 SlaveMasterSlave Small Transactions
  27. 27. ©Continuent 2014 27 Demo: Scalable Transaction Processing
  28. 28. ©Continuent 2014 28 Master/Slave Clustering Across Regions
  29. 29. ©Continuent 2014 Primary/Disaster Recovery Clustering 29 App Server + Connector App Server + Connector Master Slave San Jose NYC
  30. 30. ©Continuent 2014 Further Cross-Site Topologies 30 Data Warehouse All Masters Fan-In
  31. 31. ©Continuent 2014 31 Failover and Maintenance
  32. 32. ©Continuent 2014 Automatic Failover 32 Application Tungsten Connector
  33. 33. ©Continuent 2014 Automatic Failover 33 Application Tungsten Connector 1. Detect non- responsive master 2. Halt master connections
  34. 34. ©Continuent 2014 Automatic Failover 34 Application Tungsten Connector 3. Select most advanced slave and wait for pending transactions to apply
  35. 35. ©Continuent 2014 Automatic Failover 35 Application Tungsten Connector 4. Promote to master, reconfigure slave(s), and re-enable connections
  36. 36. ©Continuent 2014 Automatic Failover 36 Application Tungsten Connector 5. Recover failed master
  37. 37. ©Continuent 2014 Rolling MySQL maintenance • Task: change the innodb log file size • Problem: it requires a database server restart • Constraint: we do not want to restart • Solution: rolling maintenance • shun node2 (slave) - do maintenance - put node online • shun node3 - do maintenance - put node online • promote node2 to master • shun node1 - do maintenance - put node online • promote again node1 to master 37
  38. 38. ©Continuent 2014 Rolling MySQL Maintenance 38 Final UpgradeSwitchSlave Upgrade Slave Upgrade • Shun slave 1, run upgrade, and return to cluster. Discard and recover on failure • Upgrade old master. Cluster schema is now fully upgraded • Repeat for additional slave(s) • Switch master to promote an upgraded slave
  39. 39. ©Continuent 2014 39 Demo: Zero-Downtime Upgrade and Failover
  40. 40. ©Continuent 2014 40 Wrap-Up
  41. 41. ©Continuent 2014 Master/Slave Performs Very Well! 41 Server A Server B Server C Updates propagated without cost to applications Very High Performance Very High Availability Transparent connectivity with full SQL semantics
  42. 42. ©Continuent 2014 In Conclusion: Tungsten Offers... • Transparent access to a highly available cluster of off-the-shelf MySQL replicas • High performance transaction processing regardless of cluster size or distance between nodes • Zero-downtime maintenance and upgrade of DBMS and applications 42
  43. 43. ©Continuent 2014 www.continuent.com Follow us on Twitter @continuent ! Tungsten Replicator: http://code.google.com/p/tungsten-replicator Our Blogs: http://scale-out-blog.blogspot.com http://datacharmer.org/blog http://www.continuent.com/news/blogs http://flyingclusters.blogspot.com/ 560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009 e-mail: sales@continuent.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×