It is about Driving High Performance with MySQL Cluster in a Hong Kong Monthly Tech Talk

1. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster
Driving high performance
Ivan Ma
11th October 2014
Open Source Tech Talk 2014.10

2. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
2

3. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster Overview
•Auto-Sharding, Multi-Master
•ACID Compliant, OLTP + Real-Time Analytics
HIGH SCALE, READS + WRITES
•Shared nothing, no Single Point of Failure
•Self Healing + On-Line Operations
99.999% AVAILABILITY
•In-Memory Optimization + Disk-Data
•Predictable Low-Latency, Bounded Access Time
REAL-TIME
•Key/Value + Complex, Relational Queries
•SQL + Memcached + JavaScript + Java + HTTP/REST & C++
SQL + NoSQL
•Open Source + Commercial Editions
•Commodity hardware + Management, Monitoring Tools
LOW TCO

5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer

7. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster Scaling
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer

8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster HA
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer

9. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster 7.3: 1.2 Billion UPDATEs per Minute
•NoSQL C++ API, flexaSynch benchmark
•30 x Intel E5-2600 Intel Servers, 2 socket, 64GB
•ACID Transactions, with Synchronous Replication
0
5
10
15
20
25
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
Millions of UPDATEs per Second
MySQL Cluster Data Nodes

10. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Scaling & Linear Scaling
•Partitioning & Sharding
•Push Down Query
•Explain Query
•Multi-threaded Data Node – ndbd vs ndbmtd
•SQL and NoSQL

11. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1
Data Node 2
F1
Primary Fragment
Secondary Fragment
Data Node 3
Fx Data Node 4
Fx
Table T1
P2
P3
P4
Px Partition
4 Partitions * 2 Replicas = 8 Fragments
P1
Automatic Data Partitioning
Nodes & Node Groups
Copyright 2011 Oracle Corporation 11

12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1
Data Node 2
F1
Primary Fragment
Secondary Fragment
F1
Data Node 3
Fx Data Node 4
Fx
Table T1
P2
P3
P4
Px Partition
4 Partitions * 2 Replicas = 8 Fragments
P1
Automatic Data Partitioning
Nodes & Node Groups
Copyright 2011 Oracle Corporation 12

13. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1
Data Node 2
F1
Primary Fragment
Secondary Fragment
F3 F1
Data Node 3
Fx Data Node 4
Fx
Table T1
P2
P3
P4
Px Partition
4 Partitions * 2 Replicas = 8 Fragments
P1
Automatic Data Partitioning
Nodes & Node Groups
Copyright 2011 Oracle Corporation 13

14. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1
Data Node 2
F1 F3
Primary Fragment
Secondary Fragment
F3 F1
Data Node 3
Fx Data Node 4
Fx
Table T1
P2
P3
P4
Px Partition
4 Partitions * 2 Replicas = 8 Fragments
P1
Automatic Data Partitioning
Nodes & Node Groups
Copyright 2011 Oracle Corporation 14

15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1
Data Node 2
F1 F3
Primary Fragment
Secondary Fragment
F3 F1
Data Node 3
Data Node 4
F2 F4
F4 F2
Node Group 1
Node Group 2
Fx
Fx
Table T1
P2
P3
P4
Px Partition
4 Partitions * 2 Replicas = 8 Fragments
P1
Automatic Data Partitioning
Nodes & Node Groups
Copyright 2011 Oracle Corporation 15

16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Scalability: Auto-Sharding
•Partitioning happens automatically & transparent to the application
•Transparency maintained during failover, upgrades and scale-out
•No need for application-layer sharding logic
•Transactional across shards (COMMIT transaction across shards/partitions)
http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster_perfomance.php

17. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Adaptive Query Localization Scaling Distributed Joins
•Perform Complex Queries across Shards
•JOINs pushed down to data nodes
•Executed in parallel
•Returns single result set to MySQL
•Opens Up New Use-Cases
•Real-time analytics
•Recommendations engines
•Fraud-detection
mysqld
Data Nodes
mysqld
AQL
Data Nodes

18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
AQL – How to Use it
•Activated when ndb_join_pushdown is on (default)
•Run ANALYZE TABLE <tab-name> on each table once
•Use EXPLAIN to see what components are being pushed down:
–Extra: Child of 'd' in pushed join@1

19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1 Data Node 2
Need more throughput? Scale Out

20. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1 Data Node 2
Need more throughput? Scale Out

21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1 Data Node 2
Need more throughput? Scale Out
Oops, need to increase capacity as well!

22. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Data Node 1 Data Node 2
Scale Out
Data Node 3 Data Node 4

23. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

24. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
NoSQL
Simple access patterns
Compromise on consistency for performance
Ad-hoc data format
Simple operation
SQL
Complex queries with joins
ACID transactions
Well defined schemas
Rich set of tools
•No best single solution fits all
•Mix and match
Still a role for SQL (RDBMS)?
Scalability
Performance
HA
Ease of use
SQL/Joins
ACID Transactions

25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Apps
Apps
Apps
Apps
Apps
Apps
Apps
Apps
Apps
Apps
Apps
Apps
JPA
Cluster JPA
PHP
Perl
Python
Ruby
JDBC
Cluster J
JS
Apache
Memcached
MySQL
JNI
Node.JS
mod_ndb
ndb_eng
NDB API (C++)
NoSQL Access to MySQL Cluster data
MySQL Cluster Data Nodes

26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster 7.4.1 DMR
Available Now!

27. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Better performance and operational simplicity
MySQL Cluster 7.4.1 DMR
• Performance gain over 7.3
– 47% (Read-Only)
– 38% (Read-Write)
• Faster maintenance operations
– Nodal & Rolling restarts
– Upgrades

28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
•Asynchronous replication between MySQL Clusters
•Active-Active
–Update anywhere
–Conflict detection
•Application notified through exception tables
•Can opt to have conflicts resolved automatically
–Auto-conflict-resolution
•Conflicting transaction and dependent ones are rolled-back
•No changes to application schema
Active-Active Geo-Replication

29. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Restart Times
•Restarting data node with locally checkpointed data
–Major improvement
•Restarting data node which must recover data from peer
–Major improvement
–Further speedups to come in 7.4.X (greater parallelization)
•Upgraded/rolling restarts
–Major improvement
•Cluster shutdown and restart
–Minor improvement
What operations benefit?

30. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle MySQL HA & Scaling Solutions
MySQL Replication
MySQL Fabric
Oracle VM Template
Oracle Clusterware
Solaris Cluster
Windows Cluster
DRBD
MySQL Cluster
App Auto-Failover
✖
✔
✔
✔
✔
✔
✔
✔
Data Layer Auto-Failover
✖
✔
✔
✔
✔
✔
✔
✔
Platform Support
All
All
Linux
Linux
Solaris
Windows
Linux
All
Clustering Mode
Master + Slaves
Master + Slaves
Active/Passive
Active/Passive
Active/Passive
Active/Passive
Active/Passive
Multi- Master
Failover Time
N/A
Secs
Secs +
Secs +
Secs +
Secs +
Secs +
< 1 Sec
Scale-out
Reads
✔
✖
✖
✖
✖
✖
✔
Cross-shard operations
N/A
✖
N/A
N/A
N/A
N/A
N/A
✔
Transparent routing
✖
For HA
✔
✔
✔
✔
✔
✔
Shared Nothing
✔
✔
✖
✖
✖
✖
✔
✔
Storage Engine
InnoDB+
InnoDB+
InnoDB+
InnoDB+
InnoDB+
InnoDB+
InnoDB+
NDB
Single Vendor Support
✔
✔
✔
✔
✔
✖
✔
✔
30

31. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
When to Consider MySQL Cluster
•Scalability demands
–Sharding for write performance?
•Latency demands
–Cost of each millisecond?
•Uptime requirements
–Cost per minute of downtime?
–Failure versus maintenance?
•Application agility
–Developer languages and frameworks?
–SQL or NoSQL?

32. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
32