A look at the mechanisms that ensure that metadata is synchronized across all the MySQL Servers that are connected to a MySQL NDB Cluster

1. Metadata Synchronization in MySQL
NDB Cluster 8.0
Lead Member Technical Staff
MySQL Cluster
March 05, 2020
Arnab Ray

2. 2 Copyright – © 2020 Oracle and/or its affiliates
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon in
making purchasing decisions. The development, release, timing, and pricing of any features
or functionality described for Oracle’s products may change and remains at the sole
discretion of Oracle Corporation.
Statements in this presentation relating to Oracle’s future plans, expectations, beliefs,
intentions and prospects are “forward-looking statements” and are subject to material risks
and uncertainties. A detailed discussion of these factors and other risks that affect our
business is contained in Oracle’s Securities and Exchange Commission (SEC) filings,
including our most recent reports on Form 10-K and Form 10-Q under the heading “Risk
Factors.” These filings are available on the SEC’s website or on Oracle’s website at
http://www.oracle.com/investor. All information in this presentation is current as of
September 2019 and Oracle undertakes no duty to update any statement in light of new
information or future events.
Safe Harbour Statement

3. 5
4
3
2
1
Post-restore Synchronization
Synchronization On Connection
DDL Distribution
MySQL Cluster Architecture
Introduction to MySQL NDB Cluster
Agenda
Copyright – © 2020 Oracle and/or its affiliates3

4. Massive linear scale
99.9999% Availability
Distributed In-Memory Datasets
Always Consistent
Parallel real-time performance,
auto-partitioning, data distribution,
and replication built-in
Read and Write Scale-Out
to many TB on commodity hardware
Designed for mission critical
systems. Masterless, shared-nothing
with no single point of failure
Transactional consistency across
distributed and partitioned dataset
Out of the box straightforward
application programming
Ease of use
Open Source
Written in C++. Can be used standalone
or with MySQL as a SQL front-end
4 Copyright – © 2020 Oracle and/or its affiliates

5. 5
4
3
2
1
Post-restore Synchronization
Synchronization On Connection
DDL Distribution
MySQL Cluster Architecture
Introduction to MySQL NDB Cluster
Agenda
Copyright – © 2020 Oracle and/or its affiliates5

6. Clients
MySQL Server & Custom
Applications
Data Nodes
NDBAPI
MySQL Cluster Architecture
6 Copyright – © 2020 Oracle and/or its affiliates

7. Data Dictionary
Pre-8.0 World
SQL
.FRM .TRG .OPT
System tables (mysql.*)
User Proc Events
InnoDB System tables
NDB
File system
MyISAM
InnoDB
Native Dictionary
Copyright – © 2020 Oracle and/or its affiliates7

8. 8 Copyright – © 2020 Oracle and/or its affiliates
Native NDB Dictionary
• Distributed Dictionary
• Describes all NDB metadata objects
• NDB SE level
• Accessed and modified directly using
NDB API
• Tools such as ndb_restore,
ndb_drop_table, and ndb_show_tables
• Source of truth
MySQL Server Transactional Data Dictionary
• Introduced in 8.0
• Stores information about metadata
objects in InnoDB dictionary tables
• Atomic DDL
• Crash-safe DDL
• INFORMATION_SCHEMA improvements
• Cached copy of NDB metadata objects
Tale Of Two Dictionaries

9. Data Dictionary
SQL
InnoDB
Data
Dictionary
Tables
NDB
Tale Of Two Dictionaries
Native Dictionary
Copyright – © 2020 Oracle and/or its affiliates9

10. • Metadata information must be
consistent between the MySQL Server
DD and NDB Dictionary
• MySQL Cluster usually comprises of
multiple MySQL Servers
• Every MySQL Server should contain the
same “view” of NDB metadata
information
Metadata Synchronization
Local DDLocal DD
NDBAPI
NDB Dictionary
Copyright – © 2020 Oracle and/or its affiliates10

11. 5
4
3
2
1
Post-restore Synchronization
Synchronization On Connection
DDL Distribution
MySQL Cluster Architecture
Introduction to MySQL NDB Cluster
Agenda
Copyright – © 2020 Oracle and/or its affiliates11

12. • DDL statements involving NDB
metadata are distributed across all
MySQL Servers connected to the Cluster
• DDL is sent to a MySQL Server i.e. the
Originating MySQL Server
• Other MySQL Servers are referred to as
Participant MySQL Servers
• Communication faciliated by NDB API
table events on a special system table:
mysql.ndb_schema
DDL Distribution
Local DDLocal DD
NDBAPI
Copyright – © 2020 Oracle and/or its affiliates12
ndb_schema

13. • MySQL Servers subscribe to table events
on the mysql.ndb_schema table on
connection to the Cluster
• Distribution Client
• Distribution Coordinator
• Distribution Participant
• Originating MySQL Server plays the role
of Client and Coordinator
• Every MySQL Server connected to the
Cluster performs the role of a Participant
DDL Distribution
Local DDLocal DD
NDBAPI
Table
Events
+
Copyright – © 2020 Oracle and/or its affiliates13
ndb_schema

14. • Client is responsible for writing to the
mysql.ndb_schema table
• DDL statement is encapsulated in event
data form
• Coordinator receives the request from
the event stream
• Maintains a list of the current
subscribers
DDL Distribution
Local DDLocal DD
NDBAPI
Table
Events
+
Copyright – © 2020 Oracle and/or its affiliates14
ndb_schema

15. • Participant receives the event
notification and handles the event
• Uses NDB API and DD API to propagate
the DDL change to the Local DD
• Sends an ack to the Coordinator along
with the result (new in 8.0!)
• Result is sent via another special NDB
system table: mysql.ndb_schema_result
• Parallel acknowledgements
• No Participant limit
• Support for error return
DDL Distribution
Local DDLocal DD
NDBAPI
Table
Events
+
Copyright – © 2020 Oracle and/or its affiliates15
ndb_schema ndb_schema_result

16. • Coordinator receives an ack from all the
Participants in its list of subscribers
• Client inspects the results returned by
the Participants
• DDL statement is not considered to
have failed even if it failed on Participant
Server(s)
• Failed results are pushed as warnings in
the originating MySQL Server
DDL Distribution
Local DDLocal DD
NDBAPI
ndb_schema
Table
Events
+
ndb_schema_result
Copyright – © 2020 Oracle and/or its affiliates16

17. 17 Copyright – © 2020 Oracle and/or its affiliates
DDL Distribution

18. 5
4
3
2
1
Post-restore Synchronization
Synchronization On Connection
DDL Distribution
MySQL Cluster Architecture
Introduction to MySQL NDB Cluster
Agenda
Copyright – © 2020 Oracle and/or its affiliates18

19. • MySQL Server must synchronize all NDB
related contents when it (re)connects to
a Cluster
• Done by the way of comparison with the
NDB Dictionary on a per-object basis
• NDB metadata might have changed
while the MySQL Server was
disconnected
• Ensures that the MySQL Server is ready
to read/write data and perform DDL
Synchronization On Connection
Local DDLocal DD
NDBAPI
NDB Dictionary
Copyright – © 2020 Oracle and/or its affiliates19

20. • Synchronization is protected by a global
lock
• Responsible for checking that the special
system NDB tables exist
• Tables created if missing
• NDB tables remain read-only until the
synchronization concludes
Synchronization On Connection
Local DDLocal DD
NDBAPI
NDB Dictionary
Copyright – © 2020 Oracle and/or its affiliates20

21. 21 Copyright – © 2020 Oracle and/or its affiliates
The synchronization mechanism has been enhanced in 8.0:
• Rewritten to use DD API
• Synchronization of logfile groups
• Synchronization of tablespaces
• Improved synchronization of schemata
• Synchronization of NDB stored users and roles along with their grants
• Modern C++
Synchronization On Connection

22. 5
4
3
2
1
Post-restore Synchronization
Synchronization On Connection
DDL Distribution
MySQL Cluster Architecture
Introduction to MySQL NDB Cluster
Agenda
Copyright – © 2020 Oracle and/or its affiliates22

23. 23 Copyright – © 2020 Oracle and/or its affiliates
• The ndb_restore utility can create metadata in NDB Dictionary
• Changes should be propagated to the MySQL Server’s DD
• Done using SHOW TABLES query in older versions
• Equivalent of Synchronization On Connection
• Global locking
• Additional work done by SHOW TABLES query
• User thread performing synchronization
Post-restore Synchronization

24. • Metadata Change Monitor introduced in
NDB Cluster 8.0
• Detects mismatches
• Compares the contents of the NDB
Dictionary with that of the MySQL Server
DD
• Runs in the background
• User-configurable intervals of time
Post-restore Synchronization
Local DD
NDBAPI
NDB Dictionary
Change
Monitor
Copyright – © 2020 Oracle and/or its affiliates24

25. • Change Monitor submits any
mismatched objects to a queue
• Eventually synchronized by the NDB
Event Handling component
• Discovery and synchronization is
asynchronous by design
• Synchronization rate is throttled to avoid
performance overhead
Post-restore Synchronization
Local DD
NDBAPI
NDB Dictionary
Change
Monitor
Event
Handler
Copyright – © 2020 Oracle and/or its affiliates25

26. 26 Copyright – © 2020 Oracle and/or its affiliates
Post-restore Synchronization

27. 27 Copyright – © 2020 Oracle and/or its affiliates
Post-restore Synchronization

28. 28 Copyright – © 2020 Oracle and/or its affiliates
The new mechanism in 8.0 compares favourably to its predecessor:
• Less intrusive global locks
• Locks are short-lived and employ a try-lock strategy
• SHOW TABLES improvement
• Cleaner design
• Potential Usability Improvement
Post-restore Synchronization

29. 29 Copyright – © 2020 Oracle and/or its affiliates
• Manual: https://dev.mysql.com/doc/refman/8.0/en/mysql-cluster.html
• Packages: https://dev.mysql.com/downloads/cluster
• Source: https://github.com/mysql/mysql-server
• MySQL High Availability: https://mysqlhighavailability.com
• Mikael Ronstrom’s Blog: mikaelronstrom.blogspot.com
• Frazer Clement’s Blog: messagepassing.blogspot.com
References

30. 30 Copyright – © 2020 Oracle and/or its affiliates
Thank You