Severalnines Self-Training: MySQL® Cluster - Part II

MySQL Cluster Training presented by severalnines.com Address: Contact: SeveralninesAB Jean-Jérôme Schmidt c/o SICS, Box 1263 Email: services@severalnines.com Isafjordsgatan22 SE-164-29 KistaCopyright 2011 Severalnines AB Control your database infrastructure 1

Introduction • At Severalnines, we believe in sharing information and knowledge; we all come from an open source background • We know a lot of things about MySQL Cluster and think that MySQL Cluster is a great technology • These free MySQL Cluster Training slides are a contribution of ours to the knowledge and information sharing that‟s common practice in the open source community • If you have any questions on these slides or would like to book an actual training class, please contact us at: services@severalnines.comCopyright 2011 Severalnines AB Control your database infrastructure 2

Training Slides - Concept • Over the coming weeks we will be chronologically releasing slides for the different sections of our MySQL Cluster Training program on our website. • The full agenda of the training with all of its modules is outlined in the next slides so that you can see what topics will be covered over the coming weeks. • Particularly specialised topics such as Cluster/J or NDB API are not fully covered in the slides. We recommend our instructor-led training classes for such topics. • Please contact us for more details: services@severalnines.comCopyright 2011 Severalnines AB Control your database infrastructure 3

Full Training Agenda (1/4) • MySQL Cluster Introduction – MySQL eco system – Scale up, scale out, and sharding – MySQL Cluster Architecture – Use cases – Features – Node types and Roles • Detailed Concepts – Data Distribution – Verifying data distribution – Access Methods – Partitioning – Node failures and failure detection – Network Partitioning – Transactions and Locking – Consistency Model – Redo logging and CheckpointingCopyright 2011 Severalnines AB Control your database infrastructure 4

Agenda (2/4) • Installing MySQL Cluster – Setting up MySQL Cluster – Starting/stopping nodes – Recovery and restarts – Upgrading configuration – Upgrading Cluster • Performance Tuning (instructor-led only; contact us at services@severalnines.com) – Differences compared to Innodb/MyISAM – Designing efficient and fast applications – Identifying bottlenecks – Tweaking configuration (OS and MySQL Cluster) – Query Tuning – Schema Design – Index TuningCopyright 2011 Severalnines AB Control your database infrastructure 5

Agenda (3/4) • Management and Administration – Backup and Restore – Geographical Replication – Online and offline operations – Ndbinfo tables – Reporting – Single user mode – Scaling Cluster • Disk Data – Use cases – Limitations – Best practice configuration • Designing a Cluster – Capacity Planning and Dimensioning – Hardware recommendations – Best practice Configuration – Storage calculationsCopyright 2011 Severalnines AB Control your database infrastructure 6

Agenda (4/4) • Resolving Issues – Common problems – Error logs and Tracefiles – Recovery and Escalation procedures • Connectivity Overview – NDBAPI – Cluster/J – LDAP • Severalnines Tools – Monitoring and Management – Benchmarking – Sandboxes – Configuration and capacity planning • ConclusionCopyright 2011 Severalnines AB Control your database infrastructure 7

Agenda: Lab Exercises (only applicable to instructor-led training classes) • Lab Exercises – Installing and Loading data into MySQL Cluster – Starting/stopping nodes, recovery – Query tuning – Backup and Restore – Configuration Upgrade • Would you like to try something particular? – This is possible too, speak with your instructorCopyright 2011 Severalnines AB Control your database infrastructure 8

Prerequisites • Readers / Participants have understanding of SQL and basic database concepts. • Laptops/PCs for hands-on exercises • Linux: 1GB RAM • Windows: 2GB RAM • Approx. 20GB disk space and Virtualbox installed. • Virtualbox can be downloaded for free at http://www.virtualbox.org/wiki/Downloads • MySQL Cluster version 7.1 or laterCopyright 2011 Severalnines AB Control your database infrastructure 9

MySQL Cluster Architecture Application Application Application R/I/U/D R/I/U/D R/I/U/D Java, C++, LDAP Application R/I/U/D Native API DATA DATA NODE NODE Management Management Server DATA DATA Server NODE NODECopyright 2011 Severalnines AB Control your database infrastructure 13

Data Distribution • MySQL Cluster automatically partitions tables and distributes the data on the partitions. – It is possible to use user-defined partitioning. • A table is divided into as many partitions as you have data nodes and data is “striped” on the partitions. • Hashing of the Primary Key determines the Partition that should hold the data - automatic distribution. • It is possible to hash on part of the primary key – great for sharding and locality of data!Copyright 2011 Severalnines AB Control your database infrastructure 14

Partitioning and Data Distribution • With two data nodes, data node 1 and data node 2, each table is divided into two Partitions. • Subidis the Primary Key. • Hashing on the Primary Key (subid) determines the Partition. It is also possible to hash on part of the PK. – Odd PKs Green – Even PKs Red STORAGE LAYER subid data DATA DATA NODE 1 NODE 2 1 A Partition 0 3 B 2 C Partition 1 P0 P1 4 DCopyright 2011 Severalnines AB Control your database infrastructure 15

Replicas • To provide for redundancy and fast failover partitions are synchronouslyreplicated. • Two replicas (two copies of data) is most common – it is possible to have one (previous slide), two, three and four replicas. – NoOfReplicas=2 STORAGE LAYER DATA DATA NODE 1 NODE 2 P0 S0Copyright 2011 Severalnines AB Control your database infrastructure 16

Replicas • To provide for redundancy and fast failover partitions are synchronouslyreplicated from a PRIMARY to a SECONDARY partition. – When a change ( ) is made on P0 it is synchronously replicated to S0. – The change is persisted at transaction COMMIT. – Either both P0 or S0 is update or NOTHING AT ALL STORAGE LAYER DATA DATA NODE 1 NODE 2 P0 S0Copyright 2011 Severalnines AB Control your database infrastructure 17

Data Distribution - disk logging • After the COMMIT, the data is in main memory. – But changes are REDO LOGGED, but the REDO LOG is synced to disk every xth ms (1000 ms is recommended). • Controlled by TimeBetweenGlobalCheckpoints • Similar to innodb-flush-log-at-trx_commit=2 – The data is also checkpointed to disk. • More on this later. • The disk logging makes it possible to recovery a completely failed cluster.Copyright 2011 Severalnines AB Control your database infrastructure 18

Replicas • To provide for redundancy and fast failover partitions are synchronouslyreplicated. • Two replicas (two copies of data) is most common – it is possible to have one (previous slide), two, three and four replicas. – NoOfReplicas=2 STORAGE LAYER subid data DATA DATA NODE 1 NODE 2 1 A Partition 0 3 B 2 C Partition 1 P0 P1 4 D S1 S0 Px == PRIMARY Partition x Sx == SECONDARY Partition xCopyright 2011 Severalnines AB Control your database infrastructure 19

Node groups • Data Nodes sharing the same data data belongs to a node group. • A node group contains as many nodes as you have replicas (NoOfReplicas=2 is used in the picture below). STORAGE LAYER subid data DATA DATA NODE 1 NODE 2 1 A Partition 0 3 B 2 C Partition 1 P0 P1 4 D S1 S0 Node group 0 Px == PRIMARY Partition x Sx == SECONDARY ParttionxCopyright 2011 Severalnines AB Control your database infrastructure 20

Two Replicas – Four Data Nodes • Four data nodes – four partitions – two replicas • Four data nodes and two replicas  two node groups – No of node groups = #data nodes / #replicas STORAGE LAYER subid data 1 A Partition 0 2 B DATA DATA DATA DATA NODE 1 NODE 2 NODE 3 NODE 4 3 C Partition 1 4 D P0 P1 P2 P3 5 E Partition 2 6 F S1 S0 S3 S2 7 G Partition 3 Node group 0 Node group 1 8 H Px == PRIMARY Partition x Sx == SECONDARY ParttionxCopyright 2011 Severalnines AB Control your database infrastructure 21

Three Replicas – Three Data Nodes • Three data nodes – Three partitions – More replicas gives slower writes. • Three data nodes and three replicas  one node group. This is not common. – Two replicas is common practice/best practice STORAGE LAYER DATA DATA DATA subid data NODE 1 NODE 2 NODE 3 1 A Partition 0 2 B P0 P1 P2 3 C Partition 1 S1 S0 S0 4 D S2 S2 S1 5 E Partition 6 F 2 Node group 0 Px == PRIMARY Partition x Sx == SECONDARY ParttionxCopyright 2011 Severalnines AB Control your database infrastructure 22

Recommendations • Two Replicas recommended - gives best tradeoff between Performance and Availability • Three Replicas (or Four) will give slower writes (, is less tested (not so many deployments), and less deployments are made with this. • Cost of write (approx) – 1 replica (no redundancy): cost X – 2 replicas: cost 2X – 3 replicas: cost 3XCopyright 2011 Severalnines AB Control your database infrastructure 23

Verifying Data Distribution • A helper program called ndb_desc shows how many records are in each partition: ndb_desc –c<mgmt_host:1186> –d<database> -p<tablename>Copyright 2011 Severalnines AB Control your database infrastructure 24

View of the Data Node • Each node has as active Transaction Coordinator (TC). – Every Transaction starts at the TC – One transaction consists of one or more Operations. • Operation is SELECT, INSERT, UPDATE, DELETE • Each node has 1-4 Local Query Handlers (LQH) – Executes the operations TC • ACCess stores a hash table LQH – Primary Key and Unique • TUPle stores data records ACC TUP – ACC hash index links to record in TUPCopyright 2011 Severalnines AB Control your database infrastructure 26

Two Phase Commit Protocol • 2PC facilitates the synchronous replication – To safeguard that both Primary and Secondary Partitions are modified in a Consistent Way – Locking (Row level) • 2PC consists of two phases – Prepare Phase • Grab locks • Make a update to a shadow copy – Commit Phase • Make the shadow copy the real copy • Release locks • 2PC protocol is only involved in – Update, Delete, Insert • Reads are not 2PC (would be very slow and pointless)Copyright 2011 Severalnines AB Control your database infrastructure 27

Two Phase Commit Protocol • INSERT INTO t1(id, data) VALUES (1, „hello‟) – Two data nodes - PK hashes to Primary P0. – The same applies to UPDATES and DELETEs TC TC LQH LQH ACC TUP ACC TUP P0 P0 S0 S0 IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 28

Phase 1 - Prepare Phase • INSERT INTO t1(id, data) VALUES (1, „hello‟) • The protocol starts at the TC that has the Primary Partition for the PK TC TC LQH LQH ACC TUP ACC TUP P0 P0 S0 S0 IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 29

Phase 1 - Prepare Phase • TC starts by calculating a hash (md5sum) on the Entire Primary key. – md5sum(PK) – This gives 128 bit hash value, divided up in 2 x 64-bit INDEX_HASH PARTITION_HASH – Actually further calculations are made • PARTITION= PARTITION_HASH % NO_OF_PARTITIONS • INDEX_POS=INDEX_HASH % NO_OF_BUCKETS – LH*3 Algorithm is what is really being used. • Now MySQL Cluster will know what nodes and partitions are involved. • INSERT INTO t1(id, data) VALUES (1, „hello‟) – INDEX_POS=23 – PARTITION=P0Copyright 2011 Severalnines AB Control your database infrastructure 30

Phase 1 - Prepare Phase • INSERT INTO t1(id, data) VALUES (1, „hello‟) • Prepare on the local LQH TC TC LQH LQH ACC TUP ACC TUP P0 P0 S0 S0 IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 31

Phase 1 - Prepare Phase • Insert into the hash table in ACC and write record in TUP (allocate space for the record). – The Index Entry is locked now TC TC PREPARE LQH LQH ACC TUP ACC TUP P0 P0 S0 S0 1 helloPOS=23 IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2 Copyright 2011 Severalnines AB Control your database infrastructure 32

Phase 1 - Prepare Phase • Send PREPARE message to the other LQH for the secondary Partition (S0), and do exactly the same thing. TC TC PREPARE PREPARE LQH LQH ACC TUP ACC TUP P0 P0 S0 S0 1 hello 1 hello IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 33

Phase 1 - Prepare Phase • Send back PREPARE OK to TC TC PREPARE OK TC PREPARE PREPARE LQH LQH ACC TUP ACC TUP P0 P0 S0 S0 1 hello 1 hello IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 34

Phase 2 - Commit Phase • Send COMMIT message to REMOTE LQH first – Release locks, simplifies error handling, ops waiting on lock can resume faster. TC COMMIT TC LQH LQH ACC TUP ACC TUP P0 P0 S0 S0 1 hello 1 hello IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 35

Phase 2 - Commit Phase • Send COMMIT to LQH of P0 – Release locks TC COMMIT TC COMMIT LQH LQH ACC TUP ACC TUP P0 P0 S0 S0 1 hello 1 hello IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 36

Phase 2 - Commit Phase • Send COMMIT OK to TC TC COMMIT TC COMMIT OK COMMIT LQH LQH ACC TUP ACC TUP P0 P0 S0 S0 1 hello 1 hello IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 37

Phase 2 - Commit Phase • Send COMMIT ACK to application COMMIT ACK TC COMMIT TC COMMIT OK COMMIT LQH LQH ACC TUP ACC TUP P0 P0 S0 S0 1 hello 1 hello IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 38

Failure Handling • If the Transaction Coordinator fails another TC will take over. • The new TC will ask the nodes “Send me a list of outstanding transactions from the failed TC” • The new TC will – ABORT transactions in the Prepare Phase – Force COMMIT of transactions IFF another node has seen a COMMIT message. • The Node that has failed will get the changes when it recovers.Copyright 2011 Severalnines AB Control your database infrastructure 39

Transaction Timers • Deadlock detection – waiting for lock – If a transaction is waiting too long time for a lock it is aborted • TransactionDeadLockDetectionTimeout=3000 [ms] • Inactive Transactions – holding a lock – If a transaction is holding a lock too long it is aborted • TransactionInactiveTimeout=60000 [ms] – Never set TransactioInactiveTimeout=0Copyright 2011 Severalnines AB Control your database infrastructure 40

Locking • Committed Read Isolation level is used • This means that a transaction – Will read last committed data unless it has itself modified this data. • Locks are acquired in the Prepare Phase and released in the Commit Phase.Copyright 2011 Severalnines AB Control your database infrastructure 41

Transaction and Operation Records • Each transaction requires on Transaction Record (TR) – When it was started – Controlled by MaxNoOfConcurrentTransactions • Also each Operation requires one Operation Record (OR) in TC and one Local Operation Record (LOR) in LQH – What transaction it belongs to – Operation Type – Controlled by MaxNoOfConcurrentOperations – and MaxNoOfLocalOperationsCopyright 2011 Severalnines AB Control your database infrastructure 43

Transaction and Operation Records • Assume – MaxNoOfConcurrentTransactions=3 – MaxNoOfConcurrentOperations=3 – MaxNoOfLocalOperations=4 • By default there are 10% more Local Ops than Conc Ops. • Query – INSERT INTO t1 (id,data) VALUES(1,‟x‟); • One transaction • One operationCopyright 2011 Severalnines AB Control your database infrastructure 44

Transaction Starts (prepare phase) • INSERT INTO t1 (id,data) VALUES(1,‟x‟);TR TC TCOR PREPARE PREPARELOR LQH LQH LOR ACC TUP ACC TUP P0 P0 S0 S0 IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 45

Transaction Starts (prepare phase) • Allocate a Transaction Record at TCTR TC TCOR PREPARE PREPARELOR LQH LQH LOR ACC TUP ACC TUP P0 P0 S0 S0 IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 46

Transaction Starts (prepare phase) • Allocate a Operation Record at TCTR TC TCOR PREPARE PREPARELOR LQH LQH LOR ACC TUP ACC TUP P0 P0 S0 S0 IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 47

Transaction Starts (prepare phase) • Allocate a Local Operation Record at LQHTR TC TCOR PREPARE PREPARELOR LQH LQH LOR ACC TUP ACC TUP P0 P0 S0 S0 IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 48

Transaction Starts (prepare phase) • With many parallel transactionsTR TR TC TCOR OR PREPARE PREPARELOR LQH LQH LOR ACC TUP ACC TUP P0 P0 S0 S0 IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 49

Transaction Starts (prepare phase) • Wrongly dimensioned for failover • Not enough TRs / ORsTR TR TC TCOR OR PREPARE PREPARELOR LQH LQH LOR ACC TUP ACC TUP P0 P0 S0 S0 IndexMemory DataMemory IndexMemory DataMemory DATA NODE 1 DATA NODE 2Copyright 2011 Severalnines AB Control your database infrastructure 50

Transaction and Operation Records • Define – MaxNoOfConcurrentTransactions= 2 x<max_threads> – MaxNoOfConcurrentOperations= MaxConcurrentTransactionsx<ops in trans> – MaxNoOfLocalOperations=1.1 x MaxNoOfConcurrentOperations • 1.1x is by default • Seldom needed to change this.Copyright 2011 Severalnines AB Control your database infrastructure 51

Coming next in Part 3: Detailed Concepts Access Methods Redo Logging and Check-pointing Recovery Node Failures and Error Detection Network PartitioningCopyright 2011 Severalnines AB Control your database infrastructure 52

We hope these training slides are useful to you! Please visit our website to view the next section of this training. For any questions, comments, feedback or to book a training class, please contact us at: services@severalnines.com Thank you!Copyright 2011 Severalnines AB Control your database infrastructure 53

Disclaimer © Copyright 2011 Severalnines AB. All rights reserved. Severalnines& the Severalnineslogo(s) are trademarks of Severalnines AB. MySQL is a registered trademark of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.Copyright 2011 Severalnines AB Control your database infrastructure 54

http://www.severalnines.com	533
http://severalnines.com	349

Severalnines Self-Training: MySQL® Cluster - Part II

by Severalnines AB on Nov 14, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 882

Statistics