MySQL Cluster Training                                 presented by severalnines.com                                      ...
Full Training Agenda (1/4)       •    MySQL Cluster Introduction              –   MySQL eco system              –   Scale ...
Agenda (2/4)       •    Installing MySQL Cluster              –   Setting up MySQL Cluster              –   Starting/stopp...
Agenda (3/4)       •    Management and Administration              –   Backup and Restore              –   Geographical Re...
Agenda (4/4)       •    Resolving Issues              –   Common problems              –   Error logs and Tracefiles      ...
Agenda: Lab Exercises            (only applicable to instructor-led training classes)       •    Lab Exercises            ...
Prerequisites       •    Readers / Participants have understanding of SQL and basic database concepts.       •    Laptops/...
8th Installment              Severalnines Cluster Self-Training                                 Part 7: Disk DataCopyright...
Topics covered in Installment 8       • Use cases       • Limitations       • Best practice configurationCopyright 2011 Se...
Use Cases For Disk Data       • By default MySQL Cluster tables store data in RAM       • If there is more data than RAM t...
Use Cases For Disk Data       • Avoid long running transactions and big         UPDATEs/DELETEs. Instead, divide work into...
In-Memory vs Disk Data Tables       • Transparency              – Accessing disk data/in-memory tables is the same from th...
Use Cases For Disk Data       • Disk data tables are usually used for applications              – That does not require <5...
Concepts       • Data is stored in TABLE SPACEs              – There can be several table spaces       • A TABLE SPACE con...
Concepts       • Non-indexed columns are stored on DISK.       • Indexed columns are stored in RAM (DataMemory).       • V...
Disk Data Tables       • DiskPageBufferMemory is a LRU page cache                                                         ...
Disk Data Tables - READ       • Cache miss – read page from TABLESPACE                                                    ...
Disk Data Tables - WRITE       • Cache miss – read page from TABLESPACE                                                   ...
Configuration Parameters       • DiskPageBufferMemory=4096M              – Similar to the Innodb_buffer_pool_size         ...
CREATING THE LOGFILE GROUP       CREATE LOGFILE GROUP lg ADD UNDOFILE       ‘undo_0.dat’ INITIAL_SIZE=20G       UNDO_BUFFE...
CREATING THE TABLESPACE       CREATE TABLESPACE ts_0 ADD DATAFILE       data_0.dat USE LOGFILE GROUP lg       INITIAL_SIZE...
CREATING A TABLE IN A TABLESPACE       CREATE TABLE t1 (          a INTEGER AUTO PRIMARY KEY,          b VARBINARY(2048)) ...
Storage Subsystem       • There are for parts that wants to be written to disk:              – LCP              – REDOLOG ...
Storage Subsystem       • SAS 10K RPM or 15K RPM disks are proven to work         very well.       • A good disk layout ca...
Monitor DiskData with ClusterControl       • ClusterControl monitors your tablespaceCopyright 2011 Severalnines AB        ...
Summary       • UNDOLOG – written “all the time”              – Circular, WAL (write ahead logging)       • Tablespace – p...
Coming next in Installment 9:                                 Designing a ClusterCopyright 2011 Severalnines AB           ...
We hope these training slides are                        useful to you!            Please visit our website to view the   ...
Disclaimer © Copyright 2011 Severalnines AB. All rights reserved. Severalnines& the Severalnineslogo(s) are trademarks of ...
Upcoming SlideShare
Loading in...5
×

Severalnines Self-Training: MySQL® Cluster - Part VIII

9,909

Published on

Part VIII of our self-training slides on MySQL Cluster.

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
9,909
On Slideshare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
0
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Transcript of "Severalnines Self-Training: MySQL® Cluster - Part VIII"

    1. 1. MySQL Cluster Training presented by severalnines.com Address: SeveralninesAB Contact: c/o SICS, Box 1263 services@severalnines.com Isafjordsgatan22 SE-164-29 KistaCopyright 2011 Severalnines AB Control your database infrastructure 1
    2. 2. Full Training Agenda (1/4) • MySQL Cluster Introduction – MySQL eco system – Scale up, scale out, and sharding – MySQL Cluster Architecture – Use cases – Features – Node types and Roles • Detailed Concepts – Data Distribution – Verifying data distribution – Access Methods – Partitioning – Node failures and failure detection – Network Partitioning – Transactions and Locking – Consistency Model – Redo logging and Checkpointing • Internals – NDB Design InternalsCopyright 2011 Severalnines AB Control your database infrastructure 2
    3. 3. Agenda (2/4) • Installing MySQL Cluster – Setting up MySQL Cluster – Starting/stopping nodes – Recovery and restarts – Upgrading configuration – Upgrading Cluster • Performance Tuning (instructor-led only; contact us at services@severalnines.com) – Differences compared to Innodb/MyISAM – Designing efficient and fast applications – Identifying bottlenecks – Tweaking configuration (OS and MySQL Cluster) – Query Tuning – Schema Design – Index TuningCopyright 2011 Severalnines AB Control your database infrastructure 3
    4. 4. Agenda (3/4) • Management and Administration – Backup and Restore – Geographical Replication – Online and offline operations – Ndbinfo tables – Reporting – Single user mode – Scaling Cluster • Disk Data – Use cases – Limitations – Best practice configuration • Designing a Cluster – Capacity Planning and Dimensioning – Hardware recommendations – Best practice Configuration – Storage calculationsCopyright 2011 Severalnines AB Control your database infrastructure 4
    5. 5. Agenda (4/4) • Resolving Issues – Common problems – Error logs and Tracefiles – Recovery and Escalation procedures • Connectivity Overview – NDBAPI – Cluster/J – LDAP • Severalnines Tools – Monitoring and Management – Benchmarking – Sandboxes – Configuration and capacity planning • ConclusionCopyright 2011 Severalnines AB Control your database infrastructure 5
    6. 6. Agenda: Lab Exercises (only applicable to instructor-led training classes) • Lab Exercises – Installing and Loading data into MySQL Cluster – Starting/stopping nodes, recovery – Query tuning – Backup and Restore – Configuration Upgrade • Would you like to try something particular? – This is possible too, speak with your instructorCopyright 2011 Severalnines AB Control your database infrastructure 6
    7. 7. Prerequisites • Readers / Participants have understanding of SQL and basic database concepts. • Laptops/PCs for hands-on exercises • Linux: 1GB RAM • Windows: 2GB RAM • Approx. 20GB disk space and Virtualbox installed. • Virtualbox can be downloaded for free at http://www.virtualbox.org/wiki/Downloads • MySQL Cluster version 7.1 or laterCopyright 2011 Severalnines AB Control your database infrastructure 7
    8. 8. 8th Installment Severalnines Cluster Self-Training Part 7: Disk DataCopyright 2011 Severalnines AB Control your database infrastructure 8
    9. 9. Topics covered in Installment 8 • Use cases • Limitations • Best practice configurationCopyright 2011 Severalnines AB Control your database infrastructure 9
    10. 10. Use Cases For Disk Data • By default MySQL Cluster tables store data in RAM • If there is more data than RAM then disk data tables can be used. • Disk data tables works in a similar way as innodb with – Innodb_flush_log_at_tx_commit=2 – Both use a bufferpool (innodb_buffer_poolvsdiskpagebuffer) to cache data (LRU). – Innodb disk tables are usually performing better for index scans that hits the disk.Copyright 2011 Severalnines AB Control your database infrastructure 10
    11. 11. Use Cases For Disk Data • Avoid long running transactions and big UPDATEs/DELETEs. Instead, divide work into chunks: – DELETE … LIMIT 1000; • Or TRUNCATE TABLE to empty the table – UPDATE nnn SET .. WHERE x=y LIMIT 1000; • JOINs, even pushed will be slow on Disk Data tables.Copyright 2011 Severalnines AB Control your database infrastructure 11
    12. 12. In-Memory vs Disk Data Tables • Transparency – Accessing disk data/in-memory tables is the same from the users point of view • Performance – In-memory tables should be used for RT traffic – Queries hitting the disk will be slow, like any DB. Avoid the disk with caches and/or us SSD.Copyright 2011 Severalnines AB Control your database infrastructure 12
    13. 13. Use Cases For Disk Data • Disk data tables are usually used for applications – That does not require <5ms response time • Examples: – Log / archival data – MMS/SMS messages • There are sites with > 3TB of data in disk data tables. • It is possible to have both memory and disk data tables at the same time. • Don’t store e.g, images/videos in BLOBs, regardless if you use Disk Data tables or In-memory tables. They are better kept on the filesystem, and only meta data should be kept inside the database.Copyright 2011 Severalnines AB Control your database infrastructure 13
    14. 14. Concepts • Data is stored in TABLE SPACEs – There can be several table spaces • A TABLE SPACE contains – one or more data files – more data files can be added online • One LOGFILE GROUP is associated with each table space and contains – One UNDO BUFFER • The size cannot be changed later on! – One or more UNDO LOG files – There can be only one LOGFILE GROUP – More undo files can be added onlineCopyright 2011 Severalnines AB Control your database infrastructure 14
    15. 15. Concepts • Non-indexed columns are stored on DISK. • Indexed columns are stored in RAM (DataMemory). • VAR* columns on disk are treated as fixed size (up to and including MySQL Cluster 7.1) – VARBINARY(1024)  1024B of storage INDEXED COLUMN DATATYPE YES ID BIGINT NO DATA1 VARBINARY(1024) YES TS TIMESTAMP ID TS DATA1 1 1111 aaaa 2 2222 bbbbbbCopyright 2011 Severalnines AB Control your database infrastructure 15
    16. 16. Disk Data Tables • DiskPageBufferMemory is a LRU page cache Data Node 0 LQH PGMAN LGMAN DiskPageBufferMemory ACC TUP #ref DATA1 DataMemory IM ID TS #ref P0 1 1111 &11 REDOBUFFER 1… &11 aaaa 1… REDO LOG LCP TABLESPACE UNDOLOGCopyright 2011 Severalnines AB Control your database infrastructure 16
    17. 17. Disk Data Tables - READ • Cache miss – read page from TABLESPACE Data Node 0 LQH PGMAN LGMAN DiskPageBufferMemory ACC TUP #ref DATA1 P0 DataMemory &11 aaaa IM ID TS #ref P0 1 1111 &11 REDOBUFFER 1… &11 aaaa 1… REDO LOG LCP TABLESPACE UNDOLOGCopyright 2011 Severalnines AB Control your database infrastructure 17
    18. 18. Disk Data Tables - WRITE • Cache miss – read page from TABLESPACE Data Node 0 LQH PGMAN LGMAN DiskPageBufferMemory ACC TUP #ref DATA1 P0 DataMemory &11 aaaa IM &22 bbbb ID TS #ref P0 1 1111 &11 2 2222 &22 REDOBUFFER 1… 2… &11 &22bbbb 1… 2… REDO LOG LCP TABLESPACE UNDOLOGCopyright 2011 Severalnines AB Control your database infrastructure 18
    19. 19. Configuration Parameters • DiskPageBufferMemory=4096M – Similar to the Innodb_buffer_pool_size – The bigger, the more records can be cached, increases probability the data is cached, which increases performance – Updates/inserts are perfomed in the cache and later checkpointed to the table space. • SharedGlobalMemory=512M – Allocate between 64-128M for the UNDO_BUFFER – Meta data structures are also allocated from this resource.Copyright 2011 Severalnines AB Control your database infrastructure 19
    20. 20. CREATING THE LOGFILE GROUP CREATE LOGFILE GROUP lg ADD UNDOFILE ‘undo_0.dat’ INITIAL_SIZE=20G UNDO_BUFFER_SIZE=128M ENGINE=NDB; • This creates a LOGFILE GROUP with one UNDOFILE that is 20GB in size. – More Undo files can be added later (online, no downtime): – ALTER LOGFILE GROUP lg ADD UNDOFILE ‘undo_1.dat’ INITIAL_SIZE 20GB ENGINE=NDB; • The UNDO_BUFFER_SIZE is set to 128M RAM, requires a SharedGlobalMemory=512MCopyright 2011 Severalnines AB Control your database infrastructure 20
    21. 21. CREATING THE TABLESPACE CREATE TABLESPACE ts_0 ADD DATAFILE data_0.dat USE LOGFILE GROUP lg INITIAL_SIZE=100G ENGINE=NDB; • This creates a TABLESPACE ts_0 with one DATAFILE that is 100GB in size. • More datafiles and more tablespaces can be created/added online: – ALTER TABLESPACE ts_0 ADD DATAFILE data_1.dat INITIAL_SIZE=100GB ENGINE=NDB;Copyright 2011 Severalnines AB Control your database infrastructure 21
    22. 22. CREATING A TABLE IN A TABLESPACE CREATE TABLE t1 ( a INTEGER AUTO PRIMARY KEY, b VARBINARY(2048)) ENGINE=ndb TABLESPACE ts_0 STORAGE DISK; • Column ’b’ will be stored in table space ts_0. Column ’a’ will be stored in DataMemory.Copyright 2011 Severalnines AB Control your database infrastructure 22
    23. 23. Storage Subsystem • There are for parts that wants to be written to disk: – LCP – REDOLOG (the same redo log files are used for disk data tables and in memory tables – UNDO LOG – TABLESPACE • In many cases you can put: – On one disk partition: • LCP/REDO LOG/ UNDO LOG – On another disk partition • TABLESPACE • If you are going to READ a lot from the TABLESPACE, then use SSD for the TABLESPACE data filesCopyright 2011 Severalnines AB Control your database infrastructure 23
    24. 24. Storage Subsystem • SAS 10K RPM or 15K RPM disks are proven to work very well. • A good disk layout can look like: – RAID 1+0 (4 disks) for LCP/REDO/UNDO – RAID 1+0 (4 disks) for TABLESPACE OR – RAID 1+0 (4 disks) for LCP/REDO – RAID 1+0 (4 disks SAS) for TABLESPACE/UNDO • The main point here is that LCP/REDO/UNDO files are only written too. TABLESPACE files are both READ/WRITE, and READs are most commonly RANDOM. – Again, use SSD or a separate set of disks for TABLESPACE data if you will issue READ requests.Copyright 2011 Severalnines AB Control your database infrastructure 24
    25. 25. Monitor DiskData with ClusterControl • ClusterControl monitors your tablespaceCopyright 2011 Severalnines AB Control your database infrastructure 25
    26. 26. Summary • UNDOLOG – written “all the time” – Circular, WAL (write ahead logging) • Tablespace – pages checkpointed from DiskPageBufferMemory – First DataMemory is LCPd – Then DiskPageBufferMemory • A large DiskPageBufferMemory cache is very important for performace – After dimensioning as much RAM as possible should be allocated to the DiskPageBufferMemory. • Use in-memory tables when response times are critical. • A good fast disk subsystem is neededCopyright 2011 Severalnines AB Control your database infrastructure 26
    27. 27. Coming next in Installment 9: Designing a ClusterCopyright 2011 Severalnines AB Control your database infrastructure 27
    28. 28. We hope these training slides are useful to you! Please visit our website to view the next section of this training. For any questions, comments, feedback or to book a training class, please contact us at: services@severalnines.com Thank you!Copyright 2011 Severalnines AB Control your database infrastructure 28
    29. 29. Disclaimer © Copyright 2011 Severalnines AB. All rights reserved. Severalnines& the Severalnineslogo(s) are trademarks of Severalnines AB. MySQL is a registered trademark of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.Copyright 2011 Severalnines AB Control your database infrastructure 29

    ×