• Save
Severalnines Self-Training: MySQL® Cluster - Part VIII
 

Like this? Share it with your network

Share

Severalnines Self-Training: MySQL® Cluster - Part VIII

on

  • 8,167 views

Part VIII of our self-training slides on MySQL Cluster.

Part VIII of our self-training slides on MySQL Cluster.

Statistics

Views

Total Views
8,167
Views on SlideShare
724
Embed Views
7,443

Actions

Likes
3
Downloads
0
Comments
0

4 Embeds 7,443

http://www.severalnines.com 7216
http://severalnines.com 222
http://translate.googleusercontent.com 3
http://webcache.googleusercontent.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com

Severalnines Self-Training: MySQL® Cluster - Part VIII Presentation Transcript

  • 1. MySQL Cluster Training presented by severalnines.com Address: SeveralninesAB Contact: c/o SICS, Box 1263 services@severalnines.com Isafjordsgatan22 SE-164-29 KistaCopyright 2011 Severalnines AB Control your database infrastructure 1
  • 2. Full Training Agenda (1/4) • MySQL Cluster Introduction – MySQL eco system – Scale up, scale out, and sharding – MySQL Cluster Architecture – Use cases – Features – Node types and Roles • Detailed Concepts – Data Distribution – Verifying data distribution – Access Methods – Partitioning – Node failures and failure detection – Network Partitioning – Transactions and Locking – Consistency Model – Redo logging and Checkpointing • Internals – NDB Design InternalsCopyright 2011 Severalnines AB Control your database infrastructure 2
  • 3. Agenda (2/4) • Installing MySQL Cluster – Setting up MySQL Cluster – Starting/stopping nodes – Recovery and restarts – Upgrading configuration – Upgrading Cluster • Performance Tuning (instructor-led only; contact us at services@severalnines.com) – Differences compared to Innodb/MyISAM – Designing efficient and fast applications – Identifying bottlenecks – Tweaking configuration (OS and MySQL Cluster) – Query Tuning – Schema Design – Index TuningCopyright 2011 Severalnines AB Control your database infrastructure 3
  • 4. Agenda (3/4) • Management and Administration – Backup and Restore – Geographical Replication – Online and offline operations – Ndbinfo tables – Reporting – Single user mode – Scaling Cluster • Disk Data – Use cases – Limitations – Best practice configuration • Designing a Cluster – Capacity Planning and Dimensioning – Hardware recommendations – Best practice Configuration – Storage calculationsCopyright 2011 Severalnines AB Control your database infrastructure 4
  • 5. Agenda (4/4) • Resolving Issues – Common problems – Error logs and Tracefiles – Recovery and Escalation procedures • Connectivity Overview – NDBAPI – Cluster/J – LDAP • Severalnines Tools – Monitoring and Management – Benchmarking – Sandboxes – Configuration and capacity planning • ConclusionCopyright 2011 Severalnines AB Control your database infrastructure 5
  • 6. Agenda: Lab Exercises (only applicable to instructor-led training classes) • Lab Exercises – Installing and Loading data into MySQL Cluster – Starting/stopping nodes, recovery – Query tuning – Backup and Restore – Configuration Upgrade • Would you like to try something particular? – This is possible too, speak with your instructorCopyright 2011 Severalnines AB Control your database infrastructure 6
  • 7. Prerequisites • Readers / Participants have understanding of SQL and basic database concepts. • Laptops/PCs for hands-on exercises • Linux: 1GB RAM • Windows: 2GB RAM • Approx. 20GB disk space and Virtualbox installed. • Virtualbox can be downloaded for free at http://www.virtualbox.org/wiki/Downloads • MySQL Cluster version 7.1 or laterCopyright 2011 Severalnines AB Control your database infrastructure 7
  • 8. 8th Installment Severalnines Cluster Self-Training Part 7: Disk DataCopyright 2011 Severalnines AB Control your database infrastructure 8
  • 9. Topics covered in Installment 8 • Use cases • Limitations • Best practice configurationCopyright 2011 Severalnines AB Control your database infrastructure 9
  • 10. Use Cases For Disk Data • By default MySQL Cluster tables store data in RAM • If there is more data than RAM then disk data tables can be used. • Disk data tables works in a similar way as innodb with – Innodb_flush_log_at_tx_commit=2 – Both use a bufferpool (innodb_buffer_poolvsdiskpagebuffer) to cache data (LRU). – Innodb disk tables are usually performing better for index scans that hits the disk.Copyright 2011 Severalnines AB Control your database infrastructure 10
  • 11. Use Cases For Disk Data • Avoid long running transactions and big UPDATEs/DELETEs. Instead, divide work into chunks: – DELETE … LIMIT 1000; • Or TRUNCATE TABLE to empty the table – UPDATE nnn SET .. WHERE x=y LIMIT 1000; • JOINs, even pushed will be slow on Disk Data tables.Copyright 2011 Severalnines AB Control your database infrastructure 11
  • 12. In-Memory vs Disk Data Tables • Transparency – Accessing disk data/in-memory tables is the same from the users point of view • Performance – In-memory tables should be used for RT traffic – Queries hitting the disk will be slow, like any DB. Avoid the disk with caches and/or us SSD.Copyright 2011 Severalnines AB Control your database infrastructure 12
  • 13. Use Cases For Disk Data • Disk data tables are usually used for applications – That does not require <5ms response time • Examples: – Log / archival data – MMS/SMS messages • There are sites with > 3TB of data in disk data tables. • It is possible to have both memory and disk data tables at the same time. • Don’t store e.g, images/videos in BLOBs, regardless if you use Disk Data tables or In-memory tables. They are better kept on the filesystem, and only meta data should be kept inside the database.Copyright 2011 Severalnines AB Control your database infrastructure 13
  • 14. Concepts • Data is stored in TABLE SPACEs – There can be several table spaces • A TABLE SPACE contains – one or more data files – more data files can be added online • One LOGFILE GROUP is associated with each table space and contains – One UNDO BUFFER • The size cannot be changed later on! – One or more UNDO LOG files – There can be only one LOGFILE GROUP – More undo files can be added onlineCopyright 2011 Severalnines AB Control your database infrastructure 14
  • 15. Concepts • Non-indexed columns are stored on DISK. • Indexed columns are stored in RAM (DataMemory). • VAR* columns on disk are treated as fixed size (up to and including MySQL Cluster 7.1) – VARBINARY(1024)  1024B of storage INDEXED COLUMN DATATYPE YES ID BIGINT NO DATA1 VARBINARY(1024) YES TS TIMESTAMP ID TS DATA1 1 1111 aaaa 2 2222 bbbbbbCopyright 2011 Severalnines AB Control your database infrastructure 15
  • 16. Disk Data Tables • DiskPageBufferMemory is a LRU page cache Data Node 0 LQH PGMAN LGMAN DiskPageBufferMemory ACC TUP #ref DATA1 DataMemory IM ID TS #ref P0 1 1111 &11 REDOBUFFER 1… &11 aaaa 1… REDO LOG LCP TABLESPACE UNDOLOGCopyright 2011 Severalnines AB Control your database infrastructure 16
  • 17. Disk Data Tables - READ • Cache miss – read page from TABLESPACE Data Node 0 LQH PGMAN LGMAN DiskPageBufferMemory ACC TUP #ref DATA1 P0 DataMemory &11 aaaa IM ID TS #ref P0 1 1111 &11 REDOBUFFER 1… &11 aaaa 1… REDO LOG LCP TABLESPACE UNDOLOGCopyright 2011 Severalnines AB Control your database infrastructure 17
  • 18. Disk Data Tables - WRITE • Cache miss – read page from TABLESPACE Data Node 0 LQH PGMAN LGMAN DiskPageBufferMemory ACC TUP #ref DATA1 P0 DataMemory &11 aaaa IM &22 bbbb ID TS #ref P0 1 1111 &11 2 2222 &22 REDOBUFFER 1… 2… &11 &22bbbb 1… 2… REDO LOG LCP TABLESPACE UNDOLOGCopyright 2011 Severalnines AB Control your database infrastructure 18
  • 19. Configuration Parameters • DiskPageBufferMemory=4096M – Similar to the Innodb_buffer_pool_size – The bigger, the more records can be cached, increases probability the data is cached, which increases performance – Updates/inserts are perfomed in the cache and later checkpointed to the table space. • SharedGlobalMemory=512M – Allocate between 64-128M for the UNDO_BUFFER – Meta data structures are also allocated from this resource.Copyright 2011 Severalnines AB Control your database infrastructure 19
  • 20. CREATING THE LOGFILE GROUP CREATE LOGFILE GROUP lg ADD UNDOFILE ‘undo_0.dat’ INITIAL_SIZE=20G UNDO_BUFFER_SIZE=128M ENGINE=NDB; • This creates a LOGFILE GROUP with one UNDOFILE that is 20GB in size. – More Undo files can be added later (online, no downtime): – ALTER LOGFILE GROUP lg ADD UNDOFILE ‘undo_1.dat’ INITIAL_SIZE 20GB ENGINE=NDB; • The UNDO_BUFFER_SIZE is set to 128M RAM, requires a SharedGlobalMemory=512MCopyright 2011 Severalnines AB Control your database infrastructure 20
  • 21. CREATING THE TABLESPACE CREATE TABLESPACE ts_0 ADD DATAFILE data_0.dat USE LOGFILE GROUP lg INITIAL_SIZE=100G ENGINE=NDB; • This creates a TABLESPACE ts_0 with one DATAFILE that is 100GB in size. • More datafiles and more tablespaces can be created/added online: – ALTER TABLESPACE ts_0 ADD DATAFILE data_1.dat INITIAL_SIZE=100GB ENGINE=NDB;Copyright 2011 Severalnines AB Control your database infrastructure 21
  • 22. CREATING A TABLE IN A TABLESPACE CREATE TABLE t1 ( a INTEGER AUTO PRIMARY KEY, b VARBINARY(2048)) ENGINE=ndb TABLESPACE ts_0 STORAGE DISK; • Column ’b’ will be stored in table space ts_0. Column ’a’ will be stored in DataMemory.Copyright 2011 Severalnines AB Control your database infrastructure 22
  • 23. Storage Subsystem • There are for parts that wants to be written to disk: – LCP – REDOLOG (the same redo log files are used for disk data tables and in memory tables – UNDO LOG – TABLESPACE • In many cases you can put: – On one disk partition: • LCP/REDO LOG/ UNDO LOG – On another disk partition • TABLESPACE • If you are going to READ a lot from the TABLESPACE, then use SSD for the TABLESPACE data filesCopyright 2011 Severalnines AB Control your database infrastructure 23
  • 24. Storage Subsystem • SAS 10K RPM or 15K RPM disks are proven to work very well. • A good disk layout can look like: – RAID 1+0 (4 disks) for LCP/REDO/UNDO – RAID 1+0 (4 disks) for TABLESPACE OR – RAID 1+0 (4 disks) for LCP/REDO – RAID 1+0 (4 disks SAS) for TABLESPACE/UNDO • The main point here is that LCP/REDO/UNDO files are only written too. TABLESPACE files are both READ/WRITE, and READs are most commonly RANDOM. – Again, use SSD or a separate set of disks for TABLESPACE data if you will issue READ requests.Copyright 2011 Severalnines AB Control your database infrastructure 24
  • 25. Monitor DiskData with ClusterControl • ClusterControl monitors your tablespaceCopyright 2011 Severalnines AB Control your database infrastructure 25
  • 26. Summary • UNDOLOG – written “all the time” – Circular, WAL (write ahead logging) • Tablespace – pages checkpointed from DiskPageBufferMemory – First DataMemory is LCPd – Then DiskPageBufferMemory • A large DiskPageBufferMemory cache is very important for performace – After dimensioning as much RAM as possible should be allocated to the DiskPageBufferMemory. • Use in-memory tables when response times are critical. • A good fast disk subsystem is neededCopyright 2011 Severalnines AB Control your database infrastructure 26
  • 27. Coming next in Installment 9: Designing a ClusterCopyright 2011 Severalnines AB Control your database infrastructure 27
  • 28. We hope these training slides are useful to you! Please visit our website to view the next section of this training. For any questions, comments, feedback or to book a training class, please contact us at: services@severalnines.com Thank you!Copyright 2011 Severalnines AB Control your database infrastructure 28
  • 29. Disclaimer © Copyright 2011 Severalnines AB. All rights reserved. Severalnines& the Severalnineslogo(s) are trademarks of Severalnines AB. MySQL is a registered trademark of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.Copyright 2011 Severalnines AB Control your database infrastructure 29