Your SlideShare is downloading. ×
MySQL Cluster Training                                 presented by severalnines.com                                      ...
Introduction       • At Severalnines, we believe in sharing information         and knowledge; we all come from an open so...
Training Slides - Concept       • Over the coming weeks we will be chronologically         releasing slides for the differ...
Full Training Agenda (1/4)       •    MySQL Cluster Introduction              –   MySQL eco system              –   Scale ...
Agenda (2/4)       •    Installing MySQL Cluster              –   Setting up MySQL Cluster              –   Starting/stopp...
Agenda (3/4)       •    Management and Administration              –   Backup and Restore              –   Geographical Re...
Agenda (4/4)       •    Resolving Issues              –   Common problems              –   Error logs and Tracefiles      ...
Agenda: Lab Exercises            (only applicable to instructor-led training classes)       •    Lab Exercises            ...
Prerequisites       •    Readers / Participants have understanding of SQL and basic database concepts.       •    Laptops/...
6th Installment              Severalnines Cluster Self-Training                 Part 4: Installing MySQL ClusterCopyright ...
Topics covered in Installment 6     • Configuration     • InstallationCopyright 2011 Severalnines AB             Control y...
Part 4: Installing MySQL Cluster                    Configuration and InstallationCopyright 2011 Severalnines AB          ...
Topics       •    MySQL Cluster configuration (config.ini)       •    MySQL Server configuration (my.cnf)       •    Impor...
MySQL Cluster configuration (config.ini)       • There are no parameter in config.ini that affects         performance.   ...
MySQL Cluster configuration (config.ini)       [ndb_mgmd default]       Datadir=/data/mysqlcluster       Portnumber=1186  ...
MySQL Server (my.cnf)       [mysqld]       basedir=/usr/local/mysql/       datadir=/data/mysql       socket=/tmp/mysql.soc...
Preparations       • Download MySQL Cluster from         dev.mysql.com/downloads/cluster       • On every machine         ...
Preparations       • On all Management nodes              – Put config.ini in /etc/mysql/       • On all MySQL servers    ...
Minimal [ndbd default]       [ndbd default]       Datadir=/data/mysqlcluster/       NoOfReplicas=2       DataMemory=80M   ...
MySQL Server parameters       • The following parameters affects performance              – Ndb_cluster_connection_pool=8 ...
Installing ClusterCopyright 2011 Severalnines AB                        Control your database infrastructure   21
Topics       •    Setting up MySQL Cluster       •    Starting/stopping nodes       •    Recovery and restarts       •    ...
Using the Configurator       • We do recommend to use the Severalnines Cluster         Configurator              – www.sev...
ClusterControl       • Assists and automates mundane tasks:              –   Configuration Management              –   Upg...
ClusterControlCopyright 2011 Severalnines AB                    Control your database infrastructure   25
ClusterControlCopyright 2011 Severalnines AB                    Control your database infrastructure   26
ClusterControlCopyright 2011 Severalnines AB                    Control your database infrastructure   27
Starting MySQL Cluster       • Management nodes must be started first              – Otherwise no other node can join the ...
Simple management       • The Management Client (ndb_mgm) can be used to         to perform simple management:            ...
Rolling Restarts       • Rolling restarts are used to perform changes on a         running MySQL Cluster              – Co...
Rolling Restarts       • Stop the management servers              – kill -15 `pidof ndb_mgmd`       • Start the management...
Rolling Restarts       • Rolling restarts are also implemented in         ClusterControl.              – Launch from the G...
Lab exercises                        Contact us for more details       • Setting up MySQL Cluster       • Recovery & Resta...
Upgrading Cluster Version       • Install new binaries on all hosts       • Perform a rolling restartCopyright 2011 Severa...
Coming next in Installment 7:                    Management & AdministrationCopyright 2011 Severalnines AB               C...
We hope these training slides are                        useful to you!            Please visit our website to view the   ...
Disclaimer © Copyright 2011 Severalnines AB. All rights reserved. Severalnines & the Severalnines logo(s) are trademarks o...
Upcoming SlideShare
Loading in...5
×

Severalnines Self-Training: MySQL® Cluster - Part VI

2,640

Published on

Part VI of our free self-training slides on MySQL Cluster.

In this part we cover ’Configuration and Installation'

* Data Node configuration
* SQL Node configuration
* Important parameters
* Installation
* Upgrading

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,640
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • Dear Jury, Just as the electric grid revolutionized access to electricity 100 years ago, we at Severalnines believe that Cloud Computing will revolutionize IT where organizations will be able to plug into extremely powerful computing resources over the network. We have already seen the beginnings of this new wave, where the current infrastructure stack is being challenged and disrupted by a whole set of new technologies. For instance, in the database market, over 40 startups have received funding over the past 18 months. Severalnines is not building yet another database product, we believe there are already a lot of good technologies available. To manage a database costs 4 times the purchase price, and yet, very few companies are addressing this problem. Severalnines focuses on solutions to address this underserved segment. The founders of the company have a solid background in databases, having been at MySQL since 2003. The company develops a management platform which is database and cloud agnostic.We are database independent since we do not know who, if anybody, will be the next MySQL of the cloud.We are cloud independent, since we do not want to depend on any cloud vendor (e.g. Amazon or Rackspace) to avoid vendor lock-in. After the Amazon EC2 downtime during the Easter break, hundreds of affected companies have realized the importance of this. There is also a commercial aspect for avoiding vendor lock-in. Severalnines enhances productivity of organizations by attacking the biggest cost associated with database systems. We are a Swedish startup, hosted by SICS in Kista. There is also a small but very efficient development capacity in Singapore. Just as MySQL became a major brand and placed Sweden on the global software infrastructure map, we believe Severalnines can become a serious global player in the emerging Cloud space. We are very thankful that an organization like Eurocloud exists, and would like to thank the jury for considering our application. Kind regards,Vinay Joosery Severalnines AB
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Dear Jury, Just as the electric grid revolutionized access to electricity 100 years ago, we at Severalnines believe that Cloud Computing will revolutionize IT where organizations will be able to plug into extremely powerful computing resources over the network. We have already seen the beginnings of this new wave, where the current infrastructure stack is being challenged and disrupted by a whole set of new technologies. For instance, in the database market, over 40 startups have received funding over the past 18 months. Severalnines is not building yet another database product, we believe there are already a lot of good technologies available. To manage a database costs 4 times the purchase price, and yet, very few companies are addressing this problem. Severalnines focuses on solutions to address this underserved segment. The founders of the company have a solid background in databases, having been at MySQL since 2003. The company develops a management platform which is database and cloud agnostic.We are database independent since we do not know who, if anybody, will be the next MySQL of the cloud.We are cloud independent, since we do not want to depend on any cloud vendor (e.g. Amazon or Rackspace) to avoid vendor lock-in. After the Amazon EC2 downtime during the Easter break, hundreds of affected companies have realized the importance of this. There is also a commercial aspect for avoiding vendor lock-in. Severalnines enhances productivity of organizations by attacking the biggest cost associated with database systems. We are a Swedish startup, hosted by SICS in Kista. There is also a small but very efficient development capacity in Singapore. Just as MySQL became a major brand and placed Sweden on the global software infrastructure map, we believe Severalnines can become a serious global player in the emerging Cloud space. We are very thankful that an organization like Eurocloud exists, and would like to thank the jury for considering our application. Kind regards,Vinay Joosery Severalnines AB
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
  • Transcript of "Severalnines Self-Training: MySQL® Cluster - Part VI"

    1. 1. MySQL Cluster Training presented by severalnines.com Address: Contact: Severalnines AB Jean-Jérôme Schmidt c/o SICS, Box 1263 Email: services@severalnines.com Isafjordsgatan 22 SE-164-29 KistaCopyright 2011 Severalnines AB Control your database infrastructure 1
    2. 2. Introduction • At Severalnines, we believe in sharing information and knowledge; we all come from an open source background • We know a lot of things about MySQL Cluster and think that MySQL Cluster is a great technology • These free MySQL Cluster Training slides are a contribution of ours to the knowledge and information sharing that’s common practice in the open source community • If you have any questions on these slides or would like to book an actual training class, please contact us at: services@severalnines.comCopyright 2011 Severalnines AB Control your database infrastructure 2
    3. 3. Training Slides - Concept • Over the coming weeks we will be chronologically releasing slides for the different sections of our MySQL Cluster Training program on our website. • The full agenda of the training with all of its modules is outlined in the next slides so that you can see what topics will be covered over the coming weeks. • Particularly specialised topics such as Cluster/J or NDB API are not fully covered in the slides. We recommend our instructor-led training classes for such topics. • Please contact us for more details: services@severalnines.comCopyright 2011 Severalnines AB Control your database infrastructure 3
    4. 4. Full Training Agenda (1/4) • MySQL Cluster Introduction – MySQL eco system – Scale up, scale out, and sharding – MySQL Cluster Architecture – Use cases – Features – Node types and Roles • Detailed Concepts – Data Distribution – Verifying data distribution – Access Methods – Partitioning – Node failures and failure detection – Network Partitioning – Transactions and Locking – Consistency Model – Redo logging and Checkpointing • Internals – NDB Design InternalsCopyright 2011 Severalnines AB Control your database infrastructure 4
    5. 5. Agenda (2/4) • Installing MySQL Cluster – Setting up MySQL Cluster – Starting/stopping nodes – Recovery and restarts – Upgrading configuration – Upgrading Cluster • Performance Tuning (instructor-led only; contact us at services@severalnines.com) – Differences compared to Innodb/MyISAM – Designing efficient and fast applications – Identifying bottlenecks – Tweaking configuration (OS and MySQL Cluster) – Query Tuning – Schema Design – Index TuningCopyright 2011 Severalnines AB Control your database infrastructure 5
    6. 6. Agenda (3/4) • Management and Administration – Backup and Restore – Geographical Replication – Online and offline operations – Ndbinfo tables – Reporting – Single user mode – Scaling Cluster • Disk Data – Use cases – Limitations – Best practice configuration • Designing a Cluster – Capacity Planning and Dimensioning – Hardware recommendations – Best practice Configuration – Storage calculationsCopyright 2011 Severalnines AB Control your database infrastructure 6
    7. 7. Agenda (4/4) • Resolving Issues – Common problems – Error logs and Tracefiles – Recovery and Escalation procedures • Connectivity Overview – NDBAPI – Cluster/J – LDAP • Severalnines Tools – Monitoring and Management – Benchmarking – Sandboxes – Configuration and capacity planning • ConclusionCopyright 2011 Severalnines AB Control your database infrastructure 7
    8. 8. Agenda: Lab Exercises (only applicable to instructor-led training classes) • Lab Exercises – Installing and Loading data into MySQL Cluster – Starting/stopping nodes, recovery – Query tuning – Backup and Restore – Configuration Upgrade • Would you like to try something particular? – This is possible too, speak with your instructorCopyright 2011 Severalnines AB Control your database infrastructure 8
    9. 9. Prerequisites • Readers / Participants have understanding of SQL and basic database concepts. • Laptops/PCs for hands-on exercises • Linux: 1GB RAM • Windows: 2GB RAM • Approx. 20GB disk space and Virtualbox installed. • Virtualbox can be downloaded for free at http://www.virtualbox.org/wiki/Downloads • MySQL Cluster version 7.1 or laterCopyright 2011 Severalnines AB Control your database infrastructure 9
    10. 10. 6th Installment Severalnines Cluster Self-Training Part 4: Installing MySQL ClusterCopyright 2011 Severalnines AB Control your database infrastructure 10
    11. 11. Topics covered in Installment 6 • Configuration • InstallationCopyright 2011 Severalnines AB Control your database infrastructure 11
    12. 12. Part 4: Installing MySQL Cluster Configuration and InstallationCopyright 2011 Severalnines AB Control your database infrastructure 12
    13. 13. Topics • MySQL Cluster configuration (config.ini) • MySQL Server configuration (my.cnf) • Important parameters • Installation • UpgradingCopyright 2011 Severalnines AB Control your database infrastructure 13
    14. 14. MySQL Cluster configuration (config.ini) • There are no parameter in config.ini that affects performance. • Each node and node type must be configured, and each node type has a group. – [ndb_mgmd] X <nodes> -- typically 1 or 2 – [ndbd] X <nodes> -- between 2 and 48 – [mysqld] / [api] X <nodes> -- from 2 and up • Remember, you can have 255 nodes in total in MySQL Cluster. • There are also default groups that applies to a group. – [TCP DEFAULT] – [NDB_MGMD DEFAULT] – [NDBD DEFAULT] – [MYSQLD DEFAULT] • An item in [NDBD DEFAULT] applies to all [NDBD]Copyright 2011 Severalnines AB Control your database infrastructure 14
    15. 15. MySQL Cluster configuration (config.ini) [ndb_mgmd default] Datadir=/data/mysqlcluster Portnumber=1186 [ndb_mgmd] Hostname=app01 [ndb_mgmd] Hostname=app02 [ndbd default] NoOfReplicas=2 Datadir=/data/mysqlcluster [ndbd] Hostname=storage01 [ndbd] Hostname=storage01 [mysqld] [mysqld] [mysqld] [mysqld]Copyright 2011 Severalnines AB Control your database infrastructure 15
    16. 16. MySQL Server (my.cnf) [mysqld] basedir=/usr/local/mysql/ datadir=/data/mysql socket=/tmp/mysql.sock port=3306 ndbcluster ndb-connectstring=app01:1186,app02:1186 ## the usual MySQL Server settings [mysql] Socket=/tmp/mysql.sockCopyright 2011 Severalnines AB Control your database infrastructure 16
    17. 17. Preparations • Download MySQL Cluster from dev.mysql.com/downloads/cluster • On every machine – Copy it to every machine • Using e.g scp – cd /usr/local – tar xvfz mysql-cluster-gpl-7.2.6-linux-x86_64-glibc23.tar.gz – ln –s mysql-cluster-gpl-7.2.6-linux-x86_64-glibc23 mysql – create data directories • mkdir –p /data/mysql && mkdir –p /data/mysqlcluster – Create config directories • mkdir –p /etc/mysql/Copyright 2011 Severalnines AB Control your database infrastructure 17
    18. 18. Preparations • On all Management nodes – Put config.ini in /etc/mysql/ • On all MySQL servers – Put my.cnf in /etc/mysql/ – Create the mysql user and group • useradd mysql • groupadd mysql – Install the system tables • cd /usr/local/mysql • scripts/mysql_install_db --defaults-file=/etc/mysql/my.cnf • chown mysql:mysql –R /data/mysqlCopyright 2011 Severalnines AB Control your database infrastructure 18
    19. 19. Minimal [ndbd default] [ndbd default] Datadir=/data/mysqlcluster/ NoOfReplicas=2 DataMemory=80M IndexMemory=16M FragmentLogFileSize=16M NoOfFragmentLogfiles=3 MaxNoOfConcurrentOperations=32786 MaxNoOfConcurrentTransactions=1024 RedoBuffer=8M TransactionInactiveTimeout=0 TransactionDeadlockDetectionTimeout=1200 DiskCheckpointSpeed=5M #MaxNoOfTables #MaxNoOfOrderedIndexes #MaxNoOfUniqueHashIndexes #SharedGlobalMemory #DiskPageBufferMemoryCopyright 2011 Severalnines AB Control your database infrastructure 19
    20. 20. MySQL Server parameters • The following parameters affects performance – Ndb_cluster_connection_pool=8 • Each connection requires one free [mysqld] slot in config.ini • With Ndb_cluster_connection_pool=1 the mysql server can scale up to 4-5 simultaneous connections. – Ndb_auto_increment_prefetch_size=1024 • Each MYSQL server will cache ranges of 1024 numbers, and then request a new range from the data node. • Ndb_auto_increment_prefetch_size=1 means an extra roundtrip for every insert on an auto_inc , really bad for performance • This is most most important config parameter tuning.Copyright 2011 Severalnines AB Control your database infrastructure 20
    21. 21. Installing ClusterCopyright 2011 Severalnines AB Control your database infrastructure 21
    22. 22. Topics • Setting up MySQL Cluster • Starting/stopping nodes • Recovery and restarts • Upgrading configuration • Upgrading ClusterCopyright 2011 Severalnines AB Control your database infrastructure 22
    23. 23. Using the Configurator • We do recommend to use the Severalnines Cluster Configurator – www.severalnines.com/cluster-configurator – Rapid deployment – ./deploy.sh - automatically installs MySQL Cluster (or Galera or Replication) complete with monitoring and management (ClusterControl). • The Configurator generates configurations that has been proven in for a long time in Production environments.Copyright 2011 Severalnines AB Control your database infrastructure 23
    24. 24. ClusterControl • Assists and automates mundane tasks: – Configuration Management – Upgrades – Backup and Backup schedules – Query Monitoring and Histograms – Performance Management – Scaling (add remove MySQL servers on demand)Copyright 2011 Severalnines AB Control your database infrastructure 24
    25. 25. ClusterControlCopyright 2011 Severalnines AB Control your database infrastructure 25
    26. 26. ClusterControlCopyright 2011 Severalnines AB Control your database infrastructure 26
    27. 27. ClusterControlCopyright 2011 Severalnines AB Control your database infrastructure 27
    28. 28. Starting MySQL Cluster • Management nodes must be started first – Otherwise no other node can join the Cluster – ndb_mgmd -f /etc/mysql/config.ini --configdir=/etc/mysql -- reload • Start the data nodes – ndbmtd or ndbd – ndbmtd –c “management_node1, management_node2” • Start the mysql servers – mysqld –defaults-file=/etc/mysql/my.cnf &Copyright 2011 Severalnines AB Control your database infrastructure 28
    29. 29. Simple management • The Management Client (ndb_mgm) can be used to to perform simple management: – ndb_mgm> ALL STATUS – ndb_mgm> SHOW • It is also possible to stop nodes (data nodes), but not starting them: – ndb_mgm> X STOP • Backups can also be started: – ndb_mgm> START BACKUPCopyright 2011 Severalnines AB Control your database infrastructure 29
    30. 30. Rolling Restarts • Rolling restarts are used to perform changes on a running MySQL Cluster – Configuration Upgrade – Version Upgrade – Hardware UpgradeCopyright 2011 Severalnines AB Control your database infrastructure 30
    31. 31. Rolling Restarts • Stop the management servers – kill -15 `pidof ndb_mgmd` • Start the management servers – Make sure you start them with --reload • For each data node, one by one: – Stop the Data nodes one by one • kill -15 `pidof ndbmtd` – Start the Data node • For each mysqld node, one by one: – Stop the mysqld nodes one by one • kill -15 `pidof mysqld` – Start the mysqld nodeCopyright 2011 Severalnines AB Control your database infrastructure 31
    32. 32. Rolling Restarts • Rolling restarts are also implemented in ClusterControl. – Launch from the GUI – Launch from script: • ./rolling-restart.shCopyright 2011 Severalnines AB Control your database infrastructure 32
    33. 33. Lab exercises Contact us for more details • Setting up MySQL Cluster • Recovery & Restarts • Configurations UpdatesCopyright 2011 Severalnines AB Control your database infrastructure 33
    34. 34. Upgrading Cluster Version • Install new binaries on all hosts • Perform a rolling restartCopyright 2011 Severalnines AB Control your database infrastructure 34
    35. 35. Coming next in Installment 7: Management & AdministrationCopyright 2011 Severalnines AB Control your database infrastructure 35
    36. 36. We hope these training slides are useful to you! Please visit our website to view the next section of this training. For any questions, comments, feedback or to book a training class, please contact us at: services@severalnines.com Thank you!Copyright 2011 Severalnines AB Control your database infrastructure 36
    37. 37. Disclaimer © Copyright 2011 Severalnines AB. All rights reserved. Severalnines & the Severalnines logo(s) are trademarks of Severalnines AB. MySQL is a registered trademark of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.Copyright 2011 Severalnines AB Control your database infrastructure 37

    ×