• Save
Severalnines Self-Training: MySQL® Cluster - Part I

Like this? Share it with your network

Share

Severalnines Self-Training: MySQL® Cluster - Part I

  • 14,705 views
Uploaded on

If you are planning to deploy MySQL® Cluster, then you should consider our MySQL® Cluster training. Our MySQL® Cluster experts, all former MySQL AB staff, can help you and your business get the......

If you are planning to deploy MySQL® Cluster, then you should consider our MySQL® Cluster training. Our MySQL® Cluster experts, all former MySQL AB staff, can help you and your business get the most out of your MySQL® Cluster solution. As a first step, you can now read through our free online training. We will be releasing slides of this training on MySQL® Cluster in sections over the coming weeks. Make sure to tune in here regularly to catch all of the sections of this expert training on MySQL® Cluster.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Thanks a lot for presentation.
    Very nice and helpful.

    I would be happy if you give us permission to download the presentation filei.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
14,705
On Slideshare
4,267
From Embeds
10,438
Number of Embeds
9

Actions

Shares
Downloads
1
Comments
1
Likes
11

Embeds 10,438

http://www.severalnines.com 8,853
http://severalnines.com 1,565
http://localhost 7
http://translate.googleusercontent.com 4
https://twitter.com 3
https://si0.twimg.com 2
http://twitter.com 2
http://www.linkedin.com 1
http://webcache.googleusercontent.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com

Transcript

  • 1. MySQL Cluster Trainingpresented by severalnines.com
    Address:
    c/o Severalnines AB
    SICS, Box 1263
    Isafjordsgatan22
    SE-164-29 Kista
    Contact:
    Jean-Jérôme Schmidt
    Email: services@severalnines.com
  • 2. Introduction
    At Severalnines, we believe in sharing information and knowledge; we all come from an open source background
    We know a lot of things about MySQL Cluster and think that MySQL Cluster is a great technology
    These free MySQL Cluster Training slides are a contribution of ours to the knowledge and information sharing that’s common practice in the open source community
    If you have any questions on these slides or would like to book an actual training class, please contact us at: services@severalnines.com
  • 3. Training Slides - Concept
    Over the coming weeks we will be chronologically releasing slides for the different sections of our MySQL Cluster Training program on our website.
    The full agenda of the training with all of its modules is outlined in the next slides so that you can see what topics will be covered over the coming weeks.
    Particularly specialised topics such as Cluster/J or NDB API are not fully covered in the slides. We recommend our instructor-led training classes for such topics.
    Please contact us for more details: services@severalnines.com
  • 4. Full Training Agenda (1/4)
    MySQL Cluster Introduction
    MySQL eco system
    Scale up, scale out, and sharding
    MySQL Cluster Architecture
    Use cases
    Features
    Node types and Roles
    Detailed Concepts
    Data Distribution
    Verifying data distribution
    Access Methods
    Partitioning
    Node failures and failure detection
    Network Partitioning
    Transactions and Locking
    Consistency Model
    Redo logging and Checkpointing
  • 5. Agenda (2/4)
    Installing MySQL Cluster
    Setting up MySQL Cluster
    Starting/stopping nodes
    Recovery and restarts
    Upgrading configuration
    Upgrading Cluster
    Performance Tuning (instructor-led only; contact us at services@severalnines.com)
    Differences compared to Innodb/MyISAM
    Designing efficient and fast applications
    Identifying bottlenecks
    Tweaking configuration (OS and MySQL Cluster)
    Query Tuning
    Schema Design
    Index Tuning
  • 6. Agenda (3/4)
    Management and Administration
    Backup and Restore
    Geographical Replication
    Online and offline operations
    Ndbinfo tables
    Reporting
    Single user mode
    Scaling Cluster
    Disk Data
    Use cases
    Limitations
    Best practice configuration
    Designing a Cluster
    Capacity Planning and Dimensioning
    Hardware recommendations
    Best practice Configuration
    Storage calculations
  • 7. Agenda (4/4)
    Resolving Issues
    Common problems
    Error logs and Tracefiles
    Recovery and Escalation procedures
    Connectivity Overview
    NDBAPI
    Cluster/J
    LDAP
    Severalnines Tools
    Monitoring and Management
    Benchmarking
    Sandboxes
    Configuration and capacity planning
    Conclusion
  • 8. Agenda: Lab Exercises(only applicable to instructor-led training classes)
    Lab Exercises
    Installing and Loading data into MySQL Cluster
    Starting/stopping nodes, recovery
    Query tuning
    Backup and Restore
    Configuration Upgrade
    Would you like to try something particular?
    This is possible too, speak with your instructor
  • 9. Prerequisites
    Readers / Participants have understanding of SQL and basic database concepts.
    Laptops/PCs for hands-on exercises
    Linux: 1GB RAM
    Windows: 2GB RAM
    Approx. 20GB disk space and Virtualbox installed.
    Virtualbox can be downloaded for free at http://www.virtualbox.org/wiki/Downloads
    MySQL Cluster version 7.1 or later
  • 10. Week 1 – Training SlidesSection 1: MySQL Cluster Introduction
  • 11. Topics
    Origins of MySQL Cluster
    Scale up, scale out, and sharding
    MySQL Cluster Architecture
    Use cases
    Features
    Node types and Roles
  • 12. Origins of MySQL Cluster
    Ericsson started development of NDB Cluster in 1996
    Focus on HA/performance requirements of Telecom
    Offered as a product in 2001 by Alzato, fully owned by Ericsson
    Acquisition by MySQL in September 2003
    Core team is transferred
    Intellectual Property transferred (9 patents)
    New product called MySQL Cluster
    Original NDB Cluster + SQL engine + MySQL connectors
    Major investment in usability
    Co-developed with some telecom vendors
  • 13. Telco-grade Data Management
    High Availability
    99.999% (< 5 mins/yr) or 99.9999% (< 30 secs/yr)
    Scalability
    Throughput in the range of 1000s or 10,000s per second
    Predictable Real-time Performance
    Typically 5 to 10 milliseconds
    In-memory performance
    Cost effective
    ScaleOut vs. ScaleUp
    COTS hardware
  • 14. Scale up, scale out, sharding
    Scale up means having one beefy machine handling load.
    At some point it won’t scale
    Depending on implementation, mutexes and semaphores these systems scales up to about 16 processors/cores.
    With the MySQL Server you will be limited by the number of incoming connections at some stage.
    Hard to increase the capacity
    Reconfigure with more cores
    Examples
    Sybase. Customer has a Sun E10000, next step is to get a Sun E15000, and then license cost for additional processors
    Gets expensive.
    Scale up is old school.
  • 15. Scale up, scale out, sharding
    Scale-out means that instead of using one beefy machine you use many.
    This can be accomplished with
    MySQL Replication
    Master -> slave replication
    Pros: easy to add slaves
    Cons: Write scaling is hard, one single master, read/write split
    MySQL Cluster
    Will be addressed in this training.
    Read
    Read/writes
    Asynch. replication
    Read
  • 16. Scale up, scale out, sharding
    Shardingmeans that instead of using one beefy machine you use many and partitions the data on the machines.
    Problems:
    Hard do find a good key to partition by
    Adding nodes are hard – how to redistribute the data?
    MySQL Cluster addresses a lot of these problems
    Sharding Aware Layer in Application Code
    Users1-1000
    Users1001-2000
    Users2001-3000
  • 17. Application R/I/U/D
    Application R/I/U/D
    Application R/I/U/D
    Java, C++, LDAP
    Application R/I/U/D
    MySQL Cluster Architecture
    Native API
    DATANODE
    DATANODE
    Management Server
    Management Server
    DATANODE
    DATANODE
  • 18.
    • Parallel Database Server
    • 19. Shared-nothing architecture
    • 20. Data partitioned across nodes
    • 21. In a RAID10 fashion
    • 22. Redundancy
    • 23. Synchronous replication
    • 24. Automatic failover and repair
    • 25. Performance
    • 26. Ability to mix and match disk and memory tables
    • 27. Different access methods (SQL, C++, Java,LDAP)
    MySQL Cluster basics
  • 28.
    • No single point of failure
    • 29. Any node can fail without affecting the system
    • 30. Configurable number of data copies
    • 31. Up to 4 copies
    • 32. Synchronous replication
    • 33. Automatic failure detection and failover
    • 34. Detection time determined by heartbeat
    • 35. Failover is instantaneous
    • 36. Automatic resynchronization
    • 37. Delta or Full synchronization
    Availability Features (1/2)
  • 38.
    • Handling of network partitioning
    • 39. Uses an Arbitrator
    • 40. Online Software Upgrade
    • 41. Supports upgrades and downgrade of system without outages
    • 42. Disaster Recovery
    • 43. Geographical Replication using Asynchronous protocol
    • 44. Online Backup
    Availability Features (2/2)
  • 45. Use Cases
    Good fit
    OLTP apps with short running queries
    Application with real-time characteristics and requirements
    A lot of concurrent requests
    Write intensive applications
    Typically the following are a poor fit:
    Heavy reporting type (OLAP)
    Data Warehouse
    Complex JOINs (much improved in 7.2) scale badly.
    A couple of tables and with about 1000 records meeting the JOIN criteria is just fine)
    Complex and long running transactions in a realtime system is just wrong.
    However, replicate from MySQL Cluster to regular MySQL (innodb) which runs the reporting.
  • 46. Application R/I/U/D
    Application R/I/U/D
    Application R/I/U/D
    Java, C++, LDAP
    Application R/I/U/D
    MySQL Cluster Architecture
    Native API
    DATANODE
    DATANODE
    Management Server
    Management Server
    DATANODE
    DATANODE
  • 47. Node Types and Roles
    MySQL Cluster is comprised of the following node types:
    Management Nodes
    Maintains the cluster configuration
    Involved in handling network partitioning
    Writes a cluster log containing events occurring in Cluster.
    Typically two are deployed for redundancy
    Data Nodes
    Transaction, query handling and data storage
    Atleast two for redundancy
    Application Nodes
    Mysql server, Cluster/J
    Requests are ultimately converted to NDBAPI calls
  • 48. Starting a MySQL Cluster
    Pretend that we have four hosts:
    app01
    app02
    storage01
    storage02
    We will start a HA cluster consisting of four nodes:
    2 data nodes, one each on storage01 and storage02
    2 mysql servers to access the data nodes, one on each of app01 and app02
    2 management servers to deal with configuration and network partitioning, one of each on app01 and app02.
  • 49. Management Nodes/Servers
    Owns the Cluster configuration and loads a file called config.ini.
    Config.ini describes the Cluster
    Number of management nodes, data nodes access (mysql/ndbapi) nodes
    Ports, Resources, Data Directories
    The management node is started with
    ndb_mgmd –f /etc/mysql/config.ini --configdir=/etc/mysql
    The management server generates a binary versioned configuration file that is stored in configdir.
    All nodes must first connect to the management nodes to fetch the configuration
    Listens by default on port 1186
    Writes a cluster log in Datadir (e.gDatadir=/data/)
  • 50. MySQLClusterTwo management servers are started
    The management servers loads in the config.ini file.
    Compare with each other they have the same version. If not, they won’t start.
    The management servers start writing a cluster log
    Ndb_X_cluster.log
    Located in the datadir.
    Hostname=app01datadir=/data/Listens on *:1186
    Hostname=app02datadir=/data/Listens on *:1186
    Management Server1
    Management Server2
    Compareconfig
    app01> ndb_mgmd –f /etc/mysql/config.ini –configdir=/etc/mysql --reload
    app02> ndb_mgmd –f /etc/mysql/config.ini –configdir=/etc/mysql --reload
  • 51. Data Nodes
    Connects to the management nodes and fetch the configuration
    Needs a connectstring pointing to the management nodes
    The data nodes are the processes
    Storing data
    Managing transaction, indexing, data distribution etc etc.
    When the data node gets the configuration it will try and setup connections to other nodes.
    Started with
    ndbd –c “hostname1:1186,hostname2:1186”
    ndbmtd –c “hostname1:1186,hostname2:1186”
    Ndbmtd is the multithreaded daemon and can use up to 8 cores. Ndbd is for single/dual core machines.
  • 52. MySQL ClusterData Node 1 Joins
    Hostname=app01datadir=/data/Listens on *:1186
    Hostname=app02datadir=/data/Listens on *:1186
    Hostname=storage01datadir=/data/
    Management Server1
    Management Server2
    DATANODE1
    Get config
    Starts listening for other data node
    storage01> ndbd –c “app01:1186,app02:1186”
  • 53. MySQL ClusterData Node 2 Joins
    Hostname=app01datadir=/data/Listens on *:1186
    Hostname=app02datadir=/data/Listens on *:1186
    Hostname=storage01datadir=/data/
    Management Server1
    Management Server2
    DATANODE1
    Get config
    Hostname=storage01datadir=/data/
    DATANODE1
    storage02> ndbd –c “app01:1186,app02:1186”
  • 54. API Nodes
    NDBAPI programs
    Direct API (no SQL)
    C++ programs, uses libndbclient
    Cluster/J
    Java, has a JNI interface that uses libndbclient
    MySQL nodes/server
    Convert SQL into NDBAPI code
    Uses libndbclient
    Stores data/logs in a data directory
    Configuration for the server in a my.cnf file
    In any case
    A connectstring is a always needed pointing to the management nodes.
  • 55. MySQL ClusterA MySQL Server Joins
    Get config
    Hostname=app01datadir=/data/Listens on *:1186
    Hostname=app02datadir=/data/Listens on *:1186
    Hostname=storage01datadir=/data/
    Hostname=app03datadir=/data/
    Management Server1
    Management Server2
    DATANODE1
    Hostname=storage01datadir=/data/
    DATANODE1
    app03> mysqld &
  • 56. MySQL ClusterA MySQL Server Joins
    Hostname=app01datadir=/data/Listens on *:1186
    Hostname=app02datadir=/data/Listens on *:1186
    Hostname=storage01datadir=/data/
    Hostname=app03datadir=/data/
    Management Server1
    Management Server2
    DATANODE1
    Hostname=storage01datadir=/data/
    DATANODE1
    app03> mysqld &
  • 57. MySQL ClusterFullystarted
    Hostname=app01datadir=/data/mysqlclusterListens on *:1186
    Hostname=app02datadir=/data/mysqlclusterListens on *:1186
    Hostname=storage01datadir=/data/mysqlcluster
    Hostname=app01datadir=/data/mysql
    Management Server1
    Management Server2
    DATANODE1
    Hostname=storage01datadir=/data/mysqlcluster
    Hostname=app02datadir=/data/mysql
    DATANODE1
  • 58. Fully Connected
    All data nodes are connected to every other node
    Epoll is used
    Max 255 Nodes in the Cluster
    Max 48 Data Nodes
    Biggest Deployed Cluster
    48 data nodes
  • 59. Coming next in week 2:Section 2: Detailed Concepts
  • 60. Topics covered in Week 2
    Data Distribution
    Verifying data distribution
    Access Methods
    Partitioning
    Node failures and failure detection
    Network Partitioning
    Transactions and Locking
    Consistency Model
    Redo logging and Checkpointing
  • 61. We hope these training slides are useful to you!Please visit our website to view the next section of this training.For any questions, comments, feedback or to book a training class, please contact us at: services@severalnines.comThank you!
  • 62. Disclaimer
    © Copyright 2011 Severalnines AB. All rights reserved.
    Severalnines and the Severalnineslogo(s) are
    trademarks of Severalnines AB.
    MySQL is a registered trademark of Oracle and/or its
    affiliates.
    Other names may be trademarks of their
    respective owners.