MySQL Cluster Carrier Grade Edition

MySQL Cluster Carrier Grade Edition Provides the telecom industry with a true real-time database with the flexibility of a relational database plus the cost savings of open source Designed by telecom for telecom Ideally suited for the telecom industry's top carriers, operators and equipment manufactures Used by industry leaders for real-time, scalable data management

Provides the telecom industry with a true real-time database with the flexibility of a relational database plus the cost savings of open source

Designed by telecom for telecom

Ideally suited for the telecom industry's top carriers, operators and equipment manufactures

Used by industry leaders for real-time, scalable data management

Basic Cluster Architecture Application Nodes (NDB API and/or MySQL Server) ‏ Data Nodes Management Nodes Clients NDB API NDB Storage Engine

Geographic Redundancy For more information on MySQL Cluster and Replication visit: http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-replication.html Asynchronous MySQL Replication between Clusters Enables a higher degree of availability and continuity Replication Cluster 1 Cluster 2

Asynchronous MySQL Replication between Clusters

Enables a higher degree of availability and continuity

Carrier Grade Edition Overview Geographic Replication Scalability Performance High Availability Requirement Increase availability/scalability across geographies & data centers Mitigate potential network or site failures Enables Scale-Out Asynchronous data replication Incrementally “scale-out” for increased capacity, performance & scalability Linear scalability with user-defined partitioning Support for disk-based data and large objects In-memory and disk-based data storage Fast, native data access via NDB API Distributed, synchronous, “shared-nothing” data storage Automated fail-over in under a second Automatic data resynchronization in the event of a failure Online backups Rolling upgrades Feature Set

Increase availability/scalability across geographies & data centers

Mitigate potential network or site failures

Enables Scale-Out

Asynchronous data replication

Incrementally “scale-out” for increased capacity, performance & scalability

Linear scalability with user-defined partitioning

Support for disk-based data and large objects

In-memory and disk-based data storage

Fast, native data access via NDB API

Distributed, synchronous, “shared-nothing” data storage

Automated fail-over in under a second

Automatic data resynchronization in the event of a failure

Online backups

Rolling upgrades

Carrier Grade Edition Overview Lower TCO Rapid Development Interoperability Reliability Requirement Runs on commodity, Advanced TCA hardware Lower cost licensing model Minimized design complexity and associated maintenance costs Based on open source and open standards SQL, Java and NDB API data access Automatic and user-defined partitioning Professional consulting and customized application development from MySQL Leverage telecom specific features Most popular open source database in the world Runs on popular open source and proprietary technology stacks Support for ACID transactions Synchronous intra-cluster data replication Local and Global checkpoints 24x7 Technical Support Professional training and certification Feature Set

Runs on commodity, Advanced TCA hardware

Lower cost licensing model

Minimized design complexity and associated maintenance costs

Based on open source and open standards

SQL, Java and NDB API data access

Automatic and user-defined partitioning

Professional consulting and customized application development from MySQL

Leverage telecom specific features

Most popular open source database in the world

Runs on popular open source and proprietary technology stacks

Support for ACID transactions

Synchronous intra-cluster data replication

Local and Global checkpoints

24x7 Technical Support

Professional training and certification

What’s Different? Adaptability Performance Scalability Quality & Reliability Stability Characteristic Online Add Column Support for mixed (x86, SPARC, PowerPC) environments Optimizations to communication protocol enabling API to Data Node over WAN More efficient Client/Server data handling and transfers Other optimizations (Memory, BLOBs, Disk-based data, etc.) ‏ Support for larger clusters, up to 255 Nodes Conflict detection support in replication Circular replication support Support for user-defined partitioning Developed & tested with major TEMs Separate development tree from MySQL Server Permits prioritized bug fixes & enhancements for customers Benefits

Online Add Column

Support for mixed (x86, SPARC, PowerPC) environments

Optimizations to communication protocol enabling API to Data Node over WAN

More efficient Client/Server data handling and transfers

Other optimizations (Memory, BLOBs, Disk-based data, etc.) ‏

Support for larger clusters, up to 255 Nodes

Conflict detection support in replication

Circular replication support

Support for user-defined partitioning

Developed & tested with major TEMs

Separate development tree from MySQL Server

Permits prioritized bug fixes & enhancements for customers

System Requirements COTS – Advanced TCA 32 & 64-bit x86, PowerPC & SPARC Hardware Varies on size of database, # of hosts, # of replicas Memory Linux (Red Hat, SuSE), Solaris, HP-UX, Mac OSX, others… Operating System 100 Mbps or better (Gigabit recommended, SCI supported) ‏ Network Shared-Nothing - Memory & Disk Data SCSI or RAID for I/O performance Storage Maximum of 255 total nodes (48 Data Nodes) ‏ Hosts Requirement System Component

NDB Storage Engine Memory and disk-based ACID Transactions Row-Level Locks Synchronous replication across Data Nodes Linear Two Phase Commit Data automatically or user partitioned Local & Global Checkpoints to disk for persistence Automatic fail over and data resynchronization Data can be accessed via SQL or NDB API Online Backups Currently not supported: - Foreign Keys - Full-text Search - Microsoft Windows

Memory and disk-based

ACID Transactions

Row-Level Locks

Synchronous replication across Data Nodes

Linear Two Phase Commit

Data automatically or user partitioned

Local & Global Checkpoints to disk for persistence

Automatic fail over and data resynchronization

Data can be accessed via SQL or NDB API

Online Backups

Data Distribution & Partitioning P1 - Primary P2 - Replica P1 - Replica P2 - Primary data data data data col2 data data 4 data data 3 data data 2 data data 1 col3 col1 PK Two Data Nodes * Complete copy of database on each node Two Replicas * Two copies of each partition One Node Group * One set of identical Data Nodes Partitions - Automatic or User-Defined - Horizontal - MD5 hash of Primary Key Node Group

Data Distribution & Partitioning P1 - Primary P2 - Replica P1 - Replica P2 - Primary P3 - Primary P4 - Replica P3 - Replica P4 - Primary Four Data Nodes * Half of database on each node Two Replicas * Two copies of each partition Two Node Groups * Two sets of identical Data Nodes Node Group 0 Node Group 1 data data data data col2 data data 4 data data 3 data data 2 data data 1 col3 col1 PK

Design Considerations Number of Replicas Up to 4 supported, 2 is most common NoOfReplicas parameter in config.ini Partitions Automatic Horizontal MD5 hash of primary key Memory Sizing Leverage ndb_size.pl script Data Size * Replicas * 1.1 = Total Memory Example: 2 GB * 2 * 1.1 = 4.4 GB (Data Size * Replicas * 1.1)/Nodes = RAM Per Node Example: (2 GB * 2 * 1.1)/4 = 1.1 GB Disk Storage Used for data, backups, checkpoints and logs SCSI or RAID for faster backups or write-heavy apps

Number of Replicas

Up to 4 supported, 2 is most common

NoOfReplicas parameter in config.ini

Partitions

Automatic

Horizontal

MD5 hash of primary key

Memory Sizing

Leverage ndb_size.pl script

Data Size * Replicas * 1.1 = Total Memory

Example: 2 GB * 2 * 1.1 = 4.4 GB

(Data Size * Replicas * 1.1)/Nodes = RAM Per Node

Example: (2 GB * 2 * 1.1)/4 = 1.1 GB

Disk Storage

Used for data, backups, checkpoints and logs

SCSI or RAID for faster backups or write-heavy apps

Detecting Lost Connections - Heartbeats

Data Node Failure & Recovery Server 3 failure is detected Server 4 handles Server 3 requests P1 - Primary P2 - Replica P1 - Replica P2 - Primary P3 - Primary P4 - Replica P3 - Replica P4 - Primary 1 3 2 4 Node Group 0 Node Group 1

Server 3 failure is detected

Server 4 handles Server 3 requests

Data Node Failure & Recovery Server 3 begins copying data from Server 4 Complete or just changes since failure P1 - Primary P2 - Replica P1 - Replica P2 - Primary P3 - Primary P4 - Replica P3 - Replica P4 - Primary 1 3 2 4 Node Group 0 Node Group 1

Server 3 begins copying data from Server 4

Complete or just changes since failure

Data Node Failure & Recovery Data is available as it is restored Server 3 rejoins the Cluster P1 - Primary P2 - Replica P1 - Replica P2 - Primary P3 - Primary P4 - Replica P3 - Replica P4 - Primary 1 3 2 4 Node Group 0 Node Group 1

Data is available as it is restored

Server 3 rejoins the Cluster

Network Partitioning – Avoiding “Split Brain” By default, the Management Node acts as arbitrator In the event of a network failure, arbitration determines which set of nodes should continue while shutting down the rest X Application Nodes (NDB API and/or MySQL Server) ‏ Data Nodes Management Nodes NDB API

By default, the Management Node acts as arbitrator

In the event of a network failure, arbitration determines which set of nodes should continue while shutting down the rest

Queries and Transactions MySQL Server is a SQL interface for the NDB storage engine NDB API for fast, native data access Data Node answers query or with other Nodes in the Node Group Round-robin by default Transaction Coordinator Local Query Handler With write or updates, a Data Node initiates the transaction on all Nodes in the Node Group Linear two-phase commit Ensures all Data Nodes have received and acted on a “write” before committing

MySQL Server is a SQL interface for the NDB storage engine

NDB API for fast, native data access

Data Node answers query or with other Nodes in the Node Group

Round-robin by default

Transaction Coordinator

Local Query Handler

With write or updates, a Data Node initiates the transaction on all Nodes in the Node Group

Linear two-phase commit

Ensures all Data Nodes have received and acted on a “write” before committing

MySQL Cluster NDB API High performance C++ API for NDB storage engine Implements indexes, scans, transactions & events ACID-compliant Object-oriented error-handling Also, MGM API for administrative tasks For more information about the NDB API Developers’ Guide visit: http://dev.mysql.com/doc/ndbapi/en/index.html

High performance C++ API for NDB storage engine

Implements indexes, scans, transactions & events

ACID-compliant

Object-oriented error-handling

Also, MGM API for administrative tasks

Linear Scalability Primary Key look-ups are very efficient Increase the number of PK operations in a transaction, batch size Yields linear improvements in performance Multiple-record look-ups benefit from user-defined partitioning Limit full table scans and use of JOINs

Primary Key look-ups are very efficient

Increase the number of PK operations in a transaction, batch size

Yields linear improvements in performance

Multiple-record look-ups benefit from user-defined partitioning

Limit full table scans and use of JOINs

Linear Scalability Benefits

Reliability: Checkpoints Process of writing data and logs from memory to disk Local Checkpoint (LCP) ‏ Take place for all Data Nodes in the cluster concurrently Persists all of a Data Node's data to disk Interval varies by amount of data and level of cluster activity UNDO Log used to create consistent copy without any locks LCP allows Data Node to remove old REDO Logs LCP UNDO Log

Process of writing data and logs from memory to disk

Local Checkpoint (LCP) ‏

Take place for all Data Nodes in the cluster concurrently

Persists all of a Data Node's data to disk

Interval varies by amount of data and level of cluster activity

UNDO Log used to create consistent copy without any locks

LCP allows Data Node to remove old REDO Logs

Reliability: Checkpoints Global Checkpoint (GCP) ‏ Transactions on all nodes are synchronized Recent transactions stored in REDO Log flushed to disk Frequency controlled by TimeBetweenGlobalCheckpoints Performance vs. durability GCP REDO Log GCP Transactions Synchronized

Global Checkpoint (GCP) ‏

Transactions on all nodes are synchronized

Recent transactions stored in REDO Log flushed to disk

Frequency controlled by TimeBetweenGlobalCheckpoints

Performance vs. durability

Where is MySQL Cluster CGE used Subscriber Databases Management Web Session Management Service Delivery Platforms Messaging Presence VoIP Infrastructure Intelligent Networks Value Added Services IPTV Mobile Content Delivery Gaming DNS/DHCP for Broadband Payment Gateways Directory Services (LDAP) http://www.mysql.com/industry/telecom/

Subscriber Databases Management

Web Session Management

Service Delivery Platforms

Messaging

Presence

VoIP Infrastructure

Intelligent Networks

Value Added Services

IPTV

Mobile Content Delivery

Gaming

DNS/DHCP for Broadband

Payment Gateways

Directory Services (LDAP)

Nokia Siemens Networks, Alcatel Lucent, Ericsson, Nortel, HP OpenCall, Cisco, Motorola Areas include subscriber repositories, SIP infrastructure, Service Delivery Platforms, Prepaid Vodafone UK, Deutsche Telekom, Orange/FT, Telenor, Telekom Austria Areas include subscriber directories, “modern” IN platforms (SEPs), new services (VoIP, Video/IP), DHCP infrastructure for Broadband Networks, Payment Gateways, Mediation Customer Snapshots

Nokia Siemens Networks, Alcatel Lucent, Ericsson, Nortel, HP OpenCall, Cisco, Motorola

Areas include subscriber repositories, SIP infrastructure, Service Delivery Platforms, Prepaid

Vodafone UK, Deutsche Telekom, Orange/FT, Telenor, Telekom Austria

Areas include subscriber directories, “modern” IN platforms (SEPs), new services (VoIP, Video/IP), DHCP infrastructure for Broadband Networks, Payment Gateways, Mediation

Thank You