C:\fakepath\cluster 7 1

<Insert Picture Here> MySQL Cluster 7.1 Andrew Morgan Mat Keep MySQL Cluster Product Manager MySQL Product Management andrew.morgan@oracle.com mat.keep@oracle.com

Agenda • Introduction to MySQL Cluster • Deep Dive into MySQL Cluster 7.1 New Features • Resources to Get Started

Serving Key Markets & Industry Leaders Web / Web 2.0 OEM / ISV's On Demand, SaaS, Hosting Telecommunications Enterprise 2.0 Open-source powers the Web & the Network

MySQL Cluster Users and Applications HA, Real Time Services: Web & Telecoms • Subscriber Databases • Service Delivery Platforms • Application Servers • Telco Value Added Services • Web Session Stores • eCommerce • VoIP, IPTV & VoD • Mobile Content Delivery • On-Line app stores and portals • DNS/DHCP for Broadband • Payment Gateways • Data Store for LDAP Directories •http://www.mysql.com/customers/cluster/

MySQL Cluster - Key Advantages High Throughput Distributed, Parallel architecture Reads & Writes Transactional, ACID-compliant relational database Carrier-Grade Shared-nothing design, synchronous data replication Availability Sub-second failover & self-healing recovery Real-Time High-load, real-time performance Responsiveness Predictable low latency, bounded access times Incrementally scale out, scale up and scale on-line On-Line, Linear Scalability Linearly scale with distribution awareness Low TCO, GPL & Commercial editions, scale on COTS Open platform Flexible APIs: SQL, C++, Java, OpenJPA, LDAP & HTTP

MySQL Cluster Architecture Parallel Database with no SPOF: High Read & Write Performance & 99.999% uptime Clients MySQL Cluster Application Nodes MySQL MySQL Cluster Cluster Mgmt Mgmt MySQL Cluster Data Nodes

MySQL Cluster 7.0 Momentum (GA April ‘09) Leading Provider of Mobile, Fixed & 250% Increase in Download Rate Broadband Services across Europe & Asia “Telenor has found MySQL Cluster to be the best performing database in the world for our applications.” Windows Port Peter Eriksson, Manager, Network Provisioning LDAP Interface I/O Multi-Threading 2x Faster Record Handling Largest Entertainment Guide on the On-Line Cluster Scaling Mobile Web: “go2 has the ability to efficiently scale 4x Higher Throughput our platform with the growth of the mobile web” Dan Smith, Co-Founder & CEO, go2 Media

MySQL Cluster CGE 7.1 – Key Enhancements Delivering up to 10x higher Reducing Cost of Operations Java Throughput Simplified Management & MySQL Cluster Connector for Monitoring: Java: NDBINFO Native Java API MySQL Cluster Manager (part of CGE only) OpenJPA Plug-In Faster Restarts

ndbinfo • Example 1: Check memory usage/availability mysql> select * from memoryusage; +---------+--------------+------+------+ | node_id | DATA_MEMORY | used | max | +---------+--------------+------+------+ | 3 | DATA_MEMORY | 594 | 2560 | | 4 | DATA_MEMORY | 594 | 2560 | | 3 | INDEX_MEMORY | 124 | 2336 | | 4 | INDEX_MEMORY | 124 | 2336 | +---------+--------------+------+------+ • Note that there is a DATA_MEMORY and INDEX_MEMORY row for each data node in the cluster • If the Cluster is nearing the configured limit then increase the DataMemory and/or IndexMemory parameters in config.ini and then perform a rolling restart

ndbinfo • Example 2: Check how many table scans performed on each data node since the last restart mysql> select node_id as 'data node', val as 'Table Scans' from counters where counter_name='TABLE_SCANS'; +-----------+-------------+ | data node | Table Scans | +-----------+-------------+ | 3 | 3 | | 4 | 4 | +-----------+-------------+ • You might check this if your database performance is lower than anticipated • If this figure is rising faster than you expected then examine your application to understand why there are so many table scans

MySQL Cluster 7.1: ndbinfo • Example 3: Check if approaching the point at which the undo log completely fills up between local checkpoints (which could result in delayed transactions or even a database halt if not addressed): mysql> select node_id as 'data node', total as 'configured undo log buffer size', used as 'used buffer space' from logbuffers where log_type='DD- UNDO'; +-----------+---------------------------------+-------------------+ | data node | configured undo log buffer size | used buffer space | +-----------+---------------------------------+-------------------+ | 3 | 2096128 | 0 | | 4 | 2096128 | 0 | +-----------+---------------------------------+-------------------+ • If log buffer is almost full then increase size of log buffer

MySQL Cluster 7.1: ndbinfo.counters •mysql> select * from ndbinfo.counters; •+---------+------------+----------------+------------+--------------+------+ | node_id | block_name | block_instance | counter_id | counter_name | val | +---------+------------+----------------+------------+--------------+------+ • View created from hidden tables to | 3 | DBLQH | 1 | 10 | OPERATIONS | 2069 | | 3 | DBLQH | 2 | 10 | OPERATIONS | 28 | present counters for the number of | 4 | DBLQH | 1 | 10 | OPERATIONS | 2066 | | 4 | DBLQH | 2 | 10 | OPERATIONS | 27 | events since the last node restart | 3 | DBTC | 0 | 1 | ATTRINFO | 140 | | 3 | DBTC | 0 | 2 | TRANSACTIONS | 19 | • When using a single threaded data | 3 | DBTC | 0 | 3 | COMMITS | 19 | | 3 | DBTC | 0 | 4 | READS | 19 | node (ndbd), block _instance=0 | | 3 | DBTC 3 | DBTC | | 0 | 0 | 5 | SIMPLE_READS | 6 | WRITES | 0 | 0 | • If using a multi-threaded data node | | 3 | DBTC 3 | DBTC | | 0 | 0 | 7 | ABORTS 8 | TABLE_SCANS | | 0 | 0 | (ndbmtd) then there will be a DBLQH | 3 | DBTC | 0 | 9 | RANGE_SCANS | 0 | row for each Local Query Handler | 4 | DBTC | 0 | 1 | ATTRINFO | 2 | | 4 | DBTC | 0 | 2 | TRANSACTIONS | 1 | thread, for the OPERATIONS | 4 | DBTC | 0 | 3 | COMMITS | 1 | | 4 | DBTC | 0 | 4 | READS | 1 | counter, identified with different | 4 | DBTC | 0 | 5 | SIMPLE_READS | 0 | | 4 | DBTC | 0 | 6 | WRITES | 0 | values for block_instance | 4 | DBTC | 0 | 7 | ABORTS | 0 | | 4 | DBTC | 0 | 8 | TABLE_SCANS | 1 | | 4 | DBTC | 0 | 9 | RANGE_SCANS | 0 | +---------+------------+----------------+------------+--------------+------+

MySQL Cluster 7.1: ndbinfo.logbuffers • View created from hidden tables to present log buffer usage • log_type=REDO: „total‟ represents the amount of memory configured for the redo mysql> select * from ndbinfo.logbuffers; logs and „used‟ indicates how much of it is +---------+----------+--------+----------+----------+--------+ currently being used. If the buffer fills up | node_id | log_type | log_id | log_part | total | used | +---------+----------+--------+----------+----------+--------+ then will see 1221 “REDO log buffers | 3 | REDO | 0 | 0 | 67108864 | 262144 | | 3 | DD-UNDO | 4 | 0 | 2096128 | 0 | overloaded” errors – avoid this by | 4 | REDO | 0 | 0 | 67108864 | 262144 | increasing the RedoBuffer configuration | 4 | DD-UNDO | 4 | 0 | 2096128 | 0 | +---------+----------+--------+----------+----------+--------+ parameter if „used‟ approaches „total‟ • log_type=DD_UNDO: „total‟ represents the amount of memory configured for the undo logs (UNDO_BUFFER_SIZE when creating the log group) and „used‟ indicates how much of it is currently being used. If buffers fill too quickly then transactions can be delayed while they are flushed to disk; in extreme cases causing database to halt

MySQL Cluster 7.1: ndbinfo.logspaces • View created from hidden tables to present logspace usage • log_type=REDO: 1 row for each of the 4 mysql> select * from logspaces; file sets for each data node. +---------+----------+--------+----------+-----------+--------+ „total‟=NoOfFragmentLogFiles*Fragment | node_id | log_type | log_id | log_part | total | used | +---------+----------+--------+----------+-----------+--------+ LogFileSize and „used‟ is the amount | 3 | REDO | 0 | 0 | 536870912 | 0 | actually used. If the files fill up before a | 3 | REDO | 0 | 1 | 536870912 | 0 | | 3 | REDO | 0 | 2 | 536870912 | 0 | local checkpoint can complete then error | 3 | REDO | 0 | 3 | 536870912 | 0 | code 410 (Out of log file space | 3 | DD-UNDO | 4 | 0 | 78643200 | 169408 | | 4 | REDO | 0 | 0 | 536870912 | 0 | temporarily) will be observed. Avoid this | 4 | REDO | 0 | 1 | 536870912 | 0 | by increasing NoOfFragmentLogFiles | 4 | REDO | 0 | 2 | 536870912 | 0 | | 4 | REDO | 0 | 3 | 536870912 | 0 | and/or FragmentLogFileSize if used | 4 | DD-UNDO | 4 | 0 | 78643200 | 169408 | +---------+----------+--------+----------+-----------+--------+ approaches total. • log_type=DD_UNDO: „total‟=cumulative size of all of the undo log files assigned to the log group, as added using create/alter logfile group or the InitialLogFileGroup parameter. Add extra undo files if „used‟ approaches „total‟ to avoid 1501 errors

MySQL Cluster 7.1: ndbinfo.memoryusage • View created from hidden tables to present memory limits and usage mysql> select * from memoryusage; • Compare the amount of memory and index +---------+--------------+------+------+ used to the amount configured for each data | node_id | memory_type | used | max | node +---------+--------------+------+------+ | 3 | DATA_MEMORY | 632 | 3144 | • DATA_MEMORY: „max‟ = DataMemory | 4 | DATA_MEMORY | 632 | 3144 | parameter. If „used‟ approaches „max‟ then | 3 | INDEX_MEMORY | 38 | 2336 | increase DataMemory and/or delete obsolete | 4 | INDEX_MEMORY | 38 | 2336 | rows run OPTIMIZE TABLE for the Cluster +---------+--------------+------+------+ tables • INDEX_MEMORY: „max‟ = IndexMemory parameter. If „used‟ approaches „max‟ then increase IndexMemory (+ DataMemory if needed) and/or delete obsolete rows run OPTIMIZE TABLE for the Cluster tables

MySQL Cluster 7.1: ndbinfo.nodes • View created from hidden tables for the status of data nodes. mysql> select * from nodes; • For each data node that is running in the +---------+--------+---------+-------------+ cluster, a corresponding row in this table | node_id | uptime | status | start_phase | +---------+--------+---------+-------------+ provides the | 3 | 3719 | STARTED | 0 | • Uptime: time in seconds that this node has been | 4 | 3719 | STARTED | 0 | running since it was last started or restarted. +---------+--------+---------+-------------+ • Status: One of NOTHING, CMVMI, STARTING, STARTED, SINGLEUSER, STOPPING_1, STOPPING_2, STOPPING_3, or STOPPING_4. • Start-phase: If Status=STARTING then the start-phase (as seen through ndb_mgm), 0 otherwise.

MySQL Cluster Connector for Java • New Domain Object Model Persistence API (ClusterJ) : • Java API • High performance, low latency • Feature rich • JPA interface built upon this new Network Java layer: • Java Persistence API compliant • Implemented as an OpenJPA plugin • Uses ClusterJ where possible, reverts to JDBC for some operations • Higher performance than JDBC • More natural for most Java designers • Easier Cluster adoption for web Data Nodes applications

ClusterJ • High Performance, Easy to Use • In the style of Hibernate / JPA / JDO • Domain Object Model DataMapper pattern • Data is represented as domain objects • Domain objects are separate from business logic • Domain objects are mapped to database tables • Built on ndbjtie • JNI adapter • integral part of MySQL Cluster • Straight mapping of MySQL Cluster API (a.k.a Ndb API) to Java • Does not support relationships • Look at JDO / JPA for these modelling patterns

Cluster/J Interfaces Configuration Session properties Factory • SessionFactory • One per Cluster per JVM • Session Session & Session & Session & Transaction Transaction Transaction • Instance per user; represents a Cluster connection • Transaction • Instance per Session Domain Domain Domain Object Domain Object Object Domain Object Object Domain Object • Query Domain Domain Domain • Multiple instances per Session Object Domain Domain Domain Object Object Object Object Object Annotated interface

ClusterJ Configuration properties example com.mysql.clusterj.connectstring=192.168.0.44:1186 com.mysql.clusterj.database=clusterdb com.mysql.clusterj.connect.retries=4 com.mysql.clusterj.connect.delay=5 com.mysql.clusterj.connect.verbose=1 com.mysql.clusterj.connect.timeout.before=30 com.mysql.clusterj.connect.timeout.after=20 com.mysql.clusterj.max.transactions=1024

ClusterJ Annotated interface example @PersistenceCapable(table="employee") @Index(name="idx_uhash") public interface Employee { @PrimaryKey CREATE TABLE employee ( int getId(); void setId(int id); id INT NOT NULL PRIMARY KEY, String getFirst(); first VARCHAR(64) DEFAULT NULL, void setFirst(String first); last VARCHAR(64) DEFAULT NULL, String getLast(); municipality VARCHAR(64) DEFAULT NULL, void setLast(String last); started VARCHAR(64) DEFAULT NULL, @Column(name="municipality") ended VARCHAR(64) DEFAULT NULL, @Index(name="idx_municipality") department INT NOT NULL DEFAULT 1, String getCity(); UNIQUE KEY idx_u_hash (first,last) void setCity(String city) USING HASH, String getStarted(); KEY idx_municipality (municipality) void setStarted(String date); ) ENGINE=NDBCLUSTER; String getEnded(); void setEnded(String date); Integer getDepartment(); void setDepartment( Integer department);}

ClusterJ Application code example // Create a session (connection to the database) SessionFactory factory = ClusterJHelper.getSessionFactory(props); Session session = factory.getSession(); // Create and initialise an Employee Employee newEmployee = session.newInstance(Employee.class); newEmployee.setId(988); newEmployee.setFirst("John"); newEmployee.setLast("Jones"); newEmployee.setStarted("1 February 2009"); newEmployee.setDepartment(666); // Write the Employee to the database session.persist(newEmployee); // Fetch the Employee from the database Employee theEmployee = session.find(Employee.class, 988); // Make some changes to the Employee & write back to the database theEmployee.setDepartment(777); theEmployee.setCity("London"); session.updatePersistent(theEmployee); // Retrieve the set all of Employees in department 777 QueryBuilder builder = session.getQueryBuilder(); QueryDomainType<Employee> domain = builder.createQueryDefinition(Employee.class); domain.where(domain.get("department").equal(domain.param("department"))); Query<Employee> query = session.createQuery(domain); query.setParameter("department",777); List<Employee> results = query.getResultList();

ClusterJPA • Removes ClusterJ limitations: • Persistent classes • Relationships • Joins in queries • Lazy loading • Table and index creation from object model • Implemented as an OpenJPA plugin • Better JPA performance for insert, update, delete

ClusterJPA Annotated class example @Entity(name = "department") mysql> describe department; public class Department { +----------+--------------+------+-----+---------+-------+ private int Id; | Field | Type | Null | Key | Default | Extra | +----------+--------------+------+-----+---------+-------+ private String Site; | id | int(11) | NO | PRI | NULL | | | location | varchar(255) | YES | | NULL | | +----------+--------------+------+-----+---------+-------+ public Department(){} @Id public int getId() {return Id;} public void setId(int id) {Id=id;} @Column(name="location") public String getSite() {return Site;} public void setSite(String site) {Site=site;} }

ClusterJPA Application code example EntityManagerFactory entityManagerFactory = Persistence.createEntityManagerFactory("clusterdb"); EntityManager em = entityManagerFactory.createEntityManager(); EntityTransaction userTransaction = em.getTransaction(); userTransaction.begin(); Department dept = new Department(); dept.setId(101); dept.setSite(“London”); em.persist(dept); userTransaction.commit(); userTransaction.begin(); Department theDepartment = em.find(Department.class, 101); theDepartment.setSite(“Edinburgh”); userTransaction.commit(); em.close(); entityManagerFactory.close();

Performance

MySQL Cluster Manager 1.0 Features Automated Management Monitoring HA Operations Cluster-Wide Status Monitoring & Disk Persistence Management Recovery Configuration Process Management Consistency On-Line Operations HA Agent Operation (Upgrades / Reconfiguration)

Terms used by MySQL Cluster Manager • Site: the set of physical hosts which are to run Cluster processes to be managed by MySQL Cluster Manager. A site can include 1 or more clusters. • Cluster: represents a MySQL Cluster deployment. A Cluster contains 1 or more processes running on 1 or more hosts • Host: Physical machine, running the MySQL Cluster Manager agent • Agent: The MySQL Cluster Manager process running on each host • Process: an individual MySQL Cluster node; one of: ndb_mgmd, ndbd, ndbmtd, mysqld & ndbapi* • Package: A copy of a MySQL Cluster installation directory as downloaded from mysql.com, stored on each host *ndbapi is a special case, representing a slot for an external application process to connect to the cluster using the NDB API

Example configuration mysql client • MySQL Cluster Manager agent runs on each physical host 7. mysqld 8. mysqld • No central process for Cluster 1. ndb_mgmd 2. ndb_mgmd Manager – agents co-operate, each one responsible for its local nodes agent agent • Agents are responsible for managing 192.168.0.10 192.168.0.11 all nodes in the cluster • Management responsibilities 3. ndbd 4. ndbd • Starting, stopping & restarting nodes 5. ndbd 6. ndbd • Configuration changes agent agent • Upgrades 192.168.0.12 192.168.0.13 • Host & Node status reporting • Recovering failed nodes n. mysqld MySQL Server (ID=n) n. ndb_mgmd Management Node (ID=n) n. ndbd Data Node (ID=n) agent MySQL Cluster Manager agent

Installing, configuring, running & accessing MySQL Cluster Manager • The agent must be installed and run on each host in the Cluster: mysql 1. Expand the tar-ball into a known directory client (/usr/local/mcm) 2. Copy /usr/local/mcm/etc/mysql-cluster-manager.ini to /home/billy/mcm and edit: [mysql-proxy] plugins=manager manager-port = :1862 log-file = mysql-manager-agent.log log-level = message manager-directory=/home/billy/mcm/manager agent agent 3. “plugins=manager” should always be used 4. Launch the agent process: 192.168.0.10 192.168.0.11 /usr/local/mcm/bin/mysql-cluster-manager --defaults- file=/home/billy/mcm/mysql-cluster-manager.ini 5. Access any of the agents from any machine (that has the mysql client installed): agent agent C:Usersam233268>mysql -h 192.168.0.10 -P 1862 -u admin –p Enter password: super Welcome to the MySQL monitor. Commands end with ; or g. Your MySQL connection id is 1 192.168.0.12 192.168.0.13 Server version: 1.0.1-agent-manager MySQL Cluster Manager Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. mysql>

Creating & Starting a Cluster mysql client 1. Define the site: Mysql> create site --hosts=192.168.0.10,192.168.0.11, -> 192.168.0.12,192.168.0.13 mysite; 2. Expand the MySQL Cluster tar-ball(s) 7. mysqld 8. mysqld from mysql.com to known directory 1. ndb_mgmd 2. ndb_mgmd 3. Define the package(s): agent agent Mysql> add package --basedir=/usr/local/mysql_6_3_26 6.3; Mysql> add package --basedir=/usr/local/mysql_7_0_7 7.0; Note that the basedir should match the 192.168.0.10 192.168.0.11 directory used in Step 2. 3. ndbd 4. ndbd 4. Create the Cluster Mysql> create cluster --package=6.3 5. ndbd 6. ndbd -> --processhosts=ndb_mgmd@192.168.0.10,ndb_mgmd@192.168.0.11, -> ndbd@192.168.0.12,ndbd@192.168.0.13, ndbd@192.168.0.12, -> ndbd@192.168.0.13,mysqld@192.168.9.10,mysqld@192.168.9.11 agent agent -> mycluster; This is where you define what 192.168.0.12 192.168.0.13 nodes/processes make up the Cluster and where they should run 5. Start the Cluster: Mysql> start cluster mycluster;

Check the status of the Cluster mysql client 1. Request the status of the Cluster processes mysql> show status --process mycluster; •+------+----------+--------------+---------+-----------+ 7. mysqld 8. mysqld | Id | Process | Host | Status | Nodegroup | +------+----------+--------------+---------+-----------+ | 1 | ndb_mgmd | 192.168.0.10 | running | | 1. ndb_mgmd 2. ndb_mgmd | 2 | ndb_mgmd | 192.168.0.11 | running | | | 3 | ndbd | 192.168.0.12 | running | 0 | agent agent | 4 | ndbd | 192.168.0.13 | running | 0 | | 5 | ndbd | 192.168.0.12 | running | 1 | | 6 | ndbd | 192.168.0.13 | running | 1 | | 7 | mysqld | 192.168.0.10 | running | | 192.168.0.10 192.168.0.11 | 8 | mysqld | 192.168.0.11 | running | | +------+----------+--------------+---------+-----------+ 3. ndbd 4. ndbd 2. Get the status of the hosts & agents mysql> list hosts mysite; 5. ndbd 6. ndbd •+--------------+-----------+---------+ | Host | Status | Version | agent agent +--------------+-----------+---------+ | 192.168.0.10 | Available | 1.0.1 | | 192.168.0.11 | Available | 1.0.1 | | 192.168.0.12 | Available | 1.0.1 | 192.168.0.12 192.168.0.13 | 192.168.0.13 | Available | 1.0.1 | +--------------+-----------+---------+

Setting Cluster parameters mysql client • Example: Turn of privilege checking for all MySQL Servers and change the port for connecting to the mysqld with ID = 8 to 7. mysqld 8. mysqld 3307. Allow data nodes to be automatically 1. ndb_mgmd 2. ndb_mgmd restarted after they fail: mysql> set skip_grant_tables:mysqld=true,port:mysqld:8=3307, agent agent --> StopOnError:ndbd=false mycluster; MySQL Cluster Manager automatically 192.168.0.10 192.168.0.11 determines which nodes (processes) need to be restarted and in which order to make 3. ndbd 4. ndbd the change take effect but avoid loss of 5. ndbd 6. ndbd service agent agent 192.168.0.12 192.168.0.13

Upgrade Cluster mysql client • Upgrade from MySQL Cluster 6.3.26 to 7. mysqld 8. mysqld 7.0.7: 1. ndb_mgmd 2. ndb_mgmd mysql> upgrade cluster --package=7.0 mycluster; agent agent • Automatically upgrades each node and 192.168.0.10 192.168.0.11 restarts the process – in the correct order to avoid any loss of service 3. ndbd 4. ndbd • Without MySQL Cluster Manager, the 5. ndbd 6. ndbd administrator must stop each process in turn, start the process with the new version agent agent and wait for the node to restart before 192.168.0.12 192.168.0.13 moving onto the next one

"MySQL Cluster 7.1 offers the potential for users of the FreeRADIUS AAA server to enhance operational efficiency and performance in the management and access of their business critical and time sensitive applications” Alan Dekok FreeRADIUS Project Founder, CEO of Network RADIUS "Many of our users rely on OpenLDAP to run their most critical directory services, and with the addition of MySQL Cluster at the back-end, can support highly dynamic and scalable directory databases. The release of MySQL Cluster 7.1 can enhance manageability and platform choice for our communities and customers." Marty Heyman, CEO of Symas Corporation and OpenLDAP Project Member

Resources to Get Started •MySQL Cluster 7.1, Architecture and New Features • http://www.mysql.com/why-mysql/white- papers/mysql_wp_cluster7_architecture.php • MySQL Cluster Manager white paper • http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster_manager.php •MySQL Cluster Connector for Java white paper • http://www.mysql.com/why-mysql/white- papers/mysql_wp_cluster_connector_for_java.php •MySQL Cluster 7.1 Evaluation Guide • http://www.mysql.com/why-mysql/white-papers/mysql_cluster_eval_guide.php •Getting Started with MySQL Cluster • http://www.mysql.com/products/database/cluster/get-started.html •MySQL Cluster on the Web • http://www.mysql.com/products/database/cluster/

C:\fakepath\cluster 7 1

by thingsandstuff

Accessibility

Categories

Upload Details

Usage Rights

Statistics

$C:\fakepath\cluster 7 1$ C:\fakepath\cluster 7 1 Presentation Transcript