The mysqlnd replication and load balancing plugin

Ulf Wendel, Oracle Web scale made easy (just kidding...) The mysqlnd replication and load balancing plugin

MySQL scale-out solutions Part 1 - Proven and designed for web-scale

The speaker says... MySQL offers a variety of scale-out solutions. Which are they, when to use and how to use from an application developers point of view?

Developer dreams
Day 1 (Launch) – web site visitors The first customer
Systemload: 0.1 100%

The speaker says... Let's take a PHP developer centric look at a high-growth web site. The web site is build on top of the industrial leading LAMP platform: Linux, MySQL, Apache and PHP. The day before the launch, the developer dreams the dream of his options or shares rising in value: 2x, 5x, 42x ...

Developer dreams Day 2 (Growth) – web site visitors The first customer Twitter follower Facebook friend A slashdot reader A TV moderator Systemload: 0.5

The speaker says... ... a wonderful dream. The enterprise becomes popular, IPO is on the horizon. There are many reasons for growth. As a developer you should know about the MySQL scale-out solutions to be prepared. According to a MySQL Online Quickpoll from 2009 a majority of the MySQL users is using MySQL Replication.

Developer dreams Day 3 (Success) – scale out needed 408 Request Timeout 503 Service Unavailable 1040 SQLSTATE: 08004 (ER_CON_COUNT_ERROR) - Too many connections Systemload: 42

The speaker says... Dear developer, help is on the way. The PHP mysqlnd replication and load balancing plugin ( PECL/mysqlnd_ms ) is a drop-in solution for your PHP MySQL web application adding MySQL scale-out support. For basic scale-out setups no code changes are required. As of PHP 5.3 all three PHP MySQL APIs ( mysql , mysqli , PDO_MySQL ) can be compiled to use the MySQL native driver for PHP ( mysqlnd ) library. Mysqlnd is shipped with PHP as of version 5.3. It is a default as of PHP 5.4.

Scale out means clustering
Scale-out horizontally
TCO: Commodity hardware
TCO: Proven Open Source solutions
MySQL Server 1 MySQL Server 2 MySQL Server n

The speaker says... Scale out, not up. Use a cluster of commodity servers. Scale horizontally building upon proven Open Source solutions. Try MySQL Replication, MySQL Cluster or a custom MySQL scale-out solution.

Asynchronous read scale-out
Master: writes, Slaves: reads
Various topologies supported
Other use cases: backup, HA, OLTP/warehousing
MySQL Replication Writes Reads Master Slave Slave Slave

The speaker says... MySQL Replication is used for read scale-out. Writes are send to a MySQL master, reads are send to a MySQL slave. The master sends updates to the slaves in an asynchronous way. Slaves may not have the latest changes. MySQL Replication can also be used for:
Backup – perfom blocking backup on slave
High Availablility – for example, remote slaves
Warehousing – OLTP reporting on the slaves

Real-time transactional write scale-out
99,999% availability
Auto sharding, shared nothing architecture
Web use cases: session server, tele communication
MySQL Cluster SQL Node SQL Node Cluster Data Node Cluster Data Node Cluster Data Node

The speaker says... MySQL Cluster is used for real-time transactional read and write scale-out. Data is auto-partitioned on cluster data nodes resulting in 99.999% availability. The cluster is accessed through MySQL Application Cluster Nodes. The most basic MySQL Application Cluster Node is a SQL Node – a MySQL server (mysqld). If MySQL Cluster is new to you, think of MySQL Cluster as a storage engine for the network. Like InnoDB and MyISAM are storage engines for disks.

For example, Partitioning
Horizontal: rows in different tables
Vertical: columns in additional tables
... sharding
Custom Cluster Shard 1 ID % 2 = 0 Node 3 Cats Node 4 Dogs Shard 2 ID % 2 = 1

The speaker says... Special requirements may make it necessary to build a custom cluster. For example, your application may require a sharding solution.

On using any database cluster Part 2 - Required application changes

Tasks:
Choose servers to execute statement
Load balance within candidates
Maintain connection pool
Automatic fail over for High Availability
Application using any DB cluster MySQL Server 1 MySQL Server 2 MySQL Server n

The speaker says... All PHP applications talking to a cluster of MySQL database servers are facing the same tasks. Replacing a single database with a database cluster means changing the 1:1 relation between the application and the database into a 1:n relation . 1:n puts an additional task on the application: find the right n, find the right server to talk to.

The plugin for all of you MySQL Server 1 MySQL Server 2 MySQL Server n

The speaker says... PECL/mysqlnd_ms is a plugin for the MySQL native driver for PHP (mysqlnd) library . The mysqlnd library is part of PHP as of version 5.3. As of 5.4 mysqlnd is a default. All three PHP MySQL APIs (mysql, mysqli, PDO_MySQL) can be compiled to use the mysqlnd library, thus all existing PHP MySQL application are supported by the plugin. From an applications point of view a mysqlnd plugin can act as a transparent proxy. Depending on plugin an use case, no code changes are needed. A plugin is a drop-in solution.

Plugin focus: MySQL Replication Part 3 - Tasks, solutions and limitations

Using MySQL Replication
Statement redirection
Read/write split (optional: master on write)
Replication filter (table based parititioning)
Custom: callback
Load Balancing
Round robin, random, random once
Custom: callback
Connection Pooling
Lazy connections
Failover

The speaker says... The plugin is not limited to but optimized for MySQL Replication. The common clustering support feature set focusses on MySQL Replication. For example, statement redirection covers read/write splitting and MySQL Replication filter support. All read accesses are executed on the slave servers, all writes are performed on the master. MySQL Replication supports table based partitioning, replicating selected tables to selected slaves only. The plugin can be made aware of this.

Speaking code /* Connection user handle represents a connection pool */ $mysqli = new mysqli( "hostname" , $user, $pw, $db, $port); /* Read/Write split, Load Balancing */ $res = $mysqli->query("SELECT 'Slave speaking' FROM DUAL"); var_dump($res->fetch_all(MYSQLI_ASSOC)); { "hostname" :{ "master":{ "host":"localhost" }, "slave":{ "slave_0":{"host":"192.168.2.27"} }, } }

The speaker says... No code changes for basic use cases. Install the plugin following standard PHP PECL procedures. Create a configuration file. Enable the plugin and set its configuration file in your PHP configuration using mysqlnd_ms.enable=1 and mysqlnd_ms.ini_file=config.ini . In the plugin configuration create a section with the name of the host your application connects to, for example „hostname“. List master and slave servers. Any connection to „hostname“ is now handled by the plugin. Using any API.

Some code changes /* Connection user handle represents a connection pool */ $mysqli = new mysqli("hostname", $user, $pw, $db, $port); / * R/W split: no SELECT, run on master */ $mysqli->query("SET @myrole='Master'); /* R/W split: overruled by SQL hint */ $sql = sprintf("/*%s*/%s", MYSQLND_MS_MASTER_SWITCH, "SELECT @myrole AS _role"); $mysqli->query($sql); var_dump($res->fetch_all(MYSQLI_ASSOC));

The speaker says... Non-trivial use cases require support by the application. In other words: existing applications may need updates, new applications need to be designed appropriately. In the example a SQL hint is used to force executio of a SELECT statement on a MySQL Replication master server. There are SQL hints for running a statement on the master, on the slave and on the last used server. Alternative: new APIs call, no win compared to SQL hint.

Why and when code changes $mysqli = new mysqli("hostname", $user, $pw, $db, $port); $mysqli->query("SET @myrole='Master'); $mysqli->query("SELECT myrole AS _role"); Without plugin With plugin

The speaker says... An application not designed for use with a cluster assumes a connection handle always points to the same server (1:1), having the same state. When using the plugin a connection handle represents a pool of connections (1:n), potentially having different states. Connection switches may occur whenever a statement is executed. After the switch, the connection handle may point to a connection with a different state. Applications need to take care and hint the plugin.

A trouble maker in detail
Connection State
Connection settings, e.g. charsets
Transaction status
Temporary tables
Table locks
Session system variables and user variables
Prepared statements (API and SQL PREPARE)
HANDLER variables
Locks acquired with GET_LOCK
Units of work from an application point of view

The speaker says... No client library, no client proxy can monitor all state changing actions. Applications in need for a certain connection state need to control the plugins connection switches using :
SQL hints
MYSQLND_MS_SLAVE_SWITCH
MYSQLND_MS_MASTER_SWITCH
MYSQLND_MS_LAST_USED_SWITCH
Callback to replace built-in server selection logic

Trouble maker transaction
Unit of work
Part of the connection state
Application to prevent connection switches
Workaround
PHP 5.3: SQL hints must be used
PHP 5.4: API calls monitored, switch auto-disabled
Replication cluster tip
Always run on master, only master has latest data!
use trx_stickiness = master

The speaker says... A transaction is part of the connection state. For the duration of a transaction, server switches must be disabled. Transactions can be controlled by SQL statements and API calls. The plugin cannot monitor all of them with reasonable effort. Thus, you must hint the plugin. Disable load balancing by adding SQL hints (MYSQLND_MS_LAST_USED_SWITCH), you install your own load balancer logic (callback). As of PHP 5.4, the plugin monitors API calls to auto-disable switching.

Slave request
read-only: statement begins with SELECT
read-only: statement begins with slave SQL hint
custom: callback picks slave for statement
Redirection: Read/Write split Writes Reads (SELECT, SQL hint) Master Slave Slave Slave

The speaker says... MySQL Replication is for read scale-out. Read-only statements shall go to slaves, writes to the master. The plugin automatically redirects a statement to a slave, if the statement begins with SELECT or the MYSQLND_MS_SLAVE_SWITCH SQL hint. All other statements are considered as write operations and executed on the master. By default SHOW or CALL will be run on the master. It is a design principle of the plugin that all automatic decisions can be overruled. A callback can be installed to pick the server for running a statement on, if needed.

Optimization: use master only after write
MySQL Replication is asynchronous
MySQL slaves can lag behind
Most applications need current data after write
Redirection: Master on write Writes, Reads after write Reads before write Master Slave Slave

The speaker says... The master_on_write configuration option is an optimization of the R/W split. MySQL Replication slaves can lag behind the master because MySQL Replication is asynchronous. When using master_on_write all reads made after a write are executed on the master for the duration of the web request. Applications that perform a write often need to rely on fresh reads afterwards. Think of a web shop creating an order, writing to the master. If reading again from the order table, within the same web request, the application should access the master to make sure the previously written is available.

MySQL Replication filter support
Slaves replicate different databases/tables
Redirection: Partitioning Writes Reads (Filter) DB_Report DB_Shop DB_Report DB_Internal DB_Shop DB_Shop Master Slave Slave Slave

The speaker says... MySQL Replication supports partitioning: slaves can be configured to replicate only selected databases and tables using MySQL Replication filter. Filter help to reduce the amount of internal replication messages. Less frequently accessed data can be replicated to only few slaves. In future versions, plugin can be made aware of the filtering rules for appropriate statement redirection. Every statement is analyzed to find a slave for reading. If no match, the plugin redirects the statement to the master.

Built-in or user-defined through callback
Round robin: iterate over slaves for each read
Random: pick random slave for each read
Random once: pick random slave for all reads
Load Balancing Writes Load Balancing of Reads Master Slave Slave Slave

The speaker says... MySQL Replication is about read scale-out. The plugin helps distributing the load over the read-only slaves. The best and default load balancing stategy is the one with the lowest impact on the connection state: random once. Minimize switches for the life-span of your PHP script. Pick a random slave during startup and use it for all reads. The life-span of PHP is the of a web request. It is short. Two web requests will end up on two random slaves. Load balancing adaptive to server health can be achieved through a user-defined callback.

Caution: connection state change
Default: failover=disabled, raise exception
Optional: failover=master, automatic
Pooling: connection failover Writes Reads Master Slave Slave Slave

The speaker says... Automatic and transparent failover is impossible with stateful connections. The plugin does not and cannot know the full state of a connection. Parts of the state are only known to the server. By default the plugin leaves it to the application to handle a connection failure. Like without the plugin, it is up to the application to decide what to do. Applications using stateless connections can enable automatic failover to the master.

Do not open connection before needed
Delay open until statement execution
Reduce number of connections, reduce load
Pooling: lazy connections Writes Reads Master Slave Slave Slave

The speaker says... By default, to reduce the connection pool size of every PHP web request, connections are not opened before executing a statement. Imagine you have 50 concurrent web requests, you configured four slaves and, you are using random once. If not using lazy connections, the plugin opens 50 x 4 = 200 slave connections but uses at max 50 of them (random once). Every connection occupies resources – ever seen a PHP application opening connections before using? There are many badly written ones...

http://blog.ulf-wendel.de	6789
http://www.planet-php.net	2113
http://www.scoop.it	453
http://www.planet-php.org	150
http://planet-php.org	140
http://www.phpeye.com	34
http://phpeye.com	15
http://translate.googleusercontent.com	12
http://translate.googleusercontent.com	12
http://www.hanrss.com	11
http://php5.planet-php.org	10
http://127.0.0.1:8795	8
http://www.blog.ulf-wendel.de	5
http://www.twylah.com	5
http://webcache.googleusercontent.com	4
http://blog.planet-php.org	2
http://www.newsblur.com	2
http://127.0.0.1	1
http://translate.googleuser	1
http://wall.bit-tests.1and1.com	1
http://translate.google.com	1
http://paper.li	1

The mysqlnd replication and load balancing plugin

by Ulf Wendel

Accessibility

Categories

Upload Details

Usage Rights

22 Embeds 9,770

Statistics