Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The mysqlnd replication and load balancing plugin


Published on

The mysqlnd replication and load balancing plugin for mysqlnd makes using MySQL Replication from PHP much easier. The plugin takes care of Read/Write splitting, Load Balancing, Failover and Connection Pooling. Lazy Connections, a feature not only useful with replication, help reducing the MySQL server load. Like any other mysqlnd plugin, the plugin operates mostly transparent from an applications point of view and can be used in a drop-in style.

Published in: Technology
  • Be the first to comment

The mysqlnd replication and load balancing plugin

  1. 1. Ulf Wendel, Oracle Web scale made easy (just kidding...) The mysqlnd replication and load balancing plugin
  2. 2. MySQL scale-out solutions Part 1 - Proven and designed for web-scale
  3. 3. The speaker says... MySQL offers a variety of scale-out solutions. Which are they, when to use and how to use from an application developers point of view?
  4. 4. Developer dreams <ul>Day 1 (Launch) – web site visitors The first customer </ul>Systemload: 0.1 100%
  5. 5. The speaker says... Let's take a PHP developer centric look at a high-growth web site. The web site is build on top of the industrial leading LAMP platform: Linux, MySQL, Apache and PHP. The day before the launch, the developer dreams the dream of his options or shares rising in value: 2x, 5x, 42x ...
  6. 6. Developer dreams Day 2 (Growth) – web site visitors The first customer Twitter follower Facebook friend A slashdot reader A TV moderator Systemload: 0.5
  7. 7. The speaker says... ... a wonderful dream. The enterprise becomes popular, IPO is on the horizon. There are many reasons for growth. As a developer you should know about the MySQL scale-out solutions to be prepared. According to a MySQL Online Quickpoll from 2009 a majority of the MySQL users is using MySQL Replication.
  8. 8. Developer dreams Day 3 (Success) – scale out needed 408 Request Timeout 503 Service Unavailable 1040 SQLSTATE: 08004 (ER_CON_COUNT_ERROR) - Too many connections Systemload: 42
  9. 9. The speaker says... Dear developer, help is on the way. The PHP mysqlnd replication and load balancing plugin ( PECL/mysqlnd_ms ) is a drop-in solution for your PHP MySQL web application adding MySQL scale-out support. For basic scale-out setups no code changes are required. As of PHP 5.3 all three PHP MySQL APIs ( mysql , mysqli , PDO_MySQL ) can be compiled to use the MySQL native driver for PHP ( mysqlnd ) library. Mysqlnd is shipped with PHP as of version 5.3. It is a default as of PHP 5.4.
  10. 10. Scale out means clustering <ul><li>Scale-out horizontally </li></ul><ul><ul><li>TCO: Commodity hardware
  11. 11. TCO: Proven Open Source solutions </li></ul></ul>MySQL Server 1 MySQL Server 2 MySQL Server n
  12. 12. The speaker says... Scale out, not up. Use a cluster of commodity servers. Scale horizontally building upon proven Open Source solutions. Try MySQL Replication, MySQL Cluster or a custom MySQL scale-out solution.
  13. 13. <ul><li>Asynchronous read scale-out </li></ul><ul><ul><li>Master: writes, Slaves: reads
  14. 14. Various topologies supported
  15. 15. Other use cases: backup, HA, OLTP/warehousing </li></ul></ul>MySQL Replication Writes Reads Master Slave Slave Slave
  16. 16. The speaker says... MySQL Replication is used for read scale-out. Writes are send to a MySQL master, reads are send to a MySQL slave. The master sends updates to the slaves in an asynchronous way. Slaves may not have the latest changes. MySQL Replication can also be used for: <ul><li>Backup – perfom blocking backup on slave
  17. 17. High Availablility – for example, remote slaves
  18. 18. Warehousing – OLTP reporting on the slaves </li></ul>
  19. 19. <ul><li>Real-time transactional write scale-out </li></ul><ul><ul><li>99,999% availability
  20. 20. Auto sharding, shared nothing architecture
  21. 21. Web use cases: session server, tele communication </li></ul></ul>MySQL Cluster SQL Node SQL Node Cluster Data Node Cluster Data Node Cluster Data Node
  22. 22. The speaker says... MySQL Cluster is used for real-time transactional read and write scale-out. Data is auto-partitioned on cluster data nodes resulting in 99.999% availability. The cluster is accessed through MySQL Application Cluster Nodes. The most basic MySQL Application Cluster Node is a SQL Node – a MySQL server (mysqld). If MySQL Cluster is new to you, think of MySQL Cluster as a storage engine for the network. Like InnoDB and MyISAM are storage engines for disks.
  23. 23. <ul><li>For example, Partitioning </li></ul><ul><ul><li>Horizontal: rows in different tables
  24. 24. Vertical: columns in additional tables
  25. 25. ... sharding </li></ul></ul>Custom Cluster Shard 1 ID % 2 = 0 Node 3 Cats Node 4 Dogs Shard 2 ID % 2 = 1
  26. 26. The speaker says... Special requirements may make it necessary to build a custom cluster. For example, your application may require a sharding solution.
  27. 27. On using any database cluster Part 2 - Required application changes
  28. 28. <ul><li>Tasks: </li></ul><ul><ul><li>Choose servers to execute statement
  29. 29. Load balance within candidates
  30. 30. Maintain connection pool
  31. 31. Automatic fail over for High Availability </li></ul></ul>Application using any DB cluster MySQL Server 1 MySQL Server 2 MySQL Server n
  32. 32. The speaker says... All PHP applications talking to a cluster of MySQL database servers are facing the same tasks. Replacing a single database with a database cluster means changing the 1:1 relation between the application and the database into a 1:n relation . 1:n puts an additional task on the application: find the right n, find the right server to talk to.
  33. 33. The plugin for all of you MySQL Server 1 MySQL Server 2 MySQL Server n
  34. 34. The speaker says... PECL/mysqlnd_ms is a plugin for the MySQL native driver for PHP (mysqlnd) library . The mysqlnd library is part of PHP as of version 5.3. As of 5.4 mysqlnd is a default. All three PHP MySQL APIs (mysql, mysqli, PDO_MySQL) can be compiled to use the mysqlnd library, thus all existing PHP MySQL application are supported by the plugin. From an applications point of view a mysqlnd plugin can act as a transparent proxy. Depending on plugin an use case, no code changes are needed. A plugin is a drop-in solution.
  35. 35. Plugin focus: MySQL Replication Part 3 - Tasks, solutions and limitations
  36. 36. Using MySQL Replication <ul><li>Statement redirection </li></ul><ul><ul><li>Read/write split (optional: master on write)
  37. 37. Replication filter (table based parititioning)
  38. 38. Custom: callback </li></ul></ul><ul><li>Load Balancing </li></ul><ul><ul><li>Round robin, random, random once
  39. 39. Custom: callback </li></ul></ul><ul><li>Connection Pooling </li></ul><ul><ul><li>Lazy connections
  40. 40. Failover </li></ul></ul>
  41. 41. The speaker says... The plugin is not limited to but optimized for MySQL Replication. The common clustering support feature set focusses on MySQL Replication. For example, statement redirection covers read/write splitting and MySQL Replication filter support. All read accesses are executed on the slave servers, all writes are performed on the master. MySQL Replication supports table based partitioning, replicating selected tables to selected slaves only. The plugin can be made aware of this.
  42. 42. Speaking code /* Connection user handle represents a connection pool */ $mysqli = new mysqli( &quot;hostname&quot; , $user, $pw, $db, $port); /* Read/Write split, Load Balancing */ $res = $mysqli->query(&quot;SELECT 'Slave speaking' FROM DUAL&quot;); var_dump($res->fetch_all(MYSQLI_ASSOC)); { &quot;hostname&quot; :{ &quot;master&quot;:{ &quot;host&quot;:&quot;localhost&quot; }, &quot;slave&quot;:{ &quot;slave_0&quot;:{&quot;host&quot;:&quot;;} }, } }
  43. 43. The speaker says... No code changes for basic use cases. Install the plugin following standard PHP PECL procedures. Create a configuration file. Enable the plugin and set its configuration file in your PHP configuration using mysqlnd_ms.enable=1 and mysqlnd_ms.ini_file=config.ini . In the plugin configuration create a section with the name of the host your application connects to, for example „hostname“. List master and slave servers. Any connection to „hostname“ is now handled by the plugin. Using any API.
  44. 44. Some code changes /* Connection user handle represents a connection pool */ $mysqli = new mysqli(&quot;hostname&quot;, $user, $pw, $db, $port); / * R/W split: no SELECT, run on master */ $mysqli->query(&quot;SET @myrole='Master'); /* R/W split: overruled by SQL hint */ $sql = sprintf(&quot;/*%s*/%s&quot;, MYSQLND_MS_MASTER_SWITCH, &quot;SELECT @myrole AS _role&quot;); $mysqli->query($sql); var_dump($res->fetch_all(MYSQLI_ASSOC));
  45. 45. The speaker says... Non-trivial use cases require support by the application. In other words: existing applications may need updates, new applications need to be designed appropriately. In the example a SQL hint is used to force executio of a SELECT statement on a MySQL Replication master server. There are SQL hints for running a statement on the master, on the slave and on the last used server. Alternative: new APIs call, no win compared to SQL hint.
  46. 46. Why and when code changes $mysqli = new mysqli(&quot;hostname&quot;, $user, $pw, $db, $port); $mysqli->query(&quot;SET @myrole='Master'); $mysqli->query(&quot;SELECT myrole AS _role&quot;); Without plugin With plugin
  47. 47. The speaker says... An application not designed for use with a cluster assumes a connection handle always points to the same server (1:1), having the same state. When using the plugin a connection handle represents a pool of connections (1:n), potentially having different states. Connection switches may occur whenever a statement is executed. After the switch, the connection handle may point to a connection with a different state. Applications need to take care and hint the plugin.
  48. 48. A trouble maker in detail <ul><li>Connection State </li></ul><ul><ul><li>Connection settings, e.g. charsets
  49. 49. Transaction status
  50. 50. Temporary tables
  51. 51. Table locks
  52. 52. Session system variables and user variables
  53. 53. Prepared statements (API and SQL PREPARE)
  54. 54. HANDLER variables
  55. 55. Locks acquired with GET_LOCK
  56. 56. Units of work from an application point of view </li></ul></ul>
  57. 57. The speaker says... No client library, no client proxy can monitor all state changing actions. Applications in need for a certain connection state need to control the plugins connection switches using : <ul><li>SQL hints </li></ul><ul><ul><li>MYSQLND_MS_SLAVE_SWITCH
  59. 59. MYSQLND_MS_LAST_USED_SWITCH </li></ul></ul><ul><li>Callback to replace built-in server selection logic </li></ul>
  60. 60. Trouble maker transaction <ul><li>Unit of work </li></ul><ul><ul><li>Part of the connection state
  61. 61. Application to prevent connection switches </li></ul></ul><ul><li>Workaround </li></ul><ul><ul><li>PHP 5.3: SQL hints must be used
  62. 62. PHP 5.4: API calls monitored, switch auto-disabled </li></ul></ul><ul><li>Replication cluster tip </li></ul><ul><ul><li>Always run on master, only master has latest data!
  63. 63. use trx_stickiness = master </li></ul></ul>
  64. 64. The speaker says... A transaction is part of the connection state. For the duration of a transaction, server switches must be disabled. Transactions can be controlled by SQL statements and API calls. The plugin cannot monitor all of them with reasonable effort. Thus, you must hint the plugin. Disable load balancing by adding SQL hints (MYSQLND_MS_LAST_USED_SWITCH), you install your own load balancer logic (callback). As of PHP 5.4, the plugin monitors API calls to auto-disable switching.
  65. 65. <ul><li>Slave request </li></ul><ul><ul><li>read-only: statement begins with SELECT
  66. 66. read-only: statement begins with slave SQL hint
  67. 67. custom: callback picks slave for statement </li></ul></ul>Redirection: Read/Write split Writes Reads (SELECT, SQL hint) Master Slave Slave Slave
  68. 68. The speaker says... MySQL Replication is for read scale-out. Read-only statements shall go to slaves, writes to the master. The plugin automatically redirects a statement to a slave, if the statement begins with SELECT or the MYSQLND_MS_SLAVE_SWITCH SQL hint. All other statements are considered as write operations and executed on the master. By default SHOW or CALL will be run on the master. It is a design principle of the plugin that all automatic decisions can be overruled. A callback can be installed to pick the server for running a statement on, if needed.
  69. 69. <ul><li>Optimization: use master only after write </li></ul><ul><ul><li>MySQL Replication is asynchronous
  70. 70. MySQL slaves can lag behind
  71. 71. Most applications need current data after write </li></ul></ul>Redirection: Master on write Writes, Reads after write Reads before write Master Slave Slave
  72. 72. The speaker says... The master_on_write configuration option is an optimization of the R/W split. MySQL Replication slaves can lag behind the master because MySQL Replication is asynchronous. When using master_on_write all reads made after a write are executed on the master for the duration of the web request. Applications that perform a write often need to rely on fresh reads afterwards. Think of a web shop creating an order, writing to the master. If reading again from the order table, within the same web request, the application should access the master to make sure the previously written is available.
  73. 73. <ul><li>MySQL Replication filter support </li></ul><ul><ul><li>Slaves replicate different databases/tables </li></ul></ul>Redirection: Partitioning Writes Reads (Filter) DB_Report DB_Shop DB_Report DB_Internal DB_Shop DB_Shop Master Slave Slave Slave
  74. 74. The speaker says... MySQL Replication supports partitioning: slaves can be configured to replicate only selected databases and tables using MySQL Replication filter. Filter help to reduce the amount of internal replication messages. Less frequently accessed data can be replicated to only few slaves. In future versions, plugin can be made aware of the filtering rules for appropriate statement redirection. Every statement is analyzed to find a slave for reading. If no match, the plugin redirects the statement to the master.
  75. 75. <ul><li>Built-in or user-defined through callback </li></ul><ul><ul><li>Round robin: iterate over slaves for each read
  76. 76. Random: pick random slave for each read
  77. 77. Random once: pick random slave for all reads </li></ul></ul>Load Balancing Writes Load Balancing of Reads Master Slave Slave Slave
  78. 78. The speaker says... MySQL Replication is about read scale-out. The plugin helps distributing the load over the read-only slaves. The best and default load balancing stategy is the one with the lowest impact on the connection state: random once. Minimize switches for the life-span of your PHP script. Pick a random slave during startup and use it for all reads. The life-span of PHP is the of a web request. It is short. Two web requests will end up on two random slaves. Load balancing adaptive to server health can be achieved through a user-defined callback.
  79. 79. <ul><li>Caution: connection state change </li></ul><ul><ul><li>Default: failover=disabled, raise exception
  80. 80. Optional: failover=master, automatic </li></ul></ul>Pooling: connection failover Writes Reads Master Slave Slave Slave
  81. 81. The speaker says... Automatic and transparent failover is impossible with stateful connections. The plugin does not and cannot know the full state of a connection. Parts of the state are only known to the server. By default the plugin leaves it to the application to handle a connection failure. Like without the plugin, it is up to the application to decide what to do. Applications using stateless connections can enable automatic failover to the master.
  82. 82. <ul><li>Do not open connection before needed </li></ul><ul><ul><li>Delay open until statement execution
  83. 83. Reduce number of connections, reduce load </li></ul></ul>Pooling: lazy connections Writes Reads Master Slave Slave Slave
  84. 84. The speaker says... By default, to reduce the connection pool size of every PHP web request, connections are not opened before executing a statement. Imagine you have 50 concurrent web requests, you configured four slaves and, you are using random once. If not using lazy connections, the plugin opens 50 x 4 = 200 slave connections but uses at max 50 of them (random once). Every connection occupies resources – ever seen a PHP application opening connections before using? There are many badly written ones...
  85. 85. See also...
  86. 86. Documentation and resources <ul><li> </li></ul><ul><ul><li>Installation, Quickstart, Concepts, Reference </li></ul></ul><ul><li> </li></ul><ul><ul><li>Introduction, Changes, C plugin API </li></ul></ul>
  87. 87. THE END Credits: Andrey Hristov, Johannes Schlüter - Contact: