Your SlideShare is downloading. ×
0
Read / Write Splitting for MySQL and PHP    Bill Karwin, Percona Inc.
It’s About Scalability                     www.percona.com
The Problem: Scaling• One database server can handle a finite number of   queries per second (QPS).                       ...
The Strategy       More database servers!• Increase QPS capacity by spreading the load over    multiple instances.• Provid...
MySQL Replicationmaster          application writes changes                       to the master    slave             chang...
The Difficulty                 Slave lag!• Replication is asynchronous.• Changes on the master aren’t logged until commit....
A Mitigation Strategy Semi-synchronous replication.• Commit on the master waits for a semi-sync slave.• Semi-sync slaves m...
Fact of Life        Slave lag happens.• Even when using semi-synchronous replication.                                     ...
Resources on Replication• MySQL Manual:  • http://dev.mysql.com/doc/refman/5.5/en/replication.html• High Performance MySQL...
Degrees of Tolerance                  www.percona.com
Querying the Slave• Does your application need the slave’s data to be   absolutely current?  • Yes!  • No.  • Sometimes. D...
Degrees of ToleranceStrict       Time    Event        Lax                             www.percona.com
Strict Tolerance         • Must always query from the master.Strict         • Example: reading and writing data in        ...
Strict Tolerance         • Must always query from the master.Strict         • If query must always view current data,     ...
Time-Based Tolerance        • Slave must include recent changes           made by any session.Time        • Lag must be le...
Time-Sensitive• Master                         current log position                          on master = 3000• Slave      ...
Detecting Lag• Before reading on the slave, check its lag:mysql> show slave statusG. . .Seconds_Behind_Master: 5. . .     ...
How Does That Perform?mysql> show profiles;+----------+------------+-------------------+| Query_ID | Duration   | Query   ...
Event-Based Tolerance         • Data must include recent changes            made by the same session.Event         • Examp...
Event-Sensitive• Master binary log          change committed                          1234                                ...
Detecting Events• After executing a change on the master, note the   binlog position:mysql> show master statusG           ...
How Does That Perform?mysql> show profiles;+----------+------------+---------------------+| Query_ID | Duration   | Query ...
Detecting Events• Before reading on the slave, check its execution   position:mysql> show slave statusG. . .    Master_Log...
How Does That Perform?mysql> show profiles;+----------+------------+-------------------+| Query_ID | Duration   | Query   ...
Lax Tolerance      • Queries may read freely from any         slave without checking lag.Lax      • Reading data that does...
Which Method Should I Use?            All of them.• In a typical application, you have a variety of queries.• You know you...
Proof of Concept                   www.percona.com
Database Access LibraryRequirements: 1.Manage connections to a master MySQL instance    and multiple slave instances. 2.Pr...
MasterSlaveConnection• Implemented in the Doctrine Project.    namespace DoctrineDBALConnections;    class MasterSlaveConn...
Extending the Wrappernamespace DoctrineDBALConnections;class BaseMasterSlaveConnection extends  MasterSlaveConnection { .....
Choose a Subclass$wrapperClasses = array("strict" => "DoctrineDBALConnectionsStrictMasterSlaveConnection","time"    => "Do...
Declare Multiple Connections$conn_args = array(     "wrapperClass" => $wrapperClasses[$mode],     "driver" => "pdo_mysql",...
Set the Tolerance$conn = DriverManager::getConnection($conn_args);if ($mode == "time") {    $conn->setLagTolerance($lag);}...
And Connect!• You may request a slave, but if no slaves meet the   tolerance conditions, it will transparently return a   ...
Query the Slave• Only a limited set of functions will query the slave,   even if you got a slave instance: $conn->executeQ...
Query the Master• All other functions implicitly switch the connection   back to the master instance: $conn->query(...); $...
Demo Time            www.percona.com
Caveats• You as the application developer decide on which   tolerance to use, on a case-by-case.• Beware the risk of overl...
mysqlnd_ms plugin• Maps read queries (SELECT) across a configured   set of MySQL master and slaves.• Plugin supports round...
Future Todo• Commit my proof of concept Doctrine code to   Github, as a branch of Doctrine/DBAL.• Experiment with mysqlns_...
Percona Training for MySQL    Senior Industry Experts In-Person and Online Classes    Custom Onsite Training     http://pe...
Santa Clara, April 22-25, 2013                 www.percona.com/live
SQL Antipatternshttp://www.pragprog.com/titles/bksqla/                                www.percona.com
License and Copyright            Copyright 2012, 2013 Bill Karwin             www.slideshare.net/billkarwin               ...
Upcoming SlideShare
Loading in...5
×

Read / Write Splitting with MySQL and PHP

16,500

Published on

Published in: Technology
2 Comments
21 Likes
Statistics
Notes
No Downloads
Views
Total Views
16,500
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
174
Comments
2
Likes
21
Embeds 0
No embeds

No notes for slide

Transcript of "Read / Write Splitting with MySQL and PHP"

  1. 1. Read / Write Splitting for MySQL and PHP Bill Karwin, Percona Inc.
  2. 2. It’s About Scalability www.percona.com
  3. 3. The Problem: Scaling• One database server can handle a finite number of queries per second (QPS). www.percona.com
  4. 4. The Strategy More database servers!• Increase QPS capacity by spreading the load over multiple instances.• Provided the instances have identical data. www.percona.com
  5. 5. MySQL Replicationmaster application writes changes to the master slave changes propagate to slaves slave ...eventually slave www.percona.com
  6. 6. The Difficulty Slave lag!• Replication is asynchronous.• Changes on the master aren’t logged until commit.• Then slave downloads binary log.• Then slave executes changes. www.percona.com
  7. 7. A Mitigation Strategy Semi-synchronous replication.• Commit on the master waits for a semi-sync slave.• Semi-sync slaves must receive the binary logs and flush them to disk as relay logs.• But the master does not wait for slave to execute the changes. www.percona.com
  8. 8. Fact of Life Slave lag happens.• Even when using semi-synchronous replication. www.percona.com
  9. 9. Resources on Replication• MySQL Manual: • http://dev.mysql.com/doc/refman/5.5/en/replication.html• High Performance MySQL, 3rd Edition • http://www.amazon.com/High-Performance-MySQL- Optimization-Replication/dp/1449314287/• Effective MySQL Replication Techniques in Depth • http://www.amazon.com/Effective-MySQL- Replication-Techniques-Depth/dp/0071791868/ www.percona.com
  10. 10. Degrees of Tolerance www.percona.com
  11. 11. Querying the Slave• Does your application need the slave’s data to be absolutely current? • Yes! • No. • Sometimes. Depends. www.percona.com
  12. 12. Degrees of ToleranceStrict Time Event Lax www.percona.com
  13. 13. Strict Tolerance • Must always query from the master.Strict • Example: reading and writing data in the same transaction—or even the same statement. • INSERT INTO foo SELECT ... • Data isn’t written to binlog until COMMIT; the slave never sees uncommitted data. www.percona.com
  14. 14. Strict Tolerance • Must always query from the master.Strict • If query must always view current data, with no possibility of lag. • Example: bids in an auction. www.percona.com
  15. 15. Time-Based Tolerance • Slave must include recent changes made by any session.Time • Lag must be less than a specific number of seconds. • Example: chat-room messages. • Check slave lag; if it’s too great, then queries must read from the master. www.percona.com
  16. 16. Time-Sensitive• Master current log position on master = 3000• Slave slave is lagging 5+ seconds slave has executed only up to log position = 2000 www.percona.com
  17. 17. Detecting Lag• Before reading on the slave, check its lag:mysql> show slave statusG. . .Seconds_Behind_Master: 5. . . www.percona.com
  18. 18. How Does That Perform?mysql> show profiles;+----------+------------+-------------------+| Query_ID | Duration | Query |+----------+------------+-------------------+| 1 | 0.00028875 | show slave status || 2 | 0.00032550 | show slave status || 3 | 0.00042400 | show slave status || 4 | 0.00029000 | show slave status || 5 | 0.00026125 | show slave status || 6 | 0.00028900 | show slave status |+----------+------------+-------------------+ about 2-4 ten-thousandths of a second www.percona.com
  19. 19. Event-Based Tolerance • Data must include recent changes made by the same session.Event • Example: INSERT a line item to a user’s shopping cart, then SELECT contents of shopping cart. • Check slave binlog execution position; it must be caught up at least to a specific change, or else queries must read from the master. www.percona.com
  20. 20. Event-Sensitive• Master binary log change committed 1234 current log position on master = 3000• Slave relay log slave is lagging 5+ seconds 1234 slave has executed only up to log position = 2000 OK to read because the slave has executed the relevant change www.percona.com
  21. 21. Detecting Events• After executing a change on the master, note the binlog position:mysql> show master statusG File: mysql-bin.000004 Position: 18485• Save this information in $_SESSION. www.percona.com
  22. 22. How Does That Perform?mysql> show profiles;+----------+------------+---------------------+| Query_ID | Duration | Query |+----------+------------+---------------------+| 1 | 0.00010375 | show master status || 2 | 0.00010425 | show master status || 3 | 0.00010075 | show master status || 4 | 0.00010375 | show master status || 5 | 0.00012975 | show master status || 6 | 0.00010400 | show master status |+----------+------------+---------------------+ about 1 ten-thousandth of a second www.percona.com
  23. 23. Detecting Events• Before reading on the slave, check its execution position:mysql> show slave statusG. . . Master_Log_File: mysql-bin.000004Exec_Master_Log_Pos: 18485. . .• Compare these to the information you saved in $_SESSION previously. www.percona.com
  24. 24. How Does That Perform?mysql> show profiles;+----------+------------+-------------------+| Query_ID | Duration | Query |+----------+------------+-------------------+| 1 | 0.00028875 | show slave status || 2 | 0.00032550 | show slave status || 3 | 0.00042400 | show slave status || 4 | 0.00029000 | show slave status || 5 | 0.00026125 | show slave status || 6 | 0.00028900 | show slave status |+----------+------------+-------------------+ about 2-4 ten-thousandths of a second www.percona.com
  25. 25. Lax Tolerance • Queries may read freely from any slave without checking lag.Lax • Reading data that doesn’t change. • Example: lookup tables. • Reading old data is okay. • Example: count of “likes” (so far). www.percona.com
  26. 26. Which Method Should I Use? All of them.• In a typical application, you have a variety of queries.• You know your application, and how tolerant each query can be, on a case-by-case basis.• Use a data access library that makes it easy to choose any methods. www.percona.com
  27. 27. Proof of Concept www.percona.com
  28. 28. Database Access LibraryRequirements: 1.Manage connections to a master MySQL instance and multiple slave instances. 2.Provide an API to send read queries to a slave. 3.Choose one slave randomly. 4.Detect a lagging slave and automatically switch to another slave, or back to the master. 5.Detect an offline slave, and switch to another slave, or back to the master. www.percona.com
  29. 29. MasterSlaveConnection• Implemented in the Doctrine Project. namespace DoctrineDBALConnections; class MasterSlaveConnection extends Connection { ... }• Custom wrapperClass: • http://docs.doctrine-project.org/projects/doctrine-dbal/en/latest/reference/ configuration.html#wrapper-class• Satisfies requirements 1 - 3, but not 4 - 5. www.percona.com
  30. 30. Extending the Wrappernamespace DoctrineDBALConnections;class BaseMasterSlaveConnection extends MasterSlaveConnection { ... }class StrictMasterSlaveConnection extends BaseMasterSlaveConnection { ... }class TimeMasterSlaveConnection extends BaseMasterSlaveConnection { ... }class EventMasterSlaveConnection extends BaseMasterSlaveConnection { ... }class LaxMasterSlaveConnection extends BaseMasterSlaveConnection { ... } www.percona.com
  31. 31. Choose a Subclass$wrapperClasses = array("strict" => "DoctrineDBALConnectionsStrictMasterSlaveConnection","time" => "DoctrineDBALConnectionsTimeMasterSlaveConnection","event" => "DoctrineDBALConnectionsEventMasterSlaveConnection","lax" => "DoctrineDBALConnectionsLaxMasterSlaveConnection",);$wrapperClass = $wrapperClasses[ $mode ]; www.percona.com
  32. 32. Declare Multiple Connections$conn_args = array( "wrapperClass" => $wrapperClasses[$mode], "driver" => "pdo_mysql", "master" => array( ! ! "user" => "app", "password" => "xyzzy", ! ! "host" => "huey", "dbname" => "test"), "slaves" => array( "dewey" => array( ! ! ! "user" => "app", "password" => "xyzzy", ! ! ! "host" => "dewey", "dbname" => "test"), "louie" => array( ! ! ! "user" => "app", "password" => "xyzzy", ! ! ! "host" => "louie", "dbname" => "test"), "daisy" => array( ! ! ! "user" => "app", "password" => "xyzzy", ! ! ! "host" => "daisy", "dbname" => "test"), )); www.percona.com
  33. 33. Set the Tolerance$conn = DriverManager::getConnection($conn_args);if ($mode == "time") { $conn->setLagTolerance($lag);}if ($mode == "event") { $conn->setTargetBinlog( ! ! $target_binlog_file, ! ! $target_binlog_pos);} www.percona.com
  34. 34. And Connect!• You may request a slave, but if no slaves meet the tolerance conditions, it will transparently return a connection to the master anyway. $ret = $conn->connect("slave"); this is the literal word “slave” not one of the names of my slaves www.percona.com
  35. 35. Query the Slave• Only a limited set of functions will query the slave, even if you got a slave instance: $conn->executeQuery("SELECT ...", ...); $conn->executeCacheQuery("SELECT ...", ...); www.percona.com
  36. 36. Query the Master• All other functions implicitly switch the connection back to the master instance: $conn->query(...); $conn->executeUpdate(...); $conn->delete(...); $conn->update(...); $conn->insert(...); $conn->prepare(...); $conn->exec(); $conn->beginTransaction(); $conn->commit(); $conn->rollBack(); $conn->createSavePoint(...); $conn->releaseSavePoint(...); $conn->rollbackSavePoint(...); www.percona.com
  37. 37. Demo Time www.percona.com
  38. 38. Caveats• You as the application developer decide on which tolerance to use, on a case-by-case.• Beware the risk of overloading the master as traffic automatically moves off of slaves. www.percona.com
  39. 39. mysqlnd_ms plugin• Maps read queries (SELECT) across a configured set of MySQL master and slaves.• Plugin supports round-robin, random, and even a user-defined callback to choose an instance.• Download from PECL and build.• Requires PHP 5.3.6+ (RHEL/CentOS is on 5.3.3).• How to choose a tolerance level dynamically? www.percona.com
  40. 40. Future Todo• Commit my proof of concept Doctrine code to Github, as a branch of Doctrine/DBAL.• Experiment with mysqlns_ms plugin. • http://dev.mysql.com/doc/refman/5.5/en/apis-php-book.mysqlnd-ms.html• Unit testing, performance testing, load testing. www.percona.com
  41. 41. Percona Training for MySQL Senior Industry Experts In-Person and Online Classes Custom Onsite Training http://percona.com/training www.percona.com
  42. 42. Santa Clara, April 22-25, 2013 www.percona.com/live
  43. 43. SQL Antipatternshttp://www.pragprog.com/titles/bksqla/ www.percona.com
  44. 44. License and Copyright Copyright 2012, 2013 Bill Karwin www.slideshare.net/billkarwin Released under a Creative Commons 3.0 License: http://creativecommons.org/licenses/by-nc-nd/3.0/ You are free to share - to copy, distribute and transmit this work, under the following conditions: Attribution. Noncommercial. No Derivative Works.You must attribute this work You may not use this work You may not alter, transform, to Bill Karwin. for commercial purposes. or build upon this work. www.percona.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×