MySQL 5.6 Global Transaction IDs - Use case: (session) consistency

Ulf Wendel, Oracle MySQL 5.6Global Transaction IdsUse case: Consistency MySQL 5.6, PECL/mysqlnd_ms 1.3

The speaker says...We need new database cluster-aware APIs in PHP.Many of us use clusters for availability and performancereasons. No matter what data storage solution, there ishardly a one-fits-all solution.Cluster-wide consistency varies by solution. Somesolutions, such as MySQL Replication, providedifferent levels, depending on usage pattern. Our APIsshall allow requesting any level needed.Simplified consistency level definitions first before itsexplained how Global Transaction Ids (GTIDs) fit in.

Strong consistencyAll nodes have all the latest copies All cients see the latest (comitted) copies SynchronousSET X = 1 MySQL Node MySQL Node MySQL Node GET X, X = 1

The speaker says...In this presentation a cluster is considered to deliverstrong consistency if all nodes immediately serve thelatest comitted copies. Clients see each otherschanges immediately. The nodes in the cluster aresynchronous. Use, if transactions dont allow for any„fuzzyness“, for example, for financial or bookingtransactions.As will be shown, strong consistency is possible even in lazyprimary copy systems. It does not always require an eagerread-one, update-any design.

Session consistencySome nodes have the latest copies A client reads his latest (comitted) copies Other clients may read old copies Client A MySQL NodeSET X = 1 Client A MySQL Node MySQL Node GET X, X = 1 Client B GET X, X = 0

The speaker says...Session consistency ensures that one client will beable to read all his updates for the duration of asession. Clients may or may not see comittedtransactions of each other immediately. A loadbalancer must pick nodes appropriately.This relaxed consistency level is good enough to please adiscussion forum user. The user will see his latestcontributions immediately.It becomes unlikely that users resubmit their posts as theyare displayed immediately. However, other readers may notsee the very latest posts.

Eventual consistencyNodes may or may not serve the latest copies Nodes may serve stale copies Nodes may not have copies at allSET X = 1 MySQL Node MySQL Node MySQL Node GET X, X = 0 GET X, NULL

The speaker says...An eventual consistent node may or may not servethe latest copy. In fact, there is no promise that aparticular copy is available from every node. Manysystems that default to eventual consistency reach strongconsistency over time. Eventually all nodes getsynchronized. This model is similar to that of a cache.Eventual consistency is good enoug for browsing productcatalogs or other infrequently changing contents in areaswhere stale data is acceptable. It is the default consistencylevel with MySQL Replication. Whatever, why bother ?! Cantvendors provide proper solutions?

Are vendors fooling you?There is no one-fits-all replication solution The dangers of replication and a solution (Jim Gray, Pat Helland, 1996): a ten-fold increase in nodes and traffic gives a thousand fold increase in deadlocks or reconciliations CAP theorem (Eric Brewer, 2000): consistency, availability and paritition tolerance cant be achieved together … vendors are forced to offer compromises … vendors are faced with a highly complex problem … you are forced to know about consistency :-(

The speaker says...I would love to stop talking about consistency. Butreplication has limits. Keep on pushing but know thelimits. Every synchronous, update anywhere solution has ascalability limit. Partitioning (sharding) only shifts the limitsand creates other issues (rebuilding aggregates). Until itssolved...Applications shall state their QoS/consistency needs:To allow vendors to relax consistency for performancereasons, but only if feasible, if allowed by the appTo allow load balancers to make an optimal node choice= PECL/mysqlnd_ms 1.2+ build around GTIDs...

PECL/mysqlnd_ms 1.2+ APITell the plugin/load balancer what you need!/* Yes, works with PDO and mysql as well. */$mysqli = new mysqli("myapp", "username", "password", "database");/* read-write splitting: master used */$mysqli->query("INSERT INTO orders(item) VALUES (spring flowers)");/* Request session consistency: read your writes */mysqlnd_ms_set_qos($mysqli, MYSQLND_MS_QOS_CONSISTENCY_SESSION);/* Plugin picks a node which has the changes, here: master */$res = $mysqli->query("SELECT item FROM orders WHERE order_id = 1");var_dump($res->fetch_assoc());/* Back to eventual consistency: stale data allowed */mysqlnd_ms_set_qos($mysqli, MYSQLND_MS_QOS_CONSISTENCY_EVENTUAL);/* Plugin picks any slave, stale data is allowed */$res = $mysqli->query("SELECT item, price FROM specials");

The speaker says...Because PECL/mysqlnd_ms is a load balancer at the driverlevel, it can be controlled not only through SQL hints butalso provide API calls. mysqlnd_ms_set_qos() definesthe quality of service (QoS) that shall be provided. Itinstructs the load balancer to select only MySQLdatabase cluster nodes that cen deliver therequested quality of service, for example, with regardsto consistency.Without GTIDs the rules for a MySQL Replicationcluster are simple: eventual consistency – any slave,session and strong consistency – master only.

Global transaction identifierCombination of server id and sequence number Emulation: PECL/mysqlnd_ms 1.2, MySQL Proxy Built-in: MySQL 5.6 MySQL Master Log 7, Pos 34, GTID M:1: UPDATE x=1 Log 7, Pos 35, GTID M:2: UPDATE x=9 MySQL Slave 1 MySQL Slave 2 … , GTID M:1: UPDATE x=1 … , GTID M:1: UPDATE x=1 … , GTID M:2: UPDATE x=9

The speaker says...A global transaction identifier is a cluster-wideunique transaction identifier. MySQL 5.6 can generate itautomatically. MySQL Proxy and PECL/mysqlnd_ms 1.2feature client-side emulations for use with any MySQLversion. SQL can be used to access the GTIDs.GTIDs have been been created to make MySQL Replicationfailover easier. However, they are useful for load balancingas well in a primary copy system.

MySQL Replication: strong con.With or without GTID: primary only Only the primary has all comitted transactions Client A Client B MySQL MasterSET X = 1 GET X, X = 1 MySQL Slave MySQL Slave

The speaker says...Configuring PECL/mysqlnd_ms for use with a MySQLReplication cluster and calling mysqlnd_ms_set_qos($conn,MYSQLND_MS_QOS_CONSISTENCY_STRONG) instructsPECL/mysqlnd_ms to use only the MySQL Replicationmaster server for requests.In a lazy primary copy system there is only one node that isguaranteed to have all comitted updates: the primary. Notethat its possible to achieve higher consistency levels thaneventual consistency in an lazy primary copy system byappropriately choosing nodes.

MySQL Replication: session con. Use GTID to find „synchronous“ slave Check slave status using SQL Reduce read load on master SET X = 9 MySQL Master ..., GTID M:1: UPDATE x=1 ..., GTID M:2: UPDATE x=9GET X, X = 9 MySQL Slave 1 MySQL Slave 2 … , GTID M:1: UPDATE x=1 … , GTID M:1: UPDATE x=1 … , GTID M:2: UPDATE x=9

The speaker says...Global transaction identifier help to find „up-to-date“slaves that have already replicated the latestupdates of a client. Thus, session consistency cannow be achieved by reading from the master andselected „up-to-date“ slaves.This works with the GTID emulation of PECL/mysqlnd_ms1.2 and any MySQL server version as well as withPECL/mysqlnd_ms 1.3 (not yet released) and MySQL 5.6with its built-in GTIDs.Remember: only one API call for your PHP application...

MySQL Replication: eventual c.With or without GTID: all slaves Optional QoS level: upper slave lag limit MySQL estimates slave lag! SET X = 9 MySQL MasterGET X, X = 8 MySQL Slave 1 MySQL Slave 2 Slave lag = 1 second Slave lag = 7 seconds

The speaker says...A MySQL Replication slave is eventual consistent – itmay or may not have the latest updates. There is noneed to filter nodes with regards to consistency.However, slaves can be filtered by replication lag:mysqlnd_ms_set_qos($conn,MYSQLND_MS_QOS_CONSISTENCY_EVENTUAL,MYSQLND_MS_QOS_OPTION_AGE, 5) filters out all slavesthat report an estimated lag of more than five seconds.

Slave selection logicSame logic whenever slaves are to be filtered applied for: session consistency + GTID applied for: eventual consistency + Lag Stage 1: send SQL to check status to all slaves Stage 2: fetch replies in order Stage 3: apply filter logic SQL is documented in the manual

The speaker says...Whenever PECL/mysqlnd_ms has to check the slave statusfor filtering slaves, the check is done in two stages. First, allslaves are queried. Then, replies are fetched from the slavesin order. Usually, many requests can be send out in stageone before the servers reply.The SQL details are documented at php.net/mysqlnd_ms.

Stateless and shared-nothingPECL/mysqlnd_ms is a PHP solution Stateless: decisions are not „remembered“ Shared-nothing: instances dont communicate Optional: user hooks to make statefull decisions

The speaker says...Please recall, that we are talking about a PHP integratedsolution. By default PHP is stateless and promotes a shared-nothing architecture.PHP and PECL/mysqlnd_ms loose their state at the end of aweb request. State is neither persisted nor shared betweendifferent processes. Thus, there is no single point of failure.If you want PECL/mysqlnd_ms to remember decisions,install user hooks and persist their decisions.

THE ENDContact: ulf.wendel@oracle.com

http://blog.ulf-wendel.de	3356
http://www.planet-php.net	1014
http://asyncionews.com	399
http://planet-php.net	175
http://planet-php.org	105
http://www.planet-php.org	98
http://php5.planet-php.org	10
http://www.newsblur.com	8
http://www.hanrss.com	5
http://translate.googleusercontent.com	4
http://blog.planet-php.org	3
http://www.btekd.com	3
http://php.oss-tohoku.org	2
https://si0.twimg.com	2
http://wall.bit-tests.1and1.com	1
http://207.46.192.232	1
http://www.rssfindnews.com	1
http://127.0.0.1	1
http://www.blog.ulf-wendel.de	1
http://blog.ulfwendel.de	1

MySQL 5.6 Global Transaction IDs - Use case: (session) consistency

by Ulf Wendel

Accessibility

Categories

Upload Details

Usage Rights

20 Embeds 5,190

Statistics

MySQL 5.6 Global Transaction IDs - Use case: (session) consistency Presentation Transcript