The document discusses the need for new database cluster-aware APIs in PHP to handle various consistency levels in MySQL replication. It outlines three consistency models: strong, session, and eventual consistency, explaining how global transaction identifiers (GTIDs) play a role in achieving these levels and managing load balancing. It emphasizes the complexities of replication and the necessity for applications to specify their quality-of-service requirements for optimal performance.

1. Ulf Wendel, Oracle
MySQL 5.6
Global Transaction Ids
Use case: Consistency
MySQL 5.6, PECL/mysqlnd_ms 1.3

2. The speaker says...
We need new database cluster-aware APIs in PHP.
Many of us use clusters for availability and performance
reasons. No matter what data storage solution, there is
hardly a one-fits-all solution.
Cluster-wide consistency varies by solution. Some
solutions, such as MySQL Replication, provide
different levels, depending on usage pattern. Our APIs
shall allow requesting any level needed.
Simplified consistency level definitions first before its
explained how Global Transaction Ids (GTIDs) fit in.

3. Strong consistency
All nodes have all the latest copies
All cients see the latest (comitted) copies
Synchronous
SET X = 1 MySQL Node
MySQL Node MySQL Node GET X, X = 1

4. The speaker says...
In this presentation a cluster is considered to deliver
strong consistency if all nodes immediately serve the
latest comitted copies. Clients see each others
changes immediately. The nodes in the cluster are
synchronous. Use, if transactions don't allow for any
„fuzzyness“, for example, for financial or booking
transactions.
As will be shown, strong consistency is possible even in lazy
primary copy systems. It does not always require an eager
read-one, update-any design.

5. Session consistency
Some nodes have the latest copies
A client reads his latest (comitted) copies
Other clients may read old copies
Client A
MySQL Node
SET X = 1
Client A
MySQL Node MySQL Node
GET X, X = 1
Client B
GET X, X = 0

6. The speaker says...
Session consistency ensures that one client will be
able to read all his updates for the duration of a
session. Clients may or may not see comitted
transactions of each other immediately. A load
balancer must pick nodes appropriately.
This relaxed consistency level is good enough to please a
discussion forum user. The user will see his latest
contributions immediately.
It becomes unlikely that users resubmit their posts as they
are displayed immediately. However, other readers may not
see the very latest posts.

7. Eventual consistency
Nodes may or may not serve the latest copies
Nodes may serve stale copies
Nodes may not have copies at all
SET X = 1 MySQL Node
MySQL Node MySQL Node GET X, X = 0
GET X, NULL

8. The speaker says...
An eventual consistent node may or may not serve
the latest copy. In fact, there is no promise that a
particular copy is available from every node. Many
systems that default to eventual consistency reach strong
consistency over time. Eventually all nodes get
synchronized. This model is similar to that of a cache.
Eventual consistency is good enoug for browsing product
catalogs or other infrequently changing contents in areas
where stale data is acceptable. It is the default consistency
level with MySQL Replication. Whatever, why bother ?! Can't
vendors provide proper solutions?

9. Are vendors fooling you?
There is no one-fits-all replication solution
The dangers of replication and a solution (Jim Gray, Pat
Helland, 1996): a ten-fold increase in nodes and traffic
gives a thousand fold increase in deadlocks or
reconciliations
CAP theorem (Eric Brewer, 2000): consistency,
availability and paritition tolerance can't be achieved
together
… vendors are forced to offer compromises
… vendors are faced with a highly complex problem
… you are forced to know about consistency :-(

10. The speaker says...
I would love to stop talking about consistency. But
replication has limits. Keep on pushing but know the
limits. Every synchronous, update anywhere solution has a
scalability limit. Partitioning (sharding) only shifts the limits
and creates other issues (rebuilding aggregates). Until its
solved...
Applications shall state their QoS/consistency needs:
To allow vendors to relax consistency for performance
reasons, but only if feasible, if allowed by the app
To allow load balancers to make an optimal node choice
= PECL/mysqlnd_ms 1.2+ build around GTIDs...

11. PECL/mysqlnd_ms 1.2+ API
Tell the plugin/load balancer what you need!
/* Yes, works with PDO and mysql as well. */
$mysqli = new mysqli("myapp", "username", "password", "database");
/* read-write splitting: master used */
$mysqli->query("INSERT INTO orders(item) VALUES ('spring flowers')");
/* Request session consistency: read your writes */
mysqlnd_ms_set_qos($mysqli, MYSQLND_MS_QOS_CONSISTENCY_SESSION);
/* Plugin picks a node which has the changes, here: master */
$res = $mysqli->query("SELECT item FROM orders WHERE order_id = 1");
var_dump($res->fetch_assoc());
/* Back to eventual consistency: stale data allowed */
mysqlnd_ms_set_qos($mysqli, MYSQLND_MS_QOS_CONSISTENCY_EVENTUAL);
/* Plugin picks any slave, stale data is allowed */
$res = $mysqli->query("SELECT item, price FROM specials");

12. The speaker says...
Because PECL/mysqlnd_ms is a load balancer at the driver
level, it can be controlled not only through SQL hints but
also provide API calls. mysqlnd_ms_set_qos() defines
the quality of service (QoS) that shall be provided. It
instructs the load balancer to select only MySQL
database cluster nodes that cen deliver the
requested quality of service, for example, with regards
to consistency.
Without GTIDs the rules for a MySQL Replication
cluster are simple: eventual consistency – any slave,
session and strong consistency – master only.

13. Global transaction identifier
Combination of server id and sequence number
Emulation: PECL/mysqlnd_ms 1.2, MySQL Proxy
Built-in: MySQL 5.6
MySQL Master
Log 7, Pos 34, GTID M:1: UPDATE x=1
Log 7, Pos 35, GTID M:2: UPDATE x=9
MySQL Slave 1 MySQL Slave 2
… , GTID M:1: UPDATE x=1 … , GTID M:1: UPDATE x=1
… , GTID M:2: UPDATE x=9

14. The speaker says...
A global transaction identifier is a cluster-wide
unique transaction identifier. MySQL 5.6 can generate it
automatically. MySQL Proxy and PECL/mysqlnd_ms 1.2
feature client-side emulations for use with any MySQL
version. SQL can be used to access the GTIDs.
GTIDs have been been created to make MySQL Replication
failover easier. However, they are useful for load balancing
as well in a primary copy system.

15. MySQL Replication: strong con.
With or without GTID: primary only
Only the primary has all comitted transactions
Client A Client B
MySQL Master
SET X = 1 GET X, X = 1
MySQL Slave MySQL Slave

16. The speaker says...
Configuring PECL/mysqlnd_ms for use with a MySQL
Replication cluster and calling mysqlnd_ms_set_qos($conn,
MYSQLND_MS_QOS_CONSISTENCY_STRONG) instructs
PECL/mysqlnd_ms to use only the MySQL Replication
master server for requests.
In a lazy primary copy system there is only one node that is
guaranteed to have all comitted updates: the primary. Note
that its possible to achieve higher consistency levels than
eventual consistency in an lazy primary copy system by
appropriately choosing nodes.

17. MySQL Replication: session con.
Use GTID to find „synchronous“ slave
Check slave status using SQL
Reduce read load on master
SET X = 9 MySQL Master
..., GTID M:1: UPDATE x=1
..., GTID M:2: UPDATE x=9
GET X, X = 9 MySQL Slave 1 MySQL Slave 2
… , GTID M:1: UPDATE x=1 … , GTID M:1: UPDATE x=1
… , GTID M:2: UPDATE x=9

18. The speaker says...
Global transaction identifier help to find „up-to-date“
slaves that have already replicated the latest
updates of a client. Thus, session consistency can
now be achieved by reading from the master and
selected „up-to-date“ slaves.
This works with the GTID emulation of PECL/mysqlnd_ms
1.2 and any MySQL server version as well as with
PECL/mysqlnd_ms 1.3 (not yet released) and MySQL 5.6
with its built-in GTIDs.
Remember: only one API call for your PHP application...

19. MySQL Replication: eventual c.
With or without GTID: all slaves
Optional QoS level: upper slave lag limit
MySQL estimates slave lag!
SET X = 9 MySQL Master
GET X, X = 8 MySQL Slave 1 MySQL Slave 2
Slave lag = 1 second Slave lag = 7 seconds

20. The speaker says...
A MySQL Replication slave is eventual consistent – it
may or may not have the latest updates. There is no
need to filter nodes with regards to consistency.
However, slaves can be filtered by replication lag:
mysqlnd_ms_set_qos($conn,
MYSQLND_MS_QOS_CONSISTENCY_EVENTUAL,
MYSQLND_MS_QOS_OPTION_AGE, 5) filters out all slaves
that report an estimated lag of more than five seconds.

21. Slave selection logic
Same logic whenever slaves are to be filtered
applied for: session consistency + GTID
applied for: eventual consistency + Lag
Stage 1: send SQL to check status to all slaves
Stage 2: fetch replies in order
Stage 3: apply filter logic
SQL is documented in the manual

22. The speaker says...
Whenever PECL/mysqlnd_ms has to check the slave status
for filtering slaves, the check is done in two stages. First, all
slaves are queried. Then, replies are fetched from the slaves
in order. Usually, many requests can be send out in stage
one before the servers reply.
The SQL details are documented at php.net/mysqlnd_ms.

23. Stateless and shared-nothing
PECL/mysqlnd_ms is a PHP solution
Stateless: decisions are not „remembered“
Shared-nothing: instances don't communicate
Optional: user hooks to make statefull decisions

24. The speaker says...
Please recall, that we are talking about a PHP integrated
solution. By default PHP is stateless and promotes a shared-
nothing architecture.
PHP and PECL/mysqlnd_ms loose their state at the end of a
web request. State is neither persisted nor shared between
different processes. Thus, there is no single point of failure.
If you want PECL/mysqlnd_ms to remember decisions,
install user hooks and persist their decisions.

25. THE END
Contact: ulf.wendel@oracle.com