{
MySQL mysqlnd PHP driver
Serge Frezefond /
Twitter : @sfrezefond

FORUM PHP
PARIS 2013
{

MySQL User Survey: Top Languages

FORUM PHP
PARIS 2013
Application

{

MySQL Connectors

…
LISP

FORUM PHP
PARIS 2013

MySQL Server

Connector/Python*
Connector/Java
Connector/P...
{

MariaDB Connectors

LGPL connectors(non contaminating) :
•  MariaDB Client Library for Java Applications
•  MariaDB Cli...
{

MySQL Protocol
Transport Layer
•  Unix Domain Socket

– Local communication on Linux /tmp/mysql.sock
•  Windows Named P...
{

MySQL Protocol
Compression
•  MySQL compresses each packet individually

– Each returned row is (usually) one packet
• ...
{

MySQL Protocol
Encryption
•  Encryption is being enabled during the initial

handshake
•  All subsequent communication ...
{

MySQL Protocol
Character Sets
•  MySQL has a rich support for character set and

encodings
•  Character set can be chan...
{

MySQL Protocol
Support for Prepared Statements
Server

Client
SELECT foo
FROM bar
WHERE id
=?
Handle
Handle
Param 1: 42...
{

Buffered and Unbuffered Result
Sets

•  Buffered Results

– Copies the complete
result set to the client
– Minimal memo...
{

PHP mysqlnd driver

•  Under the PHP license. no licensing issues that

have been problematic in the past.
•  MySQL PHP...
{

PHP and mysqlnd
PHP
ext/mysql

mysqli

PDO_mysql

mysqlnd – MySQL native driver for PHP
PHP Memory

Infrastructure

IO:...
{

libmysql vs. mysqlnd

PHP Memory

mysqlnd

libmysql

PHP Memory

libmysql Memory

MySQL Server

MySQL Server

copy

PHP...
{

Mysqli API
Key mysqli extension benefits :
l 
Object-oriented interface
l 
Support for Prepared Statements
l 
Suppor...
{

Mysqlnd
easy Install / few dependancies
PHP 5 ChangeLog 5.4 : ext/mysql, mysqli and
pdo_mysql now use mysqlnd by defaul...
{

Mysqlnd unique features

Special features compare to MySQL Client
Library :
»  Improved persistent connections
»  The s...
{

Mysqlnd
supported features
SSL Support
MySQL Native Driver supports SSL since PHP 5.3.3
Compressed Protocol Support
As ...
{

Mysql asynchronous query
only with mysqlnd
$link1->query("SELECT 'test'",
MYSQLI_ASYNC);
...
if (!mysqli_poll($links, $...
{

client per-process statistics
only with mysqlnd

• Around 150 statistic values collected
–  mysqli_get_client_stats ()
...
{

Mysqlnd
Extensible with plugins

mysql_query()

mysqli::query()

PDO::query()

mysqlnd Query
Plugin Hook
Wire Protocol
...
{

mysqlnd Plugins
Client Proxy
– Load Balancing
•  Read / Write splitting
•  Failover
•  Sharding suport ( Fabric framewo...
{

mysqlnd Plugins through
PECL/PHP Extension Community Library

• mysqlnd_ms Replication /load balancing plugin
• mysqlnd...
{

mysqlnd Query Cache
PHP
mysql / mysqli / PDO_mysql

mysqlnd
Query Cache

Cache Backend
Local Memory, APC,
Memcache, Cus...
{

mysqlnd Query Cache
Key Properties
• Transparent
– PHP Extension hooking into mysqlnd
• Works with ext/mysql, mysqli, p...
{

mysqlnd Query Cache
Transparent ?

$mysqli = new mysqli($host, $user, $pw, $db);
$sql = sprintf(“/*%s*/SELECT SLEEP(10)...
{

mysqlnd Query Cache
User-Defined Storage Handler

public function is_select($query)
{ if (preg_match("@from employees w...
{

myslqnd_ms
• mysqlnd replication and load balancing plugin
(mysqlnd_ms = “mysqlnd master/slave
splitting”)

FORUM PHP
P...
{

mysqlnd_ms
Application

mysql_query()

mysqli::query()

PDO::query()

mysqlnd_ms

Master

Slave(s)
FORUM PHP
PARIS 2013
{

mysqlnd_ms Configuration
{
"myapp": {
"master": {
"master_0": {
"host": "localhost",
"socket": "/tmp/mysql.sock"
}
},
"...
{

mysqlnd_ms: Connecting

$mysqli = new MySQLi(“myapp”, “user”, “pwd”);
$mysqli->query(“SELECT * FROM t”);
$mysqli->query...
{

mysqlnd_ms Quality of Service

mysqlnd_ms is a load balancer at the driver level
controlled by :
–  SQL hints
– 

mysql...
{

myslqnd_mux
multiplexes MySQL connections

•  Less connections to the MySQL server.
•  Pooling saves connection time.
•...
{

MySQL 5.6:
NotOnlySQL: Memcached API .
Application

SQL

NoSQL

(MySQL Client)

mysqld

MySQL
Server

(Memcached
Protoc...
{

PHP NoSQL data Access
memcached API

$mysqli = new mysqli("localhost", "usr", "pass",
"test");
$memcache = new memcache...
{

myslqnd_uh
Write your own plugin in PHP
Monitoring
l  Auditing
l  Load Balancing connections
l  …
l 

l 

Two clas...
{

myslqnd_uh
Installing a connection proxy

class proxy extends MysqlndUhConnection {
public function connect($res, $host...
{

Write your own mysqlnd plugin !

A mysqlnd plugin is itself a PHP extension.
A mysqlnd plugin works by intercepting cal...
{

Sharding with PHP
MySQL Fabric
MySQL Fabric is a sharding framework.
Some connectors can route query based on
sharding ...
{

MySQL Fabric aware connector
Application

Fabric Server
connector

Shard 1

Shard 2

Shard n

…
FORUM PHP
PARIS 2013
{

Sharded Highly Available setup
with Fabric

…
$ mysqlfabric group create shard1
$ mysqlfabric group add shard1 127.0.0....
{

MySQL Fabric
Global and sharded tables
Application
Global Tables
Driver

Shard 1

Shard 2
Sharded table

Shard n

…
FOR...
{

PHP access to
MySQL Fabric shards

fabric.json: {
"test" : {
"fabric":{
"hosts": [ { "host": "localhost", "port”:8080 }...
{

PHP Access
to Fabric sharded database

$c = new mysqli("test", "root", "", "test");
mysqlnd_ms_fabric_select_shard($c, ...
{

PHP Access
to Fabric sharded database

$c = new mysqli("test", "root", "", "test");
mysqlnd_ms_fabric_select_global($c,...
{

Other Sharding Framework
Mature alternative to Fabric

•  Google/Youtube has Vitess (in Go open
source)
•  Twitter has ...
{

PHP Access
to MariaDB Galera Cluster

Ø Read & write access to any node
Ø Client can connect to any node
Ø There can...
{

MariaDB Galera Cluster…
mysqlnd multi master
Clients1
Read
Write

Read
Write

Read
Write

…
Galera Replication
FORUM PH...
{

MariaDB Galera Cluster
…
R/W splitting with HAproxy
Clients

Write

READ on
HAproxy

…
Galera Replication
FORUM PHP
PAR...
{

MariaDB CONNECT Storage Engine
remote and ODBC Access
Node 1
col col2
MariaDB Node 0
ODBC

ODBC
table

MySQL
Node 2
col...
{

MariaDB Connec Storage Engine
Parallel execution

MariaDB Node 0

Node 1
col col2
1

ODBC
table

MySQL
Node 2
col1 col ...
{
l 
l 

Conclusion

Mysqlnd makes MySQL & PHP stronger
Fabric sharding and HA from PHP worth
testing
Thank You
Serge Fr...
Upcoming SlideShare
Loading in...5
×

MySQL PHP native driver : Advanced Functions / PHP forum Paris 2013

13,772

Published on

mysqlnd the MySQL native driver for PHP brings a lot of value to MySQL.
There is no change for developers that can still use the mysqli and PDO API.
This driver supports a plugins extension capability. Some very useful features have been implemented :
- mysqlnd_ms replication and load balancing plugin
- mysqlnd_qc query result cache plugin
- mysqlnd_memcache innoDB memcache plugin
- mysqlnd_uh user handler plugin
- mysqlnd_mux plugin to multiplex PHP connections
MySQL Fabric is the new sharding framework for MySQL. The mysqlnd_ms plugging the MySQL native driver makes it possible to use this sharding framework from PHP.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
13,772
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
27
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of " MySQL PHP native driver : Advanced Functions / PHP forum Paris 2013 "

  1. 1. { MySQL mysqlnd PHP driver Serge Frezefond / Twitter : @sfrezefond FORUM PHP PARIS 2013
  2. 2. { MySQL User Survey: Top Languages FORUM PHP PARIS 2013
  3. 3. Application { MySQL Connectors … LISP FORUM PHP PARIS 2013 MySQL Server Connector/Python* Connector/Java Connector/PHP Connector/.Net Ruby Perl Connector/ODBC Connector/C++ libmysql
  4. 4. { MariaDB Connectors LGPL connectors(non contaminating) : •  MariaDB Client Library for Java Applications •  MariaDB Client Library for C •  MariaDB ODBC Driver (work in progress) FORUM PHP PARIS 2013
  5. 5. { MySQL Protocol Transport Layer •  Unix Domain Socket – Local communication on Linux /tmp/mysql.sock •  Windows Named Pipes – Local communication on Windows systems •  TCP/IP Socket – Network-based communication. TCP Port 3306 •  Embedded MySQL Server – Single process with application code and MySQL Daemon FORUM PHP PARIS 2013
  6. 6. { MySQL Protocol Compression •  MySQL compresses each packet individually – Each returned row is (usually) one packet •  Compression costs CPU to reduce network IO – With bad luck compression consumes more IO •  SELECT id FROM table – Will most likely show bad compression ratio, even with many rows •  SELECT complete_article_text FROM table – Might have good compression ratio FORUM PHP PARIS 2013
  7. 7. { MySQL Protocol Encryption •  Encryption is being enabled during the initial handshake •  All subsequent communication (incl. authentication) will be encrypted – Passwords will always be transferred in a scrambled safe manner only •  SSL handshake time (incl. certificate validation etc.) takes relatively long FORUM PHP PARIS 2013
  8. 8. { MySQL Protocol Character Sets •  MySQL has a rich support for character set and encodings •  Character set can be changed using –  SQL: SET NAMES –  API: mysql_set_character_set() –  In some environments (p.e. Connector/J) the character set is handled internally and should not be changed •  Always use these API calls to set the character set so FORUM PHP escape routines know about it! PARIS 2013
  9. 9. { MySQL Protocol Support for Prepared Statements Server Client SELECT foo FROM bar WHERE id =? Handle Handle Param 1: 42 Resultset(s) prepare() • Create basic Execution plan execute( ) • Query database FORUM PHP PARIS 2013
  10. 10. { Buffered and Unbuffered Result Sets •  Buffered Results – Copies the complete result set to the client – Minimal memory usage on Server Unbuffered Results - Copies rows in sequence as needed - Minimal memory usage on Client FORUM PHP PARIS 2013
  11. 11. { PHP mysqlnd driver •  Under the PHP license. no licensing issues that have been problematic in the past. •  MySQL PHP drivers part of the PHP distribution •  Actively maintained by Oracle as part of the php.net project •  Three APIs (mysql, mysqli, pdo_mysql) and one base library (mysqlnd) •  http://php.net FORUM PHP PARIS 2013
  12. 12. { PHP and mysqlnd PHP ext/mysql mysqli PDO_mysql mysqlnd – MySQL native driver for PHP PHP Memory Infrastructure IO: PHP Streams MySQL Server FORUM PHP PARIS 2013
  13. 13. { libmysql vs. mysqlnd PHP Memory mysqlnd libmysql PHP Memory libmysql Memory MySQL Server MySQL Server copy PHP PHP Memory use directly PHP copy copy FORUM PHP PARIS 2013
  14. 14. { Mysqli API Key mysqli extension benefits : l  Object-oriented interface l  Support for Prepared Statements l  Support for Multiple Statements l  Support for Transactions l  Enhanced debugging capabilities l  Embedded server support FORUM PHP PARIS 2013
  15. 15. { Mysqlnd easy Install / few dependancies PHP 5 ChangeLog 5.4 : ext/mysql, mysqli and pdo_mysql now use mysqlnd by default. $ yum install mysqlnd $ rpm -ql php-mysqlnd-5.3.13-1.fc16.x86_64 /etc/php.d/mysqlnd.ini mysqlnd_mysqli.ini pdo_mysqlnd.ini /usr/lib64/php/modules/mysqlnd.so mysqlnd_mysqli.so pdo_mysqlnd.so FORUM PHP PARIS 2013
  16. 16. { Mysqlnd unique features Special features compare to MySQL Client Library : »  Improved persistent connections »  The special function mysqli_fetch_all() »  Performance statistics calls: mysqli_get_cache_stats(), mysqli_get_client_stats(), mysqli_get_connection_stats() FORUM PHP PARIS 2013
  17. 17. { Mysqlnd supported features SSL Support MySQL Native Driver supports SSL since PHP 5.3.3 Compressed Protocol Support As of PHP 5.3.2 MySQL Native Driver supports the compressed client server protocol. Note that PDO_MYSQL does NOT support compression when used together with mysqlnd. Named Pipes Support Named pipes support for Windows was added in PHP FORUM PHP version 5.4.0. PARIS 2013
  18. 18. { Mysql asynchronous query only with mysqlnd $link1->query("SELECT 'test'", MYSQLI_ASYNC); ... if (!mysqli_poll($links, $errors, $reject, 1)) { ... $result = $link->reap_async_query(); FORUM PHP PARIS 2013
  19. 19. { client per-process statistics only with mysqlnd • Around 150 statistic values collected –  mysqli_get_client_stats () –  mysqli_get_connection_stats() • Check PS Execution Ratio –  $stats = mysqli_get_client_stats(); echo $stats['com_execute'] / $stats['com_prepare']; •  JSMysqlndBundle a symfony2 bundle FORUM PHP PARIS 2013
  20. 20. { Mysqlnd Extensible with plugins mysql_query() mysqli::query() PDO::query() mysqlnd Query Plugin Hook Wire Protocol Plugin Hook Network FORUM PHP PARIS 2013
  21. 21. { mysqlnd Plugins Client Proxy – Load Balancing •  Read / Write splitting •  Failover •  Sharding suport ( Fabric framework) – Monitoring •  Query Logging / Auditing – Performance •  Caching •  throtling •  multiplexing FORUM PHP PARIS 2013
  22. 22. { mysqlnd Plugins through PECL/PHP Extension Community Library • mysqlnd_ms Replication /load balancing plugin • mysqlnd_qc Query cache plugin • mysqlnd_uh mysqlnd Userland Handler • mysqlnd_memcache Transparent access to memcached • mysqlnd_mux Multiplexes MySQL connections • # pecl install mysqlnd_qc-beta … and you are done! FORUM PHP PARIS 2013
  23. 23. { mysqlnd Query Cache PHP mysql / mysqli / PDO_mysql mysqlnd Query Cache Cache Backend Local Memory, APC, Memcache, Custom Handler MySQL Server FORUM PHP PARIS 2013
  24. 24. { mysqlnd Query Cache Key Properties • Transparent – PHP Extension hooking into mysqlnd • Works with ext/mysql, mysqli, pdo_mysql • Pluggable storage handler – By default: local memory, APC, memcache, SQLite – PHP Userspace • Invalidation via TTL – No automatic invalidation by server – Custom handlers may use custom invalidation logic FORUM PHP PARIS 2013
  25. 25. { mysqlnd Query Cache Transparent ? $mysqli = new mysqli($host, $user, $pw, $db); $sql = sprintf(“/*%s*/SELECT SLEEP(10) FROM table”, MYSQLND_QC_ENABLE_SWITCH); ... mysqlnd_qc.cache_by_default = 1 FORUM PHP PARIS 2013
  26. 26. { mysqlnd Query Cache User-Defined Storage Handler public function is_select($query) { if (preg_match("@from employees where@ism", $query)) {return true;} .. $qc = new my_qc(); mysqlnd_qc_change_handler($qc); FORUM PHP PARIS 2013
  27. 27. { myslqnd_ms • mysqlnd replication and load balancing plugin (mysqlnd_ms = “mysqlnd master/slave splitting”) FORUM PHP PARIS 2013
  28. 28. { mysqlnd_ms Application mysql_query() mysqli::query() PDO::query() mysqlnd_ms Master Slave(s) FORUM PHP PARIS 2013
  29. 29. { mysqlnd_ms Configuration { "myapp": { "master": { "master_0": { "host": "localhost", "socket": "/tmp/mysql.sock" } }, "slave": { "slave_0": { "host": "192.168.2.27", "port": "3306" } } } } FORUM PHP PARIS 2013
  30. 30. { mysqlnd_ms: Connecting $mysqli = new MySQLi(“myapp”, “user”, “pwd”); $mysqli->query(“SELECT * FROM t”); $mysqli->query(“SET @sessionvar='value' ”); $mysqli->query(“SELECT @sessionvar”); FORUM PHP PARIS 2013
  31. 31. { mysqlnd_ms Quality of Service mysqlnd_ms is a load balancer at the driver level controlled by : –  SQL hints –  mysqlnd_ms_set_qos() defines the quality of service (QoS). tells load balancer how to select database cluster nodes . Without GTIDs the rules for a MySQL Replication cluster are simple: –  eventual consistency – any slave, session –  strong consistency – master only. FORUM PHP PARIS 2013
  32. 32. { myslqnd_mux multiplexes MySQL connections •  Less connections to the MySQL server. •  Pooling saves connection time. •  Multiplexed connection •  Persistent connection FORUM PHP PARIS 2013
  33. 33. { MySQL 5.6: NotOnlySQL: Memcached API . Application SQL NoSQL (MySQL Client) mysqld MySQL Server (Memcached Protocol) Memcached plugin InnoDB Storage Engine •  Fast, simple access to InnoDB •  Memcached for key-value operations •  Memcached daemon plug-in to mysqld FORUM PHP PARIS 2013
  34. 34. { PHP NoSQL data Access memcached API $mysqli = new mysqli("localhost", "usr", "pass", "test"); $memcache = new memcached(); $memcache->addServer("localhost", 11211); mysqlnd_memcache_set($mysqli, $memcache); …. $q1 = $mysqli->query("SELECT firstname, lastname FROM test WHERE id = 1"); FORUM PHP PARIS 2013
  35. 35. { myslqnd_uh Write your own plugin in PHP Monitoring l  Auditing l  Load Balancing connections l  … l  l  Two classes are provided by the extension: MysqlndUhConnection and MysqlndUhPreparedStatement. FORUM PHP PARIS 2013
  36. 36. { myslqnd_uh Installing a connection proxy class proxy extends MysqlndUhConnection { public function connect($res, $host, $user, $pwd, $db, $port, $socket, $mysql_flags) { printf("%s(%s)n", __METHOD__, var_export(func_get_args(), true)); … } mysqlnd_uh_set_connection_proxy(new proxy()); $mysqli = new mysqli("localhost", "root", "", FORUM PHP "test"); PARIS 2013
  37. 37. { Write your own mysqlnd plugin ! A mysqlnd plugin is itself a PHP extension. A mysqlnd plugin works by intercepting calls made to mysqlnd by extensions that use mysqlnd. This is achieved by obtaining the mysqlnd function table, backing it up, and replacing it by a custom function table, which calls the functions of the plugin as required. FORUM PHP PARIS 2013
  38. 38. { Sharding with PHP MySQL Fabric MySQL Fabric is a sharding framework. Some connectors can route query based on sharding metadata stores in Fabric Server. Fabric aware connectors : •  connector python •  connector J •  connect PHP FORUM PHP PARIS 2013
  39. 39. { MySQL Fabric aware connector Application Fabric Server connector Shard 1 Shard 2 Shard n … FORUM PHP PARIS 2013
  40. 40. { Sharded Highly Available setup with Fabric … $ mysqlfabric group create shard1 $ mysqlfabric group add shard1 127.0.0.1:3303 root secret $ mysqlfabric group promote shard1 $ mysqlfabric group add shard1 127.0.0.1:3304 root secret … FORUM PHP PARIS 2013
  41. 41. { MySQL Fabric Global and sharded tables Application Global Tables Driver Shard 1 Shard 2 Sharded table Shard n … FORUM PHP PARIS 2013
  42. 42. { PHP access to MySQL Fabric shards fabric.json: { "test" : { "fabric":{ "hosts": [ { "host": "localhost", "port”:8080 } ] }}} php.ini: extension=mysqlnd_ms.so mysqlnd_ms.enable=1 mysqlnd_ms.config_file=/path/to/fabric.json FORUM PHP PARIS 2013
  43. 43. { PHP Access to Fabric sharded database $c = new mysqli("test", "root", "", "test"); mysqlnd_ms_fabric_select_shard($c, "test.ftest", 10); $c->query("INSERT INTO ftest VALUES (10)")); mysqlnd_ms_fabric_select_shard($c, "test.ftest", 110); $r = $c->query("SELECT * FROM ftest WHERE id = 110"); $r->fetch_row(); FORUM PHP PARIS 2013
  44. 44. { PHP Access to Fabric sharded database $c = new mysqli("test", "root", "", "test"); mysqlnd_ms_fabric_select_global($c, "test.fabrictest"); $c->query("CREATE TABLE fabrictest (id INT NOT NULL) »); FORUM PHP PARIS 2013
  45. 45. { Other Sharding Framework Mature alternative to Fabric •  Google/Youtube has Vitess (in Go open source) •  Twitter has built Gizzard (in Scala open source) •  Tumblr has built Jetpants (in Ruby open source) •  Facebook has also it own sharding framework which is a range based model. The HA is addressed with MHA. they have presented their architecture at various events •  Many others : ETSY, …. FORUM PHP PARIS 2013
  46. 46. { PHP Access to MariaDB Galera Cluster Ø Read & write access to any node Ø Client can connect to any node Ø There can be several nodes Ø Automatic node provisioning Ø Replication is synchronous FORUM PHP PARIS 2013
  47. 47. { MariaDB Galera Cluster… mysqlnd multi master Clients1 Read Write Read Write Read Write … Galera Replication FORUM PHP PARIS 2013
  48. 48. { MariaDB Galera Cluster … R/W splitting with HAproxy Clients Write READ on HAproxy … Galera Replication FORUM PHP PARIS 2013
  49. 49. { MariaDB CONNECT Storage Engine remote and ODBC Access Node 1 col col2 MariaDB Node 0 ODBC ODBC table MySQL Node 2 col1 col col3table MYSQL FORUM PHP PARIS 2013
  50. 50. { MariaDB Connec Storage Engine Parallel execution MariaDB Node 0 Node 1 col col2 1 ODBC table MySQL Node 2 col1 col col3table 2 TBL MYSQL / ODBC Node col col3 col4 col1 3 2 FORUM PHP PARIS 2013
  51. 51. { l  l  Conclusion Mysqlnd makes MySQL & PHP stronger Fabric sharding and HA from PHP worth testing Thank You Serge Frezefond @sfrezefond http://serge.frezefond.com FORUM PHP PARIS 2013
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×