SlideShare a Scribd company logo
1 of 50
Ulf Wendel, Oracle Web scale made easy (just kidding...) The mysqlnd replication and load balancing plugin
MySQL scale-out solutions Part 1 - Proven and designed for web-scale
The speaker says... MySQL offers a variety of scale-out solutions. Which are they, when to use and how to use from an application developers point of view?
Developer dreams ,[object Object],Systemload: 0.1 100%
The speaker says... Let's take a PHP developer centric look at a high-growth web site. The web site is build on top of the industrial leading LAMP platform: Linux, MySQL, Apache and PHP. The day before the launch, the developer dreams the dream of his options or shares rising in value: 2x, 5x, 42x ...
Developer dreams Day 2 (Growth) – web site visitors The first customer Twitter follower Facebook friend A slashdot reader A TV moderator Systemload: 0.5
The speaker says... ... a wonderful dream. The enterprise becomes popular, IPO is on the horizon. There are many reasons for growth. As a developer you should know about the MySQL scale-out solutions to be prepared. According to a  MySQL Online Quickpoll from 2009  a majority of the MySQL users is using MySQL Replication.
Developer dreams Day 3 (Success) – scale out needed 408 Request Timeout 503 Service Unavailable 1040 SQLSTATE: 08004 (ER_CON_COUNT_ERROR) -  Too many connections Systemload: 42
The speaker says... Dear developer, help is on the way. The PHP mysqlnd replication and load balancing plugin ( PECL/mysqlnd_ms ) is a drop-in solution for your PHP MySQL web application adding MySQL scale-out support. For basic scale-out setups no code changes are required. As of PHP 5.3 all three PHP MySQL APIs ( mysql ,  mysqli ,  PDO_MySQL ) can be compiled to use the MySQL native driver for PHP ( mysqlnd ) library. Mysqlnd is shipped with PHP as of version 5.3. It is a default as of PHP 5.4.
Scale out means clustering ,[object Object],[object Object]
TCO: Proven Open Source solutions MySQL Server 1 MySQL Server 2 MySQL Server n
The speaker says... Scale out, not up. Use a cluster of commodity servers. Scale horizontally building upon proven Open Source solutions.  Try MySQL Replication, MySQL Cluster or a custom MySQL scale-out solution.
[object Object],[object Object]
Various topologies supported
Other use cases: backup, HA, OLTP/warehousing MySQL Replication Writes Reads Master Slave Slave Slave
The speaker says... MySQL Replication is used for read scale-out.  Writes are send to a MySQL master, reads are send to a MySQL slave. The master sends updates to the slaves in an  asynchronous  way. Slaves may not have the latest changes. MySQL Replication can also be used for: ,[object Object]
High Availablility – for example, remote slaves
Warehousing – OLTP reporting on the slaves
[object Object],[object Object]
Auto sharding, shared nothing architecture
Web use cases: session server, tele communication MySQL Cluster SQL Node SQL Node Cluster Data Node Cluster Data Node Cluster Data Node
The speaker says... MySQL Cluster is used for real-time transactional read and write scale-out.  Data is auto-partitioned on cluster data nodes resulting in 99.999% availability. The cluster is accessed through MySQL Application Cluster Nodes. The most basic MySQL Application Cluster Node is a SQL Node – a MySQL server (mysqld). If MySQL Cluster is new to you, think of MySQL Cluster as a storage engine for the network. Like InnoDB and MyISAM are storage engines for disks.
[object Object],[object Object]
Vertical: columns in additional tables
... sharding Custom Cluster Shard 1 ID % 2 = 0 Node 3 Cats Node 4 Dogs Shard 2 ID % 2 = 1
The speaker says... Special requirements may make it necessary to build a custom cluster.  For example, your application may require a sharding solution.
On using any database cluster Part 2 - Required application changes
[object Object],[object Object]
Load balance within candidates
Maintain connection pool
Automatic fail over for High Availability Application using any DB cluster MySQL Server 1 MySQL Server 2 MySQL Server n
The speaker says... All PHP applications talking to a cluster of MySQL database servers are facing the same tasks. Replacing a single database with a database cluster means changing the 1:1 relation between the application and the database into a  1:n relation . 1:n puts an additional task on the application:  find the right n, find the right server to talk to.
The plugin for all of you MySQL Server 1 MySQL Server 2 MySQL Server n
The speaker says... PECL/mysqlnd_ms is a plugin for the MySQL native driver for PHP (mysqlnd) library . The mysqlnd library is part of PHP as of version 5.3. As of 5.4 mysqlnd is a default. All three PHP MySQL APIs (mysql, mysqli, PDO_MySQL) can be compiled to use the mysqlnd library, thus  all existing PHP MySQL application are supported by the plugin. From an applications point of view a mysqlnd plugin can act as a transparent proxy.  Depending on plugin an use case, no code changes are needed.  A plugin is a drop-in solution.
Plugin focus: MySQL Replication Part 3 - Tasks, solutions and limitations
Using MySQL Replication ,[object Object],[object Object]
Replication filter (table based parititioning)
Custom: callback ,[object Object],[object Object]
Custom: callback ,[object Object],[object Object]
Failover
The speaker says... The plugin is not limited to but optimized for MySQL Replication.  The common clustering support feature set focusses on MySQL Replication.  For example, statement redirection covers read/write splitting and MySQL Replication filter support. All read accesses are executed on the slave servers, all writes are performed on the master. MySQL Replication supports table based partitioning, replicating selected tables to selected slaves only. The plugin can be made aware of this.
Speaking code /* Connection user handle represents a connection pool */ $mysqli = new mysqli( "hostname" , $user, $pw, $db, $port); /* Read/Write split, Load Balancing */ $res = $mysqli->query("SELECT 'Slave speaking' FROM DUAL"); var_dump($res->fetch_all(MYSQLI_ASSOC)); { "hostname" :{ "master":{ "host":"localhost" }, "slave":{ "slave_0":{"host":"192.168.2.27"} }, } }
The speaker says... No code changes for basic use cases. Install the plugin following standard PHP PECL procedures. Create a configuration file. Enable the plugin and set its configuration file in your PHP configuration using mysqlnd_ms.enable=1 and mysqlnd_ms.ini_file=config.ini .  In the plugin configuration create a section with the name of the host your application connects to, for example „hostname“. List master and slave servers. Any connection to „hostname“ is now handled by the plugin. Using any API.
Some code changes /* Connection user handle represents a connection pool */ $mysqli = new mysqli("hostname", $user, $pw, $db, $port); / * R/W split: no SELECT, run on master */ $mysqli->query("SET @myrole='Master'); /* R/W split: overruled by SQL hint */ $sql = sprintf("/*%s*/%s", MYSQLND_MS_MASTER_SWITCH, "SELECT @myrole AS _role"); $mysqli->query($sql); var_dump($res->fetch_all(MYSQLI_ASSOC));
The speaker says... Non-trivial use cases require support by the application. In other words: existing applications may need updates, new applications need to be designed appropriately. In the example a SQL hint is used to force executio of a SELECT statement on a MySQL Replication master server. There are SQL hints for running a statement on the master, on the slave and on the last used server. Alternative: new APIs call, no win compared to SQL hint.
Why and when code changes $mysqli = new mysqli("hostname", $user, $pw, $db, $port);  $mysqli->query("SET @myrole='Master'); $mysqli->query("SELECT myrole AS _role"); Without plugin With plugin
The speaker says... An application not designed for use with a cluster assumes a connection handle always points to the same server (1:1), having the same state.  When using the plugin a connection handle represents a pool of connections (1:n), potentially having different states.  Connection switches may occur whenever a statement is executed. After the switch, the connection handle may point to a connection with a different state. Applications need to take care and hint the plugin.
A trouble maker in detail ,[object Object],[object Object]
Transaction status
Temporary tables

More Related Content

What's hot

DIY: A distributed database cluster, or: MySQL Cluster
DIY: A distributed database cluster, or: MySQL ClusterDIY: A distributed database cluster, or: MySQL Cluster
DIY: A distributed database cluster, or: MySQL ClusterUlf Wendel
 
Built-in query caching for all PHP MySQL extensions/APIs
Built-in query caching for all PHP MySQL extensions/APIsBuilt-in query caching for all PHP MySQL extensions/APIs
Built-in query caching for all PHP MySQL extensions/APIsUlf Wendel
 
Intro to PECL/mysqlnd_ms (4/7/2011)
Intro to PECL/mysqlnd_ms (4/7/2011)Intro to PECL/mysqlnd_ms (4/7/2011)
Intro to PECL/mysqlnd_ms (4/7/2011)Chris Barber
 
PHP mysqlnd connection multiplexing plugin
PHP mysqlnd connection multiplexing pluginPHP mysqlnd connection multiplexing plugin
PHP mysqlnd connection multiplexing pluginUlf Wendel
 
MySQL Group Replication
MySQL Group ReplicationMySQL Group Replication
MySQL Group ReplicationUlf Wendel
 
NoSQL in MySQL
NoSQL in MySQLNoSQL in MySQL
NoSQL in MySQLUlf Wendel
 
MySQL 5.7 clustering: The developer perspective
MySQL 5.7 clustering: The developer perspectiveMySQL 5.7 clustering: The developer perspective
MySQL 5.7 clustering: The developer perspectiveUlf Wendel
 
HTTP Plugin for MySQL!
HTTP Plugin for MySQL!HTTP Plugin for MySQL!
HTTP Plugin for MySQL!Ulf Wendel
 
Award-winning technology: Oxid loves the query cache
Award-winning technology: Oxid loves the query cacheAward-winning technology: Oxid loves the query cache
Award-winning technology: Oxid loves the query cacheUlf Wendel
 
Highly Available MySQL/PHP Applications with mysqlnd
Highly Available MySQL/PHP Applications with mysqlndHighly Available MySQL/PHP Applications with mysqlnd
Highly Available MySQL/PHP Applications with mysqlndJervin Real
 
MySQL Group Replication
MySQL Group ReplicationMySQL Group Replication
MySQL Group ReplicationBogdan Kecman
 
Mysql high availability and scalability
Mysql high availability and scalabilityMysql high availability and scalability
Mysql high availability and scalabilityyin gong
 
Introduction to Galera
Introduction to GaleraIntroduction to Galera
Introduction to GaleraHenrik Ingo
 
MySQL Multi Master Replication
MySQL Multi Master ReplicationMySQL Multi Master Replication
MySQL Multi Master ReplicationMoshe Kaplan
 
Methods of Sharding MySQL
Methods of Sharding MySQLMethods of Sharding MySQL
Methods of Sharding MySQLLaine Campbell
 
High-Availability using MySQL Fabric
High-Availability using MySQL FabricHigh-Availability using MySQL Fabric
High-Availability using MySQL FabricMats Kindahl
 
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group Replication
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group ReplicationPercona XtraDB Cluster vs Galera Cluster vs MySQL Group Replication
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group ReplicationKenny Gryp
 
MySQL High Availability: Managing Farms of Distributed Servers (MySQL Fabric)
MySQL High Availability: Managing Farms of Distributed Servers (MySQL Fabric)MySQL High Availability: Managing Farms of Distributed Servers (MySQL Fabric)
MySQL High Availability: Managing Farms of Distributed Servers (MySQL Fabric)Alfranio Júnior
 
Scaling with sync_replication using Galera and EC2
Scaling with sync_replication using Galera and EC2Scaling with sync_replication using Galera and EC2
Scaling with sync_replication using Galera and EC2Marco Tusa
 

What's hot (20)

DIY: A distributed database cluster, or: MySQL Cluster
DIY: A distributed database cluster, or: MySQL ClusterDIY: A distributed database cluster, or: MySQL Cluster
DIY: A distributed database cluster, or: MySQL Cluster
 
Built-in query caching for all PHP MySQL extensions/APIs
Built-in query caching for all PHP MySQL extensions/APIsBuilt-in query caching for all PHP MySQL extensions/APIs
Built-in query caching for all PHP MySQL extensions/APIs
 
Intro to PECL/mysqlnd_ms (4/7/2011)
Intro to PECL/mysqlnd_ms (4/7/2011)Intro to PECL/mysqlnd_ms (4/7/2011)
Intro to PECL/mysqlnd_ms (4/7/2011)
 
PHP mysqlnd connection multiplexing plugin
PHP mysqlnd connection multiplexing pluginPHP mysqlnd connection multiplexing plugin
PHP mysqlnd connection multiplexing plugin
 
MySQL Group Replication
MySQL Group ReplicationMySQL Group Replication
MySQL Group Replication
 
NoSQL in MySQL
NoSQL in MySQLNoSQL in MySQL
NoSQL in MySQL
 
MySQL 5.7 clustering: The developer perspective
MySQL 5.7 clustering: The developer perspectiveMySQL 5.7 clustering: The developer perspective
MySQL 5.7 clustering: The developer perspective
 
HTTP Plugin for MySQL!
HTTP Plugin for MySQL!HTTP Plugin for MySQL!
HTTP Plugin for MySQL!
 
Award-winning technology: Oxid loves the query cache
Award-winning technology: Oxid loves the query cacheAward-winning technology: Oxid loves the query cache
Award-winning technology: Oxid loves the query cache
 
Highly Available MySQL/PHP Applications with mysqlnd
Highly Available MySQL/PHP Applications with mysqlndHighly Available MySQL/PHP Applications with mysqlnd
Highly Available MySQL/PHP Applications with mysqlnd
 
MySQL Group Replication
MySQL Group ReplicationMySQL Group Replication
MySQL Group Replication
 
Mysql high availability and scalability
Mysql high availability and scalabilityMysql high availability and scalability
Mysql high availability and scalability
 
Introduction to Galera
Introduction to GaleraIntroduction to Galera
Introduction to Galera
 
MySQL Multi Master Replication
MySQL Multi Master ReplicationMySQL Multi Master Replication
MySQL Multi Master Replication
 
Methods of Sharding MySQL
Methods of Sharding MySQLMethods of Sharding MySQL
Methods of Sharding MySQL
 
High-Availability using MySQL Fabric
High-Availability using MySQL FabricHigh-Availability using MySQL Fabric
High-Availability using MySQL Fabric
 
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group Replication
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group ReplicationPercona XtraDB Cluster vs Galera Cluster vs MySQL Group Replication
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group Replication
 
Introduction to Galera Cluster
Introduction to Galera ClusterIntroduction to Galera Cluster
Introduction to Galera Cluster
 
MySQL High Availability: Managing Farms of Distributed Servers (MySQL Fabric)
MySQL High Availability: Managing Farms of Distributed Servers (MySQL Fabric)MySQL High Availability: Managing Farms of Distributed Servers (MySQL Fabric)
MySQL High Availability: Managing Farms of Distributed Servers (MySQL Fabric)
 
Scaling with sync_replication using Galera and EC2
Scaling with sync_replication using Galera and EC2Scaling with sync_replication using Galera and EC2
Scaling with sync_replication using Galera and EC2
 

Viewers also liked

Maintaining Low Latency While Maximizing Throughput on a Single Cluster
Maintaining Low Latency While Maximizing Throughput on a Single ClusterMaintaining Low Latency While Maximizing Throughput on a Single Cluster
Maintaining Low Latency While Maximizing Throughput on a Single ClusterMapR Technologies
 
Enabling Exploratory Analytics of Data in Shared-service Hadoop Clusters
Enabling Exploratory Analytics of Data in Shared-service Hadoop ClustersEnabling Exploratory Analytics of Data in Shared-service Hadoop Clusters
Enabling Exploratory Analytics of Data in Shared-service Hadoop ClustersDataWorks Summit
 
My sql cluster case study apr16
My sql cluster case study apr16My sql cluster case study apr16
My sql cluster case study apr16Sumi Ryu
 
Buckle promotional campaign
Buckle promotional campaignBuckle promotional campaign
Buckle promotional campaignTaylor Pickering
 
Netflix: A Case Study
Netflix: A Case StudyNetflix: A Case Study
Netflix: A Case StudyMorgan Miller
 
MBA case study presentation template
MBA case study presentation templateMBA case study presentation template
MBA case study presentation templategorvis
 

Viewers also liked (7)

Maintaining Low Latency While Maximizing Throughput on a Single Cluster
Maintaining Low Latency While Maximizing Throughput on a Single ClusterMaintaining Low Latency While Maximizing Throughput on a Single Cluster
Maintaining Low Latency While Maximizing Throughput on a Single Cluster
 
Enabling Exploratory Analytics of Data in Shared-service Hadoop Clusters
Enabling Exploratory Analytics of Data in Shared-service Hadoop ClustersEnabling Exploratory Analytics of Data in Shared-service Hadoop Clusters
Enabling Exploratory Analytics of Data in Shared-service Hadoop Clusters
 
My sql cluster case study apr16
My sql cluster case study apr16My sql cluster case study apr16
My sql cluster case study apr16
 
Buckle promotional campaign
Buckle promotional campaignBuckle promotional campaign
Buckle promotional campaign
 
Netflix: A Case Study
Netflix: A Case StudyNetflix: A Case Study
Netflix: A Case Study
 
Cluster analysis
Cluster analysisCluster analysis
Cluster analysis
 
MBA case study presentation template
MBA case study presentation templateMBA case study presentation template
MBA case study presentation template
 

Similar to MySQL Replication Made Easy

The power of mysqlnd plugins
The power of mysqlnd pluginsThe power of mysqlnd plugins
The power of mysqlnd pluginsUlf Wendel
 
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQLHTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQLUlf Wendel
 
Mysqlnd, an unknown powerful PHP extension
Mysqlnd, an unknown powerful PHP extensionMysqlnd, an unknown powerful PHP extension
Mysqlnd, an unknown powerful PHP extensionjulien pauli
 
MySQL HA with PaceMaker
MySQL HA with  PaceMakerMySQL HA with  PaceMaker
MySQL HA with PaceMakerKris Buytaert
 
MySQL InnoDB Cluster: High Availability Made Easy!
MySQL InnoDB Cluster: High Availability Made Easy!MySQL InnoDB Cluster: High Availability Made Easy!
MySQL InnoDB Cluster: High Availability Made Easy!Vittorio Cioe
 
My sql crashcourse_intro_kdl
My sql crashcourse_intro_kdlMy sql crashcourse_intro_kdl
My sql crashcourse_intro_kdlsqlhjalp
 
MySQL PHP native driver : Advanced Functions / PHP forum Paris 2013
 MySQL PHP native driver  : Advanced Functions / PHP forum Paris 2013   MySQL PHP native driver  : Advanced Functions / PHP forum Paris 2013
MySQL PHP native driver : Advanced Functions / PHP forum Paris 2013 Serge Frezefond
 
My sql università di enna a.a. 2005-06
My sql   università di enna a.a. 2005-06My sql   università di enna a.a. 2005-06
My sql università di enna a.a. 2005-06YUCHENG HU
 
Mysqlppt3510
Mysqlppt3510Mysqlppt3510
Mysqlppt3510Anuja Lad
 
MySQL InnoDB Cluster HA Overview & Demo
MySQL InnoDB Cluster HA Overview & DemoMySQL InnoDB Cluster HA Overview & Demo
MySQL InnoDB Cluster HA Overview & DemoKeith Hollman
 
MySQL Shell for Database Engineers
MySQL Shell for Database EngineersMySQL Shell for Database Engineers
MySQL Shell for Database EngineersMydbops
 
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & ClusterMySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & ClusterKenny Gryp
 
Www Kitebird Com Articles Pydbapi Html Toc 1
Www Kitebird Com Articles Pydbapi Html Toc 1Www Kitebird Com Articles Pydbapi Html Toc 1
Www Kitebird Com Articles Pydbapi Html Toc 1AkramWaseem
 

Similar to MySQL Replication Made Easy (20)

The power of mysqlnd plugins
The power of mysqlnd pluginsThe power of mysqlnd plugins
The power of mysqlnd plugins
 
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQLHTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
 
Mysqlnd, an unknown powerful PHP extension
Mysqlnd, an unknown powerful PHP extensionMysqlnd, an unknown powerful PHP extension
Mysqlnd, an unknown powerful PHP extension
 
Mysql ppt
Mysql pptMysql ppt
Mysql ppt
 
MySQL HA with PaceMaker
MySQL HA with  PaceMakerMySQL HA with  PaceMaker
MySQL HA with PaceMaker
 
MySQL InnoDB Cluster: High Availability Made Easy!
MySQL InnoDB Cluster: High Availability Made Easy!MySQL InnoDB Cluster: High Availability Made Easy!
MySQL InnoDB Cluster: High Availability Made Easy!
 
My sql crashcourse_intro_kdl
My sql crashcourse_intro_kdlMy sql crashcourse_intro_kdl
My sql crashcourse_intro_kdl
 
Php mysql-tutorial-en
Php mysql-tutorial-enPhp mysql-tutorial-en
Php mysql-tutorial-en
 
MySQL PHP native driver : Advanced Functions / PHP forum Paris 2013
 MySQL PHP native driver  : Advanced Functions / PHP forum Paris 2013   MySQL PHP native driver  : Advanced Functions / PHP forum Paris 2013
MySQL PHP native driver : Advanced Functions / PHP forum Paris 2013
 
zLAMP
zLAMPzLAMP
zLAMP
 
My sql università di enna a.a. 2005-06
My sql   università di enna a.a. 2005-06My sql   università di enna a.a. 2005-06
My sql università di enna a.a. 2005-06
 
Mysqlppt3510
Mysqlppt3510Mysqlppt3510
Mysqlppt3510
 
Mysqlppt3510
Mysqlppt3510Mysqlppt3510
Mysqlppt3510
 
Pydbapi
PydbapiPydbapi
Pydbapi
 
Mysql tutorial
Mysql tutorialMysql tutorial
Mysql tutorial
 
MySQL InnoDB Cluster HA Overview & Demo
MySQL InnoDB Cluster HA Overview & DemoMySQL InnoDB Cluster HA Overview & Demo
MySQL InnoDB Cluster HA Overview & Demo
 
MySQL Shell for Database Engineers
MySQL Shell for Database EngineersMySQL Shell for Database Engineers
MySQL Shell for Database Engineers
 
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & ClusterMySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
 
Www Kitebird Com Articles Pydbapi Html Toc 1
Www Kitebird Com Articles Pydbapi Html Toc 1Www Kitebird Com Articles Pydbapi Html Toc 1
Www Kitebird Com Articles Pydbapi Html Toc 1
 
Introduction to php
Introduction to phpIntroduction to php
Introduction to php
 

More from Ulf Wendel

Data massage: How databases have been scaled from one to one million nodes
Data massage: How databases have been scaled from one to one million nodesData massage: How databases have been scaled from one to one million nodes
Data massage: How databases have been scaled from one to one million nodesUlf Wendel
 
PHPopstar der PHP Unconference 2011
PHPopstar der PHP Unconference 2011PHPopstar der PHP Unconference 2011
PHPopstar der PHP Unconference 2011Ulf Wendel
 
Mysqlnd query cache plugin benchmark report
Mysqlnd query cache plugin benchmark reportMysqlnd query cache plugin benchmark report
Mysqlnd query cache plugin benchmark reportUlf Wendel
 
mysqlnd query cache plugin: user-defined storage handler
mysqlnd query cache plugin: user-defined storage handlermysqlnd query cache plugin: user-defined storage handler
mysqlnd query cache plugin: user-defined storage handlerUlf Wendel
 
Mysqlnd query cache plugin statistics and tuning
Mysqlnd query cache plugin statistics and tuningMysqlnd query cache plugin statistics and tuning
Mysqlnd query cache plugin statistics and tuningUlf Wendel
 
Mysqlnd Async Ipc2008
Mysqlnd Async Ipc2008Mysqlnd Async Ipc2008
Mysqlnd Async Ipc2008Ulf Wendel
 

More from Ulf Wendel (6)

Data massage: How databases have been scaled from one to one million nodes
Data massage: How databases have been scaled from one to one million nodesData massage: How databases have been scaled from one to one million nodes
Data massage: How databases have been scaled from one to one million nodes
 
PHPopstar der PHP Unconference 2011
PHPopstar der PHP Unconference 2011PHPopstar der PHP Unconference 2011
PHPopstar der PHP Unconference 2011
 
Mysqlnd query cache plugin benchmark report
Mysqlnd query cache plugin benchmark reportMysqlnd query cache plugin benchmark report
Mysqlnd query cache plugin benchmark report
 
mysqlnd query cache plugin: user-defined storage handler
mysqlnd query cache plugin: user-defined storage handlermysqlnd query cache plugin: user-defined storage handler
mysqlnd query cache plugin: user-defined storage handler
 
Mysqlnd query cache plugin statistics and tuning
Mysqlnd query cache plugin statistics and tuningMysqlnd query cache plugin statistics and tuning
Mysqlnd query cache plugin statistics and tuning
 
Mysqlnd Async Ipc2008
Mysqlnd Async Ipc2008Mysqlnd Async Ipc2008
Mysqlnd Async Ipc2008
 

Recently uploaded

Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxfnnc6jmgwh
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...itnewsafrica
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationKnoldus Inc.
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructureitnewsafrica
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 

Recently uploaded (20)

Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 

MySQL Replication Made Easy

  • 1. Ulf Wendel, Oracle Web scale made easy (just kidding...) The mysqlnd replication and load balancing plugin
  • 2. MySQL scale-out solutions Part 1 - Proven and designed for web-scale
  • 3. The speaker says... MySQL offers a variety of scale-out solutions. Which are they, when to use and how to use from an application developers point of view?
  • 4.
  • 5. The speaker says... Let's take a PHP developer centric look at a high-growth web site. The web site is build on top of the industrial leading LAMP platform: Linux, MySQL, Apache and PHP. The day before the launch, the developer dreams the dream of his options or shares rising in value: 2x, 5x, 42x ...
  • 6. Developer dreams Day 2 (Growth) – web site visitors The first customer Twitter follower Facebook friend A slashdot reader A TV moderator Systemload: 0.5
  • 7. The speaker says... ... a wonderful dream. The enterprise becomes popular, IPO is on the horizon. There are many reasons for growth. As a developer you should know about the MySQL scale-out solutions to be prepared. According to a MySQL Online Quickpoll from 2009 a majority of the MySQL users is using MySQL Replication.
  • 8. Developer dreams Day 3 (Success) – scale out needed 408 Request Timeout 503 Service Unavailable 1040 SQLSTATE: 08004 (ER_CON_COUNT_ERROR) - Too many connections Systemload: 42
  • 9. The speaker says... Dear developer, help is on the way. The PHP mysqlnd replication and load balancing plugin ( PECL/mysqlnd_ms ) is a drop-in solution for your PHP MySQL web application adding MySQL scale-out support. For basic scale-out setups no code changes are required. As of PHP 5.3 all three PHP MySQL APIs ( mysql , mysqli , PDO_MySQL ) can be compiled to use the MySQL native driver for PHP ( mysqlnd ) library. Mysqlnd is shipped with PHP as of version 5.3. It is a default as of PHP 5.4.
  • 10.
  • 11. TCO: Proven Open Source solutions MySQL Server 1 MySQL Server 2 MySQL Server n
  • 12. The speaker says... Scale out, not up. Use a cluster of commodity servers. Scale horizontally building upon proven Open Source solutions. Try MySQL Replication, MySQL Cluster or a custom MySQL scale-out solution.
  • 13.
  • 15. Other use cases: backup, HA, OLTP/warehousing MySQL Replication Writes Reads Master Slave Slave Slave
  • 16.
  • 17. High Availablility – for example, remote slaves
  • 18. Warehousing – OLTP reporting on the slaves
  • 19.
  • 20. Auto sharding, shared nothing architecture
  • 21. Web use cases: session server, tele communication MySQL Cluster SQL Node SQL Node Cluster Data Node Cluster Data Node Cluster Data Node
  • 22. The speaker says... MySQL Cluster is used for real-time transactional read and write scale-out. Data is auto-partitioned on cluster data nodes resulting in 99.999% availability. The cluster is accessed through MySQL Application Cluster Nodes. The most basic MySQL Application Cluster Node is a SQL Node – a MySQL server (mysqld). If MySQL Cluster is new to you, think of MySQL Cluster as a storage engine for the network. Like InnoDB and MyISAM are storage engines for disks.
  • 23.
  • 24. Vertical: columns in additional tables
  • 25. ... sharding Custom Cluster Shard 1 ID % 2 = 0 Node 3 Cats Node 4 Dogs Shard 2 ID % 2 = 1
  • 26. The speaker says... Special requirements may make it necessary to build a custom cluster. For example, your application may require a sharding solution.
  • 27. On using any database cluster Part 2 - Required application changes
  • 28.
  • 29. Load balance within candidates
  • 31. Automatic fail over for High Availability Application using any DB cluster MySQL Server 1 MySQL Server 2 MySQL Server n
  • 32. The speaker says... All PHP applications talking to a cluster of MySQL database servers are facing the same tasks. Replacing a single database with a database cluster means changing the 1:1 relation between the application and the database into a 1:n relation . 1:n puts an additional task on the application: find the right n, find the right server to talk to.
  • 33. The plugin for all of you MySQL Server 1 MySQL Server 2 MySQL Server n
  • 34. The speaker says... PECL/mysqlnd_ms is a plugin for the MySQL native driver for PHP (mysqlnd) library . The mysqlnd library is part of PHP as of version 5.3. As of 5.4 mysqlnd is a default. All three PHP MySQL APIs (mysql, mysqli, PDO_MySQL) can be compiled to use the mysqlnd library, thus all existing PHP MySQL application are supported by the plugin. From an applications point of view a mysqlnd plugin can act as a transparent proxy. Depending on plugin an use case, no code changes are needed. A plugin is a drop-in solution.
  • 35. Plugin focus: MySQL Replication Part 3 - Tasks, solutions and limitations
  • 36.
  • 37. Replication filter (table based parititioning)
  • 38.
  • 39.
  • 41. The speaker says... The plugin is not limited to but optimized for MySQL Replication. The common clustering support feature set focusses on MySQL Replication. For example, statement redirection covers read/write splitting and MySQL Replication filter support. All read accesses are executed on the slave servers, all writes are performed on the master. MySQL Replication supports table based partitioning, replicating selected tables to selected slaves only. The plugin can be made aware of this.
  • 42. Speaking code /* Connection user handle represents a connection pool */ $mysqli = new mysqli( "hostname" , $user, $pw, $db, $port); /* Read/Write split, Load Balancing */ $res = $mysqli->query("SELECT 'Slave speaking' FROM DUAL"); var_dump($res->fetch_all(MYSQLI_ASSOC)); { "hostname" :{ "master":{ "host":"localhost" }, "slave":{ "slave_0":{"host":"192.168.2.27"} }, } }
  • 43. The speaker says... No code changes for basic use cases. Install the plugin following standard PHP PECL procedures. Create a configuration file. Enable the plugin and set its configuration file in your PHP configuration using mysqlnd_ms.enable=1 and mysqlnd_ms.ini_file=config.ini . In the plugin configuration create a section with the name of the host your application connects to, for example „hostname“. List master and slave servers. Any connection to „hostname“ is now handled by the plugin. Using any API.
  • 44. Some code changes /* Connection user handle represents a connection pool */ $mysqli = new mysqli("hostname", $user, $pw, $db, $port); / * R/W split: no SELECT, run on master */ $mysqli->query("SET @myrole='Master'); /* R/W split: overruled by SQL hint */ $sql = sprintf("/*%s*/%s", MYSQLND_MS_MASTER_SWITCH, "SELECT @myrole AS _role"); $mysqli->query($sql); var_dump($res->fetch_all(MYSQLI_ASSOC));
  • 45. The speaker says... Non-trivial use cases require support by the application. In other words: existing applications may need updates, new applications need to be designed appropriately. In the example a SQL hint is used to force executio of a SELECT statement on a MySQL Replication master server. There are SQL hints for running a statement on the master, on the slave and on the last used server. Alternative: new APIs call, no win compared to SQL hint.
  • 46. Why and when code changes $mysqli = new mysqli("hostname", $user, $pw, $db, $port); $mysqli->query("SET @myrole='Master'); $mysqli->query("SELECT myrole AS _role"); Without plugin With plugin
  • 47. The speaker says... An application not designed for use with a cluster assumes a connection handle always points to the same server (1:1), having the same state. When using the plugin a connection handle represents a pool of connections (1:n), potentially having different states. Connection switches may occur whenever a statement is executed. After the switch, the connection handle may point to a connection with a different state. Applications need to take care and hint the plugin.
  • 48.
  • 52. Session system variables and user variables
  • 53. Prepared statements (API and SQL PREPARE)
  • 56. Units of work from an application point of view
  • 57.
  • 59.
  • 60.
  • 61.
  • 62.
  • 64. The speaker says... A transaction is part of the connection state. For the duration of a transaction, server switches must be disabled. Transactions can be controlled by SQL statements and API calls. The plugin cannot monitor all of them with reasonable effort. Thus, you must hint the plugin. Disable load balancing by adding SQL hints (MYSQLND_MS_LAST_USED_SWITCH), you install your own load balancer logic (callback). As of PHP 5.4, the plugin monitors API calls to auto-disable switching.
  • 65.
  • 66. read-only: statement begins with slave SQL hint
  • 67. custom: callback picks slave for statement Redirection: Read/Write split Writes Reads (SELECT, SQL hint) Master Slave Slave Slave
  • 68. The speaker says... MySQL Replication is for read scale-out. Read-only statements shall go to slaves, writes to the master. The plugin automatically redirects a statement to a slave, if the statement begins with SELECT or the MYSQLND_MS_SLAVE_SWITCH SQL hint. All other statements are considered as write operations and executed on the master. By default SHOW or CALL will be run on the master. It is a design principle of the plugin that all automatic decisions can be overruled. A callback can be installed to pick the server for running a statement on, if needed.
  • 69.
  • 70. MySQL slaves can lag behind
  • 71. Most applications need current data after write Redirection: Master on write Writes, Reads after write Reads before write Master Slave Slave
  • 72. The speaker says... The master_on_write configuration option is an optimization of the R/W split. MySQL Replication slaves can lag behind the master because MySQL Replication is asynchronous. When using master_on_write all reads made after a write are executed on the master for the duration of the web request. Applications that perform a write often need to rely on fresh reads afterwards. Think of a web shop creating an order, writing to the master. If reading again from the order table, within the same web request, the application should access the master to make sure the previously written is available.
  • 73.
  • 74. The speaker says... MySQL Replication supports partitioning: slaves can be configured to replicate only selected databases and tables using MySQL Replication filter. Filter help to reduce the amount of internal replication messages. Less frequently accessed data can be replicated to only few slaves. In future versions, plugin can be made aware of the filtering rules for appropriate statement redirection. Every statement is analyzed to find a slave for reading. If no match, the plugin redirects the statement to the master.
  • 75.
  • 76. Random: pick random slave for each read
  • 77. Random once: pick random slave for all reads Load Balancing Writes Load Balancing of Reads Master Slave Slave Slave
  • 78. The speaker says... MySQL Replication is about read scale-out. The plugin helps distributing the load over the read-only slaves. The best and default load balancing stategy is the one with the lowest impact on the connection state: random once. Minimize switches for the life-span of your PHP script. Pick a random slave during startup and use it for all reads. The life-span of PHP is the of a web request. It is short. Two web requests will end up on two random slaves. Load balancing adaptive to server health can be achieved through a user-defined callback.
  • 79.
  • 80. Optional: failover=master, automatic Pooling: connection failover Writes Reads Master Slave Slave Slave
  • 81. The speaker says... Automatic and transparent failover is impossible with stateful connections. The plugin does not and cannot know the full state of a connection. Parts of the state are only known to the server. By default the plugin leaves it to the application to handle a connection failure. Like without the plugin, it is up to the application to decide what to do. Applications using stateless connections can enable automatic failover to the master.
  • 82.
  • 83. Reduce number of connections, reduce load Pooling: lazy connections Writes Reads Master Slave Slave Slave
  • 84. The speaker says... By default, to reduce the connection pool size of every PHP web request, connections are not opened before executing a statement. Imagine you have 50 concurrent web requests, you configured four slaves and, you are using random once. If not using lazy connections, the plugin opens 50 x 4 = 200 slave connections but uses at max 50 of them (random once). Every connection occupies resources – ever seen a PHP application opening connections before using? There are many badly written ones...
  • 86.
  • 87. THE END Credits: Andrey Hristov, Johannes Schlüter - Contact: ulf.wendel@oracle.com