High performance and high availability proxies for MySQL

www.mydbops.com mysqlsupport@mydbops.com
Presenters
Karthik P R ( CEO/ Founder Mydbops )
Kabilesh P R ( Database Consultant )
High Performance and High
Availability Proxies for MySQL

Mydbops is into MySQL Support and Consulting. It is founded by experts who have
scaled MySQL database at Yahoo! ,Percona and Datavail. We are providing an expert level
support and monitoring for MySQL databases and its related technologies like MariaDB ,
Percona ( also clustering ) . We support modern database technologies in MySQL which
includes Galera ( Clustering ), Group Replication , SQL aware Load balancers like
Maxscale / ProxySQL, MySQL Engine like InnoDB, TokuDB and MyRocks.
We offer seamless end to end DB solutions like migration, installation, up-gradation, design,
performance tuning, troubleshooting, monitoring, recovery with 24x7 on call support and
Remote DBA support inevitably decreasing the downtime by 99.9%. Our services will improve
your DB Performance and helps in saving cost on IT.
About Mydbops

Our Clients

Karthik P R
Founder of Mydbops. Has an experience of around 7 years in handling MySQL servers .
Handled large scale systems at Yahoo! ( 3 years ). Active MySQL speakers and blogger.
Currently providing solutions to many established startups and organisation to scale in
MySQL.
Email : prk@mydbops.com Twitter: @prkart
Kabilesh P R
Lead Consultant in Mydbops. Has an experience of about 5 years. Worked with large remote
database support company Datavail on multi customer platform. Expertise in monitoring of
MySQL and actively working with mysql on cloud
About US

● Need for Load Balancer
● SQL Awareness Load Balancers
● Introduction to Maxscale
● Maxscale use cases
● Introduction to ProxySQL
● ProxySQL use cases
● Maxscale Vs ProxySQL
Agenda

● Load balancer distributes the incoming traffic efficiently to underlying backend servers
● Provides efficient use of the underlying hardware
● Avoids single point of failures ie., High Availability
● Provides way for easy switchover and failover
● Additional Features
Need for a Load balancer

Types Of Load Balancers
● Reverse Proxies
○ HAProxy
○ Ngnix
● SQL aware Proxies
○ Maxscale
○ MySQL Router
○ ProxySQL
Need for a Load balancer

● SQL aware load balancer, forward the queries/statements by understanding the
underlying state of servers.
● Read/Write splitting.
● Support for external tools and scripts.
● Monitoring of backend servers.
● Query Caching.
● Data Streaming
SQL Aware Load Balancer

Maxscale

● Maxscale is an intelligent SQL Aware proxy.
● Built by MariaDB corporation
● Pluggable router
● Easy to configure
● Highly Scalable and flexible
● Provides High Availability
Introduction To Maxscale

● Maxscale comes with two license model they are GPL (Open Source License) and BSL
(Business Source License).
● Till version 1.4 Maxscale comes with GPL
● From version 2.0 Maxscale comes with BSL
● Usage Limitation of BSL : The usage of software is free for when you are using total or
less than of 3 server for production purposes.
● Change Date of BSL : 2019-01-01
“Business source is simply proprietary software released under a Microsoft-esque shared source license that magically
becomes fully open source after a period of time.”
License In Maxscale

● Maxscale Comes with a simple pluggable modular architecture
● Modules are loaded dynamically on demand
● The maxscale core provides the following
Network layer interface
Buffer management
Config Management
Synchronization
Maxscale Architecture

Maxscale Architecture

Maxscale Internal

● Maxscale support the below routing algorithms for various use cases
○ ReadConnRoute - Connection based load balancing
○ ReadWriteSplit - Read/Write aware statement based routing
○ SchemaRouter - Simple Sharding based on database level
○ Binlogrouter - As Binlog server
Maxscale Routers

● Maxscale monitor module monitors the underlying cluster/server and provides the status
information to the router based on which the query is routed to the server, the various
supported monitor modules include
○ MySQL Monitor(mysqlmon) - Monitors M-S replication and assign roles
○ Galera Monitor(galeramon) - Monitors the SYNC status of the each cluster node
○ NDBCluster Monitor(ndbclustermon) - Monitors the NDB cluster
○ Multi-Master Monitor(mmmon) - monitors the multi-master replication cluster.
Maxscale Monitor

● Configuration file to store the maxscale configurations (/etc/maxscale.cnf) default.
● Similar interface like mysql cmd line
● Has query parser similar to mysql server
● Default port for maxscaled 6603
● Default password is ‘mariadb’ for maxadmin (admin cmd line)
Maxscale Configuration

Maxscale Configuration
[MaxScale]
threads=auto
[Service Name]
type=service
router=
servers=server1,server2
user=
passwd=
routeroptions=
[listener]
type=listener
service=Service Name
protocol=MySQLClient
port=3306
socket=/tmp/cluster
[monitorName]
type=momitor
module=
servers=server1,server2
user=
passwd=
[server1]
type=server
address=127.0.0.1(IP)
port=3306
[CLI
type=service
router=cli
[CLI Listener]
type=listener
service=CLI
protocol=maxscaled
address=localhost
port=6603

Maxscale For Replication
Writes Reads

Maxscale For Galera Cluster
Writes Reads Reads

Maxscale As Binlog Router

Maxscale As Schema Router

Maxscale As Data Streamer

● Compression is not supported in mysql server handshake
● With R/W read queries routed to master when they are executed as transaction, in case
of prepared statement, procedure call and multiple statement in one query.
● Maxscale cannot manage authentication with % as hostname, the only accepted wildcard
will be with IP as ‘192.167.2.%’
● Has some limitation with client session variable
Maxscale Limitations

● From 2.1 maxscale accepts runtime changes to most of the config
● Support for Aurora Cluster (AWS)
● Selective data Masking
● Secure binlog server using SSL
● Great improvements with query caching.
● Inbuild failover management for promoting slave ( no more external scripts )
Maxscale Advanced Features

ProxySQL

● Built for DBA’s by a DBA.
● Major Contribution by René Cannaò (Founder)
● Winner MySQL Community Contributor (2017)
● Light weight application.
● Advanced Multi Core Architecture.
● Complete Open Source GPL License (GPL 3.0)
● Current Version 1.3.7 and 1.4 is Beta. ( 17-06-2017 )
ProxySQL Introduction

● More Granular Configuration
● SQL based Administration.
● Dynamic Configuration
● Advanced Metric Collections.
● Supports More Complex Topologies.
● Seamless Integrations.
● Manage database at large scale ( 1000’s )
ProxySQL Introduction

ProxySQL Architecture

● SQLite database to store config.
● MySQL Compatible interface.
● There is no real parser in ProxySQL
● Multi Layer Configuration
● Default port 6032
$mysql -u admin -padmin -h 127.0.0.1 -P6032
ProxySQL Configuration

Memory: Represent SQLite3 in memory
database
Runtime: In memory store of ProxySQL with
RUNTIME Config.
Disk: On Disk location of SQLite3. Contains
Persistent storage of config.
Config File: Load the config (proxysql.cnf)
when there is no disk file ( proxysql.db)

Loading configuration three levels:
SQL (writes ) are made to table in Memory.
Then load to RUNTIME ( Dynamically ) and
saved to DISK for persistence.
Eg)
Load MYSQL USERS TO RUNTIME;
SAVE MYSQL USERS TO DISK;

Scenario Loading a User
Host groups 101 - Writer , 102 -Reader
insert into mysql_users(username,password,active,default_hostgroup,default_schema) values
('writer_app','test',1,101,'sbtest');
insert into mysql_users(username,password,active,default_hostgroup,default_schema) values
('reader_app','test',1,102,'sbtest');
LOAD MYSQL USERS TO RUNTIME;
SAVE MYSQL USERS TO DISK;

ProxySQL Replication ( case 1 )

Loading Server Info:
INSERT INTO mysql_servers ( hostname,hostgroup_id,port,weight) VALUES ('192.168.1.1',101,3306,1000);
INSERT INTO mysql_servers (hostname,hostgroup_id,port,weight, max_replication_lag) VALUES
('192.168.1.2',102,3306,10000,2);
('192.168.1.3',102,3306,10000,2);
('192.168.1.4',102,3306,1,2);
INSERT INTO mysql_replication_hostgroups VALUES (101,102);
Loading Config
LOAD MYSQL SERVERS TO RUNTIME; SAVE MYSQL SERVERS TO DISK;

● ProxySQL can be Integrated with application server to reduce latency.
● Read_only flag defines the availability of readers
● Group of ProxySQL can be designed for complex Infra.
● Schedulers can be made for custom solutions.
● HA Solution for ProxySQL can be made with Keepalived.
● Prepared Statements are not supported in Read/write split

● ProxySQL has in built Query caching.
● Caching is based on the query Rules.
● Cache invalidation is through cache_ttl ( in mysql_query_rules)
● mysql_query_cache_size_MB variables controls it.
● mysql_threshold_resultset_size defines the Maximum size of can be buffered.
● stats_mysql_global has metric related to Query Cache too.
● More Granular control over caching ( Query , User , Schema , Hint )
Note : Query Cache inbuilt with MySQL is Retiring in MySQL 8.0
ProxySQL Caching ( case 2)

Reader - 102
ProxySQL - With Cache

● Caching Rules has to be made in Query rules. ( digest,cache_ttl )
● Rules are made based on digest ( stats_mysql_query_digest )
● TTL defines the invalidation in ms . ( cache_ttl)
● Top Queries can loaded based on Count of execution, Sum of execution time.
● Cache size helps in tuning it based on global stats.
● Background threads take care of memory cleanup and expired entries.
● It can replace Redis , memcache and other 3rd party caching for DB.

ProxySQL Caching Vs MySQL Query Cache from official proxysql website
ProxySQL Caching ( case 2 )

● Galera is cluster based on InnoDB engine.
● Developed by Codership and it is an open source.
● Synchronous replication ( Zero lag )
● Multi Threaded replication.
● Automated node Recovery in case of crash.
● Distributions - Percona XtraDB Cluster , MariaDB Cluster and MySQL ( with plugin ).
● Galera with ProxySQL makes a better High Availability.
ProxySQL For Galera ( case 3 )

ProxySQL For Galera ( case 3 )
Cluster - 103
ProxySQL ( Cluster )

● Inbuilt schedulers to supports
● Support third party schedulers
● Awares on Galera States during SST / IST
● Desync states in galera.
● Easy Maintenance with status ( ONLINE , SHUNNED,OFFLINE_SOFT,OFFLINE_HARD
)
ProxySQL For Galera ( case 3)

Integrations:
● ProxySQL is bundled in Percona XtraDB Cluster 5.7
● Proxy-admin tool by Percona to ease configuration.
● Orchestrator has integration for proxysql.
● Ansible modules helps in easy deployment ( 7 modules are present )
● Prometheus exporters for metrics.
● Integrated with PMM ( Grafana based Dashboards )www.mydbops.com mysqlsupport@mydbops.com
ProxySQL Integration & Deployment

● Query Rewrite
● Sharding ( Schema , User , Query , Hints )
● Persistent Connections
● NDB Cluster Support
● Mirroring
● Query Filter
● Data Masking and more
ProxySQL Other use cases

Maxscale Vs ProxySQL
Features Maxscale ProxySQL
License BSL/GPL GPL
Group Replication Not Supported Supported
Galera Support Supported Supported
Read/Write Split Query Module ( Classifier ) REGEX
Aurora Support Supported(2.1) Not Supported
Binlog router Supported Not Supported
Kafka Streaming Supported Not Supported

Maxscale Vs ProxySQL
Features Maxscale ProxySQL
Multiplexing Not Supported Supported
Caching Control Not Supported Supported
Query Digest Not Supported Supported
Ansible No Modules Ansible Modules
NagiOS Plugins are present No Official
Prometheus No exporters Exporters are present
Clustering Not Supported Not Supported

Queries
Reach US
www.mydbops.com
mysqlsupport@mydbops.com
080 - 48505683

Thank You

High performance and high availability proxies for MySQL

High performance and high availability proxies for MySQL

More Related Content

What's hot

Similar to High performance and high availability proxies for MySQL

More from Mydbops

Recently uploaded

High performance and high availability proxies for MySQL