ProxySQL for MySQL

www.mydbops.com mysqlsupport@mydbops.com
Presenter
Karthik P R
Founder Mydbops
ProxySQL for MySQL
An Efficient Load Balancer

● MySQL Consulting
● MySQL Support
● Remote DBA support ( 24*7 )
● Expert MySQL solutions
● Percona Cluster Support
● MySQL Load balancer Support
About Mydbops

Our Clients

● Founder Mydbops & MySQL Solution Architect .
● 7 years experience in handling MySQL and its forks.
● Currently Consulting Organizations to scale in MySQL
● Worked on large scale systems at Yahoo!
● MySQL Speaker and Blogger.
Contact : prk@mydbops.com
Twitter : @prkart
About Me

● Need for Load Balancer
● SQL Aware Load Balancers
● ProxySQL Introduction.
● ProxySQL Configuration
● ProxySQL for MySQL Replication.
● ProxySQL Caching
● ProxySQL Mirroring
● ProxySQL Monitoring and deployment
● Other Features
Agenda

● High Availability of servers
● Performance of servers
● Avoid Complexity at application
● Better Usage of Hardware
● Additional features
Need for a Load balancer

Types Of Load Balancers
● Reverse Proxies
○ HAProxy
○ Ngnix
● SQL aware Proxies
○ Maxscale
○ MySQL Router
○ ProxySQL

Reverse Proxies ( Eg, HAProxy Scenario )
● HAProxy will split the connections to the read
servers based on the algorithms ( RR/ LC )
● It serves the connections to slaves
● The slave connection are made on diff port.
● Works on TCP ( Layer 7 )

Reverse Proxies ( Eg, HAProxy Scenario )
● When everything is running fine, load balancer
will split connections based on Round Robin
or Least recently used algorithms.
● It’s good for application not for database.
● In case if any one of the slave IO thread stopped,
it won’t stop sharing queries.
● It lacks SQL Intelligence.

SQL aware Proxies ( Eg, ProxySQL Scenario )
● ProxySQL make health checks for each servers in
the loop.
● It’s database friendly which removes the failed
server from load balancer.
● In case if any one of the slave IO thread stopped,
it will take out from load balancer until the issue
get resolved.

● Designed for DBA’s
● Awareness MySQL states
● Read /Write Split
● Support for external tools and plugins
● Health of Database on query basics
● Routing of Queries
● Query Caching
SQL Aware Load Balancers

● Built for DBA’s by a DBA.
● Major Contribution by René Cannaò (Founder)
● Winner MySQL Community Contributor (2017)
● Light weight application.
● Advanced Multi Core Architecture.
● Complete Open Source GPL License (GPL 3.0)
● Current Version 1.3.6 and 1.4 is Beta. ( 09-06-2017 )
ProxySQL Introduction

● More Granular Configuration
● SQL based Administration.
● Dynamic Configuration
● Advanced Metric Collections.
● Supports More Complex Topologies.
● Seamless Integrations.
● Manage database at large scale ( 1000’s )

● REGEX Based Routing
● Support for Group Replication ( 1.4)
● Query Rewrite and Mirroring
● Easy Integration with Ansible Module
● PMM Graphs with ProxySQL Exporters

ProxySQL Architecture

Query Cache & Rewrite:
● Caching of queries & result
● Caching or mirroring
● In memory store and pattern based
User Authentication
● User Credentials of ProxySQL connectivity
● Max connections limitation & Throttling
● Credentials are encrypted

Connection Pool
● Reuses the connection and reduces the overhead
● Automatic retry of queries
● Connection Multiplexing
● Failover management
Monitoring :
● Collect metrics and monitor server healths
● Asynchronous replication support
● 1.4 has support for Group replication
● Monitor read_only variables ( Reader host groups)

Query Processor:
● Based on Query rules defined
● Timeout and delay
● Queries to be cached
● Hostgroup Targets
● Mirroring and firewall ( query blocking )

Query rules:
● Regex on Query
○ RE2 ( 1.3 )
○ PCRE ( Default) & RE2 (1.4)
● Fine granular control on incoming traffic based on
○ Username
○ Host group
○ Source IP and Database name
○ Digest ( stats_mysql_query_digest_reset )
○ Bind Address and port

Host Groups Manager
● All Backend or Grouped into hostgroups
● Hostgroups has logical function
● Tracks Server status
● Integrated with Connection pool
Eg)
HGO : Write Master ( Only one Server )
HG1 : Read Slaves ( One or Many Servers )
HG2 : Reporting Slave ( One or Many Servers )

● SQLite database to store config.
● MySQL Compatible interface.
● There is no real parser in ProxySQL
● Multi Layer Configuration
● Default port 6032
$mysql -u admin -padmin -h 127.0.0.1 -P6032
ProxySQL Configuration

Tables in ProxySQL ( Main )
Global_variables Config of proxysql
Mysql_query_rules Rules for R/w split,Caching,routing,etc
Mysql_servers Info of all MySQL Servers
Mysql_replication_hostgroups Reader_hostgroup and writer_hostgroup groups
Mysql_users Defines MySQL user to connect
DB’s
Scheduler Jobs running at regular interval
(Custom scripts)www.mydbops.com mysqlsupport@mydbops.com

Memory: Represent SQLite3 in memory
database
Runtime: In memory store of ProxySQL with
RUNTIME Config.
Disk: On Disk location of SQLite3. Contains
Persistent storage of config.
Config File: Load the config (proxysql.cnf)
when there is no disk file ( proxysql.db)

Loading configuration three levels:
SQL (writes ) are made to table in Memory.
Then load to RUNTIME ( Dynamically ) and
saved to DISK for persistence.
Eg)
Load MYSQL USERS TO RUNTIME;
SAVE MYSQL USERS TO DISK;

Scenario Loading a User
Host groups 101 - Writer , 102 -Reader
insert into mysql_users(username,password,active,default_hostgroup,default_schema) values
('writer_app','test',1,101,'sbtest');
insert into mysql_users(username,password,active,default_hostgroup,default_schema) values
('reader_app','test',1,102,'sbtest');
LOAD MYSQL USERS TO RUNTIME;
SAVE MYSQL USERS TO DISK;

Replication ( Common Use case Scenario )
● Read write Splitting
● Read scaling
● Easy Maintenance with status ( ONLINE , SHUNNED,OFFLINE_SOFT,OFFLINE_HARD )
● Zero downtime Failover
● Avoid stale data at slaves caused by lag ( max_replication_lag )
● REGEX Based Routing
● Do not support re-slaving or promotion ( mysqlrpladmin , MHA , Orchestrator )
ProxySQL Replication

Loading Server Info:
INSERT INTO mysql_servers ( hostname,hostgroup_id,port,weight) VALUES ('192.168.1.1',101,3306,1000);
INSERT INTO mysql_servers (hostname,hostgroup_id,port,weight, max_replication_lag) VALUES
('192.168.1.2',102,3306,10000,2);
('192.168.1.3',102,3306,10000,2);
('192.168.1.4',102,3306,1,2);
INSERT INTO mysql_replication_hostgroups VALUES (101,102);
Loading Config
LOAD MYSQL SERVERS TO RUNTIME; SAVE MYSQL SERVERS TO DISK;

Loading Query Rules:
UPDATE mysql_users SET default_hostgroup=101; --- All Queries Goes to writer group
INSERT INTO mysql_query_rules
(username,destination_hostgroup,active,retries,match_digest,apply)values(’writer_app’,101,1,3,'^SELECT.*FOR
UPDATE',1);
INSERT INTO mysql_query_rules
(username,destination_hostgroup,active,retries,match_digest,apply)values(‘reader_app’,102,1,3,'^SELECT ',1);
INSERT INTO mysql_query_rules (username,destination_hostgroup,active) values(’reader_app’,102,1);
Loading Config
LOAD MYSQL QUERY RULES TO RUNTIME; SAVE MYSQL QUERY RULES TO DISK;

● ProxySQL can be Integrated with application server to reduce latency.
● Read_only flag defines the availability of readers
● Group of ProxySQL can be designed for complex Infra.
● Schedulers can be made for custom solutions.
● HA Solution for ProxySQL can be made with Keepalived.
● Prepared Statements are not supported in Read/write split

● ProxySQL has in built Query caching.
● Caching is based on the query Rules.
● Cache invalidation is through cache_ttl ( in mysql_query_rules)
● mysql_query_cache_size_MB variables controls it.
● mysql_threshold_resultset_size defines the Maximum size of can be buffered.
● stats_mysql_global has metric related to Query Cache too.
Note : Query Cache inbuilt MySQL is Retiring in MySQL 8.0
ProxySQL Caching

ProxySQL Caching Vs MySQL Query Cache from official proxysql website
ProxySQL Caching

● Caching Rules has to be made in Query rules. ( digest,cache_ttl )
● Rules are made based on digest ( stats_mysql_query_digest )
● TTL defines the invalidation in ms . ( cache_ttl)
● Top Queries can loaded based on Count of execution, Sum of execution time.
● Cache size helps in tuning it based on global stats.
● Background threads take care of memory cleanup and expired entries.
ProxySQL Caching

● ProxySQL Mirroring
Will make exact query mirroring to the testing server.
Can be tested without affecting live production database.
Extensions to mysql_query_rules
Table mysql_query_rules was modified to add 2 more columns:
mirror_flagOUT
mirror_hostgroup
ProxySQL Mirroring

● The green arrow represents live production
traffic that is sent to the production database
server, which sends all responses back to the
application as if nothing changed.
● ProxySQL issues the mirrored query, represented
by the violet arrow, independently without impact
to the application, collecting metrics and errors
for both queries.
ProxySQL Mirroring

● ProxySQL stores metrics and digest info
● STATS and Monitor DB stores the information.
● Response time of queries , Connections usage , stats on traffic
● SHOW MYSQL STATUS ( Query cache, Buffer, Connections )
● Connection pool usage stats from stats_mysql_connection_pool table.
● ProxySQL exporters for Prometheus helps integration with PMM .
ProxySQL Monitoring

ProxySQL Monitoring
Query Routed

ProxySQL Monitoring
Client Connections

ProxySQL Monitoring
Endpoint Status

● Alerting can be made on Grafana 4 ( Slack/Email)
● Datadog has monitoring for ProxySQL.
● Monitor DB has info regarding
○ Connect log
○ Ping log
○ Replication lag log
○ Read only log
● ProxySQL Lacks direct NagiOS integration
ProxySQL Monitoring

● Large scale deployment can be made with Ansible
● Ansible Modules
○ proxysql_backend_server
○ proxysql_global_variables
○ proxysql_manage_config
○ proxysql_mysql_users
○ proxysql_query_rules
○ proxysql_replication_hostgroup
○ proxysql_scheduler
ProxySQL Deployment & Integration

Integrations:
● ProxySQL is bundled in Percona XtraDB Cluster 5.7
● Proxy-admin tool by Percona to ease configuration.
● Orchestrator has integration for proxysql.
ProxySQL Deployment & Integration

● Query Rewrite
● Sharding
● Persistent Connections
● NDB Cluster Support
● Galera Support Native With Schedulers .
● Query Filter
● Data Masking and more
ProxySQL Other features

Queries

Contact US
www.mydbops.com
mysqlsupport@mydbops.com
080 - 4850 5683

Thank You

ProxySQL for MySQL

ProxySQL for MySQL

More Related Content

What's hot

Similar to ProxySQL for MySQL

More from Mydbops

Recently uploaded

ProxySQL for MySQL