This document describes MySQL Proxy, a transparent proxy for MySQL that allows requests to be transformed, handled, or blocked. It provides features like load balancing, failover, query rewriting, profiling, and auditing through an embedded Lua scripting language. The goal is to make MySQL more flexible and add capabilities like read/write splitting without modifying applications or MySQL itself.

1. MySQL Proxy
Making MySQL more flexible
Jan Kneschke
jan@mysql.com

2. MySQL Proxy
●
proxy­servers forward requests to backends
and can
– transform, handle or block them
●
released under the GPL
– see http://forge.mysql.com/wiki/MySQL_Proxy
●
developed as part of the Enterprise Tools since
February 2007

3. Design Decisions
●
goal is to be transparent to the application layer
●
supports all platforms and languages
●
designed to handle thousands of parallel
connections (c10k)
●
uses a embedded scripting language for
customizations

4. Transparency
●
SHOW WARNINGS can be worked around with
Query Injection
●
SELECT USER() shows the connected user
(the proxy, not the client) which can be
corrected with result­set rewriting
●
host auth against the MySQL server

5. Latency
●
early tests via localhost
●
same script run directly and through the proxy
●
latency per mysql­packet: 0.4ms
●
ping RTT on 1Gbit: 0.1ms

6. Load Balancing
●
load balancing distributes the load across
several slaves
●
Shortest Queue First is default
– send new connections to the server with the least
number of open connections

7. Fail Over
●
dead host are detected
●
taking out of load balancing for 2min
●
uses custom load balancers to decide how to
handle a dead host
– hot + standby
– uses load balancing

8. Removing SPoF
●
one Proxy == Single Point of Failure
●
use external Heartbeat (linuxha.org) or
●
2 LB proxies + 1 Host Selecting Proxy per
application server

9. Failsafe Load Balancing

10. Flexibility
●
proxy embeds LUA
●
allows analyzing and manipulating packets
– Inspection
– Rewriting
– Blocking
– Injection

11. LUA
●
PiL http://lua.org/manual/5.1/
●
embedded, simple, efficient
●
can do OO­like programming
●
has scalars, tables, metatables and anonymous
functions

12. Query Rewriting
●
Macro Packages (ls, cd, who, ...)
●
tagging queries with SQL_CACHE
●
migrating table­names and SQL dialects
●
turn EXPLAIN UPDATE|DELETE into
equivalent EXPLAIN SELECT

13. Query Profiling
●
SHOW SESSION STATUS around a Query
Exec_time: 6749 us
.. Handler_read_rnd_next = 252
.. Handler_write = 252
.. Select_scan = 1

14. Query Statistics
●
Normalize Queries to track query usage
●
Count Table and Index usage
●
Optimize Query Cache Usage by injecting
SQL_CACHE in cachable queries
●
see lib/analyze­queries.lua

15. Auditing
●
Diagnostic Auditing
●
track which user+ip run which query or
accessed which objects when
●
assign query­costs
●
log gathered information in a central place
●
see lib/auditing.lua

16. Global Transaction ID
●
Inject a counter in all transactions
●
Answers questions like
– which slave is most current
– can I read from this slave, or do I have to read from
master
– you name it

17. Connection Pooling
●
reusing open connections between proxy and
server
●
reduces concurrency on the MySQL Server
●
external connection pool for PHP

18. Statement Routing
●
split the query stream into reading and writing
– READs go to the slaves
– WRITEs and transactions to the master
●
automatic scale­out
●
sharding

19. Tokenizer
●
turns a SQL query into a token stream
●
not a full parser, just a tokenizer for speed
reasons
●
understands KEYWORDS, /*comments*/,
“strings”, 123 and `literals`
●
later we'll add support for SQL modes

20. normalizing Queries
1: { TK_SQL_SELECT, select }
2: { TK_STAR, * }
3: { TK_SQL_FROM, from }
4: { TK_LITERAL, t1 }
5: { TK_SQL_WHERE, where }
6: { TK_LITERAL, id }
7: { TK_EQ, = }
8: { TK_INTEGER, 1 }
normalized query: SELECT * FROM `t1` WHERE
`id` = ?

21. Libraries
●
auto­config ●
parser
– SET GLOBAL ... – extract tablenames
●
balance ●
tokenizer
– load balancers – normalize()
●
commands – cleanup queries
– parse MySQL
Command Packets

22. Internals – LUA scripting
●
proxy.* is the namespace
●
proxy.connection.* is the current
connection
●
proxy.backends[...] are the backends
●
proxy.global.* is the global table
●
proxy.global.config.* is used for the
config

23. Internals ­ Scope
●
Each connection has its own script scope
●
proxy.global.* to share data between
connections
●
use local to make variables local to the
function
●
use package.seeall() to export functions
from modules

24. Internals ­ Threading
●
the global scope and threading don't play nice
by default
●
http://www.cs.princeton.edu/~diego/professional/lua
●
patches lua to apply mutexes around variable
access

25. Internals – Script Cache
●
0.6.0 we reload the script on each connection
start
●
adding a script cache with mtime check
●
lua_pushvalue(L, ­1) does the trick

26. Roadmap
●
to be released 0.6.0
– tokenizer
– read­write splitting
– Query Statistics
●
later
– parallel Queries
– proxy initiates connections

27. LUA ­ Gotchas
●
only false and nil are !true, 0 is true
●
to say “not equal” you use ~=
●
there are no shortcuts
– no a++, no a *= 4, ...
– no a > b ? a : b (there is “(a > b) and a or b)