MySQL Proxy presentation from FrosCon 2009

1. MySQL Proxy:
a MySQL toolbox
Architecture and
Concepts of Misuse
Jan Kneschke
MySQL Enterprise Tools
mailto: jan@mysql.com

2. What is MySQL Proxy
 Started Feb 2007
 Current: MySQL Proxy 0.7.0
 Source available on launchpad.net
$ bzr branch lp:mysql-proxy
 Foundation of the MEM 2.0 Agent
 Fully support for MySQL Enterprise
Customers
 2 (+ 1) Developers, 1 QA

3. How it looks for a bird
 Inspecting traffic
 Logging, auditing
 Routing connections and queries
 Rewriting queries
 Injection queries
 “Be a good man in the middle”

4. How it looks for a frog
MySQL Proxy
Lua Scripting layer
Proxy Plugin Admin Plugin
Network Core (libmysql-proxy)
libmysql libevent libmysql liblua
-chassis -proto

5. The libs - chassis
 Command Line Interface
 Config-File support
 Logging (event-log, syslog, stderr,
logfiles)
 Plugin Loading
 Threading

6. The libs - Protocols
 Documented …
 http://forge.mysql.com/wiki/MySQL_Internals_Clie
 MySQL Client Protocol
 Binary Logs
 MyISAM Row Format for RBR
 Hides the “creativity” of the
implementation

7. What you don't want to know
 How to encode a 1234 (0x04d2) ?
 In resultsets: 04 31 32 33 34
 In Prepared depends on type: d2 04 00 00
 In the rest of the protocol: 02 d2 04
 Protocol is using a random mix of those
encodings

8. Proof of Concept
 mysql-binlog-dump
 Similar to mysqlbinlog
 First implementation of RBR decoding
 mysql-myisam-dump
 Decodes .frm and .myd files
 Uncovers a little gem: screens

9. Let's wrap it
 Lua
 Small and fast
 Easy to read and write
 Easy to embed
 All the libs are nicely wrapped

10. The core – libmysql-proxy
 Protocol live-cycle (auth, old-auth, query)
 Provides the plugin hooks
 Global Lua scope
 Network interface
 MySQL Protocol encoding/decoding

11. Lua iterator for binlogs
 Expose binlog events into lua
 Reading and writing
local binlog = require(“mysql.binlog”)
local f = binlog.open(“jan-binlog.log”)
for event in f:next() do
if event.type == “QUERY_EVENT” then
print(event.query.query)
end
end

12. Filtering
 Filter statements from a binlog
 Remove or rewrite Queries
INSERT INTO answers
VALUES ( 42, “the Answer” )
UPDATE answers
SET answer = “Thanks for the Fish”
WHERE id = 2
ALTER TABLE answers
ADD INDEX (answer);
DROP TABLE answers;

13. Merging binlogs
 independent Masters like shards
 Merge on timestamp

14. Druckbetankung
 Use RBR to push data into a server
 RBR is already in the native format
 mysql-data-source
 Convert input data to a live RBR-binlog-stream
 mysqld
 CHANGE MASTER TO … our data-source
 START SLAVE

15. Slave Accelerator
 Problem: Replication is synchronized,
single thread
 Solution: move replication out of the
slave
 Don't pull, push
 Parallel connections

16. mysql-slave-accel
 Read data from master
 Maintain slave-state centrally
 Rules to detect if data is independent
 e.g. one connection per schema
 Works transparently for SBR, tricky for
RBR

17. Master Failover - Classic
 Master on stand-by
 On failure, CHANGE MASTER on
all slaves
 Promote the best slave

18. Automatic Master Failover

19. mysql-master-failover
 Track active and passive master
 Only forward data received from both
masters
 Track matching binlog-file and -position
 Active dies
 Trigger switch over
 Slaves get a continuos binlog stream
 States
 Synched, Single, Out-of-Sync

20. Thanks
 Code is on launchpad.net
 http://launchpad.net/mysql-proxy/
 Questions to: jan@mysql.com
 weigon in #mysql-proxy on
irc.freenode.net