MySQL Proxy: Architecture and concepts of misuse

MySQL Proxy: a MySQL toolbox Architecture and Concepts of Misuse Jan Kneschke MySQL Enterprise Tools mailto: jan@mysql.com

What is MySQL Proxy  Started Feb 2007  Current: MySQL Proxy 0.7.0  Source available on launchpad.net $ bzr branch lp:mysql-proxy  Foundation of the MEM 2.0 Agent  Fully support for MySQL Enterprise Customers  2 (+ 1) Developers, 1 QA

How it looks for a bird  Inspecting traffic  Logging, auditing  Routing connections and queries  Rewriting queries  Injection queries  “Be a good man in the middle”

How it looks for a frog MySQL Proxy Lua Scripting layer Proxy Plugin Admin Plugin Network Core (libmysql-proxy) libmysql libevent libmysql liblua -chassis -proto

The libs - chassis  Command Line Interface  Config-File support  Logging (event-log, syslog, stderr, logfiles)  Plugin Loading  Threading

The libs - Protocols  Documented …  http://forge.mysql.com/wiki/MySQL_Internals_Clie  MySQL Client Protocol  Binary Logs  MyISAM Row Format for RBR  Hides the “creativity” of the implementation

What you don't want to know  How to encode a 1234 (0x04d2) ?  In resultsets: 04 31 32 33 34  In Prepared depends on type: d2 04 00 00  In the rest of the protocol: 02 d2 04  Protocol is using a random mix of those encodings

Proof of Concept  mysql-binlog-dump  Similar to mysqlbinlog  First implementation of RBR decoding  mysql-myisam-dump  Decodes .frm and .myd files  Uncovers a little gem: screens

Let's wrap it  Lua  Small and fast  Easy to read and write  Easy to embed  All the libs are nicely wrapped

The core – libmysql-proxy  Protocol live-cycle (auth, old-auth, query)  Provides the plugin hooks  Global Lua scope  Network interface  MySQL Protocol encoding/decoding

Lua iterator for binlogs  Expose binlog events into lua  Reading and writing local binlog = require(“mysql.binlog”) local f = binlog.open(“jan-binlog.log”) for event in f:next() do if event.type == “QUERY_EVENT” then print(event.query.query) end end

Filtering  Filter statements from a binlog  Remove or rewrite Queries INSERT INTO answers VALUES ( 42, “the Answer” ) UPDATE answers SET answer = “Thanks for the Fish” WHERE id = 2 ALTER TABLE answers ADD INDEX (answer); DROP TABLE answers;

Merging binlogs  independent Masters like shards  Merge on timestamp

Druckbetankung  Use RBR to push data into a server  RBR is already in the native format  mysql-data-source  Convert input data to a live RBR-binlog-stream  mysqld  CHANGE MASTER TO … our data-source  START SLAVE

Slave Accelerator  Problem: Replication is synchronized, single thread  Solution: move replication out of the slave  Don't pull, push  Parallel connections

mysql-slave-accel  Read data from master  Maintain slave-state centrally  Rules to detect if data is independent  e.g. one connection per schema  Works transparently for SBR, tricky for RBR

Master Failover - Classic  Master on stand-by  On failure, CHANGE MASTER on all slaves  Promote the best slave

Automatic Master Failover

mysql-master-failover  Track active and passive master  Only forward data received from both masters  Track matching binlog-file and -position  Active dies  Trigger switch over  Slaves get a continuos binlog stream  States  Synched, Single, Out-of-Sync

Thanks  Code is on launchpad.net  http://launchpad.net/mysql-proxy/  Questions to: jan@mysql.com  weigon in #mysql-proxy on irc.freenode.net

MySQL Proxy: Architecture and concepts of misuse

by weigon

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 63

Statistics

MySQL Proxy: Architecture and concepts of misuse Presentation Transcript

http://www.phphatesme.com	54
http://www.slideshare.net	9