MySQL Proxy:
a MySQL toolbox
Architecture and
Concepts of Misuse

Jan Kneschke
MySQL Enterprise Tools
mailto: jan@mysql.com
What is MySQL Proxy

   Started Feb 2007
    Current: MySQL Proxy 0.7.0
   Source available on launchpad.net
    $ bzr ...
How it looks for a bird

    Inspecting traffic
     Logging, auditing
    Routing connections and queries
    Rewriti...
How it looks for a frog


                  MySQL Proxy

                Lua Scripting layer

         Proxy Plugin       ...
The libs - chassis

    Command Line Interface
    Config-File support
    Logging (event-log, syslog, stderr,
     log...
The libs - Protocols

   Documented …
    http://forge.mysql.com/wiki/MySQL_Internals_Clie
   MySQL Client Protocol
   ...
What you don't want to know

   How to encode a 1234 (0x04d2) ?
    In resultsets: 04 31 32 33 34
    In Prepared depen...
Proof of Concept

   mysql-binlog-dump
    Similar to mysqlbinlog
    First implementation of RBR decoding
   mysql-my...
Let's wrap it

    Lua
     Small and fast
     Easy to read and write
     Easy to embed
    All the libs are nicely...
The core – libmysql-proxy

   Protocol live-cycle (auth, old-auth, query)
   Provides the plugin hooks
   Global Lua sc...
Lua iterator for binlogs

    Expose binlog events into lua
    Reading and writing

  local binlog = require(“mysql.bin...
Filtering

    Filter statements from a binlog
    Remove or rewrite Queries
            INSERT INTO answers
           ...
Merging binlogs

   independent Masters like shards
   Merge on timestamp
Druckbetankung

   Use RBR to push data into a server
   RBR is already in the native format
   mysql-data-source
    ...
Slave Accelerator

   Problem: Replication is synchronized,
    single thread
   Solution: move replication out of the
 ...
mysql-slave-accel

   Read data from master
   Maintain slave-state centrally
   Rules to detect if data is independent...
Master Failover - Classic

   Master on stand-by
   On failure, CHANGE MASTER on
    all slaves
   Promote the best sla...
Automatic Master Failover
mysql-master-failover

   Track active and passive master
   Only forward data received from both
    masters
   Track ...
Thanks

   Code is on launchpad.net
    http://launchpad.net/mysql-proxy/
   Questions to: jan@mysql.com
   weigon in ...
Upcoming SlideShare
Loading in...5
×

MySQL Proxy: Architecture and concepts of misuse

3,100

Published on

MySQL Proxy presentation from FrosCon 2009

Published in: Education, Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,100
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
94
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

MySQL Proxy: Architecture and concepts of misuse

  1. 1. MySQL Proxy: a MySQL toolbox Architecture and Concepts of Misuse Jan Kneschke MySQL Enterprise Tools mailto: jan@mysql.com
  2. 2. What is MySQL Proxy  Started Feb 2007  Current: MySQL Proxy 0.7.0  Source available on launchpad.net $ bzr branch lp:mysql-proxy  Foundation of the MEM 2.0 Agent  Fully support for MySQL Enterprise Customers  2 (+ 1) Developers, 1 QA
  3. 3. How it looks for a bird  Inspecting traffic  Logging, auditing  Routing connections and queries  Rewriting queries  Injection queries  “Be a good man in the middle”
  4. 4. How it looks for a frog MySQL Proxy Lua Scripting layer Proxy Plugin Admin Plugin Network Core (libmysql-proxy) libmysql libevent libmysql liblua -chassis -proto
  5. 5. The libs - chassis  Command Line Interface  Config-File support  Logging (event-log, syslog, stderr, logfiles)  Plugin Loading  Threading
  6. 6. The libs - Protocols  Documented …  http://forge.mysql.com/wiki/MySQL_Internals_Clie  MySQL Client Protocol  Binary Logs  MyISAM Row Format for RBR  Hides the “creativity” of the implementation
  7. 7. What you don't want to know  How to encode a 1234 (0x04d2) ?  In resultsets: 04 31 32 33 34  In Prepared depends on type: d2 04 00 00  In the rest of the protocol: 02 d2 04  Protocol is using a random mix of those encodings
  8. 8. Proof of Concept  mysql-binlog-dump  Similar to mysqlbinlog  First implementation of RBR decoding  mysql-myisam-dump  Decodes .frm and .myd files  Uncovers a little gem: screens
  9. 9. Let's wrap it  Lua  Small and fast  Easy to read and write  Easy to embed  All the libs are nicely wrapped
  10. 10. The core – libmysql-proxy  Protocol live-cycle (auth, old-auth, query)  Provides the plugin hooks  Global Lua scope  Network interface  MySQL Protocol encoding/decoding
  11. 11. Lua iterator for binlogs  Expose binlog events into lua  Reading and writing local binlog = require(“mysql.binlog”) local f = binlog.open(“jan-binlog.log”) for event in f:next() do if event.type == “QUERY_EVENT” then print(event.query.query) end end
  12. 12. Filtering  Filter statements from a binlog  Remove or rewrite Queries INSERT INTO answers VALUES ( 42, “the Answer” ) UPDATE answers SET answer = “Thanks for the Fish” WHERE id = 2 ALTER TABLE answers ADD INDEX (answer); DROP TABLE answers;
  13. 13. Merging binlogs  independent Masters like shards  Merge on timestamp
  14. 14. Druckbetankung  Use RBR to push data into a server  RBR is already in the native format  mysql-data-source  Convert input data to a live RBR-binlog-stream  mysqld  CHANGE MASTER TO … our data-source  START SLAVE
  15. 15. Slave Accelerator  Problem: Replication is synchronized, single thread  Solution: move replication out of the slave  Don't pull, push  Parallel connections
  16. 16. mysql-slave-accel  Read data from master  Maintain slave-state centrally  Rules to detect if data is independent  e.g. one connection per schema  Works transparently for SBR, tricky for RBR
  17. 17. Master Failover - Classic  Master on stand-by  On failure, CHANGE MASTER on all slaves  Promote the best slave
  18. 18. Automatic Master Failover
  19. 19. mysql-master-failover  Track active and passive master  Only forward data received from both masters  Track matching binlog-file and -position  Active dies  Trigger switch over  Slaves get a continuos binlog stream  States  Synched, Single, Out-of-Sync
  20. 20. Thanks  Code is on launchpad.net  http://launchpad.net/mysql-proxy/  Questions to: jan@mysql.com  weigon in #mysql-proxy on irc.freenode.net
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×