LSPE @Yahoo mysql-proxy <ul><li>By Farhad Saberi </li></ul>- Overview and architecture - Making The case for a proxy - Lua...
Overview and Architecture <ul><li>Same idea as any other proxy. Sits between the application and its Database: </li></ul>
Overview and Arhitecture Read only Load Balancing with multiple –proxy-read-only-backend-addresses option
Making The case for mysql-proxy <ul><li>Easier sites management.  </li><ul><li>Hundreds of connection definitions use the ...
No need to restart for configuration or behavioral changes. It all happens in real time. </li></ul></ul>
Making the case for mysql-proxy <ul><li>Flexible, Powerful and invisible: </li><ul><li>intercept, tokenize and modify quer...
Deny/allow queries based on DB or DB User.
Deny/allow based on a regex
Construct your own results set and return it to the application without ever actually querying the backend database.
LUA embedded </li></ul></ul>
Making the case for a proxy <ul><li>It's fast and stable: </li><ul><li>Small daemon, about 35 MB in VSZ compared to a pref...
Coded in C
implements connection pooling
Doesn't need to do NS lookups every time
Has never “crashed” on me so far! </li></ul></ul>
Making the case for mysql-proxy Open source.
LUA <ul><li>Simple scripting language
Small: its whole distribution fits in a floppy disk.
Extensible; it is easy to interface with C code. </li></ul>
LUA <ul><li>Widely used in games: </li></ul>
Upcoming SlideShare
Loading in …5
×

Mysql Proxy Presentation Yahoo

1,914 views
1,855 views

Published on

Farhad Saberi's lightning talk from http://www.meetup.com/SF-Bay-Area-Large-Scale-Production-Engineering/calendar/15464663/

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,914
On SlideShare
0
From Embeds
0
Number of Embeds
27
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Mysql Proxy Presentation Yahoo

  1. 1. LSPE @Yahoo mysql-proxy <ul><li>By Farhad Saberi </li></ul>- Overview and architecture - Making The case for a proxy - Lua - Lua examples - The admin interface - currently known issues
  2. 2. Overview and Architecture <ul><li>Same idea as any other proxy. Sits between the application and its Database: </li></ul>
  3. 3. Overview and Arhitecture Read only Load Balancing with multiple –proxy-read-only-backend-addresses option
  4. 4. Making The case for mysql-proxy <ul><li>Easier sites management. </li><ul><li>Hundreds of connection definitions use the same host and port. DB Host connection changes are done at one place only.
  5. 5. No need to restart for configuration or behavioral changes. It all happens in real time. </li></ul></ul>
  6. 6. Making the case for mysql-proxy <ul><li>Flexible, Powerful and invisible: </li><ul><li>intercept, tokenize and modify queries
  7. 7. Deny/allow queries based on DB or DB User.
  8. 8. Deny/allow based on a regex
  9. 9. Construct your own results set and return it to the application without ever actually querying the backend database.
  10. 10. LUA embedded </li></ul></ul>
  11. 11. Making the case for a proxy <ul><li>It's fast and stable: </li><ul><li>Small daemon, about 35 MB in VSZ compared to a preforked apache child being 50 to 100 MB.
  12. 12. Coded in C
  13. 13. implements connection pooling
  14. 14. Doesn't need to do NS lookups every time
  15. 15. Has never “crashed” on me so far! </li></ul></ul>
  16. 16. Making the case for mysql-proxy Open source.
  17. 17. LUA <ul><li>Simple scripting language
  18. 18. Small: its whole distribution fits in a floppy disk.
  19. 19. Extensible; it is easy to interface with C code. </li></ul>
  20. 20. LUA <ul><li>Widely used in games: </li></ul>
  21. 21. LUA <ul><li>Mysql-proxy is the application code and Lua is its extension language . You change a running mysql-proxy's behavior and variables in a single Lua script in real time (no restart required). </li></ul>
  22. 22. Lua examples <ul>Mysql-proxy starts with </ul>– proxy-backend-addresses=db1.host.net, db2.host.net, db3.host.net Choose your backend in connect_server() Function connect_server() proxy.connection.backend_ndx= 2 End And you will always connect to db2.host.net. If you change 2 to 3, you will always connect to db3. Note: Lua's array starts at 1, !0
  23. 23. Lua examples Deny all queries to any backend DB: function read_query(packet) if string.byte(packet) == proxy.COM_QUERY then local fields = { } local rows = { } fields = { { name = “ message from DBA ”, type =proxy.MYSQL_TYPE_STRING}, } rows[1] = { “ maintenance ”} proxy.respose = { type = proxy.MYSQLD_PACKET_OK, resultset = { fields = fields, rows = rows } } return proxy.PROXY_SEND_RESULT –means bypass backend, return to client now end end
  24. 24. Lua examples <ul>For any query issued, this will be the result: mysql> show tables; # or anything else +--------------------------+ | message from DBA | +--------------------------+ | maintenance | +---------------------------+ </ul>
  25. 25. Lua example <ul>denying access to just one table using a regex. Return mysql error msg to client. Function read_query(packet) if packet:byte() == proxy.COM_QUERY then local query= packet:sub(2) print (“query: “ .. query ) – debugging purpose if string.match(query:lower(), “ from%s+.*my_table ” ) then proxy.response.type = proxy.MYSQLD_PACKET_ERR proxy.response.errmsg = “ access denied to table my_table ” return proxy.PROXY_SEND_RESULT end end end </ul>
  26. 26. Lua example <ul>Running any query that would match “from%s+.* my_table” will return ERROR 1105 (07000): access denied to table my_table </ul>
  27. 27. Lua example <ul>Some other things you can do: <li>Start/stop mysqld backend.
  28. 28. Choose your backend DB based on which IP address, or which mysql user the client has.
  29. 29. Modify result set by adding or removing a column from the returned table.
  30. 30. Time your queries. Gather statistics.
  31. 31. Your limits are that of the Lua language. </li></ul>
  32. 32. The admin interface <ul><li>You can monitor and show internal structures of the running mysql-proxy process by programming your own admin interface in Lua.
  33. 33. Start mysql-proxy with admin options:
  34. 34. --admin-address=127.0.0.1:10008
  35. 35. --admin-username=root
  36. 36. – admin-password=password
  37. 37. --admin-lua-script=/mysql-proxy/share/admin/admin.lua </li></ul>
  38. 38. The admin interface Function read_query(packet) If packet:byte() ~= proxy.COM_QUERY then return_error(“ [admin] we only handle COM_QUERY ”); --user defined end Local rows = { } local fields = { } – no need for a semicolon in Lua if packet:sub(2) == “ls” then – if query is an ls fields = { { name = “ backend_ndx ”, type = proxy.MYSQL_TYPE_LONG }, { name = “ address ” , type = proxy.MYSQL_TYPE_STRING } , { name = “ connected_clients ”, type = proxy.MYSQL_TYPE_LONG}, } for i = 1 , #proxy.global.backends do Local b = proxy.global.backends[i] rows[#rows + 1 ] = { i, b.dst.name, b.connected_clients } end end … Now build proxy.response and return with proxy.PROXY_SEND_RESULT
  39. 39. The admin interface Querying the previously Lua coded interface will return: mysql>msyql -u root -ppassword -h 127.0.0.1 -P 10008 mysql> ls; +------------------+--------------------------+-------------------------+ | backend_ndx | address | connected_clients | +------------------+--------------------------+-------------------------+ | 1 | 192.168.1.9:3309 | 23 | +------------------+--------------------------+-------------------------+ | 2 | 192.168.1.10:3309 | 142 | +------------------+--------------------------+-------------------------+
  40. 40. Currently known issues <ul><li>Read/Write splitting does not work if used by more than one client on the same proxy process. This is a known issue and I'm assuming it is being worked on.
  41. 41. A bit complicated. Everything has to be coded by you.
  42. 42. You need to know Lua well. (not a bad thing though) </li></ul>

×