MySQL in your laptop

1,419 views

Published on

Use multiple MySQL instances in a single host, customizing it on-the-fly

Published in: Technology

MySQL in your laptop

  1. 1. MySQL in a box (or in your laptop) Giuseppe Maxia QA Director, Continuent, Inc http://mysqlsandbox.net This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License. To view a copy of this license, visit http:// creativecommons.org/licenses/by-sa/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA. 1
  2. 2. about me - Giuseppe Maxia • • • • • • • a.k.a. The Data Charmer QA Director at Continuent, Inc Long time hacking with MySQL features Formerly, community manager,db consultant, designer, coder. A passion for QA and open source Blogger Oracle ACE Director • http://datacharmer.blogspot.com 2
  3. 3. An example from a few days ago • MySQL 5.7.2 was released during the keynote at MySQL Connect • Announced at 9:25 • Downloaded at 9:28 • Tested with replication at 9:35 • In this laptop 3
  4. 4. Another example from the same day • MySQL 5.7.2 labs preview were announced • Downloaded source at 10:30 • Built at 12:30 • Tested with replication at 12:35 • In this laptop 4
  5. 5. MySQL Sandbox lightning presentation 5
  6. 6. I used to install a lot of MySQL databases for testing MANUALLY 6
  7. 7. Then, I decided to use Perl ... 7
  8. 8. DBA pop quiz 8
  9. 9. HOW MANY KEYSTROKES to install a MySQL server? 11 msb 5.6.12 12345678901 9
  10. 10. HOW MANY KEYSTROKES to install 3 MySQL servers in replication? 12 msb r5.6.14 123456789012 10
  11. 11. HOW LONG does it take to install a MySQL server? < 5 seconds time msb 5.1.71 0m3.258s 11
  12. 12. HOW LONG does it take to install 3 MySQL servers in replication? < 10 seconds sb 5.1.71 0m6.314s 12
  13. 13. MySQL Sandbox http://mysqlsandbox.net • Free software (Perl under GPL) • One (unix) host • Many database servers • Single or multiple sandboxes • Customized scripts to use the servers • Standard or circular replication • Installs IN SECONDS 13
  14. 14. overview MySQL server Data DB2 MySQL server DB1 DB3 Data DATA DIRECTORY DB1 DB2 DB3 PORT SOCKET 14
  15. 15. overview MySQL server MySQL server Data DB2 SAME DATA DIRECTORY? DB1 DB3 /var/lib/mysql Data DB1 DB2 DB3 /var/lib/mysql DATA CORRUPTION 15
  16. 16. overview MySQL server MySQL server SAME PORT or SOCKET? 3306 /tmp/mysql.sock 3306 /tmp/mysql.sock DOES NOT START 16
  17. 17. The hard way Read the manual try to figure out what to change Install 17
  18. 18. The easy way MySQL Sandbox $ make_sandbox /path/to/mysql-5.1.54_linux.tar.gz # it should work always 18
  19. 19. The easier way Prepare once # some # preliminary # work Install many times $ make_sandbox 5.1.54 19
  20. 20. The easiest way Prepare once # some # preliminary # work Install many times $ sb 5.1.54 20
  21. 21. MySQL Sandbox VERSION MySQL server Data DB2 DB1 DB3 $SANDBOX_HOME/msb_VERSION/data VERSION /tmp/mysql_VERSION.sock 21
  22. 22. MySQL Sandbox 5.1.54 MySQL server Data DB2 DB1 DB3 $SANDBOX_HOME/msb_5_1_54/data 5154 /tmp/mysql_5154.sock 22
  23. 23. MySQL Sandbox 5.5.34 MySQL server Data DB2 DB1 DB3 $SANDBOX_HOME/msb_5_5_34/data 5534 /tmp/mysql_5534.sock 23
  24. 24. Single Sandbox MySQL server customized scripts start stop restart status clear send_kill use 24
  25. 25. Multiple Sandbox MySQL server customized scripts start_all stop_all restart_all m n1 status_all s1 n2 clear_all s2 n3 send_kill_a ll use_all 25
  26. 26. Where do you get it •from CPAN sudo su cpan MySQL::Sandbox •from launchpad http://launchpad.net/mysql-sandbox 26
  27. 27. The easy replication way MySQL Sandbox $ make_replication_sandbox /path/to/mysql-5.1.70_linux.tar.gz Prepare once # some # preparation Install many times $ make_replication_sandbox 5.1.70 27
  28. 28. default architecture $HOME /sandboxes opt expanded tarballs $SANDBOX_HOME mysql installed sandboxes $SANDBOX_BINARY 28
  29. 29. default architecture $HOME /sandboxes opt msb_5_0_91 mysql msb_5_1_48 rsandbox_5_1_48 master node1 node2 5.0.91 5.5.32 5.6.13 5.7.2 29
  30. 30. 30
  31. 31. creating a single sandbox make_sandbox /path/to/mysql-X.X.XX-OS.tar.gz 31
  32. 32. using a single sandbox # after # make_sandbox # /path/to/mysql-X.X.XX-OS.tar.gz $ cd $SANDBOX_HOME/msb_X_X_XX $ ./use 32
  33. 33. creating a single sandbox with a specific options file make_sandbox /path/to/mysql-X.X.XX-OS.tar.gz --my_file=/path/to/my.cnf 33
  34. 34. easily create a sandbox after the first one The long way $ cd $HOME/opt/mysql # $SANDBOX_BINARY $ gunzip -c /path/to/mysql-5.1.34-osx10.5-x86.tar.gz | tar -xf $ mv mysql-5.1.34-osx10.5-x86 5.1.34 $ make sandbox 5.1.34 34
  35. 35. easily create a sandbox after the first one The short way $ make_sandbox --export_binaries path/to/mysql-5.1.34-osx10.5-x86.tar.gz 35
  36. 36. starting a single sandbox $ cd $SANDBOX_HOME/msb_X_X_XX $ ./start 36
  37. 37. starting a single sandbox with temporary options $ cd $SANDBOX_HOME/msb_X_X_XX $ ./start --option=value $ ./restart --option=value $ ./start --key-buffer=20000000 37
  38. 38. creating a sandbox with custom port and directory $ make_sandbox 5.1.34 -- --sandbox_port=7800 --sandbox_directory=mickeymouse 38
  39. 39. creating a sandbox with automatic port checking $ make_sandbox 5.1.34 -- --check_port # if 5.1.34 is free # port=5134 # directory=msb_5_1_34 # else # port=5135 (or the first free) # directory=msb_5_1_34_a 39
  40. 40. create a replication sandbox $ make_replication_sandbox path/to/mysql-5.1.34-osx10.5-x86.tar.gz 40
  41. 41. create a circular replication sandbox $ make_replication_sandbox --circular=4 path/to/mysql-5.1.34-osx10.5-x86.tar.gz 41
  42. 42. changing port to an existing sandbox $ sbtool -o port -s /path/to/source/sandbox --new_port=XXXX 42
  43. 43. installing the innodb plugin $ sbtool -o plugin --plugin=innodb -s /path/to/source/sandbox 43
  44. 44. creating a replication sandbox with new base port $ make_replication_sandbox --replication_directory=newwdir --check_base_port 5.0.79 # # # # Creates a replication directory under $SANDBOX_HOME/newdir The previous one is preserved. No conflicts happen 44
  45. 45. creating a stand-alone master $ make_sandbox 5.5.16 -- --master # Creates a sandbox with binary log and # server-id enabled 45
  46. 46. creating a quick slave ~/sandboxes/rsandbox_5_5_16/m -e 'show variables like "port"' +---------------+-------+ | Variable_name | Value | +---------------+-------+ | port | 19771 | +---------------+-------+ $ make_sandbox 5.5.16 -- --slaveof='master_port=19771, master_host="xxxx"' # adds a slave to an existing master 46
  47. 47. MySQL Sandbox cookbook $ perldoc MySQL::Sandbox::Recipes 47
  48. 48. testing MySQL creatively 48
  49. 49. Breaking replication make_replication_sandbox 5.5.32 cd $HOME/sandboxes/rsandbox_5_5_32 ./m -e 'create table t1 (i int not null primary key)' test ./s1 -e 'insert into t1 values (1)' test ./m -e 'insert into t1 values (1)' test ./s1 -e 'show slave statusG' | grep 'Error|Running' Slave_IO_Running: Yes Slave_SQL_Running: No Last_Error: Error 'Duplicate entry '1' for 'PRIMARY'' on query. Default database: 'test'. Query: 'insert values (1)' Last_IO_Error: Last_SQL_Error: Error 'Duplicate entry '1' for 'PRIMARY'' on query. Default database: 'test'. Query: 'insert values (1)' key into t1 key into t1 49
  50. 50. Fixing replication ./s1 -e 'delete from t1 where i = 1' test ./s1 -e 'start slave' ./s1 -e 'show slave statusG' | grep 'Error|Running' Slave_IO_Running: Yes Slave_SQL_Running: Yes Last_Error: Last_IO_Error: Last_SQL_Error: 50
  51. 51. Making a slave lag ./s1 -e 'stop slave SQL_THREAD' ./s2 -e 'stop slave SQL_THREAD' ./m < heavy_load_commands.sql # 1 hour later ./s1 -e 'start slave SQL_THREAD' ./s2 -e 'start slave SQL_THREAD' 51
  52. 52. Options for replication nodes --master_options = name --slave_options = name --node_options = name --one_slave_options = name Options passed to the master Options passed to each slave Options passed to each node Options passed to a specific slave with the format "N:options" 52
  53. 53. customizing sandboxes during installation (1) make_replication_sandbox --node_options=--high_performance --one_slave_options='1:-c read-only' 5.5.32 53
  54. 54. customizing sandboxes during installation (2) ./use_all 'show variables like "%innodb%buffer%"' # master Variable_name Value innodb_buffer_pool_size 536870912 innodb_log_buffer_size 52428800 # server: 1: Variable_name Value innodb_buffer_pool_size 536870912 innodb_log_buffer_size 52428800 # server: 2: Variable_name Value innodb_buffer_pool_size 536870912 innodb_log_buffer_size 52428800 54
  55. 55. customizing sandboxes during installation (3) ./use_all 'show variables like "%read_only%"' # master Variable_name Value read_only OFF # server: 1: Variable_name Value read_only ON # server: 2: Variable_name Value read_only OFF 55
  56. 56. re-playing binary logs backup bin logs original server 1. new server 2. latest backup 5. make the new server a slave of the sandbox 6. remove sandbox 3. new sandbox 4. copy binlogs 56
  57. 57. Testing MySQL 5.7 make_sandbox --add_prefix=exp --export_binaries mysql-5.7.2-m12-osx10.7-x86_64.tar.gz # and later make_sandbox exp5.7.2 57
  58. 58. DEMO 58
  59. 59. Participate! 59
  60. 60. THANKS http://mysqlsandbox.net This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License. To view a copy of this license, visit http:// creativecommons.org/licenses/by-sa/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA. 60

×