Oreilly Webcast Jan 09, 2009

1,312 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,312
On SlideShare
0
From Embeds
0
Number of Embeds
66
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Oreilly Webcast Jan 09, 2009

  1. 1. MySQL Replication: Audit, Test & Verify Presented by Sean Hull, a business and technology consultant, author & speaker
  2. 2. MySQL Relication - How <ul><li>Master records changes to binlog </li></ul><ul><li>Slave copies master’s binlog to relaylog via it’s IO thread </li></ul><ul><li>Slave replays sql from relaylog via it’s SQL thread (5.0 statement-based rep) </li></ul>
  3. 3. MySQL Replication - Limitations <ul><li>Mixing transactional + non-transactional tables </li></ul><ul><li>Non-deterministic app code </li></ul><ul><ul><li>Sysdate, current_user, load_file, uuid, user, found_rows, version </li></ul></ul><ul><ul><li>Session variables </li></ul></ul><ul><ul><ul><li>Set max_join_size=1000 </li></ul></ul></ul><ul><ul><ul><li>Insert into mytable values (@@max_join_size) </li></ul></ul></ul><ul><ul><li>Locking functions - get_lock, release_lock </li></ul></ul>
  4. 4. Audit - Application Code <ul><li>dev.mysql.com/doc/refman/5.0/en/ </li></ul><ul><ul><li>“ Replication Features And Issues” </li></ul></ul><ul><li>Look for problem functions </li></ul><ul><li>Look for non-transactional tables </li></ul>
  5. 5. Test - Replication Setup <ul><li>Use Nagios </li></ul><ul><li>Look for: </li></ul><ul><ul><li>Slave_IO_Running: Yes </li></ul></ul><ul><ul><li>Slave_SQL_Running: Yes </li></ul></ul><ul><li>Monitor slave error log </li></ul>
  6. 6. Verify - Use Maatkit <ul><li>http://code.google.com/p/maatkit/ </li></ul><ul><li>mk-table-checksum </li></ul><ul><ul><li>Checksums - compare mathematically </li></ul></ul><ul><ul><li>Checksum table </li></ul></ul><ul><ul><li>Checksum timestamps </li></ul></ul><ul><ul><li>Propogate through replication </li></ul></ul>
  7. 7. Demo <ul><li>Install Maatkit, DBI & DBD-mysql </li></ul><ul><li>Install MySQL Sandbox </li></ul><ul><li>Perform Initial Checksums </li></ul><ul><li>Add Bad Row & Verify </li></ul><ul><li>Fix Bad Row & Verify </li></ul>
  8. 8. Install Maatkit <ul><li>http://www.maatkit.org/ </li></ul><ul><li>Relevant tools: </li></ul><ul><ul><li>mk-table-checksum </li></ul></ul><ul><ul><li>mk-checksum-filter </li></ul></ul><ul><ul><li>mk-slave-prefetch </li></ul></ul><ul><ul><li>mk-slave-restart </li></ul></ul><ul><ul><li>mk-slave-delay </li></ul></ul>
  9. 9. Install MySQL Sandbox <ul><li>Giuseppe Maxia </li></ul><ul><li>http://launchpad.net/mysql-sandbox </li></ul><ul><li>./make_replication_sandbox mysql-linux-5.0.67.tar.gz </li></ul>
  10. 10. Perform Initial Checksums <ul><li>Connect to master </li></ul><ul><li>Create checksum table </li></ul><ul><li>Create sean_test table </li></ul><ul><li>Insert one row </li></ul><ul><li>Use mk-table-checksum on master </li></ul>
  11. 11. Add Bad Row & Verify <ul><li>Add incorrect row </li></ul><ul><li>Verify with mk-table-checksum on slave </li></ul><ul><li>Redo mk-table-checksum on master </li></ul><ul><li>Redo mk-table-checksum on slave </li></ul>
  12. 12. Fix Bad Row & Verify <ul><li>Use mk-table-sync </li></ul><ul><li>Use mk-table-checksum on slave </li></ul><ul><li>Show fix on slave </li></ul><ul><li>Redo mk-table-checksum on master </li></ul><ul><li>Redo mk-table-checksum on slave </li></ul>
  13. 13. Attendee Questions <ul><li>Moving db? </li></ul><ul><ul><li>datadir & innodb_data_home_dir </li></ul></ul>
  14. 14. Attendee Questions <ul><li>Transaction intensive updates </li></ul><ul><ul><li>Problem of serialized writes on slave </li></ul></ul><ul><ul><li>Distribute schemas on multiple slaves </li></ul></ul><ul><ul><ul><li>replicate_do_db </li></ul></ul></ul><ul><ul><ul><li>replicate_ignore_db </li></ul></ul></ul><ul><ul><li>Prime the cache - mk-slave-prefetch </li></ul></ul><ul><ul><li>Row-based replication in 5.1 </li></ul></ul><ul><ul><li>Use MyISAM storage engine on slave </li></ul></ul><ul><ul><li>Lag on slaves is normal - what are your app’s expectations? </li></ul></ul><ul><ul><li>Data archiving - do writes outside of replication </li></ul></ul><ul><ul><li>Trade safety for speed - disable binlog etc </li></ul></ul>
  15. 15. Further Reading - Books <ul><li>High Performance MySQL 2nd Ed. - Schwartz,Zaitsev, Tkachenko, Zawodny, Lentz, Balling - O’Reilly </li></ul><ul><li>Pro MySQL - Kruckenberg & Pipes - Apress </li></ul>
  16. 16. Further Reading - Web <ul><li>http://www.xaprb.com </li></ul><ul><li>http://datacharmer.blogspot.com </li></ul><ul><li>http://planetmysql.com </li></ul><ul><li>http://www.mysqlperformanceblog.com </li></ul>
  17. 17. Conclusion <ul><li>Reliability is not automatic </li></ul><ul><li>Consistency is not automatic </li></ul><ul><li>Replication can fail without error </li></ul><ul><li>Vigilance will bear fruit </li></ul><ul><li>Use the right tools for the job </li></ul><ul><li>5.1 row-based replication may mitigate some of these issues </li></ul>
  18. 18. Thank You <ul><li>Sean Hull <hullsean@gmail.com> </li></ul><ul><ul><li>Founder, Senior Consultant </li></ul></ul><ul><ul><li>Heavyweight Internet Group </li></ul></ul><ul><ul><li>www.iheavy.com </li></ul></ul><ul><ul><li>Ph: +1-866-268-9448 </li></ul></ul><ul><ul><li>Blog: www.oracleopensource.com </li></ul></ul><ul><ul><li>www.linkedin.com/in/hullsean </li></ul></ul>

×