MySQL - Scale Out @ CloudParty 2013 Milano Talent Garden

1,102 views

Published on

MySQL scalability, from shodow instances to Master/Slave and Master/Master architectures.

Published in: Technology, Travel
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,102
On SlideShare
0
From Embeds
0
Number of Embeds
50
Actions
Shares
0
Downloads
13
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

MySQL - Scale Out @ CloudParty 2013 Milano Talent Garden

  1. 1. How to scale out MySQL databases CloudParty 2013 – Database in Cloud – Milan 27 Sep Walter Dal Mut - @walterdalmut www.corley.it
  2. 2. http://www.slideshare.net/ziobrando/ddd-dirty-harry-style CloudParty 2013 - 27 Sep Milan Italy
  3. 3. Why we are interested in: Scale-Out databases on the Cloud? CloudParty 2013 - 27 Sep Milan Italy
  4. 4. CloudParty 2013 - 27 Sep Milan Italy
  5. 5. We need more resources Single hard drives has limits Network interfaces too And so on... CloudParty 2013 - 27 Sep Milan Italy
  6. 6. We are also looking for availability CloudParty 2013 - 27 Sep Milan Italy
  7. 7. we want to ensure SERVICE availability and NOT server availability CloudParty 2013 - 27 Sep Milan Italy
  8. 8. CloudParty 2013 - 27 Sep Milan Italy Availability % Downtime per Month 90% 72 Hours 95% 36 Hours 99% 7.2 Hours 99.5% 3.6 Hours 99.8 86.23 Minutes 99.95 21.56 Minutes 99.99 4.32 Minutes
  9. 9. Ok, we can start... First of all we have to talk about replication CloudParty 2013 - 27 Sep Milan Italy
  10. 10. CloudParty 2013 - 27 Sep Milan Italy
  11. 11. How many kind of replications? CloudParty 2013 - 27 Sep Milan Italy
  12. 12. Synchronous Replication CloudParty 2013 - 27 Sep Milan Italy
  13. 13. Sync Replica is the process that writes data to the primary and secondary node at the SAME time. It is more expensive and introduce latency that slow down the application because. CloudParty 2013 - 27 Sep Milan Italy
  14. 14. Asynchronous Replication CloudParty 2013 - 27 Sep Milan Italy
  15. 15. Async Replica Is the process that writes data to the primary and secondary node at different time. Is more simple, a primary node doesn't care about others CloudParty 2013 - 27 Sep Milan Italy
  16. 16. Semi-Synchronous Replication CloudParty 2013 - 27 Sep Milan Italy
  17. 17. Semi-Sync Replica On the master side, write when at least one slave acknowledges that is has received all events CloudParty 2013 - 27 Sep Milan Italy
  18. 18. Which kind of replications can we sort up with MySQL? CloudParty 2013 - 27 Sep Milan Italy
  19. 19. ASYNC Master/Slave Replica (Read Replicas) CloudParty 2013 - 27 Sep Milan Italy
  20. 20. ASYNC Master/Master Replica (Read/Write replicas) CloudParty 2013 - 27 Sep Milan Italy
  21. 21. AlwaysAsync! That means: LAG REPLICATIME CloudParty 2013 - 27 Sep Milan Italy
  22. 22. Just this? CloudParty 2013 - 27 Sep Milan Italy
  23. 23. Semi-Sync Replicas Starting with MySQL 5.5 Implemented by plugins http://dev.mysql.com/doc/refman/5.5/en/replication-semisync.html CloudParty 2013 - 27 Sep Milan Italy
  24. 24. A slave indicates that it is semy-sync capable On the master side, write when at least one slave acknowledges that is has received all events If no one slave is available, the master revert to async replication If one slave catches up, the master returns to semi-sync replication CloudParty 2013 - 27 Sep Milan Italy
  25. 25. And if I need something SYNC? CloudParty 2013 - 27 Sep Milan Italy
  26. 26. Percona XtraDB Cluster MySQL Cluster NDB (internally) CloudParty 2013 - 27 Sep Milan Italy
  27. 27. Instead using MySQL Replication we can look for replication of network devices “Distributed Replication Block Device” DRBD http://www.drbd.org/users-guide-8.3/ch-heartbeat.html http://dev.mysql.com/doc/refman/5.0/en/ha-drbd.html CloudParty 2013 - 27 Sep Milan Italy
  28. 28. DRBD is essentially a RAID 1 on a network CloudParty 2013 - 27 Sep Milan Italy
  29. 29. DRBD has 3 replication modes: A → Async B → Semi Sync C → Sync CloudParty 2013 - 27 Sep Milan Italy
  30. 30. global { usage-count no; } common { net { protocol C; } } resource r0 { on drbd01 { device /dev/drbd0; disk /dev/sdb1; address 192.168.0.1:7788; meta-disk internal; } on drbd02 { device /dev/drbd0; disk /dev/sdb1; address 192.168.0.2:7788; meta-disk internal; } } CloudParty 2013 - 27 Sep Milan Italy
  31. 31. Exactly like a RAID1 array you cannot query the “second device” It means that: “the device” on a secondary node is not mounted/available CloudParty 2013 - 27 Sep Milan Italy
  32. 32. CloudParty 2013 - 27 Sep Milan Italy
  33. 33. DRBD is a RAID on networks your entire system doesn't switch automatically in case of failures. In order to automate the failover switch you can use Peacemaker/Corosync http://dev.mysql.com/doc/refman/5.5/en/ha-drbd.html CloudParty 2013 - 27 Sep Milan Italy
  34. 34. CloudParty 2013 - 27 Sep Milan Italy
  35. 35. AWS RDS – Multi AZ CloudParty 2013 - 27 Sep Milan Italy
  36. 36. CloudParty 2013 - 27 Sep Milan Italy
  37. 37. CloudParty 2013 - 27 Sep Milan Italy
  38. 38. Master/Slave – Read Replicas Async Replication CloudParty 2013 - 27 Sep Milan Italy
  39. 39. CloudParty 2013 - 27 Sep Milan Italy
  40. 40. CloudParty 2013 - 27 Sep Milan Italy
  41. 41. Master-Slave: Writes-Reads CloudParty 2013 - 27 Sep Milan Italy
  42. 42. CloudParty 2013 - 27 Sep Milan Italy
  43. 43. CloudParty 2013 - 27 Sep Milan Italy
  44. 44. Capacity left for READS Capacity left for WRITES Master-Slave: Write Intensive CloudParty 2013 - 27 Sep Milan Italy
  45. 45. Master/Master – Multi Master Async Replication CloudParty 2013 - 27 Sep Milan Italy
  46. 46. A primary node see the secondary node as a slave and vice-versa CloudParty 2013 - 27 Sep Milan Italy
  47. 47. Master M1 Slave M1 CloudParty 2013 - 27 Sep Milan Italy
  48. 48. Master M1 Slave M2 Slave M1 Master M2 CloudParty 2013 - 27 Sep Milan Italy
  49. 49. Super Easy! But, it works? CloudParty 2013 - 27 Sep Milan Italy
  50. 50. YES? CloudParty 2013 - 27 Sep Milan Italy
  51. 51. Why? CloudParty 2013 - 27 Sep Milan Italy
  52. 52. Because of auto increment columns CloudParty 2013 - 27 Sep Milan Italy
  53. 53. As a cause of LAG replication time two auto-increment keys conflicts quite immediately in a production environment! CloudParty 2013 - 27 Sep Milan Italy
  54. 54. 1 1 2 2 APPLICATION APPLICATIONMASTER 1 MASTER 2 INSERT INSERT Both replica stops immediately on replication (split brain) INSERT INTO table (id) VALUES (2) INSERT INTO table (id) VALUES (2) CloudParty 2013 - 27 Sep Milan Italy
  55. 55. Any solution? CloudParty 2013 - 27 Sep Milan Italy
  56. 56. Avoid auto-increments columns CloudParty 2013 - 27 Sep Milan Italy
  57. 57. CloudParty 2013 - 27 Sep Milan Italy
  58. 58. Work on auto-increment columns CloudParty 2013 - 27 Sep Milan Italy
  59. 59. lauto_increment_increment controls the increment between successive AUTO_INCREMENT values. lauto_increment_offset determines the starting point for AUTO_INCREMENT column values. CloudParty 2013 - 27 Sep Milan Italy
  60. 60. server-id=1 auto-increment-offset = 1 auto-increment-increment = 4 Produces: 1 5 9 13 17 21 25 server-id=2 auto-increment-offset = 2 auto-increment-increment = 4 Produces: 2 6 10 14 18 22 26 CloudParty 2013 - 27 Sep Milan Italy
  61. 61. CloudParty 2013 - 27 Sep Milan Italy
  62. 62. Collisions can occurs on all unique indexes User names, emails e unique fields in general causes collisions in a asynchronous replicated database. The application must ensure zero collisions CloudParty 2013 - 27 Sep Milan Italy
  63. 63. You have to deal also with inconsistencies CloudParty 2013 - 27 Sep Milan Italy
  64. 64. Master Master Application UPDATE users SET email='your.email@me.com' where id=1; UPDATE users SET email='ane@y.com' where id=1; ane@y.com your.email@me.com your.email@me.com ane@y.com CloudParty 2013 - 27 Sep Milan Italy
  65. 65. Do we have scale writes? CloudParty 2013 - 27 Sep Milan Italy
  66. 66. NO! CloudParty 2013 - 27 Sep Milan Italy
  67. 67. Master-Master: Writes-Reads Every write still has to be performed on both masters! CloudParty 2013 - 27 Sep Milan Italy
  68. 68. We can use Master/Master for: High Availability (Hot Stand-By) MaintenanceTasks CloudParty 2013 - 27 Sep Milan Italy
  69. 69. Thanks for listening Any question? walter.dalmut@corley.it @walterdalmut CloudParty 2013 - 27 Sep Milan Italy

×