MySQL High-Availability and Scale-Out architectures

  • 2,705 views
Uploaded on

 

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,705
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
107
Comments
0
Likes
5

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. MySQL High-Availability and Scale-Out architectures Oli Sennhauser Senior Consultant osennhauser@mysql.comCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 1
  • 2. Introduction • Who we are? • What we want?Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 2
  • 3. Table of Contents • Scale-Up vs. Scale-Out • MySQL Replication • MySQL HA solution • MySQL Cluster • Application architectural solutions • Other architectural stuff • Its your turn... • Let us build a replicationCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 3
  • 4. When do we think about architecture? • Performance problems • HA requirementsCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 4
  • 5. Scale-up vs. Scale-Out • Scale-up • Scale-out t Relaxation of constraintsCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 5
  • 6. MySQL Replication • The MySQL answer: Master-Slave-Replication: Application 2 types of sessions!!! rtw Master Slave1 Slave2 Slave3 ... LB ro Even Oracle uses the expression “scale-out” now!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 6
  • 7. MySQL Replication in detail Appl ro rt w asynchronous! IO thread M S SQL thread ... bin-log bin-log relay-log relay-log • wide distance possible! • thin line possible!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 7
  • 8. MySQL Replication varieties • Cascaded replication: M black hole SI SI SI ... SE! S1 S2 S3 S4 S5 S6 S7 S8 ... for example Yahoo!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 8
  • 9. MySQL Replication varieties • HA with “hot standby” Slave: Appl VIP HA solution Sbackup M Sstandby S1 S2 S3 • active – active fail over!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 9
  • 10. MySQL HA solution • HA with MySQL: Appl VIP heartbeat M M DRBD S1 S2 S3 • 99.99% HA (four nine) • active – passive fail over!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 10
  • 11. MySQL HA solution • HA with MySQL and SAN: Appl VIP heartbeat M M SAN S1 S2 S3Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 11
  • 12. Multi-Master Replication • actually Master-Master Replication Appl Appl VIP/LB M1 M2 S11 S12 S13 S21 S22 S23 • This architecture does NOT solve your write problems!!!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 12
  • 13. Multi-Master Replication • HA Multi-Master Replication Appl Appl VIP/LB VIP VIP M1 M1 M2 M2 DRBD DRBD S11 S12 S13 S21 S22 S23Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 13
  • 14. Shared disk cluster DB1 DB2 DB2 ro! shared disk • This is in 99.9% of the cases NOT your solution!!! • It looks like Oracle RAC but MySQL does NOT (yet) have the instruments needed! • Can be useful in some special kinds of reporting.Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 14
  • 15. Where are we now? • The architectures above solved: – your read problems. – some HA requirements. • But they did not solve: – your write problems! • And what when we need: – higher HA? – synchronous replication? – more write speed?Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 15
  • 16. MySQL Cluster • Shared-nothing architecture • Synchronous replication (2-Phase commit) • Fast automatic fail over • High performance (also writing) • High transactional throughput • No special components required • In-Memory database (in 5.1 also disk support) • Scalable, 1000s of transactions per second • 99.999% HA (five nine) • On-line upgrade path (at least on GA within same version)Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 16
  • 17. MySQL Cluster architecture Application Application Application Application Application Application my.cnf SQL­Nodes MySQL MySQL MySQL Server Server Server NDB Cluster (Data nodes) config.ini DB DB MGM Data­Nodes Server DB DB MGM client Mgmt­NodesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 17
  • 18. MySQL Cluster HA features MySQL XMySQL X MySQL X MySQL X MySQL Server Server Server Server Server NDB API Data NDB Data Management Node Storage Engine Node Server X Data X Data Node Node X Server Management MySQL ClusterCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 18
  • 19. Cluster with Replication • for read scale-out (Reporting): Appl rw mysqld mysqld mysqld ndbd ndbd S1 S2 S3 ndbd ndbd ReportingCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 19
  • 20. Cluster with Replication • Cluster-Cluster replication for disaster fail over (MySQL 5.0): Appl fail over rw mysqld mysqld mysqld mysqld mysqld mysqld ndbd ndbd ndbd ndbd ndbd ndbd ndbd ndbdCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 20
  • 21. Cluster with Replication • Cluster-Cluster replication for disaster fail over (MySQL 5.1): Appl fail over rw mysqld mysqld mysqld mysqld mysqld mysqld ndbd ndbd ndbd ndbd ndbd ndbd ndbd ndbdCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 21
  • 22. Cluster with Replication • circular Cluster-Cluster replication (>= MySQL 5.1.18?):Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 22
  • 23. Cluster examples • Session handling • Telecom (Mobile) • VoIP • RSS-Feed aggregation • Mail • On-line Games • Use cluster where – you need HA – you have high write load – you do little Joins and GroupingCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 23
  • 24. How to go on? • Read is a caching problem! ➔ More RAM or scale-out. ● Write is a batching problem! ➔ Batch your load, ➔ Buy stronger I/O system. ➔ Use MySQL Cluster. ➔ SSD! ● What then?Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 24
  • 25. SSD disruption • I/O system without any movable parts! • SSD = Solid State disk (Flash memory, NAND, NOR chips, RAM-SAN). • During the last year(s) • Price from 15000 USD / 160 Gbyte -> 100 USD / 32 Gbyte • 10-50 times faster than mechanical I/O systems • 1 Mio write cycles dead :-( (special FS!) • RAM: Huge amount of memory is cheap! ➔ This will disrupt the database world!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 25
  • 26. What then? • With or without SSD we will reach a physical limit. What then? • Application partitioning: – Split applications • OLTP vs. OLAP • all in one – Segment your applicationCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 26
  • 27. Application partitioning • OLTP vs. OLAP – hot business data (trx) vs. “old” reporting data • “All in one” – Sessions, user tracking, ads, chat, booking • Segment – split by for example 1 Mio users (split by user_id).Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 27
  • 28. Architecture examplesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 28
  • 29. Architecture examplesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 29
  • 30. Architecture examplesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 30
  • 31. Architecture examplesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 31
  • 32. Some more architecture stuff? • Databases are slow!!! ➔ Whenever possible try to avoid databases! Use memcached for example. • The SP trap. ➔ Stored Procedures are a lock in! Try to avoid SP! • Use Materialized Views (MV) and/or shadow tables. • VM/SAN is nice for consolidation but not for performance/scale-out! • Backup and Staging • MySQL – ProxyCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 32
  • 33. Now its your turn... • Your problems? • Let us build a replication...Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 33