MySQL High-Availability and Scale-Out architectures
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,054
On Slideshare
3,054
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
101
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. MySQL High-Availability and Scale-Out architectures Oli Sennhauser Senior Consultant osennhauser@mysql.comCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 1
  • 2. Introduction • Who we are? • What we want?Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 2
  • 3. Table of Contents • Scale-Up vs. Scale-Out • MySQL Replication • MySQL HA solution • MySQL Cluster • Application architectural solutions • Other architectural stuff • Its your turn... • Let us build a replicationCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 3
  • 4. When do we think about architecture? • Performance problems • HA requirementsCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 4
  • 5. Scale-up vs. Scale-Out • Scale-up • Scale-out t Relaxation of constraintsCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 5
  • 6. MySQL Replication • The MySQL answer: Master-Slave-Replication: Application 2 types of sessions!!! rtw Master Slave1 Slave2 Slave3 ... LB ro Even Oracle uses the expression “scale-out” now!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 6
  • 7. MySQL Replication in detail Appl ro rt w asynchronous! IO thread M S SQL thread ... bin-log bin-log relay-log relay-log • wide distance possible! • thin line possible!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 7
  • 8. MySQL Replication varieties • Cascaded replication: M black hole SI SI SI ... SE! S1 S2 S3 S4 S5 S6 S7 S8 ... for example Yahoo!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 8
  • 9. MySQL Replication varieties • HA with “hot standby” Slave: Appl VIP HA solution Sbackup M Sstandby S1 S2 S3 • active – active fail over!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 9
  • 10. MySQL HA solution • HA with MySQL: Appl VIP heartbeat M M DRBD S1 S2 S3 • 99.99% HA (four nine) • active – passive fail over!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 10
  • 11. MySQL HA solution • HA with MySQL and SAN: Appl VIP heartbeat M M SAN S1 S2 S3Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 11
  • 12. Multi-Master Replication • actually Master-Master Replication Appl Appl VIP/LB M1 M2 S11 S12 S13 S21 S22 S23 • This architecture does NOT solve your write problems!!!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 12
  • 13. Multi-Master Replication • HA Multi-Master Replication Appl Appl VIP/LB VIP VIP M1 M1 M2 M2 DRBD DRBD S11 S12 S13 S21 S22 S23Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 13
  • 14. Shared disk cluster DB1 DB2 DB2 ro! shared disk • This is in 99.9% of the cases NOT your solution!!! • It looks like Oracle RAC but MySQL does NOT (yet) have the instruments needed! • Can be useful in some special kinds of reporting.Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 14
  • 15. Where are we now? • The architectures above solved: – your read problems. – some HA requirements. • But they did not solve: – your write problems! • And what when we need: – higher HA? – synchronous replication? – more write speed?Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 15
  • 16. MySQL Cluster • Shared-nothing architecture • Synchronous replication (2-Phase commit) • Fast automatic fail over • High performance (also writing) • High transactional throughput • No special components required • In-Memory database (in 5.1 also disk support) • Scalable, 1000s of transactions per second • 99.999% HA (five nine) • On-line upgrade path (at least on GA within same version)Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 16
  • 17. MySQL Cluster architecture Application Application Application Application Application Application my.cnf SQL­Nodes MySQL MySQL MySQL Server Server Server NDB Cluster (Data nodes) config.ini DB DB MGM Data­Nodes Server DB DB MGM client Mgmt­NodesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 17
  • 18. MySQL Cluster HA features MySQL XMySQL X MySQL X MySQL X MySQL Server Server Server Server Server NDB API Data NDB Data Management Node Storage Engine Node Server X Data X Data Node Node X Server Management MySQL ClusterCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 18
  • 19. Cluster with Replication • for read scale-out (Reporting): Appl rw mysqld mysqld mysqld ndbd ndbd S1 S2 S3 ndbd ndbd ReportingCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 19
  • 20. Cluster with Replication • Cluster-Cluster replication for disaster fail over (MySQL 5.0): Appl fail over rw mysqld mysqld mysqld mysqld mysqld mysqld ndbd ndbd ndbd ndbd ndbd ndbd ndbd ndbdCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 20
  • 21. Cluster with Replication • Cluster-Cluster replication for disaster fail over (MySQL 5.1): Appl fail over rw mysqld mysqld mysqld mysqld mysqld mysqld ndbd ndbd ndbd ndbd ndbd ndbd ndbd ndbdCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 21
  • 22. Cluster with Replication • circular Cluster-Cluster replication (>= MySQL 5.1.18?):Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 22
  • 23. Cluster examples • Session handling • Telecom (Mobile) • VoIP • RSS-Feed aggregation • Mail • On-line Games • Use cluster where – you need HA – you have high write load – you do little Joins and GroupingCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 23
  • 24. How to go on? • Read is a caching problem! ➔ More RAM or scale-out. ● Write is a batching problem! ➔ Batch your load, ➔ Buy stronger I/O system. ➔ Use MySQL Cluster. ➔ SSD! ● What then?Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 24
  • 25. SSD disruption • I/O system without any movable parts! • SSD = Solid State disk (Flash memory, NAND, NOR chips, RAM-SAN). • During the last year(s) • Price from 15000 USD / 160 Gbyte -> 100 USD / 32 Gbyte • 10-50 times faster than mechanical I/O systems • 1 Mio write cycles dead :-( (special FS!) • RAM: Huge amount of memory is cheap! ➔ This will disrupt the database world!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 25
  • 26. What then? • With or without SSD we will reach a physical limit. What then? • Application partitioning: – Split applications • OLTP vs. OLAP • all in one – Segment your applicationCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 26
  • 27. Application partitioning • OLTP vs. OLAP – hot business data (trx) vs. “old” reporting data • “All in one” – Sessions, user tracking, ads, chat, booking • Segment – split by for example 1 Mio users (split by user_id).Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 27
  • 28. Architecture examplesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 28
  • 29. Architecture examplesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 29
  • 30. Architecture examplesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 30
  • 31. Architecture examplesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 31
  • 32. Some more architecture stuff? • Databases are slow!!! ➔ Whenever possible try to avoid databases! Use memcached for example. • The SP trap. ➔ Stored Procedures are a lock in! Try to avoid SP! • Use Materialized Views (MV) and/or shadow tables. • VM/SAN is nice for consolidation but not for performance/scale-out! • Backup and Staging • MySQL – ProxyCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 32
  • 33. Now its your turn... • Your problems? • Let us build a replication...Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 33