MySQL High-Availability and Scale-Out architectures
Upcoming SlideShare
Loading in...5
×
 

MySQL High-Availability and Scale-Out architectures

on

  • 2,765 views

 

Statistics

Views

Total Views
2,765
Views on SlideShare
2,765
Embed Views
0

Actions

Likes
3
Downloads
84
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

MySQL High-Availability and Scale-Out architectures MySQL High-Availability and Scale-Out architectures Presentation Transcript

  • MySQL High-Availability and Scale-Out architectures Oli Sennhauser Senior Consultant osennhauser@mysql.comCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 1
  • Introduction • Who we are? • What we want?Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 2
  • Table of Contents • Scale-Up vs. Scale-Out • MySQL Replication • MySQL HA solution • MySQL Cluster • Application architectural solutions • Other architectural stuff • Its your turn... • Let us build a replicationCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 3
  • When do we think about architecture? • Performance problems • HA requirementsCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 4
  • Scale-up vs. Scale-Out • Scale-up • Scale-out t Relaxation of constraintsCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 5
  • MySQL Replication • The MySQL answer: Master-Slave-Replication: Application 2 types of sessions!!! rtw Master Slave1 Slave2 Slave3 ... LB ro Even Oracle uses the expression “scale-out” now!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 6
  • MySQL Replication in detail Appl ro rt w asynchronous! IO thread M S SQL thread ... bin-log bin-log relay-log relay-log • wide distance possible! • thin line possible!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 7
  • MySQL Replication varieties • Cascaded replication: M black hole SI SI SI ... SE! S1 S2 S3 S4 S5 S6 S7 S8 ... for example Yahoo!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 8
  • MySQL Replication varieties • HA with “hot standby” Slave: Appl VIP HA solution Sbackup M Sstandby S1 S2 S3 • active – active fail over!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 9
  • MySQL HA solution • HA with MySQL: Appl VIP heartbeat M M DRBD S1 S2 S3 • 99.99% HA (four nine) • active – passive fail over!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 10
  • MySQL HA solution • HA with MySQL and SAN: Appl VIP heartbeat M M SAN S1 S2 S3Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 11
  • Multi-Master Replication • actually Master-Master Replication Appl Appl VIP/LB M1 M2 S11 S12 S13 S21 S22 S23 • This architecture does NOT solve your write problems!!!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 12
  • Multi-Master Replication • HA Multi-Master Replication Appl Appl VIP/LB VIP VIP M1 M1 M2 M2 DRBD DRBD S11 S12 S13 S21 S22 S23Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 13
  • Shared disk cluster DB1 DB2 DB2 ro! shared disk • This is in 99.9% of the cases NOT your solution!!! • It looks like Oracle RAC but MySQL does NOT (yet) have the instruments needed! • Can be useful in some special kinds of reporting.Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 14
  • Where are we now? • The architectures above solved: – your read problems. – some HA requirements. • But they did not solve: – your write problems! • And what when we need: – higher HA? – synchronous replication? – more write speed?Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 15
  • MySQL Cluster • Shared-nothing architecture • Synchronous replication (2-Phase commit) • Fast automatic fail over • High performance (also writing) • High transactional throughput • No special components required • In-Memory database (in 5.1 also disk support) • Scalable, 1000s of transactions per second • 99.999% HA (five nine) • On-line upgrade path (at least on GA within same version)Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 16
  • MySQL Cluster architecture Application Application Application Application Application Application my.cnf SQL­Nodes MySQL MySQL MySQL Server Server Server NDB Cluster (Data nodes) config.ini DB DB MGM Data­Nodes Server DB DB MGM client Mgmt­NodesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 17
  • MySQL Cluster HA features MySQL XMySQL X MySQL X MySQL X MySQL Server Server Server Server Server NDB API Data NDB Data Management Node Storage Engine Node Server X Data X Data Node Node X Server Management MySQL ClusterCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 18
  • Cluster with Replication • for read scale-out (Reporting): Appl rw mysqld mysqld mysqld ndbd ndbd S1 S2 S3 ndbd ndbd ReportingCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 19
  • Cluster with Replication • Cluster-Cluster replication for disaster fail over (MySQL 5.0): Appl fail over rw mysqld mysqld mysqld mysqld mysqld mysqld ndbd ndbd ndbd ndbd ndbd ndbd ndbd ndbdCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 20
  • Cluster with Replication • Cluster-Cluster replication for disaster fail over (MySQL 5.1): Appl fail over rw mysqld mysqld mysqld mysqld mysqld mysqld ndbd ndbd ndbd ndbd ndbd ndbd ndbd ndbdCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 21
  • Cluster with Replication • circular Cluster-Cluster replication (>= MySQL 5.1.18?):Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 22
  • Cluster examples • Session handling • Telecom (Mobile) • VoIP • RSS-Feed aggregation • Mail • On-line Games • Use cluster where – you need HA – you have high write load – you do little Joins and GroupingCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 23
  • How to go on? • Read is a caching problem! ➔ More RAM or scale-out. ● Write is a batching problem! ➔ Batch your load, ➔ Buy stronger I/O system. ➔ Use MySQL Cluster. ➔ SSD! ● What then?Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 24
  • SSD disruption • I/O system without any movable parts! • SSD = Solid State disk (Flash memory, NAND, NOR chips, RAM-SAN). • During the last year(s) • Price from 15000 USD / 160 Gbyte -> 100 USD / 32 Gbyte • 10-50 times faster than mechanical I/O systems • 1 Mio write cycles dead :-( (special FS!) • RAM: Huge amount of memory is cheap! ➔ This will disrupt the database world!Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 25
  • What then? • With or without SSD we will reach a physical limit. What then? • Application partitioning: – Split applications • OLTP vs. OLAP • all in one – Segment your applicationCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 26
  • Application partitioning • OLTP vs. OLAP – hot business data (trx) vs. “old” reporting data • “All in one” – Sessions, user tracking, ads, chat, booking • Segment – split by for example 1 Mio users (split by user_id).Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 27
  • Architecture examplesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 28
  • Architecture examplesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 29
  • Architecture examplesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 30
  • Architecture examplesCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 31
  • Some more architecture stuff? • Databases are slow!!! ➔ Whenever possible try to avoid databases! Use memcached for example. • The SP trap. ➔ Stored Procedures are a lock in! Try to avoid SP! • Use Materialized Views (MV) and/or shadow tables. • VM/SAN is nice for consolidation but not for performance/scale-out! • Backup and Staging • MySQL – ProxyCopyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 32
  • Now its your turn... • Your problems? • Let us build a replication...Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 33