Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
MySQL High Availability Solutions     Lenz Grimmer <lenz@grimmer.com>                  <     http://lenzg.net/ | Twitter: ...
$ whoami1998              20022008              2010
Agenda●   High Availability: Concepts &    Considerations●   MySQL Replication●   DRBD / Pacemaker●   MySQL Cluster●   Oth...
Single MySQL Server            MySQL Client   SQL (SELECT, INSERT,      UPDATE, DELETE)   MySQL Storage Engines(InnoDB, My...
Why HA?●   Something can and will fail●   Service Maintenance●   Downtime is expensive●   Adding HA to an existing system ...
Elimination of the SPOF●   Identify what will fail eventually     ●   Hard disks     ●   Fans●   Consider what might fail ...
What is HA Clustering?●   Redundancy●   One system or service goes down → others    take over●   IP address takeover, serv...
High Availability Levels
Rules of High Availability●   Prepare for failure●   Aim to ensure that no important data is    lost●   Keep it simple, st...
HA Components●   Heartbeat     ●   Checks that services that are monitored, are alive.     ●   Can check individual server...
Split-Brain●   Communications failures can lead to    separated partitions of the cluster●   If those partitions each try ...
Redundancy Using MySQL Replication           MySQL Replication
MySQL Replication●   Unidirectional●   Statement- or row-based (MySQL 5.1)●   Built into MySQL●   Easy to use and set up● ...
MySQL Replication (2)●   Master maintains Binary logs & index●   Replication on Slave is single-threaded    http://forge.m...
MySQL Replication - Overview        Read & Write            Web/App             Server                                    ...
Statement-based Replication●   Pro     ✔   Proven (around since MySQL 3.23)     ✔   Smaller log files     ✔   Auditing of ...
Row-based Replication●   Pro     ✔   All changes can be replicated     ✔   Similar technology used by other RDBMSes     ✔ ...
Replication TopologiesMaster > Slave                          Master > SlavesMaster > Slave > Slaves            Masters > ...
Master-Master Replication●   Two nodes are both master and slave to    each other●   Useful for easier failover●   Not sui...
MySQL Replication as a HA Solution●   What happens if the Master fails?●   What happens if the Slave fails?●   This doesn’...
Pacemaker (Linux-HA)●   Supports 2 or more Nodes (v2)●   Resource monitoring (Apps and HW)●   Active fencing mechanism (ST...
Replication & HA●   Combined with Pacemaker●   Virtual IP takeover●   Slave gets promoted to Master●   Side benefits: load...
Redundancy With Disk Replication           Disk Replication
DRBD●   Distributed Replicated Block Device●   “RAID-1 over network”●   Synchronous/asynchronous block replication●   Auto...
DRBD in Detail●   DRBD replicates data blocks between to    block devices●   DRBD can be combined with Linux-HA and    oth...
Redundancy Using Shared Storage
Replication vs. SAN●   Data Consistency / Integrity●   Synchronous vs. asynchronous●   SAN can become the SPOF●   Cold cac...
Redundancy with MySQL Cluster
MySQL Cluster●   “Shared-nothing”-Architecture●   Automatic partitioning●   Distributed fragments●   Synchronous replicati...
MySQL Cluster●   In-memory indexes●   Not suitable for all query patterns    (cross-table JOINs, range scans)●   No suppor...
MySQL Cluster & Replikation●   MySQL Cluster     ●   Easy failover from one MySQL node to another     ●   Scaling write lo...
http://johanandersson.blogspot.com/2009/05/ha-mysql-write-scaling-using-cluster-to.html
Galera Replication●   Patch for InnoDB plus external library●   Synchronous replication●   Single- or multi-master●   Mult...
MMM●   MySQL Master-Master Replication    Manager●   Perl-Script for monitoring, failover and    management●   Master-mast...
Continuent Tungsten Replicator●   Database-external solution●   Asynchronous, Master-Slave, Fan-out &    Fan-in●   Java●  ...
Red Hat Cluster Suite●   HA and load balancing●   Supports up to 16 nodes●   Shared storage●   Monitors services, file sys...
Solaris Cluster / OpenHA Cluster●   Provides failover and scalability services●   Solaris / OpenSolaris (Project Colorado)...
Flipper●   Perl Script managing pairs of MySQL    servers replicating to each other.●   Automatic switchover, triggered ma...
Q&AQuestions, Comments?
Thank you!Lenz Grimmer <lenz@grimmer.com>         Twitter: @lenzgr         http://lenzg.net/
Mysqlhacodebits20091203 1260184765-phpapp02
Upcoming SlideShare
Loading in …5
×

Mysqlhacodebits20091203 1260184765-phpapp02

993 views

Published on

Published in: Technology
  • Be the first to comment

Mysqlhacodebits20091203 1260184765-phpapp02

  1. 1. MySQL High Availability Solutions Lenz Grimmer <lenz@grimmer.com> < http://lenzg.net/ | Twitter: @lenzgr2011-01-26 | San Francisco MySQL Meetup | USA
  2. 2. $ whoami1998 20022008 2010
  3. 3. Agenda● High Availability: Concepts & Considerations● MySQL Replication● DRBD / Pacemaker● MySQL Cluster● Other HA tools/applications
  4. 4. Single MySQL Server MySQL Client SQL (SELECT, INSERT, UPDATE, DELETE) MySQL Storage Engines(InnoDB, MyISAM, PBXT...) Disk Storage(XFS, ReiserFS, JFS, ext3...)
  5. 5. Why HA?● Something can and will fail● Service Maintenance● Downtime is expensive● Adding HA to an existing system is complex
  6. 6. Elimination of the SPOF● Identify what will fail eventually ● Hard disks ● Fans● Consider what might fail ● Application crashes ● OOM-Situations, Kernel-Panics ● Network connections, Cables ● Power supply
  7. 7. What is HA Clustering?● Redundancy● One system or service goes down → others take over● IP address takeover, service takeover● Ensuring data availability & integrity● Not designed for high-performance
  8. 8. High Availability Levels
  9. 9. Rules of High Availability● Prepare for failure● Aim to ensure that no important data is lost● Keep it simple, stupid (KISS)● Complexity is the enemy of reliability● Automate it● Test your setup frequently!
  10. 10. HA Components● Heartbeat ● Checks that services that are monitored, are alive. ● Can check individual servers, software services, networking etc.● HA Monitor ● Configuration of the services ● Ensures proper shutdown and startup ● Allows manual control● Shared storage / Replication
  11. 11. Split-Brain● Communications failures can lead to separated partitions of the cluster● If those partitions each try and take control of the cluster, then its called a split-brain condition● If this happens, then bad things will happen● Use Fencing or Moderation/Arbitration to avoid it
  12. 12. Redundancy Using MySQL Replication MySQL Replication
  13. 13. MySQL Replication● Unidirectional● Statement- or row-based (MySQL 5.1)● Built into MySQL● Easy to use and set up● One Master, many Slaves● Asynchronous – Slaves can lag behind● New in MySQL 5.5: Semisync Replication
  14. 14. MySQL Replication (2)● Master maintains Binary logs & index● Replication on Slave is single-threaded http://forge.mysql.com/wiki/ReplicationFeatures/ParallelSlave● No automated fail-over● No heartbeat, no monitoring● New in 5.5: replication heartbeat
  15. 15. MySQL Replication - Overview Read & Write Web/App Server Write Relay Log mysqld SQL I/O Threa Thread d Index & BinlogsData Replication Binlog Data mysqld MySQL Master MySQL Slave
  16. 16. Statement-based Replication● Pro ✔ Proven (around since MySQL 3.23) ✔ Smaller log files ✔ Auditing of actual SQL statements ✔ No primary key requirement for replicated tables● Con ✗ Non-deterministic functions and UDFs ✗ LOAD_FILE(), UUID(), CURRENT_USER(), FOUND_ROWS() (but RAND() and NOW() work)
  17. 17. Row-based Replication● Pro ✔ All changes can be replicated ✔ Similar technology used by other RDBMSes ✔ Fewer locks required for some INSERT, UPDATE or DELETE statements● Con ✗ More data to be logged ✗ Log file size increases (backup/restore implications) ✗ Replicated tables require explicit primary keys ✗ Possible different result sets on bulk INSERTs
  18. 18. Replication TopologiesMaster > Slave Master > SlavesMaster > Slave > Slaves Masters > Slave (Multi-Source) Ring (Multi-Master)Master < > Master (Multi-Master)
  19. 19. Master-Master Replication● Two nodes are both master and slave to each other● Useful for easier failover● Not suitable for (write) load-balancing● Dont write to both masters simultaneously!● Use Sharding or Partitioning instead (e.g. MySQL Proxy)
  20. 20. MySQL Replication as a HA Solution● What happens if the Master fails?● What happens if the Slave fails?● This doesn’t sound like High Availability!● Yes!● Replication is only part of a HA configuration
  21. 21. Pacemaker (Linux-HA)● Supports 2 or more Nodes (v2)● Resource monitoring (Apps and HW)● Active fencing mechanism (STONITH)● Node failure detection in seconds● Supports many applications (incl. MySQL)● http://clusterlabs.org/● http://www.clusterlabs.org/wiki/Load_Balanced_MySQL_Replicated_Cluster● http://www.clusterlabs.org/wiki/DRBD_MySQL_HowTo
  22. 22. Replication & HA● Combined with Pacemaker● Virtual IP takeover● Slave gets promoted to Master● Side benefits: load balancing & backup● Can be tricky to fail back● No automatic conflict resolution● Proper failover needs to be scripted
  23. 23. Redundancy With Disk Replication Disk Replication
  24. 24. DRBD● Distributed Replicated Block Device● “RAID-1 over network”● Synchronous/asynchronous block replication● Automatic resynchronisation● Application-agnostic● Can mask local I/O-Errors● Active/passive configuration● Dual-primary mode (requires cluster file sytem like GFS or OCFS2)● http://drbd.org/
  25. 25. DRBD in Detail● DRBD replicates data blocks between to block devices● DRBD can be combined with Linux-HA and other HA solutions● MySQL runs normally Applications on primary node● MySQL is not active on Active Node Virtual IP Passive Node the secondary node● DRBD is Linux only DRBD
  26. 26. Redundancy Using Shared Storage
  27. 27. Replication vs. SAN● Data Consistency / Integrity● Synchronous vs. asynchronous● SAN can become the SPOF● Cold caches● “Split brain”-Situations● SAN/NAS I/O Overhead
  28. 28. Redundancy with MySQL Cluster
  29. 29. MySQL Cluster● “Shared-nothing”-Architecture● Automatic partitioning● Distributed fragments● Synchronous replication● Fast, automatic fail-over of data nodes● Automatic resynchronisation● Transparent to MySQL applicationen● Supports transactions● http://mysql.com/products/database/cluster/
  30. 30. MySQL Cluster● In-memory indexes● Not suitable for all query patterns (cross-table JOINs, range scans)● No support for foreign keys● Not suitable for long running transactions● Network latency crucial● Can be combined with MySQL replication (RBR)
  31. 31. MySQL Cluster & Replikation● MySQL Cluster ● Easy failover from one MySQL node to another ● Scaling write load using multiple SQL nodes● Asynchronous replication from Cluster to regular MySQL slaves● Slaves take read load (InnoDB/MyISAM)● Quick setup of new slaves (Cluster Online Backup)● Easy failover and fast recovery
  32. 32. http://johanandersson.blogspot.com/2009/05/ha-mysql-write-scaling-using-cluster-to.html
  33. 33. Galera Replication● Patch for InnoDB plus external library● Synchronous replication● Single- or multi-master● Multicast-Replication● HA plus load sharing possible● Certifikate-based replikation (instead of 2PC)● http://codership.com/products/mysql_galera
  34. 34. MMM● MySQL Master-Master Replication Manager● Perl-Script for monitoring, failover and management● Master-master replication configuration (one active writable master)● Failover by moving virtual IP-Address● Also supports read balancing● http://mysql-mmm.org/
  35. 35. Continuent Tungsten Replicator● Database-external solution● Asynchronous, Master-Slave, Fan-out & Fan-in● Java● Log-based● Events are stored in the Transaction History Log (THL)● Modular architekture (Pipelines, Stages)● http://continuent.com/community/tungsten-replicator
  36. 36. Red Hat Cluster Suite● HA and load balancing● Supports up to 16 nodes● Shared storage● Monitors services, file systems and network status● Fencing (STONITH) or distributed lock manager● Configuration synchronization● http://www.redhat.com/cluster_suite/
  37. 37. Solaris Cluster / OpenHA Cluster● Provides failover and scalability services● Solaris / OpenSolaris (Project Colorado)● Kernel-level components plus userland● Agents monitor applications● Geo Edition to provide Disaster Recovery using Replication● Open Source since June 2007● http://hub.opensolaris.org/bin/view/Community+Group+ha-clusters/WebHome● http://hub.opensolaris.org/bin/view/Project+ha-mysql/WebHome
  38. 38. Flipper● Perl Script managing pairs of MySQL servers replicating to each other.● Automatic switchover, triggered manually● Enables one half of the pair to be taken offline for maintenance work while other half carries on● Moves IP addresses based on a role ("read- only", "writable")● http://provenscaling.com/software/flipper/
  39. 39. Q&AQuestions, Comments?
  40. 40. Thank you!Lenz Grimmer <lenz@grimmer.com> Twitter: @lenzgr http://lenzg.net/

×