Successfully reported this slideshow.
Your SlideShare is downloading. ×

MySQL High Availability Solutions

MySQL High Availability Solutions

Download to read offline

A presentation about how to make MySQL highly available, presented at the San Francisco MySQL Meetup (http://www.sfmysql.org/events/15760472/) on January 26th, 2011.

A video recording of this presentation is available from Ustream: http://ustre.am/fyLk

A presentation about how to make MySQL highly available, presented at the San Francisco MySQL Meetup (http://www.sfmysql.org/events/15760472/) on January 26th, 2011.

A video recording of this presentation is available from Ustream: http://ustre.am/fyLk

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

MySQL High Availability Solutions

  1. 1. MySQL High Availability Solutions Lenz Grimmer <lenz@grimmer.com> < http://lenzg.net/ | Twitter: @lenzgr 2011-01-26 | San Francisco MySQL Meetup | USA
  2. 2. $ whoami 1998 2002 2008 2010
  3. 3. Agenda ● High Availability: Concepts & Considerations ● MySQL Replication ● DRBD / Pacemaker ● MySQL Cluster ● Other HA tools/applications
  4. 4. Single MySQL Server MySQL Client SQL (SELECT, INSERT, UPDATE, DELETE) MySQL Storage Engines (InnoDB, MyISAM, PBXT...) Disk Storage (XFS, ReiserFS, JFS, ext3...)
  5. 5. Why HA? ● Something can and will fail ● Service Maintenance ● Downtime is expensive ● Adding HA to an existing system is complex
  6. 6. Elimination of the SPOF ● Identify what will fail eventually ● Hard disks ● Fans ● Consider what might fail ● Application crashes ● OOM-Situations, Kernel-Panics ● Network connections, Cables ● Power supply
  7. 7. What is HA Clustering? ● Redundancy ● One system or service goes down → others take over ● IP address takeover, service takeover ● Ensuring data availability & integrity ● Not designed for high-performance
  8. 8. High Availability Levels
  9. 9. Rules of High Availability ● Prepare for failure ● Aim to ensure that no important data is lost ● Keep it simple, stupid (KISS) ● Complexity is the enemy of reliability ● Automate it ● Test your setup frequently!
  10. 10. HA Components ● Heartbeat ● Checks that services that are monitored, are alive. ● Can check individual servers, software services, networking etc. ● HA Monitor ● Configuration of the services ● Ensures proper shutdown and startup ● Allows manual control ● Shared storage / Replication
  11. 11. Split-Brain ● Communications failures can lead to separated partitions of the cluster ● If those partitions each try and take control of the cluster, then it's called a split-brain condition ● If this happens, then bad things will happen ● Use Fencing or Moderation/Arbitration to avoid it
  12. 12. Redundancy Using MySQL Replication MySQL Replication
  13. 13. MySQL Replication ● Unidirectional ● Statement- or row-based (MySQL 5.1) ● Built into MySQL ● Easy to use and set up ● One Master, many Slaves ● Asynchronous – Slaves can lag behind ● New in MySQL 5.5: Semisync Replication
  14. 14. MySQL Replication (2) ● Master maintains Binary logs & index ● Replication on Slave is single-threaded http://forge.mysql.com/wiki/ReplicationFeatures/ParallelSlave ● No automated fail-over ● No heartbeat, no monitoring ● New in 5.5: replication heartbeat
  15. 15. MySQL Replication - Overview Read & Write Web/App Server Write Relay Log mysqld SQL I/O Threa Thread d Index & Binlogs Data Replication Binlog Data mysqld MySQL Master MySQL Slave
  16. 16. Statement-based Replication ● Pro ✔ Proven (around since MySQL 3.23) ✔ Smaller log files ✔ Auditing of actual SQL statements ✔ No primary key requirement for replicated tables ● Con ✗ Non-deterministic functions and UDFs ✗ LOAD_FILE(), UUID(), CURRENT_USER(), FOUND_ROWS() (but RAND() and NOW() work)
  17. 17. Row-based Replication ● Pro ✔ All changes can be replicated ✔ Similar technology used by other RDBMSes ✔ Fewer locks required for some INSERT, UPDATE or DELETE statements ● Con ✗ More data to be logged ✗ Log file size increases (backup/restore implications) ✗ Replicated tables require explicit primary keys ✗ Possible different result sets on bulk INSERTs
  18. 18. Replication Topologies Master > Slave Master > Slaves Master > Slave > Slaves Masters > Slave (Multi-Source) Ring (Multi-Master) Master < > Master (Multi-Master)
  19. 19. Master-Master Replication ● Two nodes are both master and slave to each other ● Useful for easier failover ● Not suitable for (write) load-balancing ● Don't write to both masters simultaneously! ● Use Sharding or Partitioning instead (e.g. MySQL Proxy)
  20. 20. MySQL Replication as a HA Solution ● What happens if the Master fails? ● What happens if the Slave fails? ● This doesn’t sound like High Availability! ● Yes! ● Replication is only part of a HA configuration
  21. 21. Pacemaker (Linux-HA) ● Supports 2 or more Nodes (v2) ● Resource monitoring (Apps and HW) ● Active fencing mechanism (STONITH) ● Node failure detection in seconds ● Supports many applications (incl. MySQL) ● http://clusterlabs.org/ ● http://www.clusterlabs.org/wiki/Load_Balanced_MySQL_Replicated_Cluster ● http://www.clusterlabs.org/wiki/DRBD_MySQL_HowTo
  22. 22. Replication & HA ● Combined with Pacemaker ● Virtual IP takeover ● Slave gets promoted to Master ● Side benefits: load balancing & backup ● Can be tricky to fail back ● No automatic conflict resolution ● Proper failover needs to be scripted
  23. 23. Redundancy With Disk Replication Disk Replication
  24. 24. DRBD ● Distributed Replicated Block Device ● “RAID-1 over network” ● Synchronous/asynchronous block replication ● Automatic resynchronisation ● Application-agnostic ● Can mask local I/O-Errors ● Active/passive configuration ● Dual-primary mode (requires cluster file sytem like GFS or OCFS2) ● http://drbd.org/
  25. 25. DRBD in Detail ● DRBD replicates data blocks between to block devices ● DRBD can be combined with Linux-HA and other HA solutions ● MySQL runs normally Applications on primary node ● MySQL is not active on Active Node Virtual IP Passive Node the secondary node ● DRBD is Linux only DRBD
  26. 26. Redundancy Using Shared Storage
  27. 27. Replication vs. SAN ● Data Consistency / Integrity ● Synchronous vs. asynchronous ● SAN can become the SPOF ● Cold caches ● “Split brain”-Situations ● SAN/NAS I/O Overhead
  28. 28. Redundancy with MySQL Cluster
  29. 29. MySQL Cluster ● “Shared-nothing”-Architecture ● Automatic partitioning ● Distributed fragments ● Synchronous replication ● Fast, automatic fail-over of data nodes ● Automatic resynchronisation ● Transparent to MySQL applicationen ● Supports transactions ● http://mysql.com/products/database/cluster/
  30. 30. MySQL Cluster ● In-memory indexes ● Not suitable for all query patterns (cross-table JOINs, range scans) ● No support for foreign keys ● Not suitable for long running transactions ● Network latency crucial ● Can be combined with MySQL replication (RBR)
  31. 31. MySQL Cluster & Replikation ● MySQL Cluster ● Easy failover from one MySQL node to another ● Scaling write load using multiple SQL nodes ● Asynchronous replication from Cluster to regular MySQL slaves ● Slaves take read load (InnoDB/MyISAM) ● Quick setup of new slaves (Cluster Online Backup) ● Easy failover and fast recovery
  32. 32. http://johanandersson.blogspot.com/2009/05/ha-mysql-write-scaling-using-cluster-to.html
  33. 33. Galera Replication ● Patch for InnoDB plus external library ● Synchronous replication ● Single- or multi-master ● Multicast-Replication ● HA plus load sharing possible ● Certifikate-based replikation (instead of 2PC) ● http://codership.com/products/mysql_galera
  34. 34. MMM ● MySQL Master-Master Replication Manager ● Perl-Script for monitoring, failover and management ● Master-master replication configuration (one active writable master) ● Failover by moving virtual IP-Address ● Also supports read balancing ● http://mysql-mmm.org/
  35. 35. Continuent Tungsten Replicator ● Database-external solution ● Asynchronous, Master-Slave, Fan-out & Fan-in ● Java ● Log-based ● Events are stored in the Transaction History Log (THL) ● Modular architekture (Pipelines, Stages) ● http://continuent.com/community/tungsten-replicator
  36. 36. Red Hat Cluster Suite ● HA and load balancing ● Supports up to 16 nodes ● Shared storage ● Monitors services, file systems and network status ● Fencing (STONITH) or distributed lock manager ● Configuration synchronization ● http://www.redhat.com/cluster_suite/
  37. 37. Solaris Cluster / OpenHA Cluster ● Provides failover and scalability services ● Solaris / OpenSolaris (Project Colorado) ● Kernel-level components plus userland ● Agents monitor applications ● Geo Edition to provide Disaster Recovery using Replication ● Open Source since June 2007 ● http://hub.opensolaris.org/bin/view/Community+Group+ha-clusters/WebHome ● http://hub.opensolaris.org/bin/view/Project+ha-mysql/WebHome
  38. 38. Flipper ● Perl Script managing pairs of MySQL servers replicating to each other. ● Automatic switchover, triggered manually ● Enables one half of the pair to be taken offline for maintenance work while other half carries on ● Moves IP addresses based on a role ("read- only", "writable") ● http://provenscaling.com/software/flipper/
  39. 39. Q&A Questions, Comments?
  40. 40. Thank you! Lenz Grimmer <lenz@grimmer.com> Twitter: @lenzgr http://lenzg.net/

×