Mysqlhacodebits20091203 1260184765-phpapp02


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Mysqlhacodebits20091203 1260184765-phpapp02

  1. 1. MySQL High Availability Solutions Lenz Grimmer <> < | Twitter: @lenzgr2011-01-26 | San Francisco MySQL Meetup | USA
  2. 2. $ whoami1998 20022008 2010
  3. 3. Agenda● High Availability: Concepts & Considerations● MySQL Replication● DRBD / Pacemaker● MySQL Cluster● Other HA tools/applications
  4. 4. Single MySQL Server MySQL Client SQL (SELECT, INSERT, UPDATE, DELETE) MySQL Storage Engines(InnoDB, MyISAM, PBXT...) Disk Storage(XFS, ReiserFS, JFS, ext3...)
  5. 5. Why HA?● Something can and will fail● Service Maintenance● Downtime is expensive● Adding HA to an existing system is complex
  6. 6. Elimination of the SPOF● Identify what will fail eventually ● Hard disks ● Fans● Consider what might fail ● Application crashes ● OOM-Situations, Kernel-Panics ● Network connections, Cables ● Power supply
  7. 7. What is HA Clustering?● Redundancy● One system or service goes down → others take over● IP address takeover, service takeover● Ensuring data availability & integrity● Not designed for high-performance
  8. 8. High Availability Levels
  9. 9. Rules of High Availability● Prepare for failure● Aim to ensure that no important data is lost● Keep it simple, stupid (KISS)● Complexity is the enemy of reliability● Automate it● Test your setup frequently!
  10. 10. HA Components● Heartbeat ● Checks that services that are monitored, are alive. ● Can check individual servers, software services, networking etc.● HA Monitor ● Configuration of the services ● Ensures proper shutdown and startup ● Allows manual control● Shared storage / Replication
  11. 11. Split-Brain● Communications failures can lead to separated partitions of the cluster● If those partitions each try and take control of the cluster, then its called a split-brain condition● If this happens, then bad things will happen● Use Fencing or Moderation/Arbitration to avoid it
  12. 12. Redundancy Using MySQL Replication MySQL Replication
  13. 13. MySQL Replication● Unidirectional● Statement- or row-based (MySQL 5.1)● Built into MySQL● Easy to use and set up● One Master, many Slaves● Asynchronous – Slaves can lag behind● New in MySQL 5.5: Semisync Replication
  14. 14. MySQL Replication (2)● Master maintains Binary logs & index● Replication on Slave is single-threaded● No automated fail-over● No heartbeat, no monitoring● New in 5.5: replication heartbeat
  15. 15. MySQL Replication - Overview Read & Write Web/App Server Write Relay Log mysqld SQL I/O Threa Thread d Index & BinlogsData Replication Binlog Data mysqld MySQL Master MySQL Slave
  16. 16. Statement-based Replication● Pro ✔ Proven (around since MySQL 3.23) ✔ Smaller log files ✔ Auditing of actual SQL statements ✔ No primary key requirement for replicated tables● Con ✗ Non-deterministic functions and UDFs ✗ LOAD_FILE(), UUID(), CURRENT_USER(), FOUND_ROWS() (but RAND() and NOW() work)
  17. 17. Row-based Replication● Pro ✔ All changes can be replicated ✔ Similar technology used by other RDBMSes ✔ Fewer locks required for some INSERT, UPDATE or DELETE statements● Con ✗ More data to be logged ✗ Log file size increases (backup/restore implications) ✗ Replicated tables require explicit primary keys ✗ Possible different result sets on bulk INSERTs
  18. 18. Replication TopologiesMaster > Slave Master > SlavesMaster > Slave > Slaves Masters > Slave (Multi-Source) Ring (Multi-Master)Master < > Master (Multi-Master)
  19. 19. Master-Master Replication● Two nodes are both master and slave to each other● Useful for easier failover● Not suitable for (write) load-balancing● Dont write to both masters simultaneously!● Use Sharding or Partitioning instead (e.g. MySQL Proxy)
  20. 20. MySQL Replication as a HA Solution● What happens if the Master fails?● What happens if the Slave fails?● This doesn’t sound like High Availability!● Yes!● Replication is only part of a HA configuration
  21. 21. Pacemaker (Linux-HA)● Supports 2 or more Nodes (v2)● Resource monitoring (Apps and HW)● Active fencing mechanism (STONITH)● Node failure detection in seconds● Supports many applications (incl. MySQL)●●●
  22. 22. Replication & HA● Combined with Pacemaker● Virtual IP takeover● Slave gets promoted to Master● Side benefits: load balancing & backup● Can be tricky to fail back● No automatic conflict resolution● Proper failover needs to be scripted
  23. 23. Redundancy With Disk Replication Disk Replication
  24. 24. DRBD● Distributed Replicated Block Device● “RAID-1 over network”● Synchronous/asynchronous block replication● Automatic resynchronisation● Application-agnostic● Can mask local I/O-Errors● Active/passive configuration● Dual-primary mode (requires cluster file sytem like GFS or OCFS2)●
  25. 25. DRBD in Detail● DRBD replicates data blocks between to block devices● DRBD can be combined with Linux-HA and other HA solutions● MySQL runs normally Applications on primary node● MySQL is not active on Active Node Virtual IP Passive Node the secondary node● DRBD is Linux only DRBD
  26. 26. Redundancy Using Shared Storage
  27. 27. Replication vs. SAN● Data Consistency / Integrity● Synchronous vs. asynchronous● SAN can become the SPOF● Cold caches● “Split brain”-Situations● SAN/NAS I/O Overhead
  28. 28. Redundancy with MySQL Cluster
  29. 29. MySQL Cluster● “Shared-nothing”-Architecture● Automatic partitioning● Distributed fragments● Synchronous replication● Fast, automatic fail-over of data nodes● Automatic resynchronisation● Transparent to MySQL applicationen● Supports transactions●
  30. 30. MySQL Cluster● In-memory indexes● Not suitable for all query patterns (cross-table JOINs, range scans)● No support for foreign keys● Not suitable for long running transactions● Network latency crucial● Can be combined with MySQL replication (RBR)
  31. 31. MySQL Cluster & Replikation● MySQL Cluster ● Easy failover from one MySQL node to another ● Scaling write load using multiple SQL nodes● Asynchronous replication from Cluster to regular MySQL slaves● Slaves take read load (InnoDB/MyISAM)● Quick setup of new slaves (Cluster Online Backup)● Easy failover and fast recovery
  32. 32.
  33. 33. Galera Replication● Patch for InnoDB plus external library● Synchronous replication● Single- or multi-master● Multicast-Replication● HA plus load sharing possible● Certifikate-based replikation (instead of 2PC)●
  34. 34. MMM● MySQL Master-Master Replication Manager● Perl-Script for monitoring, failover and management● Master-master replication configuration (one active writable master)● Failover by moving virtual IP-Address● Also supports read balancing●
  35. 35. Continuent Tungsten Replicator● Database-external solution● Asynchronous, Master-Slave, Fan-out & Fan-in● Java● Log-based● Events are stored in the Transaction History Log (THL)● Modular architekture (Pipelines, Stages)●
  36. 36. Red Hat Cluster Suite● HA and load balancing● Supports up to 16 nodes● Shared storage● Monitors services, file systems and network status● Fencing (STONITH) or distributed lock manager● Configuration synchronization●
  37. 37. Solaris Cluster / OpenHA Cluster● Provides failover and scalability services● Solaris / OpenSolaris (Project Colorado)● Kernel-level components plus userland● Agents monitor applications● Geo Edition to provide Disaster Recovery using Replication● Open Source since June 2007●●
  38. 38. Flipper● Perl Script managing pairs of MySQL servers replicating to each other.● Automatic switchover, triggered manually● Enables one half of the pair to be taken offline for maintenance work while other half carries on● Moves IP addresses based on a role ("read- only", "writable")●
  39. 39. Q&AQuestions, Comments?
  40. 40. Thank you!Lenz Grimmer <> Twitter: @lenzgr