MySQL高可用性


                             上海爱可生信息技术有限公司 曹妙霞
                                   2010-10-16




Copyright 2010 Action Tech                       1
Agenda

   •    Questions
   •    HA Terms and Concepts
   •    HA Solutions with MySQL
   •    Other Solutions




Copyright 2010 Action Tech            2
Agenda

   •    Questions
   •    HA Terms and Concepts
   •    HA Solutions with MySQL
   •    Other Solutions




Copyright 2010 Action Tech            3
Questions
  • Do you need HA?
  • Can you afford to lose some data?
  • How qualified is your IT department or support
    company?
  • How much are you ready to invest?




Copyright 2010 Action Tech                           4
Agenda

   •    Questions
   •    HA Terms and Concepts
   •    HA Solutions with MySQL
   •    Other Solutions




Copyright 2010 Action Tech            5
Definitions
  • High Availability (HA)
    > 当出现故障时,保证资源的可用性
  • Two major categories
    > 硬件的可用性(如:硬件冗余)
    > 软件的可用性(如:解决方案)
  • Continuous Availability
    > 持续的、无间断的服务
    > 无故障停滞的时间



Copyright 2010 Action Tech                 6
Definitions
    • Fault Tolerance
      > 当硬件或软件故障时,保证资源的可用性
      > 硬件和软件冗余,如:cpu,memory,IO,MySQL
       APP
    • Single Point of Failure
     > 单台机器的硬件或软件出现故障时,停止服务
    • Fail Over
     > 当出现故障时,服务切换到可用的资源平台




Copyright 2010 Action Tech                 7
Scale-Up vs. Scale-Out
• Scale-Up
      > 垂直升级,昂贵的硬件
      > 专有软件
      > 平台绑定

• Scale-Out
      > 水平升级,普通的硬件
      > 开源软件
      > 平台独立


Copyright 2010 Action Tech                   8
The Five 9s of Availability




Copyright 2010 Action Tech                           9
Agenda

   •    Questions
   •    HA Terms and Concepts
   •    HA Solutions with MySQL
   •    Other Solutions




Copyright 2010 Action Tech            10
MySQL High Availability Solutions

   • MySQL Replication
   • MySQL Cluster
   • MySQL Cluster & Replication
   • Linux Heartbeat & MySQL Replication
   • Linux Heartbeat, DRBD & MySQL
   • MySQL w/ Shared-Storage & Clustering
     Agents


Copyright 2010 Action Tech                       11
MySQL Replication

      • 异步数据复制(Statement & Row-based、
        mixed)
      • 负载均衡,在线热备,水平扩展
      • 跨网数据复制
      • Fail Over ---- 手动切换




Copyright 2010 Action Tech                       12
MySQL Replication Topologies
                Master > Slave              Master > Slaves




  Masters > Slave (Multi-Source)          Master > Slave > Slaves




                                           Circular (Multi-Master)
Master < > Master (Multi-Master)




Copyright 2010 Action Tech                                           13
Inside MySQL Replication
  Writes & Reads


                         Web/App
                          Server
                                                                                  Writes

                                                             relay
                                                            binlog


                             mysqld                               I/O              SQL
                                                                Thread            Thread
                                   index &
                                   binlogs
         data

                                             Replication                 binlog            data
                                                           mysqld
              MySQL Master                                           MySQL Slave



Copyright 2010 Action Tech                                                                        14
MySQL Replication (Scale-Out)
                                           Web/App
                                            Server


         Writes & Reads                                  Reads


    Master Server                                     Slave Server
                                                                                   Backups




            Writes                      Replication
                                                        Writes


                             Index &                               Possible Roles
                             Bin Log                     • Fail over server
                                                         • Used for performing backups
                             Rotation
                                                         • Read load balancing
                                                         • Additional slaves allow Scale-Out




Copyright 2010 Action Tech                                                                     15
MySQL Replication (Fail-Over)
                                       Web/App
                                        Server


                                X                 Writes & Reads



              Master Server                       Slave Server




                       Writes       Replication       Writes



                                                                   Index &
                                                                   Bin Log
                                                                   Rotation




Copyright 2010 Action Tech                                                    16
MySQL Cluster
       • 无共享存储的数据存储模式
       • 实时同步数据 (2-phase commit)
       • 特有应用下的高性能,高事物吞吐量
       • 无特殊硬件需求
       • 内存存储数据 & 硬盘存储数据 (5.1)
       • Fail Over----快速、自动
       • 数据节点间心跳机制



Copyright 2010 Action Tech                   17
MySQL Cluster Components
     • 数据节点(Data Nodes)
       > 实时同步数据
       > 按组存储数据
     • 管理节点(Management Nodes)
       > 用于集群的起停
       > 状态查询、配置生效、在线热备
     • SQL节点(MySQL or NDB API Nodes)
       > 标准MySQL Client or NDB API接口
       > 在线平行扩展


Copyright 2010 Action Tech                              18
Basic Cluster Architecture
                                                                        MySQL Server or NDB
                                       Web/App       Web/App                      API
                                        Server        Server
                                                                        for all Writes & Reads




                                      MySQL             MySQL
                                      Server            Server




                                           NDB API
                                                                                     Management
          Memory                            NDB                                        Server
            &                  Data    Storage Engine            Data
           Disk                Node                              Node
                                                                                     Management
                                                                                       Server




Copyright 2010 Action Tech                                                                        19
MySQL Cluster Data Node Architecture
   ID      Capital           Country   UTC
   1    Copenhagen       Denmark 2                              Data           Data
                                             Partition 1        Node           Node
   2    Berlin           Germany 2
   3    New York City USA              -5
                                             Partition 2    P1-Primary    P1-Secondary
   4    Tokyo            Japan         9
   5    Athens           Greece        2
                                                           P2-Secondary    P2-Primary
                                             Partition 3
   6    Moscow           Russia        4
   7    Oslo             Norway        2
                                                                Node Group 1
                                             Partition 4
   8    Beijing          China         8


                                                                Data           Data
                                                                Node           Node
         • Four Data Nodes
         • Two Replicas                                     P3-Primary    P3-Secondary
                                                           P4-Secondary    P4-Primary
         • Two Node Groups
                                                                Node Group 2



Copyright 2010 Action Tech                                                               20
MySQL Cluster Data Node Architecture
     ID      Capital         Country   UTC
     1    Copenhagen         Denmark 2                         Data         Data
                                             Partition 1       Node         Node
     2    Berlin             Germany 2
     3    New York City USA            -5                                  P1-
                                             Partition 2   P1-Primary
     4    Tokyo              Japan     9                      P2-       Secondary
                                                                        P2-Primary
     5    Athens             Greece    2                   Secondary
                                             Partition 3
     6    Moscow             Russia    4                                             data still
                                                               Node Group 1          available
     7    Oslo               Norway    2
                                             Partition 4
     8    Beijing            China     8


                                                               Data         Data
                                                               Node         Node

              • Four Data Nodes                                            P3-
                                                           P3-Primary
              • Two Replicas                                  P4-       Secondary
                                                                        P4-Primary
              • Two Node Groups                            Secondary

                                                               Node Group 2



Copyright 2010 Action Tech                                                                        21
Scale-Out Cluster Architecture

                             MySQL          MySQL      MySQL    MySQL     MySQL
                             Server         Server     Server   Server    Server




                                        NDB API


                         Data              NDB          Data             Management
                         Node                           Node               Server
                                      Storage Engine

                         Data                           Data
                         Node                           Node             Management
                                                                           Server




                                              MySQL Cluster



Copyright 2010 Action Tech                                                            22
Cluster Architecture – Fail Over


                              MySQL          MySQL      MySQL    MySQL     MySQL
                              Server         Server     Server   Server    Server




                                         NDB API


                             Data           NDB          Data             Management
                             Node                        Node               Server
                                       Storage Engine


                             Data                        Data
                             Node                        Node             Management
                                                                            Server




                                               MySQL Cluster


Copyright 2010 Action Tech                                                             23
MySQL Cluster w/Replication
       Writes & Reads
                               Web/App
                                Server
                                                                                           Writes

                                                                    relay
                                                                   binlog

                                    mysqld
                                                                         I/O                SQL
                                     binlog_index                      Thread              Thread

                                         index &
                                         binlogs
                                                                                                    Data Nodes

     Data Nodes                                                                              NDB

                              NDB
                                                    Replication                   binlog

                                                                  mysqld
       apply_status


                             MySQL Master                                  MySQL Slave


                MySQL Cluster                                                   MySQL Cluster



  Global Redundancy: Business Continuity & Disaster Recovery



Copyright 2010 Action Tech                                                                                       24
Heartbeat & MySQL Replication
• 心跳机制---- 资源故障切换的安全管理
• 虚拟IP管理---- 对应用访问数据库透明化
• 方案特点
  > 配置简易性
  > 开源、低成本
  > 冗余资源自动管理
  > 自动VIP管理
  > 异步的数据复制
  > 手动或自动管理 Fail Over切换



Copyright 2010 Action Tech                         25
Heartbeat & MySQL Replication
                                    Web/App
                                     Server




                              Linux Heartbeat
                                = Virtual IP =
    Master Server                10.10.10.10     Slave Server




                                Replication        = Private IP =
         = Private IP =
          10.10.10.20                               10.10.10.21




Copyright 2010 Action Tech                                          26
Heartbeat & MySQL Replication (Fail Over)
                                     Web/App
                                      Server




                             Linux Heartbeat
                                 = Virtual IP =
    Master Server                 10.10.10.10     Slave Server


                             X
                                 Replication
         = Private IP =                           = Private IP =
          10.10.10.20                              10.10.10.21




Copyright 2010 Action Tech                                         27
Heartbeat, Block-Replication & MySQL
 • 分布式块复制 (DRBD)
   > 运行在标准网络协议上的分布式存储机制
   > 功能类似网络 RAID
   > 异步块复制
 • 特点
   > 开源的DRBD配置和安装
   > 无需专有的网络协议
   > 更高的复制性能 (blocks vs. rows of data)
   > 出现故障时自动切换接管数据
   > 结合heartbeat机制管理 fail over 和 virtual
     IPs

Copyright 2010 Action Tech                         28
Heartbeat, Block-Replication & MySQL
                                   Web/App
                                    Server




                             Linux Heartbeat
                               = Virtual IP =
    Active Server               10.10.10.10
                                                Passive Server



        = Private IP =                             = Private IP =
         10.10.10.20                                10.10.10.21

   Primary DRBD                    DRB              Secondary DRBD
                                    D




Copyright 2010 Action Tech                                           29
Heartbeat, Block-Replication & MySQL
                                    Web/App
                                     Server




                             Linux Heartbeat
                               = Virtual IP =
      Active Server             10.10.10.10     Passive Server



          = Private IP =
           10.10.10.20
                             X                     = Private IP =
                                                    10.10.10.21

     Primary DRBD                  DRB              Secondary DRBD
                                    D




Copyright 2010 Action Tech                                           30
MySQL w/ Shared Storage
     • Active/Passive
       > 多实例并行访问同一份数据文件

     • 特点
       > 较高的成本开销 (storage, hardware,
         software)
       > 自动管理Virtual IPs,Fail over,数据一致
       > fail over ,Mounting file systems时间略长
       > 众多的数据库配置可缩短接管时间
       > 严格的切换次序



Copyright 2010 Action Tech                             31
MySQL w/ Shared Storage Architecture
                                     Web/App
                                      Server




                             Cluster Management
                                 = Virtual IP =
                                  10.10.10.10
        Active Server                                Passive Server



           Cluster Agent                               Cluster Agent

            = Private IP =                              = Private IP =
             10.10.10.20                                 10.10.10.21

                                               SAN



Copyright 2010 Action Tech                                               32
MySQL w/ Shared Storage (Fail Over)
                                     Web/App
                                      Server




                             Cluster Management    Passive Server
     Active Server               = Virtual IP =
                                  10.10.10.10


                               X
        Cluster Agent
                               X
                                                     Cluster Agent


         = Private IP =                               = Private IP =
          10.10.10.20                                  10.10.10.21

                                             SAN

Copyright 2010 Action Tech                                             33
Agenda

   •    Questions
   •    HA Terms and Concepts
   •    HA Solutions with MySQL
   •    Other Solutions




Copyright 2010 Action Tech            34
Additional HA Solutions for MySQL
  • Load Balancing with MySQL Replication
    LVS Management(Similar to Linux Heartbeat)
  • Semi-synchronous Replication (google code)
  • Proxy with MySQL Replication




Copyright 2010 Action Tech                              35
Additional HA Solutions for MySQL
   •      Performance Networking for MySQL Clusters
          > Dolphin SCI Interconnect
   •      Operating System Clustering
          > Sun Solaris – Sun Cluster
          > Microsoft Windows Clustering Services
          > Red Hat Cluster Suite
   •      High Availability Middleware
          > Continuent m/Cluster http://solutions.mysql.com/
   •      Backup Software
          > Zmanda

                             HighAvailability.com


Copyright 2010 Action Tech                                     36
Additional MySQL HA Resources
     • MySQL Training
     • MySQL Network
     • White Papers
     • Case Studies
     • Press Releases, News, Events
     • Live Webinars
     • Webinars on Demand               www.mysql.com




Copyright 2010 Action Tech                              37
谢谢!


           技术服务热线            400-820-6580

           在线服务网站            www.actionsky.com




Copyright 2010 Action Tech                             38

MySQL高可用

  • 1.
    MySQL高可用性 上海爱可生信息技术有限公司 曹妙霞 2010-10-16 Copyright 2010 Action Tech 1
  • 2.
    Agenda • Questions • HA Terms and Concepts • HA Solutions with MySQL • Other Solutions Copyright 2010 Action Tech 2
  • 3.
    Agenda • Questions • HA Terms and Concepts • HA Solutions with MySQL • Other Solutions Copyright 2010 Action Tech 3
  • 4.
    Questions •Do you need HA? • Can you afford to lose some data? • How qualified is your IT department or support company? • How much are you ready to invest? Copyright 2010 Action Tech 4
  • 5.
    Agenda • Questions • HA Terms and Concepts • HA Solutions with MySQL • Other Solutions Copyright 2010 Action Tech 5
  • 6.
    Definitions •High Availability (HA) > 当出现故障时,保证资源的可用性 • Two major categories > 硬件的可用性(如:硬件冗余) > 软件的可用性(如:解决方案) • Continuous Availability > 持续的、无间断的服务 > 无故障停滞的时间 Copyright 2010 Action Tech 6
  • 7.
    Definitions • Fault Tolerance > 当硬件或软件故障时,保证资源的可用性 > 硬件和软件冗余,如:cpu,memory,IO,MySQL APP • Single Point of Failure > 单台机器的硬件或软件出现故障时,停止服务 • Fail Over > 当出现故障时,服务切换到可用的资源平台 Copyright 2010 Action Tech 7
  • 8.
    Scale-Up vs. Scale-Out •Scale-Up > 垂直升级,昂贵的硬件 > 专有软件 > 平台绑定 • Scale-Out > 水平升级,普通的硬件 > 开源软件 > 平台独立 Copyright 2010 Action Tech 8
  • 9.
    The Five 9sof Availability Copyright 2010 Action Tech 9
  • 10.
    Agenda • Questions • HA Terms and Concepts • HA Solutions with MySQL • Other Solutions Copyright 2010 Action Tech 10
  • 11.
    MySQL High AvailabilitySolutions • MySQL Replication • MySQL Cluster • MySQL Cluster & Replication • Linux Heartbeat & MySQL Replication • Linux Heartbeat, DRBD & MySQL • MySQL w/ Shared-Storage & Clustering Agents Copyright 2010 Action Tech 11
  • 12.
    MySQL Replication • 异步数据复制(Statement & Row-based、 mixed) • 负载均衡,在线热备,水平扩展 • 跨网数据复制 • Fail Over ---- 手动切换 Copyright 2010 Action Tech 12
  • 13.
    MySQL Replication Topologies Master > Slave Master > Slaves Masters > Slave (Multi-Source) Master > Slave > Slaves Circular (Multi-Master) Master < > Master (Multi-Master) Copyright 2010 Action Tech 13
  • 14.
    Inside MySQL Replication Writes & Reads Web/App Server Writes relay binlog mysqld I/O SQL Thread Thread index & binlogs data Replication binlog data mysqld MySQL Master MySQL Slave Copyright 2010 Action Tech 14
  • 15.
    MySQL Replication (Scale-Out) Web/App Server Writes & Reads Reads Master Server Slave Server Backups Writes Replication Writes Index & Possible Roles Bin Log • Fail over server • Used for performing backups Rotation • Read load balancing • Additional slaves allow Scale-Out Copyright 2010 Action Tech 15
  • 16.
    MySQL Replication (Fail-Over) Web/App Server X Writes & Reads Master Server Slave Server Writes Replication Writes Index & Bin Log Rotation Copyright 2010 Action Tech 16
  • 17.
    MySQL Cluster • 无共享存储的数据存储模式 • 实时同步数据 (2-phase commit) • 特有应用下的高性能,高事物吞吐量 • 无特殊硬件需求 • 内存存储数据 & 硬盘存储数据 (5.1) • Fail Over----快速、自动 • 数据节点间心跳机制 Copyright 2010 Action Tech 17
  • 18.
    MySQL Cluster Components • 数据节点(Data Nodes) > 实时同步数据 > 按组存储数据 • 管理节点(Management Nodes) > 用于集群的起停 > 状态查询、配置生效、在线热备 • SQL节点(MySQL or NDB API Nodes) > 标准MySQL Client or NDB API接口 > 在线平行扩展 Copyright 2010 Action Tech 18
  • 19.
    Basic Cluster Architecture MySQL Server or NDB Web/App Web/App API Server Server for all Writes & Reads MySQL MySQL Server Server NDB API Management Memory NDB Server & Data Storage Engine Data Disk Node Node Management Server Copyright 2010 Action Tech 19
  • 20.
    MySQL Cluster DataNode Architecture ID Capital Country UTC 1 Copenhagen Denmark 2 Data Data Partition 1 Node Node 2 Berlin Germany 2 3 New York City USA -5 Partition 2 P1-Primary P1-Secondary 4 Tokyo Japan 9 5 Athens Greece 2 P2-Secondary P2-Primary Partition 3 6 Moscow Russia 4 7 Oslo Norway 2 Node Group 1 Partition 4 8 Beijing China 8 Data Data Node Node • Four Data Nodes • Two Replicas P3-Primary P3-Secondary P4-Secondary P4-Primary • Two Node Groups Node Group 2 Copyright 2010 Action Tech 20
  • 21.
    MySQL Cluster DataNode Architecture ID Capital Country UTC 1 Copenhagen Denmark 2 Data Data Partition 1 Node Node 2 Berlin Germany 2 3 New York City USA -5 P1- Partition 2 P1-Primary 4 Tokyo Japan 9 P2- Secondary P2-Primary 5 Athens Greece 2 Secondary Partition 3 6 Moscow Russia 4 data still Node Group 1 available 7 Oslo Norway 2 Partition 4 8 Beijing China 8 Data Data Node Node • Four Data Nodes P3- P3-Primary • Two Replicas P4- Secondary P4-Primary • Two Node Groups Secondary Node Group 2 Copyright 2010 Action Tech 21
  • 22.
    Scale-Out Cluster Architecture MySQL MySQL MySQL MySQL MySQL Server Server Server Server Server NDB API Data NDB Data Management Node Node Server Storage Engine Data Data Node Node Management Server MySQL Cluster Copyright 2010 Action Tech 22
  • 23.
    Cluster Architecture –Fail Over MySQL MySQL MySQL MySQL MySQL Server Server Server Server Server NDB API Data NDB Data Management Node Node Server Storage Engine Data Data Node Node Management Server MySQL Cluster Copyright 2010 Action Tech 23
  • 24.
    MySQL Cluster w/Replication Writes & Reads Web/App Server Writes relay binlog mysqld I/O SQL binlog_index Thread Thread index & binlogs Data Nodes Data Nodes NDB NDB Replication binlog mysqld apply_status MySQL Master MySQL Slave MySQL Cluster MySQL Cluster Global Redundancy: Business Continuity & Disaster Recovery Copyright 2010 Action Tech 24
  • 25.
    Heartbeat & MySQLReplication • 心跳机制---- 资源故障切换的安全管理 • 虚拟IP管理---- 对应用访问数据库透明化 • 方案特点 > 配置简易性 > 开源、低成本 > 冗余资源自动管理 > 自动VIP管理 > 异步的数据复制 > 手动或自动管理 Fail Over切换 Copyright 2010 Action Tech 25
  • 26.
    Heartbeat & MySQLReplication Web/App Server Linux Heartbeat = Virtual IP = Master Server 10.10.10.10 Slave Server Replication = Private IP = = Private IP = 10.10.10.20 10.10.10.21 Copyright 2010 Action Tech 26
  • 27.
    Heartbeat & MySQLReplication (Fail Over) Web/App Server Linux Heartbeat = Virtual IP = Master Server 10.10.10.10 Slave Server X Replication = Private IP = = Private IP = 10.10.10.20 10.10.10.21 Copyright 2010 Action Tech 27
  • 28.
    Heartbeat, Block-Replication &MySQL • 分布式块复制 (DRBD) > 运行在标准网络协议上的分布式存储机制 > 功能类似网络 RAID > 异步块复制 • 特点 > 开源的DRBD配置和安装 > 无需专有的网络协议 > 更高的复制性能 (blocks vs. rows of data) > 出现故障时自动切换接管数据 > 结合heartbeat机制管理 fail over 和 virtual IPs Copyright 2010 Action Tech 28
  • 29.
    Heartbeat, Block-Replication &MySQL Web/App Server Linux Heartbeat = Virtual IP = Active Server 10.10.10.10 Passive Server = Private IP = = Private IP = 10.10.10.20 10.10.10.21 Primary DRBD DRB Secondary DRBD D Copyright 2010 Action Tech 29
  • 30.
    Heartbeat, Block-Replication &MySQL Web/App Server Linux Heartbeat = Virtual IP = Active Server 10.10.10.10 Passive Server = Private IP = 10.10.10.20 X = Private IP = 10.10.10.21 Primary DRBD DRB Secondary DRBD D Copyright 2010 Action Tech 30
  • 31.
    MySQL w/ SharedStorage • Active/Passive > 多实例并行访问同一份数据文件 • 特点 > 较高的成本开销 (storage, hardware, software) > 自动管理Virtual IPs,Fail over,数据一致 > fail over ,Mounting file systems时间略长 > 众多的数据库配置可缩短接管时间 > 严格的切换次序 Copyright 2010 Action Tech 31
  • 32.
    MySQL w/ SharedStorage Architecture Web/App Server Cluster Management = Virtual IP = 10.10.10.10 Active Server Passive Server Cluster Agent Cluster Agent = Private IP = = Private IP = 10.10.10.20 10.10.10.21 SAN Copyright 2010 Action Tech 32
  • 33.
    MySQL w/ SharedStorage (Fail Over) Web/App Server Cluster Management Passive Server Active Server = Virtual IP = 10.10.10.10 X Cluster Agent X Cluster Agent = Private IP = = Private IP = 10.10.10.20 10.10.10.21 SAN Copyright 2010 Action Tech 33
  • 34.
    Agenda • Questions • HA Terms and Concepts • HA Solutions with MySQL • Other Solutions Copyright 2010 Action Tech 34
  • 35.
    Additional HA Solutionsfor MySQL • Load Balancing with MySQL Replication LVS Management(Similar to Linux Heartbeat) • Semi-synchronous Replication (google code) • Proxy with MySQL Replication Copyright 2010 Action Tech 35
  • 36.
    Additional HA Solutionsfor MySQL • Performance Networking for MySQL Clusters > Dolphin SCI Interconnect • Operating System Clustering > Sun Solaris – Sun Cluster > Microsoft Windows Clustering Services > Red Hat Cluster Suite • High Availability Middleware > Continuent m/Cluster http://solutions.mysql.com/ • Backup Software > Zmanda HighAvailability.com Copyright 2010 Action Tech 36
  • 37.
    Additional MySQL HAResources • MySQL Training • MySQL Network • White Papers • Case Studies • Press Releases, News, Events • Live Webinars • Webinars on Demand www.mysql.com Copyright 2010 Action Tech 37
  • 38.
    谢谢! 技术服务热线 400-820-6580 在线服务网站 www.actionsky.com Copyright 2010 Action Tech 38