MHA, Murakumo & Me

8,078 views

Published on

Using MHA (Master High Availability Manager for MySQL) and Murakumo (distributed internal dns) to achieve a simple highly-available environment.

#autotechcasual

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
8,078
On SlideShare
0
From Embeds
0
Number of Embeds
5,483
Actions
Shares
0
Downloads
20
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • MHA, Murakumo & Me

    1. 1. MHA, MURAKUMO, & ME Automation Tech Casual Talks #1 7/17/2012
    2. 2. 自己紹介 (@IJIN)• Michael H. Oshita• Japanese American• In Japan since 2002• Software/Infra Engineer• http://ijin.github.com
    3. 3. DBの冗長化• Master - Slave• DRBD• MySQL Cluster• MMM• RDS
    4. 4. DBの冗長化 (MASTER/SLAVE)• シンプル• 手動フェールオーバー対応• 検知・対応まで数十分単位
    5. 5. DBの冗長化 (DBRD)• 複雑• sync_binlogでwrite IO低下
    6. 6. DBの冗長化 (MYSQL CLUSTER)• 複雑• NDB storage engine。。
    7. 7. DBの冗長化 (MMM)• メンテされてない• 過去ケースを見ても不安定(Percona) • http://www.xaprb.com/blog/2011/05/04/whats-wrong-with-mmm/
    8. 8. DBの冗長化 (RDS)• 楽で好き• IOがちょっと。• 政治的な理由で使えなかったり。。
    9. 9. MHA (Master High Availability Manager for MySQL)• シンプル (Master-slaveそのまま)• 自動フェールオーバー (各サーバで最新のbinlogを適用)• 安定性(DeNAさんで数百台のDBで運用)
    10. 10. MHA Masterで障害発生 Slave ManagerMaster Slave
    11. 11. MHA各slaveに最新binlogを適用し、Master昇格 binlog binlog Master Manager Master binlog Slave
    12. 12. MHA FAILOVERシーケンス"Starting master failover.""* Phase 1: Configuration Check Phase..n""* Phase 2: Dead Master Shutdown Phase..n" ==> ここで旧Master情報削除 (master_ip_failover)"* Phase 3: Master Recovery Phase..n" "* Phase 3.1: Getting Latest Slaves Phase..n" "* Phase 3.2: Saving Dead Masters Binlog Phase..n" "* Phase 3.3: Determining New Master Phase..n" ==> ここで新Master情報登録 (master_ip_failover)"* Phase 4: Slaves Recovery Phase..n"
    13. 13. MHA 切替方法は2種類 (master_ip_failoverで定義)• Virtual IP• Catalog DB Server (DNS)
    14. 14. MHA 切替方法は2種類• Virtual IP → 使えない場合も。。• Catalog DB Server (DNS) → 内部DNSサーバ運用面倒。。
    15. 15. そこで
    16. 16. By Genki Sugawara @ Cookpad
    17. 17. MURAKUMO 分散内部DNS• Gossip Protocolで拡散• ruby製 (1.9.2まで)• 操作が簡単• 余分なサーバがいらない
    18. 18. MURAKUMO Gossip Protocolnode nodenode node
    19. 19. MURAKUMO Syntax=== addition of a record shell> mrkmctl -A foo.bar,300,master,100 shell> mrkmctl -L IP address TTL Priority Weight Activity Hostname --------------- ------ -------- ------ -------- ---------- 10.11.12.13 60 Origin - Active my-host 10.11.12.13 300 Master 100 Active foo.bar=== deletion of a record shell> mrkmctl -D foo.bar shell> mrkmctl -L IP address TTL Priority Weight Activity Hostname --------------- ------ -------- ------ -------- ---------- 10.11.12.13 60 Origin - Active my-host
    20. 20. MURAKUMO chefで一発登録もknife bootstrap 10.119.45.233 -N db2 -j {"set_fqdn":"db2"} -r recipe[hostname], role[base], role[db]
    21. 21. DEMO

    ×