Successfully reported this slideshow.
Your SlideShare is downloading. ×

OpenStack HA

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Upcoming SlideShare
OpenStack High Availability
OpenStack High Availability
Loading in …3
×

Check these out next

1 of 26 Ad

More Related Content

Slideshows for you (19)

Similar to OpenStack HA (20)

Advertisement

More from Kenneth Hui (17)

Advertisement

OpenStack HA

  1. 1. 1 HA in OpenStack September 24th 2013
  2. 2. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Agenda • What is HA • Compute HA • Controller HA • Corosync, Pacemaker and DRBD • Galera • HAProxy, keepalived, VRRP • Resources and Summary 2
  3. 3. RACKSPACE® HOSTING | WWW.RACKSPACE.COM What is HA? 3
  4. 4. RACKSPACE® HOSTING | WWW.RACKSPACE.COM High Availability 4 Minimize data loss Minimize system downtime
  5. 5. RACKSPACE® HOSTING | WWW.RACKSPACE.COM High Availability Concepts • Stateless services – There is no dependency between requests – For example: Nova API, Nova Scheduler, etc. • Stateful services – An action typically comprises multiple requests – For example: MySQL, RabbitMQ, etc. 5
  6. 6. RACKSPACE® HOSTING | WWW.RACKSPACE.COM High Availability Concepts • Active/Passive – Redundant instances of stateless services are load balanced – For Stateful services a replacement resource can be brought online. • Active/Active – Redundant instances of stateless services are load balanced – Stateful services are managed in such a way that services are redundant, and that all instances have an identical state. – Updates to one instance of a database would also update all other instances. 6
  7. 7. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Compute HA 7
  8. 8. RACKSPACE® HOSTING | WWW.RACKSPACE.COM No vSphere Style HA with KVM 8
  9. 9. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Server Evacuation 9 Evacuation Evacuation • Without Shared Storage – The instance will be booted from a new disk, but will preserve the configuration, e.g. id, name, uid, ip...etc. • With Shared Storage – The instance will be booted from same disk and data will be preserved
  10. 10. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Instance Migration 10
  11. 11. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 11 Virtualization vs. Cloud • Virtualization needs care and feeding – Name the VM – Tune and groom regularly – Feed it with good food and supplements – Take to the vet when sick • Cloud servers are disposable – VMs are not unique – Tune and groom apps not the cows – Keep the cow upright – Shoot the cow when it is sick
  12. 12. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Scale Up vs. Scale Out 12 Traditional Cloud
  13. 13. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Application HA 13
  14. 14. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Controller HA Options 14
  15. 15. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Pacemaker, Corosync and DRBD 15 • Pacemaker – high availability and load balancing stack for the Linux platform. – Interacts with applications through Resource Agents (RA) • Corosync – Totem single-ring ordering and membership protocol – UDP and InfiniBand based messaging, quorum, and cluster membership to Pacemaker. • DRBD (Distributed Replication Block Device) – Synchronizes Data at the block device – Uses a journaling system (such as ext3 or ext4)
  16. 16. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Galera • Synchronous multi-master cluster technology for MySQL/InnoDB – MySQL patched for wsrep (Write Set REPlication) – Active/active multi-master topology – Read and write to any cluster node – True parallel replication, in row level – No slave lag or integrity issues 16
  17. 17. RACKSPACE® HOSTING | WWW.RACKSPACE.COM HA with Keepalived and HAProxy 17
  18. 18. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Keepalived, HAProxy and VRRP • HAProxy – Load Balancing and Proxying for HTTP and TCP Applications – Works over multiple connections – Used to load balance API services • VRRP (Virtual Router Redundancy Protocol) – Eliminates SPOF in a static default routed environment • Keepalived – Based on Linux Virtual Server (IPVS) kernel module to provide layer 4 Load Balancing – Implements a set of checkers to check service status and to maintain health – Leverage the VRRP Protocol to remap VIPS in event of failure 18
  19. 19. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Sample OpenStack HA Architecture 19 .… Availability Zone 1 Dedicated Firewalls BOND 0 BOND 1 BOND 0 BOND 1 Controller API Services API & Horizon Cinder API Nova Scheduler Keystone Glance RabbitMQ MYSQL Chef Server Recipes Load Balancers Redundant Network Switches Storage EMC, NetApp, or Solidfire Vols BOND 2 Redundant Network Switches Inside LB VLAN Storage Network (private) Fixed Network (private) Compute 1 KVM G2 G1 G4 G3 Compute N KVM G6 G5 G7 BOND 0 BOND 1 BOND 2 .… Availability Zone 2 BOND 0 BOND 1 BOND 2 Compute 1 KVM Compute N KVM G1 6 G15 G17 BOND 0 BOND 1 BOND 2 BOND 0 BOND 1 BOND 2 G12 G11 G14 G13 BOND 2 Controller API Services API & Horizon Cinder API Nova Scheduler Keystone Glance RabbitMQ MYSQL Chef Server Recipes
  20. 20. RACKSPACE® HOSTING | WWW.RACKSPACE.COM HA on OpenStack 20
  21. 21. RACKSPACE® HOSTING | WWW.RACKSPACE.COM HA on OpenStack 21
  22. 22. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 22 Comparison Database Replication method Strengths Weakness/Limita tions Keepalived/HAPro xy/VRRP Works on MySQL master-master replication Simple to implement and understand. Works for any storage system. Master-master replication does not work beyond 2 nodes. Pacemaker/Coros ync/DRBD Mirroring on Block Devices Well tested More complex to setup. Split Brain possibility Galera Based on write- set Replication (wsrep) No Slave lag Needs at least 3 nodes. Relatively new. Others MySQL Cluster, RHCS with DAS/SAN storage Well tested More complex setup.
  23. 23. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Resources and Contact 23
  24. 24. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Resources • OpenStack – openstack.org – launchpad.net/openstack – #openstack – #openstack on webchat.freenode.net • OpenStack HA – http://docs.openstack.org/trunk/openstack-ha/openstack-ha-guide-trunk.pdf – https://github.com/rcbops-cookbooks/ • MySQL HA – http://www.mysql.com/why-mysql/white-papers/mysql-high-availability-drbd-configuration- deployment-guide/ – http://dev.mysql.com/doc/refman/5.7/en/ha-overview.html – https://www.hastexo.com/ – http://www.drbd.org/
  25. 25. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 2 5 RACKSPACE® HOSTING | WWW.RACKSPACE.COM For More Information You can reach me at: Kenneth Hui Open Cloud Architect Rackspace E-mail: ken.hui@rackspace.com Twitter: @hui_kenneth Blog: http://cloudarchitectmusings.com
  26. 26. 26 RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218 US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN TH E UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM

Editor's Notes

  • I work as an Open Cloud Architect with Rackspace. Rackspace has the distinction of being one of the founders of OpenStack and the proprietor of the largest OpenStack-based Public Clouds in the world. As such, we have a vested desire and interest in driving the adoption of OpenStack and the growth of it’s community.However, Rackspace does not own OpenStack and has in fact given it to the community via the Linux Foundation and the the Apache 2 license. In that same spirit, I am happy to help anyone who wishes to learn more about OpenStack, regardless of rather you are now or ever will be a Rackspace customer. So please feel free to contact me and to ask questions or to have me talk about OpenStack with your company or user group. I tweet frequently about OpenStack and maintain a blog that often has OpenStack and/or Cloud Computing related posts.

×