This document discusses building a fault tolerant and highly available architecture for OpenStack. It proposes: 1. A master-master cluster architecture for MySQL and session-level replication for RabbitMQ to provide high availability for the database and message broker components. 2. Disk-level replication using DBRD for Glance, Swift, and Cinder to provide redundancy at the storage level. 3. Ensuring high availability for networking and the Horizon dashboard. 4. Developing predictive and reactive models to detect failures in Nova, Swift, and compute instances and enable recovery of all components. The document recommends using Pacemaker for cluster-level management and Corosync for reliable messaging between cluster nodes.

1. High Availability and Fault Tolerance
(OpenStack)
Deepak Mane
Cloud Architect

2. Objective & Motivation
• To Build a a Fault Tolerance and High Availability
Architecture (OpenStack)
• Motivation
– To build a fault tolerance architecture for OpenStack
– Build a Cluster Architecture for MySQL RabbitMQ
components
– To build high availability architecture for network
– To build a predictive and reactive model for detecting
failures of Nova , Swift and Compute.
–

3. Use cases
• Master-Master Cluster architecture for MySQL
• Disk Level replication for mySQL using DBRD for
Glance , Swift and Cinder
• Session level replication for RabbitMQ
• High availability for networking
• High availability for Horizon (Openstack
dashboard)
• Predictive model for detecting failure for all
components
• Reactive model for recovery for all components.

4. Non Use Cases
• Scenarios not suitable for cloud
– Redundancy of network components, such as
switches and routers,
– Redundancy of applications and automatic service
migration,
– Redundancy of storage components,
– Redundancy of facility services such as power, air
conditioning, fire protection, and others

5. Pacemaker – High availability for
OpenStack
• Cluster stack, the state of- the-art high availability
and load balancing stack for the Linux platform
• Storage- and application-agnostic, and is in no
way specific to OpenStack
• Pacemaker relies on the Corosync messaging
layer for reliable cluster communications.
• Corosync implements the Totem single-ring
ordering and membership protocol and provides
UDP and InfiniBand based messaging, quorum,
and cluster membership to Pacemaker.

6. Required packages
• pacemaker
• corosync
• cluster-glue
• resource-agents

7. Architecture and Technology
Information

8. HA Architecture – Cloud Controller
High availability
Implemented using DBRD , Pacemaker , Corosync

9. DBRD Architecture – MySQL-Cloud
Controller
Master Cloud controller Slave Cloud controller

10. DBRD Architecture – RabbitMQ- Cloud Controller
Master Cloud controller Slave Cloud controller

11. Nova - Recovery mode Approach
• Nova - Recovery mode Approach

12. Cloud Controller – Recovery approach