High availability (HA) is a major need for the vast majority of enterprise software applications. On the other hand, cloud environments are built with the rule that any instance is expected to fail. What should i do with my workloads that require high availability?

1. High Availability in OpenStack
Athens OpenStack User Group #OSATH
13th Meetup, 12th May 2015
Thanassis Parathyras,
aparathyras@stackmasters.eu, @parathyras

2. Announcements
Latest release brings bare-metal provisioning with Ironic
Greek Mailing List
• openstack-el@lists.openstack.org
• Join at http://lists.openstack.org
OpenStack Summit (https://www.openstack.org)
• 18-22 May, Vancouver
OpenStack CEE Day (http://openstackceeday.com)
• 8 June, Budapest

3. Outline
• HA in cloud environments
• OpenStack Orchestration Heat
• Example on Heat
– design structure
– web service HA
• More approaches to implement HA
– legacy and new stuff
• What about the cloud platform itself
– an operator’s view

4. Why HA is required
“cloud instances are expected to fail”
“pets vs cattles”
already established industry approach
w/ VMware DRS and HA for example

5. Where to apply HA
• Workloads
– VMs
– Applications
– Services
• OpenStack services
– Compute, Network, Storage, Identity, Image, …
– APIs, schedulers, agents
• Platform services
– Database (MySQL)
– AMQP (RabbitMQ)
We focus on HA for Workloads

6. Orchestrate with Heat
• Standard template-based mechanism
• AWS CloudFormation and HOT
(Heat Orchestration Template Language)
• Can handle HA
• Combined w/ Ceilometer supports auto-scaling

7. Heat basics
Template
Parameters
Resources
Outputs
Resource
Reference
Properties
Attributes

8. Heat concepts
Stacks are created from templates
• Single file template (design blueprint)
• OpenStack resources
– cloud instances,
– networks,
– storage devices,
– any other cloud resource
• Relationships between resources
– Volume attachments, etc.
• Nested stacks
• Actions on failure to support HA
• Auto-scaling policy and resource allocation

9. Stack structure - visual
stack
Resource
Resource
…
stack
Nova instance
Cinder volume
Attachment
Nested stack

10. Template structure - code
{ "AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "AWS CloudFormation Sample Template“,
"Parameters" : { "KeyName" : {…}, "InstanceType" : {…}, … },
"Mappings" : { "AWSInstanceType2Arch" : {…}, … },
"Resources" : { "User" : {…}, "Keys" : {…},
"WebServerRestartPolicy" : { "Type" : "OS::Heat::HARestarter", "Properties" : {…} },
"HttpFailureAlarm": { "Type": "AWS::CloudWatch::Alarm", "Properties": {…} },
"DatabaseServer" : { "Type" : "AWS::EC2::Instance", … , "Properties": {…} },
},
"Outputs" : { "WebsiteURL" : {...} } }
Source: https://github.com/openstack/heat-
templates/blob/master/cfn/F17/WordPress_Single_Instance_With_HA.template

11. Beyond OpenStack
Workloads HA may further be managed using tools and
mechanisms
• Linux HA clusters
– Continue to use existing solutions
• Configuration Management
– Triggered on failure detection
– Puppet, Chef, Ansible, SaltStack
• utilize oslo library (NEW with Kilo)
– Features heartbeat monitoring

12. On the management level
OpenStack services are treated like any other service, distinguished in:
• stateless – API servers
– Scale horizontally
• stateful – schedulers, agents
– Need for the service to be HA-aware (and capable)
• Swift has built-in data redundancy
Deploy external OSS solutions based on
• HAProxy (provides load-balancing w/ health check)
• keepalived (cluster VIP, implements VRRP)
• pacemaker (cluster management)
• zookeeper (service management)

13. On the platform level
All OpenStack services are operating on top of a Database, while
most of them communicate internally through a Message Queue
HA on this level is managed by external solutions
• Galera
– Synchronous replication across MySQL clusters
– Doesn’t require a shared storage device
• RabbitMQ Mirrored Queues
– Replicated messages over multiple RabbitMQ instances

14. Get involved
• Documentation
– http://docs.openstack.org
• Join the community
– http://www.openstack.org/community
• Greek mailing list
– http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-el
• Contribute
– code (development, blueprints, reviews, bugs)
– docs writing, translations, infrastructure support

15. Thank you!
Athens OpenStack User Group #OSATH
http://www.meetup.com/Athens-OpenStack-User-Group
Thanassis Parathyras
aparathyras@stackmasters.eu, @parathyras