Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Dockerizing OpenStack for High Availability


Published on

Presentation at the OpenStack Summit in Paris, France on November 4, 2014.

High availability in OpenStack can be achieved in many ways. In this session we will describe how Docker can be used to provide an active-active highly available OpenStack environment. We will focus the real world work that we have done to "Dockerize" OpenStack services, detail the advantages to this type of deployment (rapid deployment, rapid scale out, versioning, etc.), and walk through our design - from requirements, limitations, obstacles, and especially our decisions. We will use our experiences as examples to provide real world best practices, as well as showing a demonstration of the environment in action.

Manuel Silveyra - Senior Cloud Solutions Architect
Daniel Krook - Senior Certified IT Specialist
Shaun Murakami - Senior Cloud Solution Architect
Kalonji Bankole - Cloud Architect

Published in: Technology
  • Excellent Arcticle I would add the Port methodology and hardening on the Container to bring it to light.
    Are you sure you want to  Yes  No
    Your message goes here

Dockerizing OpenStack for High Availability

  1. 1. © 2014 IBM Corporation Dockerizing OpenStack High Availability A Practical Approach Manuel Silveyra - Senior Cloud Solutions Architect @manuel_silveyra Daniel Krook - Senior Certified IT Specialist @DanielKrook Shaun Murakami - Senior Cloud Solution Architect @stmuraka Kalonji Bankole - Cloud Architect @k_bankole
  2. 2. OpenStack Summit Atlanta May 2014 A Practical Approach to Deploying a Highly Available OpenStack © 2014 IBM Corporation
  3. 3. OpenStack high availability challenges • There were a lot of possible configuration options • Active/Active • Active/Standby • Installing and configuring is complicated • Keep track of configurations, ports, services, etc. • Scaling increases complexity • Distributing load has different requirements than availability © 2014 IBM Corporation
  4. 4. © 2014 IBM Corporation Our OpenStack HA architecture
  5. 5. That architecture leaves room for improvement © 2014 IBM Corporation • Existing challenges • Many configuration options • Installation is complex • Scaling increases complexity • Automation and visibility • Deployment • Patching • Monitoring
  6. 6. Can Docker help? • A technology that allows applications (and all related dependencies) to be packaged in individual containers. • Containers run as isolated userspace processes on the host OS. • Containers share the Linux kernel. Benefits include • Service isolation • Security • Version control • Portability • Repeatable • Rapid deployment • Very lightweight (close to bare metal) Bare metal Container Virtual machine © 2014 IBM Corporation
  7. 7. Advantages of OpenStack on Docker Faster scaling • New Docker containers start up in seconds Higher density • Lower overhead means more available resources on the host © 2014 IBM Corporation Greater flexibility • Docker standardizes the packaging, configuration, and deployment of services. Which all add up to faster response to changing business requirements for our OpenStack deployments
  8. 8. © 2014 IBM Corporation Before and after Bare Metal Docker Deployment Method Chef Cookbooks Custom Scripts Deployment Preparation Days Hours Deployment Time 15 Mins 5 Mins Scale Time 7 Mins Seconds Scaling Unit Bare Metal Node Service Containers
  9. 9. © 2014 IBM Corporation Our newly Dockerized OpenStack
  10. 10. © 2014 IBM Corporation Docker is a technology that... Leverages Linux containers • Process isolation • libcontainer (abstraction) • cgroups (resource control) • namespaces (isolation) • Host kernel reuse • eliminates redundancy Simulates a VM without overhead • Faster lifecycle operations • minimal operating system • copy, start, stop, delete • Better resource utilization • smaller footprint for both containers and images Provides additional benefits over VMs • Versioning and layering • promotes rapid collaboration and reuse • No hypervisor dependency • highly portable • high performance
  11. 11. © 2014 IBM Corporation Understanding Docker concepts Containers • create, delete, start, stop, restart, pause, resume, save • inspect – view metadata about a container • logs – view stdout and stderr from a container Images • create, delete, export, import • history – show commands used to make an image • along with Dockerfiles, the key persistent unit of Docker Registries • pull, push, tag, search • central location for sharing images • contains community or trusted images
  12. 12. Container Container Docker Daemon Isolation Host © 2014 IBM Corporation Docker Client Base OS/Kernel Container Docker Registry Expose select ports on Host Requires kernel compatible images libcontainer / LXC App Client Understanding Docker management
  13. 13. Docker managed container features • Expose from the container • Proxy through Network ports the host mapping Environment variables • Pass in to set runtime configuration values • Set DNS servers and search domains Network configuration • Set modes: bridged, none, container, host • Limit memory © 2014 IBM Corporation Resource constraints • Limit CPU • Mount from host Storage volumes • Share volumes between containers Restart policy • Set to: on failure, never, always Container privileges • Escalate container access to host resources
  14. 14. Bringing it all together: A simple workflow with Docker © 2014 IBM Corporation • Create and start a new container with docker run Start Ubuntu and run the bash shell docker run –ti ubuntu bash You're now in a new Ubuntu container running bash – experiment or iterate to develop and test apps and configuration. • Create new container using a Dockerfile: FROM ubuntu RUN apt-get update && apt-get install -y openssh-server EXPOSE 22 CMD ["/usr/sbin/sshd", "-D"] docker build –t simple:sshd . docker run -p 2222:22 simple:sshd Now the SSH server is running in a container and ready to be used on port 2222 Start with Ubuntu base image Each RUN action creates a new filesystem layer Only port 22 is available from outside container Command to run when container starts Map port 22 on container to 2222 on host
  15. 15. Running highly available OpenStack services in Docker © 2014 IBM Corporation
  16. 16. Running OpenStack services in Docker © 2014 IBM Corporation 1. Build an image 2. Start a container instance 3. Update load balancer(s) (repeat for all services)
  17. 17. OpenStack Dockerfile example (nova-api) © 2014 IBM Corporation # Create the base operating system layer FROM ubuntu:trusty MAINTAINER Shaun Murakami # Update base image RUN apt-get -y update RUN apt-get -y upgrade # Install OpenStack components RUN apt-get -y install python-software-properties python-mysqldb nova-api # Prepare filesystem for OpenStack components RUN chown -R nova:nova /etc/nova && chown -R root:root /etc/nova/root* && rm /var/lib/nova/nova.sqlite && cp /etc/nova/api-paste.ini /etc/nova/api-paste.ini.orig && echo "admin_token = oWKwDPaUWBNzif92" >> /etc/nova/api-paste.ini && cp /etc/nova/nova.conf /etc/nova/nova.conf.orig # Import nova.conf from the host ADD ./nova.conf /etc/nova/ # Customize container runtime EXPOSE 8774 8775 CMD /usr/bin/python /usr/bin/nova-api --config-file /etc/nova/nova.conf --logfile /var/log/nova/api-`hostname`.log
  18. 18. © 2014 IBM Corporation Create the Docker image docker build –t nova:api . Step 0 : FROM ubuntu:trusty ---> 6b4e8a7373fe Step 1 : MAINTAINER Shaun Murakami <> ---> Using cache ---> 96345089d832 Step 2 : RUN apt-get -y update ---> Running in fc22a3c8812b Step 6 : ADD ./nova.conf /etc/nova/ ---> ba53dd03fcf0 Removing intermediate container 910c4ff92b18 Step 7 : EXPOSE 8774 8775 ---> Running in 5cc44c54c15d ---> a8840d052474 Removing intermediate container 5cc44c54c15d Step 8 : CMD /usr/bin/python /usr/bin/nova-api --config-file /etc/nova/nova.conf --logfile /var/log/nova/api- `hostname`.log ---> Running in e876b1085db9 ---> a35112f528b0 Removing intermediate container e876b1085db9 Successfully built a35112f528b0 ...
  19. 19. OpenStack services running in containers docker run -d -P nova:api © 2014 IBM Corporation
  20. 20. Sharing images using a shared private registry 1. docker tag nova:api 2. docker push 3. docker pull © 2014 IBM Corporation
  21. 21. Scaling OpenStack services with Docker © 2014 IBM Corporation 1. Share images in Docker registry 2. Start a container instance 3. Update load balancer(s)
  22. 22. 1. Docker random port generation makes service management difficult • Fixed ports & script automation 2. Services that require multiple processes • Supervisord to manage and run multiple processes © 2014 IBM Corporation Lessons learned 3. Layer limitations • Combine commands in Dockerfile 4. Debugging isn’t easy (Docker ver. <1.3) • Consolidated logging
  23. 23. Docker processes with consolidated logging • Run command: /usr/bin/python /usr/bin/nova-api --config-file /etc/nova/nova.conf --logfile /var/log/nova/api-`hostname`.log • Export volume when starting: -v /root/openstack_logs/nova:/var/log/nova © 2014 IBM Corporation
  24. 24. OpenStack Docker container management options © 2014 IBM Corporation
  25. 25. © 2014 IBM Corporation Shipyard • Written in Python • Manages multiple Docker hosts • Provides a customizable UI (Django) • Utilizes Docker API to retrieve information • Active community
  26. 26. © 2014 IBM Corporation Summary • Docker improves our highly available architecture in several areas without a major redesign • Faster scaling • Higher density • Greater flexibility • OpenStack services can be encapsulated very easily within Docker containers • Easy to test iteratively • Easy to declare in a Dockerfile • Easy to run and scale • Orchestration of a Docker based OpenStack cluster needs improvement • Many fast moving options are available • Customization of Shipyard worked best for us
  27. 27. IBM technical sessions at the Paris Summit IBM Sessions on Monday, November 3rd 15:20 R.251 When Disaster Strikes the Cloud: Who, What, When, Where and How to recover Ronen Kat, Michael Factor, and Red Hat 11:40 A.Blue IPv6 Features in OpenStack Juno Xu Han Peng, Comcast, and Cisco 15:20 R252 Why is my Volume in 'ERROR' State!?! An Introduction to Troubleshooting Your Cinder Configuration Jay Bryant 16:20 A.Blue Group Based Policy Extension for Networking Mohammad Banikazemi, Cisco, Midokura, and One Convergence IBM Sessions on Tuesday. November 4th 11:15 R252 The perfect match: Apache Spark meets Swift Gil Vernik, Michael Factor, and Databricks 15:40 R242 Docker Meets Swift: A Broadcaster's Experience Eran Rom, and RAI 16:40 Maillot User Group Panel: India, Japan, China Ying Chun Guo, Guang Ya Liu, Qiang Guo Tong 14:50 Passy A Practical Approach to Dockerizing OpenStack High Availability Manuel Silveyra, Shaun Murakami, Kalonji Bankole, Daniel Krook IBM Sessions on Wednesday, November 5th 09:00 R241 Monasca DeepDive: Monitoring at scale Tong Li , Rob Basham, HP and Rackspace 09:00 R242 Beyond 86: Managing multi-platform environments with OpenStack Shaun Murakami, Philip Estes 09:50 R253 Troubleshooting Problems in Heat Deployments Fabio Oliveira, Ton Ngo, Priya Nagpurkar, Winnie Tsang 11:50 R251 Keystone to Keystone Federation Enhancements for Hybrid Cloud Enablement Steve Martinelli, Brad Topol, CERN, and Rackspace 17:50 R253 Practical advice on deployment and management of enterprise workloads Jarek Miszczyk, Venkata Jagana © 2014 IBM Corporation
  28. 28. Learn more at these IBM sponsored sessions on Wednesday: 9:50 Room 243 Step on the Gas: See how Open Technologies are driving the future of the enterprise 11:50 Room 212/213 IBM and OpenStack: Collaborations beyond the code 1:50 Room 212/213 A Use Case Driven view of IBM’s OpenStack based Offerings 2:40 Room 212/213 IBM OpenStack Offerings in Action © 2014 IBM Corporation Stop by the IBM Booth (B4) Demos, games and FREE tee shirt.
  29. 29. © 2014 IBM Corporation Legal Disclaimer • © IBM Corporation 2011. All Rights Reserved. • The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. • References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. • If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete: Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. • If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete: All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. • Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both. • If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete: Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. • If you reference Java™ in the text, please mark the first use and include the following; otherwise delete: Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. • If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete: Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. • If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete: Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. • If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete: UNIX is a registered trademark of The Open Group in the United States and other countries. • If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete: Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. • If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes only.