Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

To kayobe or not to kayobe?


Published on

Deploying, operating and upgrading a production OpenStack cloud can be challenging.

Containers offer a compelling solution for isolating OpenStack services, but do we really want the complexity of running on Kubernetes or Docker Swarm? We want an automated cloud provisioning process, but do we really need a full OpenStack undercloud? Why should upgrades be all or nothing?

This presentation introduces kayobe, an OpenStack deployment tool that aims to answer these and other questions.

Kayobe stands on the shoulders of giants:

* OpenStack bifrost discovers and provisions the cloud
* OpenStack kolla builds container images for OpenStack services
* OpenStack kolla-ansible delivers painless deployment and upgrade of containerised OpenStack services

To this solid base, kayobe adds:

* Configuration of cloud host OS & flexible networking
* Management of physical network devices
* A friendly openstack-like CLI
* All this and more, automated from top to bottom using Ansible.

Published in: Software
  • Nice !! Download 100 % Free Ebooks, PPts, Study Notes, Novels, etc @
    Are you sure you want to  Yes  No
    Your message goes here

To kayobe or not to kayobe?

  1. 1. To Kayobe or not to Kayobe? Mark Goddard Manchester OpenStack Meetup 12th March 2018
  2. 2. Overview ● Kayobe ○ Deployment of containerised OpenStack to bare metal ● Background ● Existing Solutions ● Kayobe ● Demo
  3. 3. Background
  4. 4. Me --- name: Mark Goddard education: degree: Electronic & Communications Engineering institution: University of Bristol career: - tech: Video conferencing bridges company: Tandberg Cisco - tech: High Performance Ethernet switches company: Gnodal - tech: OpenStack & Software Defined Networking (SDN) company: Cray - tech: Scientific OpenStack company: StackHPC
  5. 5. StackHPC ● Bristol-based software consultancy ● “The convergence of HPC & cloud” ● Started ~2 years ago by Stig Telfer & John Taylor ● Current headcount: 9 ● Involved with University of Cambridge, Square Kilometre Array (SKA), Verne Global (hpcDIRECT), Data Direct Networks (DDN), Cray, others... ●
  6. 6. A Kayobe is Born ● Square Kilometre Array (SKA) ● Science Data Processor (SDP) ● Performance Prototype Platform (P3) / ALaSKA ○ Hardware & software test bed ○ Two racks of bare metal compute ○ Heterogeneous nodes ■ High memory, NVMe, GPU ○ Lots of networks ■ 1G, 10G, 25G Ethernet ■ 100G EDR Infiniband ○ Managed by OpenStack ● Kayobe built to address pain points of existing tools
  7. 7. Existing Solutions
  8. 8. Surely OpenStack has enough deployment tools already? ● Yes, but... ● Existing tools have several pain points ● Many new deployment tools are based on Kubernetes
  9. 9. TripleO - OpenStack on OpenStack ● Basis of Red Hat’s OpenStack Platform (RHOS) ● Undercloud ○ Bare metal deployment of control plane ○ A separate, single node OpenStack installation ■ Heat, nova, neutron, ironic, etc. ● Overcloud ○ Multi-node, Highly Available (HA) OpenStack control plane ○ Multiple roles - controller, compute, blockstorage, etc. ○ Now using kolla container images ○ Deployed using a giant monolithic heat stack ■ OpenStack resources (networks, instances) ■ Service installation & configuration
  10. 10. TripleO - Common Pain Points ● Undercloud is complicated ● Monolithic heat stack ○ Places limits on scalability (~300 nodes?) ○ Problems are hard to diagnose ○ Deploy/reconfigure/upgrade are all-or-nothing (scary) ■ The entire heat stack is updated ■ Hard to prove it won’t change something unexpected ● Technologies are easy to add, less easy to remove ○ Heat, nova, Puppet, Pacemaker, mistral, Ansible (buried in heat templates…), Docker, Kubernetes (eventually) ● No isolation between services ○ Switch to containers should help here ● Puppet adds a layer of abstraction to config
  11. 11. Kubernetes-based Tools ● Deploy an OpenStack control plane using Kubernetes ● Kolla Kubernetes ● OpenStack helm ○ Seems to have the most momentum ○ Mostly single vendor (AT&T) ● TripleO (in future) ● Stackanetes (unmaintained) ● Fuel CCP (unmaintained)
  12. 12. What’s Wrong With Kubernetes? Disclaimer: I’ve not used any of these tools… ● A great way to run scalable, reliable microservices ● But... ● Don’t we have enough turtles already? ● How to deploy Kubernetes? ● OpenStack isn’t very “cloud native” ● Considerable scaffolding required to run OpenStack on Kubernetes
  13. 13. Kolla Ansible ● Containerised control plane using kolla images ● Ansible keeps things (relatively) simple ● Configure any* configuration option using native INI files ● Three principal operations ○ Deploy ○ Reconfigure ○ Upgrade ● Target specific services ○ kolla-ansible deploy --tags <tags> ● Target specific hosts ○ kolla-ansible upgrade --limit <hosts | groups>
  14. 14. What’s Wrong Missing from Kolla Ansible? ● Bare metal deployment of control plane* ○ Equivalent of TripleO’s undercloud ● Configuration of control plane host OS* ○ Kolla ansible largely expects hosts to be ready to deploy containers ● Bare metal compute node management * Limited support provided
  15. 15. Kayobe
  16. 16. Kayobe ● Etymology ○ Kolla On Bifrost (K-O-B) ● Adds some missing pieces to kolla ansible ○ Bare metal deployment of control plane using bifrost ○ K-a inventory generation & service placement ○ Configuration of control plane host OS ○ Management of physical network devices ○ Bare metal compute node management ○ A friendly openstack-like CLI ● Ansible playbooks & python CLI
  17. 17. Kayobe Software Stack kayobe CLI bifrost ironic kayobe playbooks kolla Docker images kolla ansible Docker containers
  18. 18. Kayobe Architecture Ansible control host [kayobe, kolla ansible] Seed host [bifrost, ironic] Control plane hosts [OpenStack] Bare metal compute hosts [user workloads] Controllers, network hosts, virtualised compute hypervisors, storage, etc. provisions provisions manages Network devices configures
  19. 19. Physical Network Device Management ● Uses Ansible’s network configuration modules ● Applies static network configuration ● Currently supports Dell OS 6, OS 9 & JunOS ● Network devices added to Ansible inventory ○ Can be used to configure neutron plugins e.g. genericswitch ● Configuration defined via host variables
  20. 20. Seed Host ● Host that runs bifrost services ● Roughly equivalent to TripleO’s undercloud ● Supports the same host configuration as the control plane ● Can be deployed as a VM ○ Allows for reuse of the hypervisor ○ Automation of: ■ Libvirt/KVM hypervisor ■ Seed VM provisioning
  21. 21. Bifrost ● Minimal* bare metal provisioning environment ○ OpenStack ironic ○ OpenStack ironic inspector ○ Dnsmasq ○ Nginx ○ MariaDB ○ RabbitMQ ○ Disk Image Builder (DIB) ● Deployed using ansible ● Discovers & inspects control plane nodes ● Deploys an image to a static inventory * Minimal for OpenStack
  22. 22. Host Configuration ● From provisioned image to container-ready ● Uses roles from Ansible Galaxy ● User bootstrapping ● Network configuration ○ Static IP address allocation ○ Interfaces (Ethernet, bridge, bond) ○ DNS ● NTP ● Disk wiping & LVM ● Docker storage ● User accounts ● Ceph block device preparation
  23. 23. Control Plane Service Placement ● Multiple standard host types supported ○ Controller ○ Compute (virtualised) ○ Storage ○ Monitoring ● Custom host types possible via group/host variables ○ e.g. Database, load balancer ● Discovered hosts mapped to a top level kayobe group ○ e.g. compute-01 is in the [compute] group ● Kayobe groups mapped to top level kolla ansible groups ○ e.g. hosts in the kayobe [compute] group should be added to the kolla ansible [compute] group ● Support for customisation of kolla ansible inventory ○ e.g. place MariaDB containers on hosts in the [database] group
  24. 24. Control Plane Service Placement control-01 controllers control nova-api, mariadb, rabbitmq, etc. compute-01 compute compute nova-compute, libvirt, etc. Control plane host [ironic inventory] kayobe top level group [kayobe host mapping] kolla ansible top level group [kayobe group mapping] kolla ansible service groups [kolla ansible inventory]
  25. 25. Usage Examples ● Provision control plane hosts ○ kayobe overcloud provision ● Configure host OS services on compute hosts ○ kayobe overcloud host configure --limit compute ● Reconfigure ironic ○ # Modify ironic.conf template ○ kayobe overcloud service reconfigure --kolla-tags ironic ● Upgrade keystone ○ kayobe overcloud service upgrade --kolla-tags keystone ● Snapshot the currently applied configuration ○ kayobe overcloud service configuration save
  26. 26. Demonstration!
  27. 27. Community
  28. 28. Kayobe Community ● StackHPC ● Square Kilometre Array (SKA) ○ Performance Prototype Platform (P3) ● Verne Global ○ hpcDIRECT ● Devoteam ○ Continuous Integration (CI) platform ● OpenStack Kolla community ○ Showed interest in collaboration during recent Project Teams Gathering (PTG) ● You?
  29. 29. Get Involved! ● Documentation: ● Try it out: ● IRC: #openstack-kayobe
  30. 30. Thanks! Mark Goddard IRC: mgoddard | #openstack-kayobe @markgoddard