Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deploying Scientific OpenStack

101 views

Published on

A lightning talk about how we at StackHPC deploy Scientific OpenStack. Includes coverage of Kayobe, Bifrost, Kolla & Kolla Ansible.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Deploying Scientific OpenStack

  1. 1. Deploying Scientific OpenStack Mark Goddard
  2. 2. An OpenStack Cloud - Sky High View Physical Infrastructure OpenStack Control Plane User Workloads Project Resources Physical servers, switches Nova, Keystone, etc. Images, networks, flavors, etc. Server instances, clusters, etc. Monitoring&Logging Metrics, log messages
  3. 3. Once Upon a Time...
  4. 4. IT Begins Hi, We’ve racked and cabled the servers and switches for the Super Science Cloud (SSC). The seed has been provisioned with an OS, as requested. I’ve attached a spreadsheet containing cabling details. Over to you!
  5. 5. Gulp...
  6. 6. Kayobe ● Provisions physical infrastructure using Bifrost ● Configures control plane hosts ● Deploys containerised OpenStack using Kolla & Kolla Ansible ● Python & Ansible ● Version controlled YAML configuration
  7. 7. Bifrost ● Deploys Ironic in a standalone configuration ● Discovers, inspects and provisions bare metal servers ● Ansible
  8. 8. Physical Infrastructure Provisioning Kayob e Bifrost
  9. 9. Kolla & Kolla Ansible ● Kolla builds Docker container images for OpenStack ● Kolla Ansible deploys containers using Kolla images ● Scale out & high availability
  10. 10. Monasca ● Metrics, alarms & notifications ● Log aggregation & processing ● Multi-tenant ● Scalable
  11. 11. OpenStack Control Plane Deployment Kayobe Kolla Ansible Kolla Monasca
  12. 12. ssc-config ● Projects, users, roles ● Images ● Flavors ● Networks, subnets, routers ● Ansible Galaxy ● Declarative YAML definitions ● Cookiecutter generated
  13. 13. Project Resources ssc- config
  14. 14. ssc-appliances ● Creates clusters running on OpenStack ● Uses Heat (servers) or Magnum (containers) ● Configures clusters ○ User accounts, packages, access to storage ● SLURM, Dask, Daliuge, Redis, BeeGFS, GlusterFS ● Ansible ● Some roles included, some pulled from Ansible Galaxy
  15. 15. User Workloads scc-appliances Heat Magnum
  16. 16. Notice a Theme?
  17. 17. Super Science Cloud Physical Infrastructure OpenStack Control Plane User Workloads OpenStack Resources Monasc a Kayobe, ssc-kayobe- config, Bifrost Kolla, Kolla Ansible ssc-config ssc-appliances Monitoring&Logging
  18. 18. References & Further Information
  19. 19. Kayobe Resources Docs: http://kayobe.readthedocs.io/en/latest/ Source: https://github.com/openstack/kayobe Configuration source: https://github.com/openstack/kayobe-config Slides: https://www.slideshare.net/MarkGoddard2/to-kayobe-or-not-to-kayobe Demo: https://asciinema.org/a/176888 Blog: http://www.stackhpc.com/kayobe-update.html Example configurations: ● Dev/test: https://github.com/openstack/kayobe-config-dev ● ALaSKA (SKA): https://github.com/SKA-ScienceDataProcessor/alaska-kayobe-config ● Evolution of Darwin (Cambridge): https://github.com/stackhpc/eod-kayobe-config
  20. 20. Bifrost Resources Docs: https://docs.openstack.org/bifrost/latest/ Source: https://github.com/openstack/bifrost
  21. 21. Kolla Resources Kolla: Docs: https://docs.openstack.org/kolla/latest/ Source: https://github.com/openstack/kolla Kolla Ansible: Docs: https://docs.openstack.org/kolla-ansible/latest/ Source: https://github.com/openstack/kolla-ansible Demo: https://www.youtube.com/watch?v=BKYJuYsT4z4
  22. 22. ssc-config Cookie cutter: https://github.com/stackhpc/stackhpc-config-cookiecutter Examples: ● ALaSKA (SKA): https://github.com/SKA-ScienceDataProcessor/p3-config ● Evolution of Darwin (Cambridge): https://github.com/stackhpc/eod-config
  23. 23. ssc-appliances Examples: ● ALaSKA (SKA) https://github.com/stackhpc/p3-appliances
  24. 24. Ansible Galaxy Roles - https://galaxy.ansible.com/stackhpc ● cluster-infra ● configdrive ● dell-powerconnect-switch ● drac ● drac-facts ● gluster-cluster ● grafana-conf ● libvirt-host ● libvirt-vm ● mellanox-switch ● mlnx-neo ● mlnx-ufm ● monasca-agent ● monasca-rsyslog ● os-config ● os-container-clusters ● os-container-infra ● os-flavors ● os-images ● os-ironic-state ● os-keypair-login ● os-keystone-pam ● os-manila-mount ● os-networks ● os-openstackclient ● os-projects ● os-shade ● os-volumes ● parted-1-1 ● repo-server
  25. 25. Diskimage Builder ● StackHPC image elements provides custom elements for HPC ● Source: https://github.com/stackhpc/stackhpc-image-elements

×