Tools and technologies used in our journey from HPC to OpenStack, Presented at Manchester OpenStack Meetup

1. HPC to OpenStack
Our journey
Tuesday 16th
February 2016

2. Outline
●
Key Questions
●
Provisioning Systems
●
Configuration Management
●
Futures
●
Q&A

3. How do we??
●
Install bare metal
●
Configure networking, and base OS
●
Install software
●
Manage the configuration
●
Make it redundant

4. Provisioning Systems
●
Which one to choose
– Foreman
– Cobbler
– Oscar
– Rocks
– xCAT
– PCM
– Bright
– Others...

5. What is xCAT
●
eXtreme Cluster/Cloud Administration Toolkit
●
Management of clusters
●
Baremetal Provisioning
●
Scriptable
●
Large scale management (Lightsout, remote console,
distributed shell)
●
Configures key services based on tables

6. Why we're using xCAT
●
xCAT provides bare metal management and provisioning of
hardware.
●
Automatic Discovery
●
EPL Licencing
– Adaptable
– No licencing cost
●
Close relationship with developers
●
Hardware management
– Lights out management
– Remote console/Serial over Lan
– Remote inventory, vitals, power

7. Pros and Cons
●
Pros
– Flexible
– Open-Source
– HW Support
– OS Support
●
Cons
– No GUI
– No Configuration Management
– Very Steep Learning Curve
– Only maintained by IBM/Lenovo

8. How is this relevant to OpenStack
●
The installation process is very similar
●
The quantity of nodes are very similar
●
We still need to provision nodes from bare-metal
●
XCAT has an Ironic driver :)

9. Why NOW!!!!
●
KVM performance
●
SR-IOV support
●
VXLAN offloading

10. Why OpenStack for HPC
●
Users want flexibility
●
Users have a choice
●
Users don't depend on Administrators as much
●
Not all applications are available on the system
●
Sequential jobs
●
Embarrassingly parallel jobs

11. Performance
●
MVAPICH2-virt performance is approx 1-4% slower
compared to native (source Mellanox booth SC15)
●
8% slower w/SR-IOV vs Native IB (source SDSC) compared
to 28% 2 years ago on KVM
http://mvapich.cse.ohio-state.edu/static/media/talks/slide/dk_mellanox_theatre_sc15_1.pdf

12. Our Dilemma
●
OpenStack is so new
●
Not many people know it
●
Where do we start

13. Our Choices
●
Initially (Havana - Icehouse)
– CentOS
– RDO
– Packstack
●
Middle (Icehouse – Juno)
– CentOS
– RDO
– Crudini (openstack-config)
●
Then (Juno - Liberty)
– CentOS
– RDO
– SaltStack

14. Why!!!
●
CentOS
– Because a lot of our customers used it
– Compatible with GPFS
●
RDO
– Natural choice after choosing CentOS
– Easy way of getting all packages installed via package manager
●
SaltStack
– Customer introduction to the SW

15. How – Strike 1
●
Packstack
●
Great for PoC environment
●
Test any specific features
●
Simple to get everything running

16. How – Strike 2
●
Crudini (openstack-config)
●
A lot of work required to get things up and running
●
Everything was scripted
●
Doesn't work with multiple values for any option
●
Very difficult to maintain and version control

17. How – Strike 3
●
SaltStack
●
Created many salt formulas specific for our requirements
●
Moved all our configs
– OpenStack
– GPFS
– Icinga (Nagios)
– Ganglia
– 3rd
-party applications
– HPC specific formulas
– InfiniBand

18. Preparing the System
●
Decide the version of OpenStack
●
Sync repos by using reposync from the repos
– epel
– RDO
– CentOS
●
Sample yum.conf and repo files below
https://gitlab.arif-ali.co.uk/arif/openstack-lab

19. Preparing the xCAT
●
Install xCAT RPMs
●
Configure xCAT tables
– nodes and their configurations
– networks
– switch mappings
– Etc…
●
Install nodes with base OS
●
Use salt to configure all nodes depending on xCAT groups
https://gitlab.arif-ali.co.uk/arif/openstack-lab

20. Future Development
●
Update configs to use salt formulas
●
Contribute to the salt-formula-* projects
– Add any experience from field
– Creating spec files for the salt-formulas
●
Keystone v3 API integration
– Federation support
●
Use openstack-ansible ??

21. Finally
●
Use heat to deploy virtual HPC environments
●
Create private networks
●
Install custom scheduler and software
●
Create as many as you want
https://gitlab.arif-ali.co.uk/arif/openstack-lab/tree/master/heat_templates

22. Links
●
https://gitlab.arif-ali.co.uk/arif/openstack-lab
●
http://blog.arif-ali.co.uk/blog/categories/openstack
●
https://gitlab.arif-ali.co.uk/arif/openstack-
lab/wikis/home

23. Q & A