Saverio Proto from SWITCH presented this deck at the Switzerland HPC Conference. "OpenStack is an open source software ecosystem for Cloud Computing. It is more than man- aging virtual machines on a fleet of bare-metal servers, OpenStack enables Cloud Computing providing new abstractions that help users to deploy their services with agility. Features like floating IPs, object storage, instance autoscaling and orchestration make the difference between a traditional Virtual Infrastructure and an IaaS. In this talk, we will overview the various OpenStack components, and highlight with simple examples the differences between a traditional virtualization approach." Watch the video presentation: https://www.youtube.com/watch?v=xh9X9dUvs7Q See more talks in the Swiss Conference Video Gallery: http://insidehpc.com/2016-swiss-hpc-conference/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

1. Openstack
Cloud computing with Openstack
Lugano, 23/03/2016
Saverio Proto
saverio.proto@switch.ch

2. © 2016 SWITCH
• SWITCH role in Openstack and Cloud Computing
• What is Virtualization ?
• Why is Cloud computing more than Virtualization ?
• How Openstack implements Cloud Computing ?
Agenda
2

3. © 2016 SWITCH
• Openstack public cloud with two regions in Switzerland
– https://www.switch.ch/engines/
• SWITCHengines provides compute and storage services
to Swiss academics. Users of participating institutions
have on-demand access to create and manage virtual
machines and storage.
• Two Regions
– Zurich
– Lausanne
SWITCHengines
3

4. © 2016 SWITCH
• SWITCHengines has been in “public beta” and “internal
production” since December 2014
• Productive service since January 2016
• Several SWITCH services run on it (SWITCHdrive,
SWITCHfilesender, SWITCHtube)
• Over 630 individual users and around 40 research projects
online
• Around 700 VMs running on the cloud right now (2200
cores, 4900 GB RAM, 400 TB of storage allocated)
• Started with Openstack Icehouse, we have upgraded the
system to Juno (8/2015) and then to Kilo (3/2016)
Openstack Current status at SWITCH
4

5. © 2016 SWITCH
• Key idea: Hypervisor and virtual machines
– Commodity technology today
– Typically GUI driven
– Scale-UP approach
– Operators manage each Hypervisor independently
• Well known Hypervisors, and date of birth
– VMWare 2001
– Xen 2003
– Virtualbox 2007
– KVM 2007
https://en.wikipedia.org/wiki/Timeline_of_virtualization_development
The starting point: Virtualization
5

6. © 2016 SWITCH
• How to manage a datacenter with many hypervisors ?
– Most of the management is by hand with GUI/WEB based tools
• What happens if I lose a physical host ?
– Hypervisors are built with a lot of redundancy to prevent failures
– All the hypervisors in a legacy datacenter are Pets, you really have
to make sure they are healthy
• How to manage multi-tenant use cases ?
– Not all the VMs belong to the same project
– Separation is possible but configured by hand
Virtualization before Cloud Computing
6

7. © 2016 SWITCH
Virtualization – networking
7
Network
Hypervisor
ETH0 br0
VM1
eth0
VM2
eth0

8. © 2016 SWITCH
Virtualization – Reference Network
8
DatacenterSwitch
• Network configuration can
become complex and hard
to manage by hand
• In case of multi-tenancy
VLAN coordination needed
by network-admin and
server-admin

9. © 2016 SWITCH
• Needs redundancy for hypervisors hardware
• Needs redundancy for networking hardware
• A lot of manual configuration
• No standard strategy for multi-tenancy
Legacy Virtualization - Cons
9

10. © 2016 SWITCH
• AWS: Started in March 2006 (10 years ago)
• Openstack: Started in 2010
Cloud is already 10 years old
10

11. © 2016 SWITCH
• In a Public cloud the services are offered to anyone over
the Internet.
– Operators and users of the cloud are usually in different domains
• In a Private cloud the access is restricted to some users
for private use
– Operators and users of the cloud are usually from the same company
Public and Private Cloud
11

12. © 2016 SWITCH
• It is design to fail
– Hardware is divided in two main categories, Pets and Cattle
– You will need to introduce redundancy only for your Pets
– Cattle are allowed to fail
• API based Control Plane
– The configuration of new VMs is done interacting with a Controller
– API interfaces make possible to automate the configuration.
– Requests for new virtual machines are processed by a controller that
will schedule the machine to right hypervisor.
What is new with Cloud Computing?
12

13. © 2016 SWITCH
• It is a change regarding Uptime responsibility.
• Hypervisors can fail, this means that VMs should be
disposable.
• Developers are required to design the application in a way
that a server can be lost.
• The application should exploit the new abstractions
provided by the cloud.
Pet and Cattle
13

14. © 2016 SWITCH
• Scale-up mindset
– Make a server redundant and powefull
Web Application Example: Legacy
14
Web Server VM
Database VM
Hypervisor1
RAID5 disks
Hostname www.mysite.com
Standby VM
Database Backup
Hypervisor2 backup
RAID5 disks

15. © 2016 SWITCH
Web Application Example: Cloud
15
vm01 vm02 vm03
LBaaS
FloatingIP
DBaaS Object Storage
• Scale-out mindset
• VMs are disposable
• Database is provided by
the Cloud Infrastructure
• Object Storage is
persistent
Autoscaling VMs on load
Hostname
www.mysite.com

16. © 2016 SWITCH
• API can be accessed via CLI, Web based interface, and
libraries are provided to implement own tools
openstack server create --flavor c1.micro
--image “Ubuntu 14.04”
--nic net-id=private
myVMname
API based Control Plane
16

17. © 2016 SWITCH
• The best hypervisor is selected automatically
– User starting a VM do not know how many hypervisors
• Multi-tenancy is by default
– Each user have visibility only of his own VMs
– VMs from the same user will appear on the same network
API based Control Plane
17

18. © 2016 SWITCH
• To make scale-out possible, a cloud computing provides
the following abstractions:
• Web Services
– FloatingIPs and LBaaS
• Databases
– DBaaS : the user receives the endpoint and credentials
– Real DB is managed by Cloud Ops in multi master replication
• Storage
– Object store
Scale out
18

19. © 2016 SWITCH
• Scalable
• Access natively via HTTP
• Replication of objects
New abstraction: Object Storage
19
Server
File SystemFile SystemFile System File System File System
Server
Object storage
User

20. © 2016 SWITCH
• Quickly move an IP address to another instance or to
another service. The backend is independent.
New abstraction: floating IP
20
vm01
FloatingIP FloatingIP
vm01 Bigger VM
vm01 vm02 vm03
LBaaS
FloatingIP

21. © 2016 SWITCH
• Main components
– Keystone: authentication and authorization
– Nova: Virtual Machine orchestrator
– Cinder: Volume manager
– Glance: Image manager
– Neutron: Network controller
• The openstack big tent
– Horizon
– Heat
– Trove
– Ceilometer
Openstack components
21

22. © 2016 SWITCH
• Keystone provides a token based authorization to the
Openstack API services.
• export OS_USERNAME=saverio.proto@switch.ch
• export OS_PASSWORD=<secret>
• export OS_TENANT_NAME=saverio.proto@switch.ch
• export OS_AUTH_URL=https://keystone.cloud.switch.ch:5000/v2.0
• export OS_REGION_NAME=LS
• Keystone publishes a catalog of the current API endpoint
for services
Keystone
22

23. © 2016 SWITCH
• Cloud Federation is possible with keystone
– However requires Keystone V3 API
– Federation enables Cloud Bursting
• Keystone development is today very active
– Token structures changed
Keystone
23

24. © 2016 SWITCH
Glance
24
macsp:~ proto$ openstack image list
+--------------------------------------+------------------------------------------+--------+
| ID | Name | Status |
+--------------------------------------+------------------------------------------+--------+
| 382099dd-61b5-4866-9956-bc68abb417dd | RStudio Appliance (SWITCHengines) | active |
| 8bc628a2-a54c-4ee4-bda5-3325a1d12cb0 | Ubuntu Trusty 14.04 (SWITCHengines) | active |
| a4e35b28-b1fe-40a0-ba33-f23f0a9a7440 | CentOS 7.1 (SWITCHengines) | active |
| 1102bb9c-0256-4a4f-a104-f57c9715ae1b | Debian Wheezy 7.8 (SWITCHengines) | active |
| 31a9beca-acc5-43de-9e52-01256ef211d3 | Fedora release 20 (SWITCHengines) | active |
| 7ac1f5f9-d54c-43ef-9c4c-2b1e69603a03 | Debian Jessie 8.1 (SWITCHengines) | active |
| 8e4c68d0-6d20-4767-9d5a-f72db43995b0 | mybackup-19-Jan-16 | active |
| 0a4daefd-83d0-48e9-b3ee-e1a5e6dcbf9d | CoreOS (SWITCHengines) | active |
| dd0659cc-ee08-45c0-9658-7d5441d726b9 | Windows 2012 Server EVAL (SWITCHengines) | active |
+--------------------------------------+------------------------------------------+--------+
• Glance is operating system image repository
– It works with different storage backends

25. © 2016 SWITCH
• Cinder is the volume manager in Openstack
• Because we said that VMs are disposable, we have two
kind of volumes
– Ephemeral volumes
– Persistent volumes
• Ephemeral volumes
– Controlled directly by Nova, the disk of the VMs is destroyed when
the VM is destroyed. Created when we boot from image.
• Persistent
– Is a volume independent from a VM. You can attach the volume to
multiple VMs
Cinder
25

26. © 2016 SWITCH
• VMs from the same project appear on the same network
regardless of the hypervisor where they are scheduled
• Hypervisors are running OpenVSWITCH
• An Overlay network is built among the Hypervisors
– Different tunnels are possible (VXLAN, GRE)
Neutron
26

27. © 2016 SWITCH
Network architecture
27

28. © 2016 SWITCH
• Nova is a cloud computing fabric controller
• It is software layer that hides to the user the hypervisor
technology chosen to implement the cloud.
• The cloud can use even different hypervisor technology at
the same time.
• Supports KVM, XEN, VMWare, Hyper-V, Bare-metal,
Containers
Nova
28

29. © 2016 SWITCH
Nova API
29
KVM hypervisor
VM
NOVA API
VMVMVMVM
XEN hypervisor
User
• User requests a VM to Nova
• Based in the flavor nova schedules the VM to the
appropriate hypervisor

30. © 2016 SWITCH
• http://docs.openstack.org/liberty/networking-guide/
scenario-classic-ovs.html
Reference scenario - Hardware
30

31. © 2016 SWITCH
Reference scenario - Network
31

32. © 2016 SWITCH
Reference scenario – Service
32

33. © 2016 SWITCH
• Openstack is a great Open Source software ecosystem
– It makes possible to anyone to deploy a cloud
• Application should be designed to exploit the new
abstractions
– Cloud Native Applications
• Cloud design with Pets and Cattles reduces infrastructure
costs
Conclusion
33

34. © 2016 SWITCH
Questions ?
34

35. © 2016 SWITCH
• ATLAS Experiment at CERN, UNIBE is using
SWITCHengines 24/7 with 300 Cores
• ATLAS Instances on SWITCHengines
– 8cores
– 16GB RAM
– 20 GB Disk
Use case on SWITCHengines
35