This talk provides an insight into the way in which we at EWERK run OpenShift (kubernetes) Clusters on Cloudstack.

1. Made by EWERK
Running Cloudstack and OpenShift with
NetApp on KVM

2. − Founded in 1995
− Owner-led since the start
− Headquarters in Leipzig, project-
offices in Bayreuth, Berlin, Wolfsburg
and Zurich
> 600
Customers
across the EU
> 200
Employees
87 %
University
graduates
50000 MD
project
volumes
2019
> 250
Projects a
year
4 International
certifications
3 Data Center
Locations +
different Cloud
Solutions
+ 20 %
Annual
Growth
> 20 Mio. €
Revenue
EWERK in Numbers

4. Global IT Outsourcing
IT Architecture Consulting
Release Management
> 5.000 MD
Project volume per
year
European Energy
Exchange
60
Core applications in
operation
> 100
Project involved
90
Releases per
year

5. A
central platform
MDK IT
15
national associations
Operational service for the MDK
industry software and the peripheral
systems
A highly available and flexible
system architecture provides the
basis for sufficient stability and
scalability of the new MDK
industry software.
medical service of the health insurance

6. Certified according to
DIN ISO/IEC 9001, 20000-1 & 27001
Outsourcing accoring to
ISAE 3402 Type II
Certified to international standards in IT
security management, quality management
and IT service management
In-depth experience and regular check-ups
Assured highest standards in provision of
digital solutions
International assurance standard for
documentation, reporting and internal
control systems for service providers and
processes
EWERK Outsourcing services are compliant
with client standards and reporting cycles.
Type II: Documentation over a period of
time, assuring dependable management
controls

7. „ „We are an active contributor of the CloudStack community“
„We submit issues, fix bugs and pull requests“
„We support and develop with the community actively on the new
Cloudstack Primate UI“
AND
„We support the community when we can“

8. Goals

9. Goals
VLAN separation3 different customers, capsulated security, at the
moment no real SDN in Cloudstack
Performance2 noisy neighborhood, no performance degradation for
VMs and Containers
Open Source1 use Open Source when useful
4 Container high performance, snapshot functionality for the PV
5 Virtual Machines high performance, snapshot functionality for the
disks

10. Components we use

11. Our Cloudstack installation
KVM (CentOS/RHEL 7, later 8)1
Network: OVS (OpenvSwitch, VLAN separation)2
Primary Storage: Netapp HCI
Secondary Storage: Active/Passive NFS
3
very close to Github - Master5
4 MariaDB Galera Cluster as Cloudstack Management Server Database
Backend
7 Use of VPC features (private gateway, ACL, NAT, soon redundant)
6 Domains, Projects, LDAP integration

12. Hardware Overview
Server and Network
Hosts
- - 1 RU Server
- - 2 Socket with 20 CPU / 40 HT
- - 768 GB RAM
- - 2x 10Gbit SFP+ Standard Networking
- - 1x local NVME or SATA DOM
Switches for Leaf and Spine
- - Edgecore 10Gbit and 100Gbit
- - Cumulus Linux NOS

13. © 2018 NetApp, Inc. All Rights Reserved. Limited Use© 2018 NetApp, Inc. All Rights Reserved. Limited Use
Compute Node Compute NodeD B
DECONSTRUCTING THE ARCHITECTURENetApp HCI
1GbE RJ-45
cables for
management
Power
Cables 10/25GbE SFP+
or
SFP28 cables for
iSCSI

14. © 2018 NetApp, Inc. All Rights Reserved. Limited Use© 2018 NetApp, Inc. All Rights Reserved. Limited Use
DECONSTRUCTING THE ARCHITECTURENetApp HCI
Node A Node B Node C Node D
A
Power Button
B
C
D
SSD-E
SSD = Solid State SATA
E = Encrypted
Drive size printed on each
drive carrier
24 x 2.5” Hot-Swap SSD Bays
Blank Drive slots when
only compute nodes

15. DECONSTRUCTING THE ARCHITECTURENetApp HCI
APP BEHAVIOR
Burst
Max
Min

16. © 2018 NetApp, Inc. All Rights Reserved. Limited Use© 2018 NetApp, Inc. All Rights Reserved. Limited Use
DECONSTRUCTING THE ARCHITECTURENetApp HCI
PERFORMANCE
TIME
SYSTEM PERFORMANCE
BEFORE QoS
AFTER QoS
BOOT STORM
3
1
2
BOOT STORM
3
1
2
BOOT STORM
PERFORMANCEPERFORMANCE

17. What is Trident?
Trident: Kubernetes Integration for NetApp Storage
▪ NetApp's open-source dynamic storage provisioner
supporting:
▪ NetApp® ONTAP® NFS and iSCSI
▪ NetApp Element® iSCSI
▪ Automates volume creation and mapping
▪ Compatible with:
▪ Kubernetes
▪ OpenShift Origin & Enterprise
▪ Available on GitHub: https://github.com/NetApp/trident

18. Datacenter Design
Chassis/Rack awareness

19. Network Design Overview
Cloudstack Domain - Projects, VPC Overview Ingress

20. Provisioning Design Overview
PXE Boot with Foreman

21. CloudStack OpenShift Cluster

22. Storage Design Overview

23. Practical

24. Practical
MVIP=10.24.48.230;SVIP=10.24.50.230;
clusterAdminUsername=admin;
clusterAdminPassword=admin;
clusterDefaultMinIops=500;
clusterDefaultMaxIops=2250;
clusterDefaultBurstIopsPercentOfMaxIops=6.5

25. Practical

26. Practical
NetApp HCI - CloudStack Volume View

27. Practical
MVIP=10.24.48.230;
SVIP=10.24.50.230;
clusterAdminUsername=admin;
clusterAdminPassword=admin;
clusterDefaultMinIops=500;
clusterDefaultMaxIops=2250;
clusterDefaultBurstIopsPercentOfMaxIops=6.5

28. Practical
URL: https://github.com/NetApp/trident/releases

29. Practical
[root@master01 setup]# cat backend.json
{
"version": 1,
"storageDriverName": "solidfire-san",
"Endpoint": "https://trident:[Password]@10.24.48.230/json-rpc/8.0",
"SVIP": "10.32.24.230:3260",
"TenantName": "trident-ew",
"UseCHAP": true,
"InitiatorIFace": "default",
"Types": [
{"Type": "bronze", "Qos": {"minIOPS": 100, "maxIOPS": 2000, "burstIOPS": 4000}},
{"Type": "silver", "Qos": {"minIOPS": 100, "maxIOPS": 6000, "burstIOPS": 8000}},
{"Type": "gold", "Qos": {"minIOPS": 100, "maxIOPS": 8000, "burstIOPS": 10000}},
]
}
[root@master01 setup]# cat storageclass-bronze.json
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: solidfire-bronze
annotations:
provisioner: netapp.io/trident
parameters:
backendType: "solidfire-san"
IOPS: "1050"
fsType: "ext4"
reclaimPolicy: Retain
./tridentctl create backend -f "setup/backend.json" -n trident-ew -d
Trident installer

30. Practical
NetApp HCI - Container PV View

31. Practical
OpenShift View

32. Practical
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source dev='/dev/disk/by-path/ip-10.24.50.230:3260-iscsi-iqn.2010-01.com.solidfire:nhjj.root-2025.3284-lun-0'/>
<backingStore/>
<target dev='hda' bus='ide'/>
<serial>4b13feca31d44a769531</serial>
<alias name='ide0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
libvirt xml file Solidfire VSS Provider

33. Upcoming Projects!

34. Sven Vogel
Lead Cloud Solution Architect / Cloudstack PMC
Member
EWERK Group
Brühl 24, D-04109 Leipzig
P +49 341 42649-0
F +49 341 42649-98
s.vogel@ewerk.com