Made by EWERK
Running Cloudstack and OpenShift with
NetApp on KVM
− Founded in 1995
− Owner-led since the start
− Headquarters in Leipzig, project-
offices in Bayreuth, Berlin, Wolfsburg
and Zurich
> 600
Customers
across the EU
> 200
Employees
87 %
University
graduates
50000 MD
project
volumes
2019
> 250
Projects a
year
4 International
certifications
3 Data Center
Locations +
different Cloud
Solutions
+ 20 %
Annual
Growth
> 20 Mio. €
Revenue
EWERK in Numbers
Global IT Outsourcing
IT Architecture Consulting
Release Management
> 5.000 MD
Project volume per
year
European Energy
Exchange
60
Core applications in
operation
> 100
Project involved
90
Releases per
year
A
central platform
MDK IT
15
national associations
Operational service for the MDK
industry software and the peripheral
systems
A highly available and flexible
system architecture provides the
basis for sufficient stability and
scalability of the new MDK
industry software.
medical service of the health insurance
Certified according to
DIN ISO/IEC 9001, 20000-1 & 27001
Outsourcing accoring to
ISAE 3402 Type II
Certified to international standards in IT
security management, quality management
and IT service management
In-depth experience and regular check-ups
Assured highest standards in provision of
digital solutions
International assurance standard for
documentation, reporting and internal
control systems for service providers and
processes
EWERK Outsourcing services are compliant
with client standards and reporting cycles.
Type II: Documentation over a period of
time, assuring dependable management
controls
„ „We are an active contributor of the CloudStack community“
„We submit issues, fix bugs and pull requests“
„We support and develop with the community actively on the new
Cloudstack Primate UI“
AND
„We support the community when we can“
Goals
Goals
VLAN separation3 different customers, capsulated security, at the
moment no real SDN in Cloudstack
Performance2 noisy neighborhood, no performance degradation for
VMs and Containers
Open Source1 use Open Source when useful
4 Container high performance, snapshot functionality for the PV
5 Virtual Machines high performance, snapshot functionality for the
disks
Components we use
Our Cloudstack installation
KVM (CentOS/RHEL 7, later 8)1
Network: OVS (OpenvSwitch, VLAN separation)2
Primary Storage: Netapp HCI
Secondary Storage: Active/Passive NFS
3
very close to Github - Master5
4 MariaDB Galera Cluster as Cloudstack Management Server Database
Backend
7 Use of VPC features (private gateway, ACL, NAT, soon redundant)
6 Domains, Projects, LDAP integration
Hardware Overview
Server and Network
Hosts
- - 1 RU Server
- - 2 Socket with 20 CPU / 40 HT
- - 768 GB RAM
- - 2x 10Gbit SFP+ Standard Networking
- - 1x local NVME or SATA DOM
Switches for Leaf and Spine
- - Edgecore 10Gbit and 100Gbit
- - Cumulus Linux NOS
© 2018 NetApp, Inc. All Rights Reserved. Limited Use© 2018 NetApp, Inc. All Rights Reserved. Limited Use
Compute Node Compute NodeD B
DECONSTRUCTING THE ARCHITECTURENetApp HCI
1GbE RJ-45
cables for
management
Power
Cables 10/25GbE SFP+
or
SFP28 cables for
iSCSI
© 2018 NetApp, Inc. All Rights Reserved. Limited Use© 2018 NetApp, Inc. All Rights Reserved. Limited Use
DECONSTRUCTING THE ARCHITECTURENetApp HCI
Node A Node B Node C Node D
A
Power Button
B
C
D
SSD-E
SSD = Solid State SATA
E = Encrypted
Drive size printed on each
drive carrier
24 x 2.5” Hot-Swap SSD Bays
Blank Drive slots when
only compute nodes
DECONSTRUCTING THE ARCHITECTURENetApp HCI
APP BEHAVIOR
Burst
Max
Min
© 2018 NetApp, Inc. All Rights Reserved. Limited Use© 2018 NetApp, Inc. All Rights Reserved. Limited Use
DECONSTRUCTING THE ARCHITECTURENetApp HCI
PERFORMANCE
TIME
SYSTEM PERFORMANCE
BEFORE QoS
AFTER QoS
BOOT STORM
3
1
2
BOOT STORM
3
1
2
BOOT STORM
PERFORMANCEPERFORMANCE
What is Trident?
Trident: Kubernetes Integration for NetApp Storage
▪ NetApp's open-source dynamic storage provisioner
supporting:
▪ NetApp® ONTAP® NFS and iSCSI
▪ NetApp Element® iSCSI
▪ Automates volume creation and mapping
▪ Compatible with:
▪ Kubernetes
▪ OpenShift Origin & Enterprise
▪ Available on GitHub: https://github.com/NetApp/trident
Datacenter Design
Chassis/Rack awareness
Network Design Overview
Cloudstack Domain - Projects, VPC Overview Ingress
Provisioning Design Overview
PXE Boot with Foreman
CloudStack OpenShift Cluster
Storage Design Overview
Practical
Practical
MVIP=10.24.48.230;SVIP=10.24.50.230;
clusterAdminUsername=admin;
clusterAdminPassword=admin;
clusterDefaultMinIops=500;
clusterDefaultMaxIops=2250;
clusterDefaultBurstIopsPercentOfMaxIops=6.5
Practical
Practical
NetApp HCI - CloudStack Volume View
Practical
MVIP=10.24.48.230;
SVIP=10.24.50.230;
clusterAdminUsername=admin;
clusterAdminPassword=admin;
clusterDefaultMinIops=500;
clusterDefaultMaxIops=2250;
clusterDefaultBurstIopsPercentOfMaxIops=6.5
Practical
URL: https://github.com/NetApp/trident/releases
Practical
[root@master01 setup]# cat backend.json
{
"version": 1,
"storageDriverName": "solidfire-san",
"Endpoint": "https://trident:[Password]@10.24.48.230/json-rpc/8.0",
"SVIP": "10.32.24.230:3260",
"TenantName": "trident-ew",
"UseCHAP": true,
"InitiatorIFace": "default",
"Types": [
{"Type": "bronze", "Qos": {"minIOPS": 100, "maxIOPS": 2000, "burstIOPS": 4000}},
{"Type": "silver", "Qos": {"minIOPS": 100, "maxIOPS": 6000, "burstIOPS": 8000}},
{"Type": "gold", "Qos": {"minIOPS": 100, "maxIOPS": 8000, "burstIOPS": 10000}},
]
}
[root@master01 setup]# cat storageclass-bronze.json
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: solidfire-bronze
annotations:
provisioner: netapp.io/trident
parameters:
backendType: "solidfire-san"
IOPS: "1050"
fsType: "ext4"
reclaimPolicy: Retain
./tridentctl create backend -f "setup/backend.json" -n trident-ew -d
Trident installer
Practical
NetApp HCI - Container PV View
Practical
OpenShift View
Practical
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source dev='/dev/disk/by-path/ip-10.24.50.230:3260-iscsi-iqn.2010-01.com.solidfire:nhjj.root-2025.3284-lun-0'/>
<backingStore/>
<target dev='hda' bus='ide'/>
<serial>4b13feca31d44a769531</serial>
<alias name='ide0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
libvirt xml file Solidfire VSS Provider
Upcoming Projects!
Sven Vogel
Lead Cloud Solution Architect / Cloudstack PMC
Member
EWERK Group
Brühl 24, D-04109 Leipzig
P +49 341 42649-0
F +49 341 42649-98
s.vogel@ewerk.com

Sven Vogel: Running CloudStack and OpenShift with NetApp on KVM

  • 1.
    Made by EWERK RunningCloudstack and OpenShift with NetApp on KVM
  • 2.
    − Founded in1995 − Owner-led since the start − Headquarters in Leipzig, project- offices in Bayreuth, Berlin, Wolfsburg and Zurich > 600 Customers across the EU > 200 Employees 87 % University graduates 50000 MD project volumes 2019 > 250 Projects a year 4 International certifications 3 Data Center Locations + different Cloud Solutions + 20 % Annual Growth > 20 Mio. € Revenue EWERK in Numbers
  • 4.
    Global IT Outsourcing ITArchitecture Consulting Release Management > 5.000 MD Project volume per year European Energy Exchange 60 Core applications in operation > 100 Project involved 90 Releases per year
  • 5.
    A central platform MDK IT 15 nationalassociations Operational service for the MDK industry software and the peripheral systems A highly available and flexible system architecture provides the basis for sufficient stability and scalability of the new MDK industry software. medical service of the health insurance
  • 6.
    Certified according to DINISO/IEC 9001, 20000-1 & 27001 Outsourcing accoring to ISAE 3402 Type II Certified to international standards in IT security management, quality management and IT service management In-depth experience and regular check-ups Assured highest standards in provision of digital solutions International assurance standard for documentation, reporting and internal control systems for service providers and processes EWERK Outsourcing services are compliant with client standards and reporting cycles. Type II: Documentation over a period of time, assuring dependable management controls
  • 7.
    „ „We arean active contributor of the CloudStack community“ „We submit issues, fix bugs and pull requests“ „We support and develop with the community actively on the new Cloudstack Primate UI“ AND „We support the community when we can“
  • 8.
  • 9.
    Goals VLAN separation3 differentcustomers, capsulated security, at the moment no real SDN in Cloudstack Performance2 noisy neighborhood, no performance degradation for VMs and Containers Open Source1 use Open Source when useful 4 Container high performance, snapshot functionality for the PV 5 Virtual Machines high performance, snapshot functionality for the disks
  • 10.
  • 11.
    Our Cloudstack installation KVM(CentOS/RHEL 7, later 8)1 Network: OVS (OpenvSwitch, VLAN separation)2 Primary Storage: Netapp HCI Secondary Storage: Active/Passive NFS 3 very close to Github - Master5 4 MariaDB Galera Cluster as Cloudstack Management Server Database Backend 7 Use of VPC features (private gateway, ACL, NAT, soon redundant) 6 Domains, Projects, LDAP integration
  • 12.
    Hardware Overview Server andNetwork Hosts - - 1 RU Server - - 2 Socket with 20 CPU / 40 HT - - 768 GB RAM - - 2x 10Gbit SFP+ Standard Networking - - 1x local NVME or SATA DOM Switches for Leaf and Spine - - Edgecore 10Gbit and 100Gbit - - Cumulus Linux NOS
  • 13.
    © 2018 NetApp,Inc. All Rights Reserved. Limited Use© 2018 NetApp, Inc. All Rights Reserved. Limited Use Compute Node Compute NodeD B DECONSTRUCTING THE ARCHITECTURENetApp HCI 1GbE RJ-45 cables for management Power Cables 10/25GbE SFP+ or SFP28 cables for iSCSI
  • 14.
    © 2018 NetApp,Inc. All Rights Reserved. Limited Use© 2018 NetApp, Inc. All Rights Reserved. Limited Use DECONSTRUCTING THE ARCHITECTURENetApp HCI Node A Node B Node C Node D A Power Button B C D SSD-E SSD = Solid State SATA E = Encrypted Drive size printed on each drive carrier 24 x 2.5” Hot-Swap SSD Bays Blank Drive slots when only compute nodes
  • 15.
    DECONSTRUCTING THE ARCHITECTURENetAppHCI APP BEHAVIOR Burst Max Min
  • 16.
    © 2018 NetApp,Inc. All Rights Reserved. Limited Use© 2018 NetApp, Inc. All Rights Reserved. Limited Use DECONSTRUCTING THE ARCHITECTURENetApp HCI PERFORMANCE TIME SYSTEM PERFORMANCE BEFORE QoS AFTER QoS BOOT STORM 3 1 2 BOOT STORM 3 1 2 BOOT STORM PERFORMANCEPERFORMANCE
  • 17.
    What is Trident? Trident:Kubernetes Integration for NetApp Storage ▪ NetApp's open-source dynamic storage provisioner supporting: ▪ NetApp® ONTAP® NFS and iSCSI ▪ NetApp Element® iSCSI ▪ Automates volume creation and mapping ▪ Compatible with: ▪ Kubernetes ▪ OpenShift Origin & Enterprise ▪ Available on GitHub: https://github.com/NetApp/trident
  • 18.
  • 19.
    Network Design Overview CloudstackDomain - Projects, VPC Overview Ingress
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
    Practical NetApp HCI -CloudStack Volume View
  • 27.
  • 28.
  • 29.
    Practical [root@master01 setup]# catbackend.json { "version": 1, "storageDriverName": "solidfire-san", "Endpoint": "https://trident:[Password]@10.24.48.230/json-rpc/8.0", "SVIP": "10.32.24.230:3260", "TenantName": "trident-ew", "UseCHAP": true, "InitiatorIFace": "default", "Types": [ {"Type": "bronze", "Qos": {"minIOPS": 100, "maxIOPS": 2000, "burstIOPS": 4000}}, {"Type": "silver", "Qos": {"minIOPS": 100, "maxIOPS": 6000, "burstIOPS": 8000}}, {"Type": "gold", "Qos": {"minIOPS": 100, "maxIOPS": 8000, "burstIOPS": 10000}}, ] } [root@master01 setup]# cat storageclass-bronze.json apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: solidfire-bronze annotations: provisioner: netapp.io/trident parameters: backendType: "solidfire-san" IOPS: "1050" fsType: "ext4" reclaimPolicy: Retain ./tridentctl create backend -f "setup/backend.json" -n trident-ew -d Trident installer
  • 30.
    Practical NetApp HCI -Container PV View
  • 31.
  • 32.
    Practical <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <disk type='block' device='disk'> <drivername='qemu' type='raw' cache='none'/> <source dev='/dev/disk/by-path/ip-10.24.50.230:3260-iscsi-iqn.2010-01.com.solidfire:nhjj.root-2025.3284-lun-0'/> <backingStore/> <target dev='hda' bus='ide'/> <serial>4b13feca31d44a769531</serial> <alias name='ide0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> libvirt xml file Solidfire VSS Provider
  • 33.
  • 34.
    Sven Vogel Lead CloudSolution Architect / Cloudstack PMC Member EWERK Group Brühl 24, D-04109 Leipzig P +49 341 42649-0 F +49 341 42649-98 s.vogel@ewerk.com