MH
Uploaded byMiroslav Halas
445 views

Comparison of control plane deployment architectures in the scope of hyperconverged OpenStack infrastructure

The document discusses control plane deployment architectures in hyper-converged OpenStack infrastructure, authored by Miroslav Halas of Lenovo Cloud Technology Center. It outlines the necessary components for deploying and operating Software Defined Data Centers (SDDC) and emphasizes the significance of control planes for enhanced resilience and efficiency in cloud infrastructure. The document also highlights Lenovo's engagement in open-source contributions and various deployment patterns within the cloud management landscape.

Embed presentation

Downloaded 21 times
2017 Lenovo. All rights reserved. Comparison of control plane deployment architectures in the scope of hyperconverged OpenStack infrastructure Miroslav Halas Lenovo Cloud Technology Center May 2017
2 About me  Director of SW Architecture, Cloud Infrastructure  SW Engineer, Architect and Leader  Designed, implemented and operated Desktop, Mobile, SaaS, Cloud Applications and Large Private Cloud Platforms  10 years of engineering leadership for information security, public and private cloud for one of top 3 financials in US  Holds patents in security, private and public cloud areas  Carries picture of his home server in his pocket 2017 Lenovo. All rights reserved.. linkedin.com/in/miroslavhalas/
3 Outline Introduction to Lenovo Cloud Technology Center What It Takes to Deploy and Operate SDDC What Is Control Plane and Why Does It Matter Resilient Converged Infrastructure Control Plane Evaluation and Testing using Rally and Phoronix Test Suite 2017 Lenovo. All rights reserved..
4 Standing on the Shoulders of Giants Lenovo DCG Research & Technology • Focused research in Systems, Storage, Cloud, AI, Big Data, etc. Lenovo System Technology Innovation Center • Co-innovating with partners and customers to deliver Next-Next ideas such as Abstract hybrid data centers, Dynamic reconfiguration of IT and Deployment of IIoT Lenovo Cloud Technology Center • Pioneering cloud advancements, deployment and management experiences with community, partners and customers Lenovo DCG Product & Development • Engineer and support production ready enterprise solutions 2017 Lenovo. All rights reserved..
5 Lenovo Cloud Technology Center (LCTC) 2017 Lenovo. All rights reserved.. Mission: Align and Drive Lenovo entry and leadership in Enterprise Cloud Infrastructure, building core capabilities and partnerships for a complete portfolio of Solutions – focus on open source • Deep customer engagement and support • Business/Partner development, Technical Product evaluations • Open source ecosystem visibility and credibility Customer and Market Engagement • OpenStack and Open Systems development – distros, containers and integrations with solutions • SDDC – development of software defined storage, networking • Professional and Managed services Portfolio Development • Cloud architecture and advisory board • e2e Cloud roadmap Center of Competence Credible Cloud Market Presence Complete Cloud Solutions Portfolio Depth and Alignment of Expertise Long Term GoalKey Activities MAJOR RESEARCH CENTERS IN RALEIGH, BEIJING, SHANGHAI
6 OpenStack Upstream Engagement 2017 Lenovo. All rights reserved.. Reviews Commits
7 ManageIQ Upstream Engagement 2017 Lenovo. All rights reserved..  Initial provider of ManageIQ Physical Infrastructure  Features/Use cases  Physical Server Data Model  New XClarity Provider  XClarity Provider Summary View  Physical Server Inventory via REST  Vital Product Data (VPD)  Firmware Levels  Server to Platform Host Relationships  Outstanding development support from Red Hat ManageIQ / CloudForms team  Open Source Contributions Summary  ManageIQ Lenovo Provider  https://github.com/ManageIQ/manageiq-providers-lenovo  Lenovo XClarity Client (Ruby)  https://github.com/lenovo/xclarity_client …
8 What It Takes to Deploy and Operate SDDC 2017 Lenovo. All rights reserved.
9 Simple Deployment Pattern 2017 Lenovo. All rights reserved.. Under cloud Overcloud Management and Operations Controllers Computes Storage Hyperconverged All in One OSP ? ? ? ? ? ?
10 Management Cloud Realistic Deployment Pattern 2017 Lenovo. All rights reserved.. Undercloud Overcloud Management and Operations Managed Cloud Under cloud Overcloud Managed Cloud Under cloud Overcloud Managing
11 Management Cloud Large Deployment Pattern 2017 Lenovo. All rights reserved.. Managed Cloud Managed Cloud Very Large Central Site Cloud Mid Size Regional Deployment Mid Size Regional Deployment Mid Size Regional Deployment Mid Size Regional Cloud Mid Size Regional Deployment Mid Size Regional Deployment Mid Size Regional Deployment Mid Size Regional Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud
12 Size / Cost / Resource Optimized Cloud Infrastructure All deployment patterns have element that is sensitive to cost / space / power / skills / complexity Objectives achieved by compressing and collocating distinctive functions on the same set of HW and utilizing familiar concepts Converged Infrastructure Management Plane (CIMP) OpenStack and non OpenStack management functions share the same HW resources Compressed Data Plane Hyperconverged Compute Node collocating compute and storage on the same node Resource protection of shared resources 2017 Lenovo. All rights reserved..
13 What Is Control Plane and Why Does It Matter 2017 Lenovo. All rights reserved.. Under cloud Overcloud Management and Operations Controllers Computes Storage All in One OSP Control Plane Data / Hosting Plane Hyperconverged
14 Identical Controller Nodes Identical Controller Nodes Monolithic Controller 2017 Lenovo. All rights reserved.. Identical Controller Nodes Identity Bare Metal Networking Image Services Orchestration Telemetry Workflow Pacemaker* Block Storage* Clustering* Messaging Object Storage Compute Mongo Shared Filesys* MariaDB OpenvSwitch HAProxy Galera* Dashboard*
15 Control Plane Beyond OpenStack Services Required to support full set of SDDC features Provide resiliency and autonomy for remote locations SDN – Contrail, NSX, Nuage Configuration Management – Ansible Tower, Pupper, SaltStack Logging – Logstash, Splunk Analytics – Elasticsearch Visualization – Kibana, Graphana, Prometheus Monitoring – Nagios, Zabbix, DataDog Performance Monitoring – Telegraf, CollectD, Graphite, InfluxDB Security – PowerBroker, ESM, CyberArk Capacity planning and optimization – Cirba, ManageIQ, 2017 Lenovo. All rights reserved..
16 Compute Scheduler Controller Disaggregated Controller 2017 Lenovo. All rights reserved.. Generic Services Controller Identity Bare Metal Networking Image Services Orchestration Workflow PacemakerBlock Storage Clustering Messaging Object Storage Compute Shared Filesys OpenvSwitch HAProxy Database Controller Telemetry Pacemaker Mongo MariaDB Galera HAProxy UI Services Controller Dashboard HAProxy HAProxy  OpenStack Performance Team, Barcelona Summit 2016
17 Hyperconverged Compute and Storage Node 2017 Lenovo. All rights reserved.. Compute Node Compute Telemetry OpenvSwitch KVM Block Storage Node Block Storage Telemetry OpenvSwitch Object Storage Node Object Storage Telemetry OpenvSwitch Ceph* Hyperconverged Compute and Storage Node Compute Telemetry OpenvSwitchKVM Block Storage Ceph*
18 C+C+S NodeC+C+S NodeC+C+S NodeC+C+S Node Hyperconverged Controller Compute and Storage Node 2017 Lenovo. All rights reserved.. C+C+S Node Identity Networking Telemetry Compute Messaging OpenvSwitch MariaDB Compute / KVM Ceph OSD Swift C+C+S Node Dashboard Networking Image Service Compute Compute OpenvSwitch MariaDB Compute / KVM Ceph OSD Swift C+C+S Node Image Service Networking Telemetry Identity Messaging OpenvSwitch MariaDB Compute / KVM Ceph OSD Swift C+C+S Node Identity Telemetry Dashboard Compute Image Service OpenvSwitch Mongo Compute / KVM Ceph OSD Swift
19 Containerized Controller 2017 Lenovo. All rights reserved.. Containerized AIO Node Identity Bare Metal Networking Image Services Orchestration Telemetry Workflow Pacemaker Block Storage Clustering Messaging Object Storage Compute Mongo Shared Filesys MariaDB OpenvSwitch HAProxy Galera Dashboard Compute / KVM CephSwift Kolla, OpenStack Helm, OpenStack LOCI Container https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.html helm install kolla-kubernetes/helm/service/mariadb --name mariadb helm install kolla-kubernetes/helm/service/rabbitmq --name rabbitmq --values ./cloud.yaml helm install kolla-kubernetes/helm/service/memcached --name memcached --values ./cloud.yaml helm install kolla-kubernetes/helm/service/keystone --name keystone --values ./cloud.yaml helm install kolla-kubernetes/helm/service/glance --name glance --values ./cloud.yaml helm install kolla-kubernetes/helm/service/cinder-control --name cinder-control --values ./cloud.yaml helm install kolla-kubernetes/helm/service/horizon --name horizon --values ./cloud.yaml helm install kolla-kubernetes/helm/service/openvswitch --name openvswitch --values ./cloud.yaml helm install kolla-kubernetes/helm/service/neutron --name neutron --values ./cloud.yaml helm install kolla-kubernetes/helm/service/nova-control --name nova-control --values ./cloud.yaml helm install kolla-kubernetes/helm/service/nova-compute --name nova-compute --values ./cloud.yaml helm install kolla-kubernetes/helm/microservice/nova-cell0-create-db-job --name nova-cell0- create-db-job --values ./cloud.yaml helm install kolla-kubernetes/helm/microservice/nova-api-create- helm install kolla-kubernetes/helm/service/cinder-volume-lvm watch -d -n 5 -c kubectl get pods --all-namespaces
20 Containerized OpenStack Infrastructure 2017 Lenovo. All rights reserved.. DCOS Node DCOS Node DCOS Node DCOS (K8S, Docker, Mesos) ? ? ? DCOS Management components (etcd, K8S Master, etc.) provisioned, operated and managed with production SLAs VM VM VM OpenStack Services VM VM VM OpenStack Services VM VM VM OpenStack Services
21 Resilient Converged Infrastructure Control Plane Use familiar and production ready technologies to provide cost / space / power / skills / complexity effective control plane required to support full set of SDDC features Provide resiliency at all levels for lights out operations Network Redundant management and data plane switches Redundant dual port NICs Servers and storage Redundant PDUs and power supplies 3+ node bare metal server cluster sizes Software Off the shelf, commodity software Live Migration, Shared storage, Snapshots, Backup2017 Lenovo. All rights reserved..
22 Control Plane for Resource Constrained Deployments 2017 Lenovo. All rights reserved.. 3 Bare Metal Servers Converged Infrastructure Control Plane Rack Converged Infrastructure Control Plane Compute/Storage Hosting Plane HW Mgmt VM Undercloud VM 3 x Controller VMs Each Compute / Storage Bare Metal Server Nova-compute process Ceph OSD process HDD HDD HDD HDD VM VM VM VM  Standard Lenovo servers and network switches  Management components (XClarity, Undercloud, ManageIQ, etc.) deployed in virtualized fashion)  Distributed storage provided by Ceph  Ceph deployed in hyper- converged mode alongside the KVM hypervisor  Easy expansibility to host future management and operations functions Ops VM 1-N
23 Virtualized Control Plane Structure 2017 Lenovo. All rights reserved.. CIMP Node 1 CIMP Node 2 CIMP Node 3 Virtual Switches/networks Shared Storage Pool Hypervisor Guest VMGuest VMGuest VMControl Plane VM Guest VMGuest VMGuest VMControl Plane VM Guest VMGuest VMGuest VMControl Plane VM Hypervisor KVM, Libvirt Live Migration Snapshots Networking Linux Bridge VLANs Storage GlusterFS Shared storage File system replication HA
24 Lenovo Integrated HW Platform in 2 Form Factors 2017 Lenovo. All rights reserved.. 10U Front 10U Rear Half or Full Width
25 Lenovo HW Management Platform - XClarity 2017 Lenovo. All rights reserved..
26 Management Network Customer Network x3550 x3650 IMM/PXE/Internal Crossconnect Uplink G7052 ENOS G8272 CNOS ISL 51- 52 ISL 53- 54 Network Topology Data Network x3550 x3650 G8272 CNOS ISL 53- 54 Customer Network 2017 Lenovo. All rights reserved.. IMM/PXE/Internal Stg/Ext/Tenant Crossconnect Uplink Stg/Ext/Tenant
27 Virtualized Control Plane Networking Resiliency 2017 Lenovo. All rights reserved..
28 Physical Network Setup and Resiliency 2017 Lenovo. All rights reserved..
29 Virtualized Control Plane Networking 2017 Lenovo. All rights reserved.. Undercloud VM Controller VM XClarity VM
30 Virtualized Control Plane Storage 2017 Lenovo. All rights reserved.. CIMP Node 1 CIMP Node 2 CIMP Node 3 brick brick brick File1 File2 replications local local local 1. GlusterFS Installation  # yum update –y  # yum install glusterfs-server  # systemctl enable glusterd  # systemctl start glusterd  # systemctl status glusterd  Configure firewall to enable traffic on ports used by gluster 2. Build XFS bricks  # pvcreate /dev/vdb  # vgcreate vg_gluster /dev/vdb  # lvcreate -L 1000G -n brick1 vg_gluster  # mkfs.xfs /dev/vg_gluster/brick1  # mkdir -p /bricks/brick1  # mount /dev/vg_gluster/brick1 /var/bricks/images  Add the following line at end of /etc/fstab:  /dev/vg_gluster/brick1 /bricks/brick1 xfs defaults 0 0 3. Configure trusted pool  #gluster peer probe cimp-node2  #gluster peer probe cimp-node3  #gluster peer status 4. Create GlusterFS volumes  # sudo gluster volume create vol-1 replica 3 cimp-node1:/var/bricks/running cimp-node2:/var/bricks/running cimp-node3:/var/bricks/running  # sudo gluster volume start vol-1  Confirm the Gluster volume running:  # sudo gluster volume info all 5. Use the GlusterFS volume as a shared storage pool  # sudo mkdir –p /var/images/running  # sudo mount -t glusterfs cimp-node1:/vol-1 /var/images/running  # echo "127.0.0.1:vol-1 /var/images/running glusterfs defaults,_netdev,noauto,x-systemd.automount 0 0" >> /etc/fstab  # sudo setsebool –P virt_use_fusefs 1
31 Virtualized Controller VM Definition 2017 Lenovo. All rights reserved.. <domain type="kvm" id="1"> <name>controller1</name> <memory unit="GB">64</memory> ... <os> <type machine="pc" arch="x86_64">hvm</type> <boot dev="network"/> <boot dev="hd"/> ... <devices> ... <disk device="disk" type="file"> <driver type="qcow2" name="qemu"/> <source file="/var/images/controller/controller1_os.qcow2"/> ... </disk> <disk device="disk" type="file"> <driver type="qcow2" name="qemu"/> <source file="/var/images/controller/controller1_mongo.qcow2"/> ... </disk> <interface type="bridge"> <source bridge="br-pxe"/> <target dev="vnet0"/> ... </interface> <interface type="bridge"> <source bridge="br-cloud1"/> <target dev="vnet1"/> ... </interface> <interface type="bridge"> <source bridge="br-cloud2"/> <target dev="vnet2"/> ... </interface> </devices> </domain> virsh create controller1.xml
32 Virtualized OpenStack Controllers Considerations Power Control Undercloud to use virsh to control power management of other nodes Pre Pike release pxe_ipmitool => pxe_ssh Starting from Pike transition to virtualbmc High Availability Core services – Galera, RabbitMQ, Redis Active-Passive services – Cinder-Volume service SystemD services – independent and able to withstand service interruption Isolating a faulty node to protect a cluster and its resources Pacemaker + Shoot-The-Other-Node-In-The-Head Fencing agent – fence_ipmilan / fence_xvm, fence_virt2017 Lenovo. All rights reserved..
33 Hyperconverged Compute Node Deploy Ceph OSD alongside nova-compute on the same node Tuning - Compute shares memory and CPU with Ceph OSD Nova.conf – set aside CPU and memory for Ceph OSD cpu_allocation_ratio / reserved_host_memory_mb Ceph.conf – balance system resources needed for ceph recovery and rebalancing and guest workloads Osd_recovery_op_priority, osd_recovery_max_active, osd_max_backfilles Tune overal system performance - throughput-performance NUMA Pinning of Ceph OSD processes <=> NIC PCIe Slots Operations Nova / Compute operational worklows must account for Ceph OSD processes and vice versa2017 Lenovo. All rights reserved..
34 Testing using Rally 2017 Lenovo. All rights reserved.. Tenant creation with user  Conditions 20-concurrency tests (create a tenant with user) and totally accomplish 100 times of tests  Overall Results All of the tasks were successfully finished. Majority of them accomplished the tasks within 4 seconds and the maximum consumed 6 seconds
35 Testing using Rally 2017 Lenovo. All rights reserved.. Snapshot creation and deletion  Conditions 10-concurrency tests (create a snapshot and delete it, set it to 10 since the snapshot limitation is 10) and totally accomplish 100 times of tests  Overall Results All of the tasks were successfully finished. Majority of them accomplished the tasks within 17 seconds and the maximum consumed 23 seconds
36 Testing using Rally 2017 Lenovo. All rights reserved.. Small instance creation and deletion  Conditions 1. Image: TestVM (around 30M) 2. Flavor: 1 core + 512M RAM+ 20G Disk 3. 20-concurrency tests (create a instance and then delete it) and totally accomplish 100 times of tests  Overall Results All of the tasks were successfully finished. Majority of them accomplished the tasks within 28 seconds and the maximum consumed 30 seconds
37 Testing using Rally 2017 Lenovo. All rights reserved.. Large instance creation and deletion  Conditions: 1. Image: Redhat 7.2 (around 4G) 2. Flavor: 8 core + 19G RAM+ 100G Disk 3. 5-concurrency tests (create a instance and then delete it, set concurrency to 5 due to quota limitation) and totally accomplish 100 times of tests  Overall Results Almost all of the tasks were successfully finished, some of them failed because the RAM of coexisted instances exceeded the quota limitation. Majority of them accomplished the tasks within 58 seconds and the maximum consumed almost 60 seconds
38 Testing using Phoronix Test Suite  Open source testing suites including test development framework, test runner, management and reporting  Benchmark catalog at OpenBenchmarking.org is comprehensive 984 tests & suite  Many real-world workloads wrapped for benchmarking  Corpus of shared results for comparison / initial settings  Easy to extend and share 2017 Lenovo. All rights reserved.. Development environment to deploy, configure and package PTS OpenStack Image with PTS Installed VMs in OpenStack running PTS test suites Client launching and controlling PTS VMs
39 Testing using Phoronix Test Suite 2017 Lenovo. All rights reserved.. 0 5000 10000 15000 20000 25000 30000 35000 40000 1 11 21 31 41 51 Average IOPS per Number of VMs (randread) IOPS_4k IOPS_64k IOPS_8k 0 20000 40000 60000 80000 100000 120000 140000 160000 0 5 10 15 20 25 30 35 40 45 50 IOPS Number of VMs Total IOPS per Number of VMs (randread) IOPS_64k IOPS_4k IOPS_8k Number of VMs 1 pts/fio-1.9.0 - randread libaio 0 1 8k 1 64 / 3 pts/fio-1.9.0 - randread libaio 0 1 8k 1 64 / 6 pts/fio-1.9.0 - randread libaio 0 1 8k 1 64 / 15 pts/fio-1.9.0 - randread libaio 0 1 8k 1 64 10g / 30 pts/fio-1.9.0 - randread libaio 0 1 8k 1 64 10g / 45 pts/fio-1.9.0 - randread libaio 0 1 8k 1 64 10g /
40 Testing using Phoronix Test Suite 2017 Lenovo. All rights reserved.. 0 100 200 300 400 500 600 700 0 10 20 30 40 50 CompilationTime(inseconds) Number of VMs Compilation time vs number of VMs Build Linux kernel Build Php Build Image Magick Build Apache 0 100 200 300 400 500 600 700 800 0 10 20 30 40 50 CompilationTime(inseconds) Number of VMs Compilation time vs number of VMs Build Linux kernel Build Linux Kernel Min Build Linux kernel max pts/build-apache-1.5.1 - pts/build-imagemagick-1.7.2 - pts/build-linux-kernel-1.7.0 - pts/build-php-1.3.1 -
41 Virtualized Control Plane Operations Similar concept as containers but ready to use today Reuse existing expertise in operating virtualized infrastructure Using the Host HA capabilities in addition to the app level HA Better resource utilization for resource limited deployments, can pack more components on the same HW VM snapshots before patching VM migration before HW upgrades VM resizing to provide additional resources VM restore after failure Implication on VM structure and services deployments, “OS/services” stateless disk and data on separate disk Should be combined with and is not replacement for backup strategy2017 Lenovo. All rights reserved..
42 Potential Future Work Reusable VM images for Undercloud and Overcloud Containerized Control Plane 2017 Lenovo. All rights reserved..
43 Resources  https://access.redhat.com/articles/2922421  https://access.redhat.com/articles/2360321  https://access.redhat.com/articles/2861641  https://access.redhat.com/documentation/en- us/red_hat_openstack_platform/10/html- single/understanding_red_hat_openstack_platform_high_availability/  http://docs.openstack.org/developer/performance-docs/  http://tripleo.org/environments/virtualbmc.html  https://docs.openstack.org/developer/kolla-kubernetes/deployment- guide.html  https://www.openstack.org/assets/presentation-media/Chasing-1000-nodes- scale.pdf  https://www.phoronix-test-suite.com/  https://openbenchmarking.org 2017 Lenovo. All rights reserved..
Comparison of control plane deployment architectures in the scope of hyperconverged OpenStack infrastructure

More Related Content

PDF
Storage based on_openstack_mariocho
byMario Cho
 
PDF
From Kubernetes to OpenStack in Sydney
bySK Telecom
 
PDF
Kubernetes meetup 101
byJakir Patel
 
PDF
Kubernetes Day 2017 - Build, Ship and Run Your APP, Production !!
bysmalltown
 
PDF
Building stateful applications on Kubernetes with Rook
byRoberto Hashioka
 
PDF
Docker Networking with Container Orchestration Engines [Docker Meetup Santa C...
byDebra Robertson
 
PDF
[OpenInfra Days Korea 2018] Day 2 - E6 - OpenInfra monitoring with Prometheus
byOpenStack Korea Community
 
PDF
DevOps in AWS with Kubernetes
byOleg Chunikhin
 
Storage based on_openstack_mariocho
byMario Cho
 
From Kubernetes to OpenStack in Sydney
bySK Telecom
 
Kubernetes meetup 101
byJakir Patel
 
Kubernetes Day 2017 - Build, Ship and Run Your APP, Production !!
bysmalltown
 
Building stateful applications on Kubernetes with Rook
byRoberto Hashioka
 
Docker Networking with Container Orchestration Engines [Docker Meetup Santa C...
byDebra Robertson
 
[OpenInfra Days Korea 2018] Day 2 - E6 - OpenInfra monitoring with Prometheus
byOpenStack Korea Community
 
DevOps in AWS with Kubernetes
byOleg Chunikhin
 

What's hot

PPTX
Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.
byOpcito Technologies
 
PPTX
Using Rook to Manage Kubernetes Storage with Ceph
byCloudOps2005
 
PDF
Introduction to Kubernetes
byRoss Kukulinski
 
PDF
Kubernetes Node Deep Dive
byLei (Harry) Zhang
 
PDF
Network plugins for kubernetes
byinwin stack
 
PPTX
Kubernetes - State of the Union (Q1-2016)
byDoiT International
 
PDF
K8s storage-glusterfs-20180210
byChe-Chia Chang
 
PDF
Kubernetes intro public - kubernetes meetup 4-21-2015
byRohit Jnagal
 
PDF
OpenShift v3 Internal networking details
byEtsuji Nakai
 
PPTX
Introduction kubernetes 2017_12_24
bySam Zheng
 
PDF
Docker Summit 2016 - Kubernetes: Sweets and Bitters
bysmalltown
 
PDF
OpenStack Tokyo Meeup - Gluster Storage Day
byDan Radez
 
PDF
Running Legacy Applications with Containers
byLinuxCon ContainerCon CloudOpen China
 
PPT
Kubernetes on CloudStack with coreOS
bySebastien Goasguen
 
PPTX
How to Achieve Canary Deployment on Kubernetes
byHanLing Shen
 
PDF
Kubernetes Networking
byCJ Cullen
 
PPTX
Introduction to Kubernetes
byVishal Biyani
 
PDF
Build Your Own CaaS (Container as a Service)
byHungWei Chiu
 
PDF
[OpenInfra Days Korea 2018] Day 2 - E5-1: "Invited Talk: Kubicorn - Building ...
byOpenStack Korea Community
 
PPTX
Lifecycle of a pod
byHarshal Shah
 
Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.
byOpcito Technologies
 
Using Rook to Manage Kubernetes Storage with Ceph
byCloudOps2005
 
Introduction to Kubernetes
byRoss Kukulinski
 
Kubernetes Node Deep Dive
byLei (Harry) Zhang
 
Network plugins for kubernetes
byinwin stack
 
Kubernetes - State of the Union (Q1-2016)
byDoiT International
 
K8s storage-glusterfs-20180210
byChe-Chia Chang
 
Kubernetes intro public - kubernetes meetup 4-21-2015
byRohit Jnagal
 
OpenShift v3 Internal networking details
byEtsuji Nakai
 
Introduction kubernetes 2017_12_24
bySam Zheng
 
Docker Summit 2016 - Kubernetes: Sweets and Bitters
bysmalltown
 
OpenStack Tokyo Meeup - Gluster Storage Day
byDan Radez
 
Running Legacy Applications with Containers
byLinuxCon ContainerCon CloudOpen China
 
Kubernetes on CloudStack with coreOS
bySebastien Goasguen
 
How to Achieve Canary Deployment on Kubernetes
byHanLing Shen
 
Kubernetes Networking
byCJ Cullen
 
Introduction to Kubernetes
byVishal Biyani
 
Build Your Own CaaS (Container as a Service)
byHungWei Chiu
 
[OpenInfra Days Korea 2018] Day 2 - E5-1: "Invited Talk: Kubicorn - Building ...
byOpenStack Korea Community
 
Lifecycle of a pod
byHarshal Shah
 

Similar to Comparison of control plane deployment architectures in the scope of hyperconverged OpenStack infrastructure

PDF
Banv meetup 04162014
byozkan01
 
PDF
tcp cloud - Advanced Cloud Computing
byMarketingArrowECS_CZ
 
PPTX
CloudComp 2015 - SDN-Cloud Testbed with Hyper-convergent SmartX Boxes
byGIST (Gwangju Institute of Science and Technology)
 
PDF
Lenovo midokura
byAshmar Kalangottil
 
PPTX
Cloudstack conference open_contrail v4
byozkan01
 
PDF
Meetup
byVictor Estival
 
PDF
[OpenStack Day in Korea 2015] Track 3-2 - Huawei Cloud Computing Powered by O...
byOpenStack Korea Community
 
PPTX
Cloud computing and OpenStack
byEdgar Magana
 
PDF
OpenStack at the speed of business with SolidFire & Red Hat
byNetApp
 
PDF
Banv meetup-contrail
bynvirters
 
PPTX
Hybrid Cloud and Hyper Cloud
byEshed Gal-Or
 
PPTX
aCloud Services Architecture - Harshal Pimpalkhute, Sr. Prod. Mktg. Mgr., A1...
byMirantis
 
PDF
Openstack - Enterprise cloud management platform
byNagaraj Shenoy
 
PDF
OpenStack Toronto Q2 MeetUp - June 1st 2017
byStacy Véronneau
 
PDF
Montreal OpenStack Q2 MeetUp - May 30th 2017
byStacy Véronneau
 
PPTX
CloudStack vs Openstack
byMadan Ganesh Velayudham
 
PDF
OpenStack Ottawa Q2 MeetUp - May 31st 2017
byStacy Véronneau
 
PDF
Automated Deployment and Management of Edge Clouds
byJay Bryant
 
PDF
Deterministic capacity planning for OpenStack as elastic cloud infrastructure
bySean Cohen
 
PPTX
OpenStack Ecosystem (10000 feet oveview)
byAnil Bidari ( CEO , Cloud Enabled)
 
Banv meetup 04162014
byozkan01
 
tcp cloud - Advanced Cloud Computing
byMarketingArrowECS_CZ
 
CloudComp 2015 - SDN-Cloud Testbed with Hyper-convergent SmartX Boxes
byGIST (Gwangju Institute of Science and Technology)
 
Lenovo midokura
byAshmar Kalangottil
 
Cloudstack conference open_contrail v4
byozkan01
 
Meetup
byVictor Estival
 
[OpenStack Day in Korea 2015] Track 3-2 - Huawei Cloud Computing Powered by O...
byOpenStack Korea Community
 
Cloud computing and OpenStack
byEdgar Magana
 
OpenStack at the speed of business with SolidFire & Red Hat
byNetApp
 
Banv meetup-contrail
bynvirters
 
Hybrid Cloud and Hyper Cloud
byEshed Gal-Or
 
aCloud Services Architecture - Harshal Pimpalkhute, Sr. Prod. Mktg. Mgr., A1...
byMirantis
 
Openstack - Enterprise cloud management platform
byNagaraj Shenoy
 
OpenStack Toronto Q2 MeetUp - June 1st 2017
byStacy Véronneau
 
Montreal OpenStack Q2 MeetUp - May 30th 2017
byStacy Véronneau
 
CloudStack vs Openstack
byMadan Ganesh Velayudham
 
OpenStack Ottawa Q2 MeetUp - May 31st 2017
byStacy Véronneau
 
Automated Deployment and Management of Edge Clouds
byJay Bryant
 
Deterministic capacity planning for OpenStack as elastic cloud infrastructure
bySean Cohen
 
OpenStack Ecosystem (10000 feet oveview)
byAnil Bidari ( CEO , Cloud Enabled)
 

Recently uploaded

PPTX
Unit-4-ARTIFICIAL NEURAL NETWORKS.pptx ANN ppt Artificial neural network
byssuserd4481a
 
PPTX
From Backup to Resilience: How MSPs Are Preparing for 2026
byMSP360
 
PDF
Day 2 - Network Security ~ 2nd Sight Lab ~ Cloud Security Class ~ 2020
by2nd Sight Lab
 
PPTX
Building Cyber Resilience for 2026: Best Practices for a Secure, AI-Driven Bu...
byYasir Naveed Riaz
 
PDF
Data Virtualization in Action: Scaling APIs and Apps with FME
bySafe Software
 
DOCX
iRobot Post‑Mortem and Alternative Paths - Discussion Document for Boards and...
byDave Litwiller
 
PDF
Day 5 - Red Team + Blue Team in the Cloud - 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
PDF
TrustArc Webinar - Looking Ahead: The 2026 Privacy Landscape
byTrustArc
 
PDF
Cybersecurity: Safeguarding Digital Assets
byYasir Naveed Riaz
 
PDF
Real-Time Data Insight Using Microsoft Forms for Business
byElevate
 
PDF
Day 1 - Cloud Security Strategy and Planning ~ 2nd Sight Lab ~ Cloud Security...
by2nd Sight Lab
 
PPTX
DYNAMICALLY.pptx good for the teachers or students to do seminars and for tea...
bympoorvika031
 
PDF
Energy Storage Landscape Clean Energy Ministerial
bySurajitBanerjee38
 
PDF
Security Forum Sessions from Houston 2025 Event
byMark Simos
 
PPTX
AI in Cybersecurity: Digital Defense by Yasir Naveed Riaz
byYasir Naveed Riaz
 
PDF
Internet_of_Things_IoT_for_Next_Generation_Smart_Systems_Utilizing.pdf
byRoshanSyed12
 
PDF
DevFest El Jadida 2025 - Product Thinking
byElmehdi AMLOU
 
PPT
software-security-intro in information security.ppt
byismaeelsahib032
 
PPTX
Cloud-and-AI-Platform-FY26-Partner-Playbook.pptx
byiuiuiuiu1
 
PPTX
AI's Impact on Cybersecurity - Challenges and Opportunities
byYasir Naveed Riaz
 
Unit-4-ARTIFICIAL NEURAL NETWORKS.pptx ANN ppt Artificial neural network
byssuserd4481a
 
From Backup to Resilience: How MSPs Are Preparing for 2026
byMSP360
 
Day 2 - Network Security ~ 2nd Sight Lab ~ Cloud Security Class ~ 2020
by2nd Sight Lab
 
Building Cyber Resilience for 2026: Best Practices for a Secure, AI-Driven Bu...
byYasir Naveed Riaz
 
Data Virtualization in Action: Scaling APIs and Apps with FME
bySafe Software
 
iRobot Post‑Mortem and Alternative Paths - Discussion Document for Boards and...
byDave Litwiller
 
Day 5 - Red Team + Blue Team in the Cloud - 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
TrustArc Webinar - Looking Ahead: The 2026 Privacy Landscape
byTrustArc
 
Cybersecurity: Safeguarding Digital Assets
byYasir Naveed Riaz
 
Real-Time Data Insight Using Microsoft Forms for Business
byElevate
 
Day 1 - Cloud Security Strategy and Planning ~ 2nd Sight Lab ~ Cloud Security...
by2nd Sight Lab
 
DYNAMICALLY.pptx good for the teachers or students to do seminars and for tea...
bympoorvika031
 
Energy Storage Landscape Clean Energy Ministerial
bySurajitBanerjee38
 
Security Forum Sessions from Houston 2025 Event
byMark Simos
 
AI in Cybersecurity: Digital Defense by Yasir Naveed Riaz
byYasir Naveed Riaz
 
Internet_of_Things_IoT_for_Next_Generation_Smart_Systems_Utilizing.pdf
byRoshanSyed12
 
DevFest El Jadida 2025 - Product Thinking
byElmehdi AMLOU
 
software-security-intro in information security.ppt
byismaeelsahib032
 
Cloud-and-AI-Platform-FY26-Partner-Playbook.pptx
byiuiuiuiu1
 
AI's Impact on Cybersecurity - Challenges and Opportunities
byYasir Naveed Riaz
 

Comparison of control plane deployment architectures in the scope of hyperconverged OpenStack infrastructure

  • 1.
    2017 Lenovo. Allrights reserved. Comparison of control plane deployment architectures in the scope of hyperconverged OpenStack infrastructure Miroslav Halas Lenovo Cloud Technology Center May 2017
  • 2.
    2 About me  Directorof SW Architecture, Cloud Infrastructure  SW Engineer, Architect and Leader  Designed, implemented and operated Desktop, Mobile, SaaS, Cloud Applications and Large Private Cloud Platforms  10 years of engineering leadership for information security, public and private cloud for one of top 3 financials in US  Holds patents in security, private and public cloud areas  Carries picture of his home server in his pocket 2017 Lenovo. All rights reserved.. linkedin.com/in/miroslavhalas/
  • 3.
    3 Outline Introduction to LenovoCloud Technology Center What It Takes to Deploy and Operate SDDC What Is Control Plane and Why Does It Matter Resilient Converged Infrastructure Control Plane Evaluation and Testing using Rally and Phoronix Test Suite 2017 Lenovo. All rights reserved..
  • 4.
    4 Standing on theShoulders of Giants Lenovo DCG Research & Technology • Focused research in Systems, Storage, Cloud, AI, Big Data, etc. Lenovo System Technology Innovation Center • Co-innovating with partners and customers to deliver Next-Next ideas such as Abstract hybrid data centers, Dynamic reconfiguration of IT and Deployment of IIoT Lenovo Cloud Technology Center • Pioneering cloud advancements, deployment and management experiences with community, partners and customers Lenovo DCG Product & Development • Engineer and support production ready enterprise solutions 2017 Lenovo. All rights reserved..
  • 5.
    5 Lenovo Cloud TechnologyCenter (LCTC) 2017 Lenovo. All rights reserved.. Mission: Align and Drive Lenovo entry and leadership in Enterprise Cloud Infrastructure, building core capabilities and partnerships for a complete portfolio of Solutions – focus on open source • Deep customer engagement and support • Business/Partner development, Technical Product evaluations • Open source ecosystem visibility and credibility Customer and Market Engagement • OpenStack and Open Systems development – distros, containers and integrations with solutions • SDDC – development of software defined storage, networking • Professional and Managed services Portfolio Development • Cloud architecture and advisory board • e2e Cloud roadmap Center of Competence Credible Cloud Market Presence Complete Cloud Solutions Portfolio Depth and Alignment of Expertise Long Term GoalKey Activities MAJOR RESEARCH CENTERS IN RALEIGH, BEIJING, SHANGHAI
  • 6.
    6 OpenStack Upstream Engagement 2017Lenovo. All rights reserved.. Reviews Commits
  • 7.
    7 ManageIQ Upstream Engagement 2017Lenovo. All rights reserved..  Initial provider of ManageIQ Physical Infrastructure  Features/Use cases  Physical Server Data Model  New XClarity Provider  XClarity Provider Summary View  Physical Server Inventory via REST  Vital Product Data (VPD)  Firmware Levels  Server to Platform Host Relationships  Outstanding development support from Red Hat ManageIQ / CloudForms team  Open Source Contributions Summary  ManageIQ Lenovo Provider  https://github.com/ManageIQ/manageiq-providers-lenovo  Lenovo XClarity Client (Ruby)  https://github.com/lenovo/xclarity_client …
  • 8.
    8 What It Takesto Deploy and Operate SDDC 2017 Lenovo. All rights reserved.
  • 9.
    9 Simple Deployment Pattern 2017Lenovo. All rights reserved.. Under cloud Overcloud Management and Operations Controllers Computes Storage Hyperconverged All in One OSP ? ? ? ? ? ?
  • 10.
    10 Management Cloud Realistic DeploymentPattern 2017 Lenovo. All rights reserved.. Undercloud Overcloud Management and Operations Managed Cloud Under cloud Overcloud Managed Cloud Under cloud Overcloud Managing
  • 11.
    11 Management Cloud Large Deployment Pattern 2017Lenovo. All rights reserved.. Managed Cloud Managed Cloud Very Large Central Site Cloud Mid Size Regional Deployment Mid Size Regional Deployment Mid Size Regional Deployment Mid Size Regional Cloud Mid Size Regional Deployment Mid Size Regional Deployment Mid Size Regional Deployment Mid Size Regional Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud Small Local Cloud
  • 12.
    12 Size / Cost/ Resource Optimized Cloud Infrastructure All deployment patterns have element that is sensitive to cost / space / power / skills / complexity Objectives achieved by compressing and collocating distinctive functions on the same set of HW and utilizing familiar concepts Converged Infrastructure Management Plane (CIMP) OpenStack and non OpenStack management functions share the same HW resources Compressed Data Plane Hyperconverged Compute Node collocating compute and storage on the same node Resource protection of shared resources 2017 Lenovo. All rights reserved..
  • 13.
    13 What Is ControlPlane and Why Does It Matter 2017 Lenovo. All rights reserved.. Under cloud Overcloud Management and Operations Controllers Computes Storage All in One OSP Control Plane Data / Hosting Plane Hyperconverged
  • 14.
    14 Identical Controller Nodes IdenticalController Nodes Monolithic Controller 2017 Lenovo. All rights reserved.. Identical Controller Nodes Identity Bare Metal Networking Image Services Orchestration Telemetry Workflow Pacemaker* Block Storage* Clustering* Messaging Object Storage Compute Mongo Shared Filesys* MariaDB OpenvSwitch HAProxy Galera* Dashboard*
  • 15.
    15 Control Plane BeyondOpenStack Services Required to support full set of SDDC features Provide resiliency and autonomy for remote locations SDN – Contrail, NSX, Nuage Configuration Management – Ansible Tower, Pupper, SaltStack Logging – Logstash, Splunk Analytics – Elasticsearch Visualization – Kibana, Graphana, Prometheus Monitoring – Nagios, Zabbix, DataDog Performance Monitoring – Telegraf, CollectD, Graphite, InfluxDB Security – PowerBroker, ESM, CyberArk Capacity planning and optimization – Cirba, ManageIQ, 2017 Lenovo. All rights reserved..
  • 16.
    16 Compute Scheduler Controller DisaggregatedController 2017 Lenovo. All rights reserved.. Generic Services Controller Identity Bare Metal Networking Image Services Orchestration Workflow PacemakerBlock Storage Clustering Messaging Object Storage Compute Shared Filesys OpenvSwitch HAProxy Database Controller Telemetry Pacemaker Mongo MariaDB Galera HAProxy UI Services Controller Dashboard HAProxy HAProxy  OpenStack Performance Team, Barcelona Summit 2016
  • 17.
    17 Hyperconverged Compute andStorage Node 2017 Lenovo. All rights reserved.. Compute Node Compute Telemetry OpenvSwitch KVM Block Storage Node Block Storage Telemetry OpenvSwitch Object Storage Node Object Storage Telemetry OpenvSwitch Ceph* Hyperconverged Compute and Storage Node Compute Telemetry OpenvSwitchKVM Block Storage Ceph*
  • 18.
    18 C+C+S NodeC+C+S NodeC+C+SNodeC+C+S Node Hyperconverged Controller Compute and Storage Node 2017 Lenovo. All rights reserved.. C+C+S Node Identity Networking Telemetry Compute Messaging OpenvSwitch MariaDB Compute / KVM Ceph OSD Swift C+C+S Node Dashboard Networking Image Service Compute Compute OpenvSwitch MariaDB Compute / KVM Ceph OSD Swift C+C+S Node Image Service Networking Telemetry Identity Messaging OpenvSwitch MariaDB Compute / KVM Ceph OSD Swift C+C+S Node Identity Telemetry Dashboard Compute Image Service OpenvSwitch Mongo Compute / KVM Ceph OSD Swift
  • 19.
    19 Containerized Controller 2017 Lenovo.All rights reserved.. Containerized AIO Node Identity Bare Metal Networking Image Services Orchestration Telemetry Workflow Pacemaker Block Storage Clustering Messaging Object Storage Compute Mongo Shared Filesys MariaDB OpenvSwitch HAProxy Galera Dashboard Compute / KVM CephSwift Kolla, OpenStack Helm, OpenStack LOCI Container https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.html helm install kolla-kubernetes/helm/service/mariadb --name mariadb helm install kolla-kubernetes/helm/service/rabbitmq --name rabbitmq --values ./cloud.yaml helm install kolla-kubernetes/helm/service/memcached --name memcached --values ./cloud.yaml helm install kolla-kubernetes/helm/service/keystone --name keystone --values ./cloud.yaml helm install kolla-kubernetes/helm/service/glance --name glance --values ./cloud.yaml helm install kolla-kubernetes/helm/service/cinder-control --name cinder-control --values ./cloud.yaml helm install kolla-kubernetes/helm/service/horizon --name horizon --values ./cloud.yaml helm install kolla-kubernetes/helm/service/openvswitch --name openvswitch --values ./cloud.yaml helm install kolla-kubernetes/helm/service/neutron --name neutron --values ./cloud.yaml helm install kolla-kubernetes/helm/service/nova-control --name nova-control --values ./cloud.yaml helm install kolla-kubernetes/helm/service/nova-compute --name nova-compute --values ./cloud.yaml helm install kolla-kubernetes/helm/microservice/nova-cell0-create-db-job --name nova-cell0- create-db-job --values ./cloud.yaml helm install kolla-kubernetes/helm/microservice/nova-api-create- helm install kolla-kubernetes/helm/service/cinder-volume-lvm watch -d -n 5 -c kubectl get pods --all-namespaces
  • 20.
    20 Containerized OpenStack Infrastructure 2017Lenovo. All rights reserved.. DCOS Node DCOS Node DCOS Node DCOS (K8S, Docker, Mesos) ? ? ? DCOS Management components (etcd, K8S Master, etc.) provisioned, operated and managed with production SLAs VM VM VM OpenStack Services VM VM VM OpenStack Services VM VM VM OpenStack Services
  • 21.
    21 Resilient Converged InfrastructureControl Plane Use familiar and production ready technologies to provide cost / space / power / skills / complexity effective control plane required to support full set of SDDC features Provide resiliency at all levels for lights out operations Network Redundant management and data plane switches Redundant dual port NICs Servers and storage Redundant PDUs and power supplies 3+ node bare metal server cluster sizes Software Off the shelf, commodity software Live Migration, Shared storage, Snapshots, Backup2017 Lenovo. All rights reserved..
  • 22.
    22 Control Plane forResource Constrained Deployments 2017 Lenovo. All rights reserved.. 3 Bare Metal Servers Converged Infrastructure Control Plane Rack Converged Infrastructure Control Plane Compute/Storage Hosting Plane HW Mgmt VM Undercloud VM 3 x Controller VMs Each Compute / Storage Bare Metal Server Nova-compute process Ceph OSD process HDD HDD HDD HDD VM VM VM VM  Standard Lenovo servers and network switches  Management components (XClarity, Undercloud, ManageIQ, etc.) deployed in virtualized fashion)  Distributed storage provided by Ceph  Ceph deployed in hyper- converged mode alongside the KVM hypervisor  Easy expansibility to host future management and operations functions Ops VM 1-N
  • 23.
    23 Virtualized Control PlaneStructure 2017 Lenovo. All rights reserved.. CIMP Node 1 CIMP Node 2 CIMP Node 3 Virtual Switches/networks Shared Storage Pool Hypervisor Guest VMGuest VMGuest VMControl Plane VM Guest VMGuest VMGuest VMControl Plane VM Guest VMGuest VMGuest VMControl Plane VM Hypervisor KVM, Libvirt Live Migration Snapshots Networking Linux Bridge VLANs Storage GlusterFS Shared storage File system replication HA
  • 24.
    24 Lenovo Integrated HWPlatform in 2 Form Factors 2017 Lenovo. All rights reserved.. 10U Front 10U Rear Half or Full Width
  • 25.
    25 Lenovo HW ManagementPlatform - XClarity 2017 Lenovo. All rights reserved..
  • 26.
    26 Management Network Customer Network x3550 x3650 IMM/PXE/Internal Crossconnect Uplink G7052 ENOS G8272 CNOS ISL51- 52 ISL 53- 54 Network Topology Data Network x3550 x3650 G8272 CNOS ISL 53- 54 Customer Network 2017 Lenovo. All rights reserved.. IMM/PXE/Internal Stg/Ext/Tenant Crossconnect Uplink Stg/Ext/Tenant
  • 27.
    27 Virtualized Control PlaneNetworking Resiliency 2017 Lenovo. All rights reserved..
  • 28.
    28 Physical Network Setupand Resiliency 2017 Lenovo. All rights reserved..
  • 29.
    29 Virtualized Control PlaneNetworking 2017 Lenovo. All rights reserved.. Undercloud VM Controller VM XClarity VM
  • 30.
    30 Virtualized Control PlaneStorage 2017 Lenovo. All rights reserved.. CIMP Node 1 CIMP Node 2 CIMP Node 3 brick brick brick File1 File2 replications local local local 1. GlusterFS Installation  # yum update –y  # yum install glusterfs-server  # systemctl enable glusterd  # systemctl start glusterd  # systemctl status glusterd  Configure firewall to enable traffic on ports used by gluster 2. Build XFS bricks  # pvcreate /dev/vdb  # vgcreate vg_gluster /dev/vdb  # lvcreate -L 1000G -n brick1 vg_gluster  # mkfs.xfs /dev/vg_gluster/brick1  # mkdir -p /bricks/brick1  # mount /dev/vg_gluster/brick1 /var/bricks/images  Add the following line at end of /etc/fstab:  /dev/vg_gluster/brick1 /bricks/brick1 xfs defaults 0 0 3. Configure trusted pool  #gluster peer probe cimp-node2  #gluster peer probe cimp-node3  #gluster peer status 4. Create GlusterFS volumes  # sudo gluster volume create vol-1 replica 3 cimp-node1:/var/bricks/running cimp-node2:/var/bricks/running cimp-node3:/var/bricks/running  # sudo gluster volume start vol-1  Confirm the Gluster volume running:  # sudo gluster volume info all 5. Use the GlusterFS volume as a shared storage pool  # sudo mkdir –p /var/images/running  # sudo mount -t glusterfs cimp-node1:/vol-1 /var/images/running  # echo "127.0.0.1:vol-1 /var/images/running glusterfs defaults,_netdev,noauto,x-systemd.automount 0 0" >> /etc/fstab  # sudo setsebool –P virt_use_fusefs 1
  • 31.
    31 Virtualized Controller VMDefinition 2017 Lenovo. All rights reserved.. <domain type="kvm" id="1"> <name>controller1</name> <memory unit="GB">64</memory> ... <os> <type machine="pc" arch="x86_64">hvm</type> <boot dev="network"/> <boot dev="hd"/> ... <devices> ... <disk device="disk" type="file"> <driver type="qcow2" name="qemu"/> <source file="/var/images/controller/controller1_os.qcow2"/> ... </disk> <disk device="disk" type="file"> <driver type="qcow2" name="qemu"/> <source file="/var/images/controller/controller1_mongo.qcow2"/> ... </disk> <interface type="bridge"> <source bridge="br-pxe"/> <target dev="vnet0"/> ... </interface> <interface type="bridge"> <source bridge="br-cloud1"/> <target dev="vnet1"/> ... </interface> <interface type="bridge"> <source bridge="br-cloud2"/> <target dev="vnet2"/> ... </interface> </devices> </domain> virsh create controller1.xml
  • 32.
    32 Virtualized OpenStack ControllersConsiderations Power Control Undercloud to use virsh to control power management of other nodes Pre Pike release pxe_ipmitool => pxe_ssh Starting from Pike transition to virtualbmc High Availability Core services – Galera, RabbitMQ, Redis Active-Passive services – Cinder-Volume service SystemD services – independent and able to withstand service interruption Isolating a faulty node to protect a cluster and its resources Pacemaker + Shoot-The-Other-Node-In-The-Head Fencing agent – fence_ipmilan / fence_xvm, fence_virt2017 Lenovo. All rights reserved..
  • 33.
    33 Hyperconverged Compute Node DeployCeph OSD alongside nova-compute on the same node Tuning - Compute shares memory and CPU with Ceph OSD Nova.conf – set aside CPU and memory for Ceph OSD cpu_allocation_ratio / reserved_host_memory_mb Ceph.conf – balance system resources needed for ceph recovery and rebalancing and guest workloads Osd_recovery_op_priority, osd_recovery_max_active, osd_max_backfilles Tune overal system performance - throughput-performance NUMA Pinning of Ceph OSD processes <=> NIC PCIe Slots Operations Nova / Compute operational worklows must account for Ceph OSD processes and vice versa2017 Lenovo. All rights reserved..
  • 34.
    34 Testing using Rally 2017Lenovo. All rights reserved.. Tenant creation with user  Conditions 20-concurrency tests (create a tenant with user) and totally accomplish 100 times of tests  Overall Results All of the tasks were successfully finished. Majority of them accomplished the tasks within 4 seconds and the maximum consumed 6 seconds
  • 35.
    35 Testing using Rally 2017Lenovo. All rights reserved.. Snapshot creation and deletion  Conditions 10-concurrency tests (create a snapshot and delete it, set it to 10 since the snapshot limitation is 10) and totally accomplish 100 times of tests  Overall Results All of the tasks were successfully finished. Majority of them accomplished the tasks within 17 seconds and the maximum consumed 23 seconds
  • 36.
    36 Testing using Rally 2017Lenovo. All rights reserved.. Small instance creation and deletion  Conditions 1. Image: TestVM (around 30M) 2. Flavor: 1 core + 512M RAM+ 20G Disk 3. 20-concurrency tests (create a instance and then delete it) and totally accomplish 100 times of tests  Overall Results All of the tasks were successfully finished. Majority of them accomplished the tasks within 28 seconds and the maximum consumed 30 seconds
  • 37.
    37 Testing using Rally 2017Lenovo. All rights reserved.. Large instance creation and deletion  Conditions: 1. Image: Redhat 7.2 (around 4G) 2. Flavor: 8 core + 19G RAM+ 100G Disk 3. 5-concurrency tests (create a instance and then delete it, set concurrency to 5 due to quota limitation) and totally accomplish 100 times of tests  Overall Results Almost all of the tasks were successfully finished, some of them failed because the RAM of coexisted instances exceeded the quota limitation. Majority of them accomplished the tasks within 58 seconds and the maximum consumed almost 60 seconds
  • 38.
    38 Testing using PhoronixTest Suite  Open source testing suites including test development framework, test runner, management and reporting  Benchmark catalog at OpenBenchmarking.org is comprehensive 984 tests & suite  Many real-world workloads wrapped for benchmarking  Corpus of shared results for comparison / initial settings  Easy to extend and share 2017 Lenovo. All rights reserved.. Development environment to deploy, configure and package PTS OpenStack Image with PTS Installed VMs in OpenStack running PTS test suites Client launching and controlling PTS VMs
  • 39.
    39 Testing using PhoronixTest Suite 2017 Lenovo. All rights reserved.. 0 5000 10000 15000 20000 25000 30000 35000 40000 1 11 21 31 41 51 Average IOPS per Number of VMs (randread) IOPS_4k IOPS_64k IOPS_8k 0 20000 40000 60000 80000 100000 120000 140000 160000 0 5 10 15 20 25 30 35 40 45 50 IOPS Number of VMs Total IOPS per Number of VMs (randread) IOPS_64k IOPS_4k IOPS_8k Number of VMs 1 pts/fio-1.9.0 - randread libaio 0 1 8k 1 64 / 3 pts/fio-1.9.0 - randread libaio 0 1 8k 1 64 / 6 pts/fio-1.9.0 - randread libaio 0 1 8k 1 64 / 15 pts/fio-1.9.0 - randread libaio 0 1 8k 1 64 10g / 30 pts/fio-1.9.0 - randread libaio 0 1 8k 1 64 10g / 45 pts/fio-1.9.0 - randread libaio 0 1 8k 1 64 10g /
  • 40.
    40 Testing using PhoronixTest Suite 2017 Lenovo. All rights reserved.. 0 100 200 300 400 500 600 700 0 10 20 30 40 50 CompilationTime(inseconds) Number of VMs Compilation time vs number of VMs Build Linux kernel Build Php Build Image Magick Build Apache 0 100 200 300 400 500 600 700 800 0 10 20 30 40 50 CompilationTime(inseconds) Number of VMs Compilation time vs number of VMs Build Linux kernel Build Linux Kernel Min Build Linux kernel max pts/build-apache-1.5.1 - pts/build-imagemagick-1.7.2 - pts/build-linux-kernel-1.7.0 - pts/build-php-1.3.1 -
  • 41.
    41 Virtualized Control PlaneOperations Similar concept as containers but ready to use today Reuse existing expertise in operating virtualized infrastructure Using the Host HA capabilities in addition to the app level HA Better resource utilization for resource limited deployments, can pack more components on the same HW VM snapshots before patching VM migration before HW upgrades VM resizing to provide additional resources VM restore after failure Implication on VM structure and services deployments, “OS/services” stateless disk and data on separate disk Should be combined with and is not replacement for backup strategy2017 Lenovo. All rights reserved..
  • 42.
    42 Potential Future Work ReusableVM images for Undercloud and Overcloud Containerized Control Plane 2017 Lenovo. All rights reserved..
  • 43.
    43 Resources  https://access.redhat.com/articles/2922421  https://access.redhat.com/articles/2360321 https://access.redhat.com/articles/2861641  https://access.redhat.com/documentation/en- us/red_hat_openstack_platform/10/html- single/understanding_red_hat_openstack_platform_high_availability/  http://docs.openstack.org/developer/performance-docs/  http://tripleo.org/environments/virtualbmc.html  https://docs.openstack.org/developer/kolla-kubernetes/deployment- guide.html  https://www.openstack.org/assets/presentation-media/Chasing-1000-nodes- scale.pdf  https://www.phoronix-test-suite.com/  https://openbenchmarking.org 2017 Lenovo. All rights reserved..