Whitepaper nebucom intelligent application broking and provisioning in a hybrid cloud environment
Intelligent Application Broking and Provisioning in a Hybrid Cloud
Pieter-JanMaenhaut, JorenInghelbrecht, BrunoVolckaert
The carbon footprintof data centersincreasesfurtherasevermore servicesare hostedin the cloud,
leveraging the needfor an efficient management of the available resources.State-of-the-art servers
exhibit a linear relationshipbetween resource and power usage, but software optimizations such as
operational costs. Furthermore, cloud providers struggle with the heterogeneity of virtualization
technologies, cloud management platforms and the underlying physical resources, and with the
upcoming popularity of container-based virtualization technologies, the question arises how well
these technologies perform regarding performance and isolation compared to traditional VM
Within IDLab, the Internet Technology and Data Science Lab, we have been working on the
developmentof ahighlyinteroperableplatformthatsupports applicationdeploymentonavarietyof
cloud management frameworks. The platform can be used for developing and evaluating multiple
intelligent VM/ container / resource allocation algorithms on top of the supported cloudplatforms,
and allows researchers to quickly setup experiments without having to dive deep into the complex
details of the underlying technologies.
In general, virtualization introduces an additional software layer between the host platform and the
virtualized resources. Cloudmanagementplatforms such as OpenStack, VMWare vCloudand Amazon
AWS make use of different virtualization technologies to manage the available cloud resources. For
example, a partition of a physical machine’s computational resources can be assigned to a Virtual
Machine(VM) orcontainer, andrentedtoacloud userasasingle computationalunit. Applicationsfrom
multiple usersare typically consolidatedonasingle physicalserver, leveraging theneed for a clear data
and performance isolation between the different units hosted on the same physical machine.
System-Level Virtualization vs OS-Level Virtualization
System-level virtualization creates an abstraction of the physical machine’s hardware and executes
one or more virtual machines on top of this virtualized hardware. This abstraction layer is typically
called the hypervisor, and can either be deployed on a host operating system (OS) or directly on
hardware (bare metal).System-level virtualizationoftenfaceshighperformance overheadandsuffers
fromI/Olimitations.IthoweverprovidesagoodisolationbetweenVMshosted onthe same machine,
which can be a stringent requirement for critical applications. Server consolidation can be achieved
by assigning a specific amount of resources to each VM, and efficiently collocating the VMs over a
minimumsetof physical servers,withoutviolatinganyof the guaranteedSLAs(e.g.maximumallowed
latencyand/orresponse time).A VMcan be savedas a single file,butthese filescanbe big in size as
the whole operatingsystemandsoftwarestackisincluded.Furthermore,snapshotscanbe takenof a
VM, which eases recovery and backup management.
OS-levelorcontainer-basedvirtualizationoperatesonahigherlevel,since containersshareanOS and
possibly libraries and binaries. The container engine provides the necessary abstraction layer for a
container to be deployable on different operating systems. OS-level virtualization is based on two
increase the manageability and security of the containers. Control groups allow processes to be
grouped together, and ensure that each group gets a share of memory, CPU and disk I/O. Kernel
namespaces ensure isolation between processes executed by different containers. As multiple
containersshare the underlyingOS,containerimagesare muchsmaller,makingcontainermigrations
much easier than VM migrations. Due to the OS-level virtualization,there is no need to virtualize
hardware, and therefore the overhead is limited to a minimum. Since containers are so lightweight,
an applicationcanbe orchestratedasmultipleinteractingcomponents(service-orientedarchitecture)
and each component can be deployed on a different container. This divide and conquer strategy is
also often called a micro service architecture.
Many concerns have been raised to the use of containers, in contrast to traditional VMs. As
virtualization happens on a higher level, the operating system, it is clear that the attack surface of
containers is much larger compared to VMs. As long as containers cannot provide isolation on the
same level as VMs, VMs will continue to play an important role within cloudcomputing. Containers
howevercan alsobe deployedwithinVMs,meaningthatusingone technologydoesnotexclude the
When hostingan applicationinthe cloud,the questionariseswhichvirtualizationtechnologyisbest
of the scenariosisprovedin Figure 1. In the firstscenario,whichwe referto as Bare Metal (BM), the
experimentsare executeddirectlyonthe hardware.In the Bare Metal – Container(BM-CT) andBare
Metal – Virtual Machine (BM-VM) scenarios,experimentsare executedinside acontainerandavirtual
machine respectively. The last scenario, Bare Metal – Virtual Machines – Container combines both
technologies by using a container engine inside a virtual machine.
Fig. 1. Overview of test scenarios used for the performance comparison.
The evaluated experiment consists of three performance tests. The first test is a CPU benchmarking
test, the second measures the disk I/O throughput and the last test determines the maximum RAM
copying bandwidth. All tests were executed on a physical machine with 4 CPU cores and 16 GB of
The CPU benchmarkingtest utilizesSysbench,anopensource tool for evaluating the performance of
Linux-basedsystems. Theresultsare illustratedinFigure2.Runningthe CPUbenchmarkinacontainer
adds no overhead compared to running it on bare metal. Equivalently, there is no performance
difference betweenthe scenariosBM-VMandBM-VM-CT. The performance overheadforusingaVM
is 16% (1 thread) up to 30% (16 threads).
Fig. 2. Total execution time of the Sysbench CPU test in function of the number of threads for the different
scenarios. The two bottom lines represent BM & BM-CT, the two top lines BM-VM & BM-VM-CT.
Sysbenchwasalsousedfor measuringthe diskI/Othroughput,andFigure 3 illustratesthe measured
throughput (in bytes per second) in function of the number of threads for all evaluated scenarios.
Again, there is a clear overlap between BM-VMand BM-BM-CT, and also between BMand BM-CT,
indicating that containerizationhas no significant impact on the performance of disk I/O. Using a
virtual machine howeverhasa significantimpactondiskI/Operformance,asforthisexperimentthe
or a different storage technology might result in a lower overhead.
Fig. 3. Throughput of the Sysbench disk I/O test in function of the number of threads for all scenarios.
The two bottom lines represent BM-VM & BM-VM-CT, the two top lines BM & BM-CT.
For measuring the RAM bandwidth, the open source MWB tool was used, which measures the
memorybandwidth availabletouserspace programs.Inthe test,anarrayof a configurable amountof
bytes(1- 64 MiB) iscopiedfromone variabletoanotherusingmemcpy. Figure4illustrates theresults,
beingthe bandwidth(MiB/s) infunctionof the array size.Again,there isnearperfectoverlapforthe
scenarios BMand BM-CT. The same is valid for BM-VMand BM-VM-CT when the array size is larger
than or equal to 8 MiB. The drop in memory (RAM) throughput is roughly 20% when using system-
Fig. 4. Memory bandwidth (using memcpy) in function of the array size for each scenario.
The two bottom lines represent BM-VM & BM-VM-CT, the two top lines BM & BM-CT.
In general, we can conclude that using containers introduces a negligible overhead. Using virtual
machines however introduces a significant overhead, especially regarding the disk I/O. As a result,
containers should be considered instead of virtual machines for deploying applications requiring a
performance,virtual machinesshouldstill be consideredasusingcontainersonbare metal resultsin
a larger attack surface.
A Platform for Intelligent Application Broking and Provisioning
To facilitate the deployment of applications on top of one or more cloud platforms, we developed a
platformforthe intelligentbrokingandprovisioningontopof oneormore virtualizationtechnologies.
Figure 5illustratesthe architectureof ourplatform.The platformconsistsof 3mainlayers,theBroking
Layer used for selecting the appropriate virtualizationtechnology,the Provisioning Layer used for
provisioningthe requiredresourceswithinthe selectedvirtualizationtechnology,andthe Framework
Abstraction Layerwhichtranslatesthe genericrequestsfromthe provisioninglayertowardsthe APIs
of the differentunderlying frameworks(e.g. OpenStackor Docker). Each layeronlyinteractswith its
adjacent layers resulting in a 3-tier architecture.
Fig. 5. General 3-tier architecture of the developed platform.
The twomostimportantnon-functional requirements of theplatformare modifiabilityandscalability.
The use of intermediariesandencapsulationensures thatmultiple external softwaretools,algorithms
and frameworks can be easily integrated withinthe platform, without having to modify the existing
code base.Additionally, the platformoperatesinacloud environment,andthereforeascalable core
architecture isessential. Asaresult,thefront-endisimplementedinanon-blockingway,andmultiple
requests can be handled simultaneously. The platform was built to scale out to a container cluster
with numerous worker nodes to process the incomingrequests.The platformprovidesa web-based
frontend, which interacts with a Django-powered backend. The asynchronous execution of
backgroundprocesses(e.g.the applicationbrokingorprovisioningprocess) ishandledthroughDjango
Channels,whichoffersthe managementof multipleworkernodesloadbalancedbyanASGIprotocol
The Broking Layer selects the appropriate technology for hosting the application or one of its
componentusingan applicationbrokingalgorithm.Asan example,analgorithmbasedona decision
tree model was implemented within the platform.Every application or component that needsto be
deployed is assigned a value between 0 and 1 for 5 different features, being security, CPU, ram,
scalabilityandavailability.These valuesare assignedbytheuser,andthealgorithmselectstheoptimal
technology based on the values for these metrics, by mapping the features on a 2D latent variable
space usingPrincipal ComponentAnalysis.Thisalgorithmonlyservesasanexample,andcanbe easily
replaced within the platform by any other broking algorithm. A second algorithm was also added,
which selects one of the available provisioning algorithms at random, and can be used a baseline
The Provisioning Layer invokes a resource provisioning algorithm to select the best suitedhost, and
the goal is to minimize the requirednumber of hosts to reduce energyconsumptionandoperational
costs.Whenan estimationof the requiredCPUandRAMusage isavailable forthe application(orone
of its components),binpackingalgorithmscanbe usedas provisioningalgorithms.Asan example,a
heuristic First-Fit Decreasing (FFD) algorithm was implemented within the platform as provisioning
algorithm, but again this algorithm can easily be replaced by any other resource provisioning
algorithm. A secondalgorithmwasalsoadded,whichselectsarandomhostand/orVM(if applicable),
mostly for comparison purposes.
Most resource allocation algorithms however require some input from the hosts. Therefore, an
additional component called Notifier was implemented, which is deployed onto each host managed
bythe platform,andthiscomponentgathersthe actual local resource usageinformationwhichissent
back to the platform. The returned results from the different hosts over time can also be visualized
within the platform through the web-based frontend.
The Framework Abstraction Layer communicates with the selected virtualization technology
framework. For example, if Docker is selected by the broking platform, the framework abstraction
layerwill be usedto deploythe applicationinside aDockercontaineronthe Dockerhost selectedby
the provisioning layer. On the other hand, if a VM is more appropriate, the framework abstraction
layer will communicate with a host running VirtualBox or with an OpenStack cluster, to provision a
newvirtual machine,anddeploytheapplicationwithinthismachine.Currently,we haveimplemented
integration with both VirtualBox and Docker as an example, and are working on the translation
towardsan OpenStackcluster. Itshould howeverbe clearthatsupportforadditional frameworkscan
easily be integrated within the platform.
To verify the correctness of the platform, 3 example applications were provisioned. The first
application is a simple Heartbeat application, which sends a log message to the platform on a
configurable time interval, and is usedto verifythat the provisioning of applications works correctly
within the platform. The two other applications are a CPU- and RAM-intensive application
respectively.The CPU-intensiveapplicationexecutesamachinelearningalgorithm,whereasthe RAM-
platformaftersome of the tasksare finished.Three scenarios were testedusingthe availablebroking
applications,anddeployingbothalternately.Fromthe obtainedresults,we verifiedthatthe platform
isworkingcorrectly,andthe example algorithmsalreadyillustratedthebenefitsof usinganintelligent
algorithmoverthe baseline random algorithms in terms of performance and server consolidation.
Conclusions and Future Work
When migrating applications or application components to the cloud, the question arises which
virtualizationtechnologyisbestsuitedforprovisioningthe requiredresources.The applicationcanbe
directly executed on bare metal, it can be executed within a virtual machine (system-level
virtualization) or it can be executed within a container (OS-level virtualization). Both methods have
advantagesanddrawbacks,butsystem-level virtualizationwill typicallyofferahigherlevelof security
and performance isolation,whereas OS-level virtualizationwill keep the performance overhead to a
minimum. The experiments described in this paper confirm this, as there is a near-perfect overlap
between the results obtained from executing the tests on bare metal (bm) and executing the tests
inside a container (bm-ct) and also between the results obtained from executing the tests inside a
virtual machine (bm-vm) andexecutingthe testsinside acontainerinside avirtual machine (bm-vm-
ct). Asa result,containersshouldbe consideredinsteadof VMswhenhighperformance is required.
To facilitate the deploymentonaphysical cloudtestbed,we introducedanintelligentplatformforthe
brokingand provisioningof applicationsandapplicationcomponentsontop of various virtualization
technologies. The platform consists of 3 core layers; the Broking Layer selects the best suited
technologybasedonthe characteristicsof the applicationthatneedstobe deployed,the Provisioning
Layerselectsasuitable workernodeforhostingthe application,andthe FrameworkAbstractionLayer
translatesthe genericrequeststowardsthe specificframework.We implementedanexample broking
algorithm and provisioning algorithm within the broking and provisioning layers respectively, and
deployed several test applications using the platform to verify the correctness of the platform.
In future work, we will be working on the implementation of the OpenStack adapter within the
Framework Abstraction Layer, and look at the possibilities for adding a feedback loop betweenthe
first two layers. In some cases, it might be beneficial to select a different type of virtualization
technology to consolidate servers, e.g. if there are still VM’s available that can host the application
although a container would be better suited. The feedback loopcan be used to implement thistype
of behavior within the platform.