Complexity of cloud infrastructures needs models and tools for process management, configuration, scaling, elastic computing and healthiness control. This paper presents a Smart Cloud solution based on a Knowledge Base, KB, with the aim of modeling cloud resources, Service Level Agreements and their evolution, and enabling the reasoning on structures by implementing strategies of efficient smart cloud management and intelligence. The solution proposed provides formal verification tools and intelligence for cloud control. It can be easily integrated with any cloud configuration manager, cloud orchestrator, and monitoring tool, since the connections with these tools are performed by using REST calls and XML files. It has been validated in the large ICARO Cloud project with a national cloud service provider.
Smart Cloud Engine and Solution based on Knowledge Base
1. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
Smart Cloud Engine and Solution
based on Knowledge Base
Pierfrancesco Bellini, Daniele Cenni, Paolo Nesi
DISIT Lab, Distributed Systems and Internet Technologies Lab
Dipartimento di Ingegneria dell’Informazione, DINFO
Università degli Studi di Firenze
Via S. Marta 3, 50139, Firenze, Italy
Tel: +39-055-2758515, fax: +39-055-2758570
http://www.disit.dinfo.unifi.it alias http://www.disit.org
Paolo.Nesi@unifi.it
Presented at: Cloud Forward 2015, October 6‐8 Pisa, Scuola Normale Superiore
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 1
2. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
Context and Motivations
• Relevant ICT infrastructures are presently deployed
on cloud to manage resources in an efficient
manner
– Cloud Customer vs Cloud Service Provider, CSP
– Service Level Agreements, SLA
• Smart Cloud Solutions are becoming mandatory to
increase resilience, reliability in cloud automation:
– configuration/reconfiguration
– dynamic scaling, elastic computing, migration, cloning, ..
– cloud resources healthiness control
– SLA management
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 2
3. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
Automation of cloud management
• formal verification and validation of cloud
configuration in terms of resources, their
relationships
• verification and reasoning about cloud security
• interoperability among public and private clouds
• discovering and brokering services and resources
• reasoning about cloud workload conditions, may be
via simulation
• computing capability for horizontal and/or vertical
scaling, thus elastic computing
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 3
4. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
Smart Cloud Engine, SCE
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 4
Supervisor &
Monitor, SM
CCM, Cloud
Configuration
Manager
Smart Cloud
Administration
Orchestrator
PaaS
IaaS
SaaS
Smart Cloud Engine, SCE
Knowledge Base on RDF Store
Knowledge base
Services
Linked Open
Graph Service
Distributed SCE
Scheduler
Cloud Services and
Administrators
Cloud
Services
Final Users
SCE Management Interface
. and Tools
Graphs Editors
5. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
SCE Main Features
• Receiving business configurations, BC, and their SLA
• Receiving requests for verification and validation of specific
BCs, providing back:
– Ack about consistency and completeness of a BC, and config.
– suggestions, hints related to feasibility,
• Automatically Assessing sporadic and/or periodic tasks for
– monitoring of the SLA associated with each single contract
element
– control of the health/security of any: BC, resource, service on
the cloud at any level
– controlling firing conditions activating reconfiguration strategies
associated with BCs: scaling, cloning, migration, alarms, etc.
• Allowing the integration of SCE with a range of Cloud
Management solutions: CCM and Orchestrators
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 5
6. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.itSequence diagram
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 6
CCM KB SM SCE
deployRequest
putBusinessConf
putBusinessConf
querySLA
storeMetrics
storeMetrics
getMetrics
updateSLA
nagiosSetup
SLA Check
Process
evalSLA
storeAlarm
getMetrics
evalSLA
storeAlarm
storeMetrics
reconfigure
putBusinessConf
putBusinessConf
nagiosSetup
7. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
Smart Cloud Engine
• SCE Management
interface
– Monitoring status/Logs of
activated processes
– Manual Action Rules
– Decision support, alarms
• DISCES: a distributed
scheduler for executing
agents for automated
computing
– Monitoring processes
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 7
Smart Cloud Engine, SCE
Distributed SCE
Scheduler
SCE Management Interface
. and Tools
Graphs Editors
8. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.itICARO Cloud Ontology of KB
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 9
Metrics Service MetricsHardware elements
SW ServicesApplications
Operating Systems
Monitoring
Virtual Things
Users
SLA
9. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
KB Internal Architecture
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 10
Apache Tomcat
ICARO Cloud KB API REST
DataCenter Metrics BCs & SLA Applications
RDF Store
(OWLIM‐SE 4.3)
SPARQL 1.1 Protocol
SPARQL 1.1 Graph Store Protocol
Verification Support
SPARQL
Proxy
Supervisor &
Monitor, SM
Linked
Open Graph
http://log.disit./org
10. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
Linked Open Graph,
LOD tool http://log.disit.org
For knowledge base browsing
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 11
11. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
Metric graphs per: BC/SLA, VM, Service,..
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 12
• ECLAP social
network:
– http://www.eclap.eu
– 1 balancer
– 2 web front ends
– 1 back office for DB
– Grid Scheduler and
nodes for content
ingestion and
processing (N VM, 2
in this case)
12. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
Experimental Results
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 13
Service Metric Monitored as the
average value of 30 minutes on
values assessed every 5 minutes
% of overvalues with
respect to reference
values expected as
defined in the SLA
Incidence of overvalues with
respect to the total number of
measures (time slots)
performed for the business
configuration
Memory Usage 73.76% 11.86%
Disk Usage 54.03% 8.57%
Network workload 24.29% 3.85%
MySQL DB Size 59.55% 2.12%
CPU Usage 0.07% 0.01%
Apache HTTP response time 0.1% 0.008%
MySQL Connections resp. time 0.1% 0.004%
Tomcat HTTP response time 0.08% 0.003%
• 26.44% of the assessed time slots were affected by some overvalue, that may be
associated to alarms for the detection of critical conditions
• collecting 3 months of data about service metrics, with about 3800 evaluations
per metric per day
13. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 14
• Strategy Condition Editor for Elastic Cloud programming
Boolean rules with arbitrary complexity to activate procedures for scaling,
migration, cloning, thus actuating solutions of control, balancing, self
regulation, etc.
Conditions are evaluated on the basis of HLM and SLA for Services, VM, and
Business Configurations with respect to absolute thresholds or percentages
14. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
ICARO Cloud project
• Partners: ComputerGross CSP and LiberoLogico
• Objectives: Reduce costs for managing on cloud
complex configurations Business Configurations
with Service Level Agreements, SLA
– Provides simple solutions for moving SW applications on
Cloud
– Automating cloud activities as: configurations,
management, monitoring, control, reconfigurations
– Managing high level metrics, HLM, at applicative level
and managing BC with HLM on SLA
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 15
15. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
ICARO Cloud Architecture
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 16
CMW SDK
Smart Cloud
Supervisor & Monitor
Subscription
Portal
Configuration
Manager
Access to BPaaS,
Services Purchase
Cloud
Management
Application Access on
iCaro cloud Applications
Business
Producer
Knowledge Base
App/Srv
Store
Cloud Simulator
PaaS
Cloud MiddleWare Services
IaaS
SaaS New
New
New
Developers
PaaS
SME App Producers
Smart Cloud Engine
16. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 17
17. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
ICARO architecture with ICLOS
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 18
Cloud Resources
Simulator Engine
Resource Group Controllers
ICLOS Resource Allocator
Smart Cloud EngineCloud
Configuration
Manager
Smart Cloud Administration
Knowledge
Base
on RDF Store
and LOD
PaaS
IaaS
SaaS
Knowledge
base Services
Linked Open
Graph Service
Scheduler Manag. Interf.
and Tools
Cloud Administration
OnLine
Cloud
Services
For Final
Users
Real Cloud
Data
Traces
Nagios
Model Cloud
Data Traces
Orchestrator
Supervisor & Monitoring
Cloud
Simulator
Simulation
Config. GUI
Simulator GUI
Simulated
Cloud Traces
Cloud Under Management
18. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
Conclusion: The proposed SCE solution
• is based on innovative Knowledge Base: ICARO Cloud
Ontology and reasoning tools: BC, SLA, HLM, etc.
• enables a flexible management of cloud resources:
verification, control, management: scaling, cloning, migrating,
etc. dynamic cloud control
• is scalable and provides smart reasoning for different contexts
• can be easily exploited in connection with any cloud
management tools such as configurators, orchestrators, and
monitoring tools
• has been validated in ICARO Cloud project with CSPs on
complex cloud configurations, and on DISIT cloud multitier
solutions with good performance in terms of low operating
workload and scalability.
• Supports scalability at level of distributed scheduler for the
Smart Cloud Engine, on KB services and on Monitoring.
DISIT Lab (DINFO UNIFI), Cloud Forward, 2015 19