Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Supporting Autonomic Management of
Clouds: Service-Level-Agreement, Cloud
Monitoring and Similarity Learning
by
Rafael Bru...
Contents
1 Introduction
2 SLA for Clouds
3 Cloud Monitoring
4 Similarity Learning
5 Polus Framework
6 Conclusions
Rafael B...
Introduction
Introduction Rafael Brundo Uriarte 2/51
Cloud Computing
Introduction Rafael Brundo Uriarte 3/51
Cloud Characteristics
Services
Heterogeneity
Virtualization
Large-Scale
Complexity
Introduction Rafael Brundo Uriarte 4/51
Autonomic Computing
Introduction Rafael Brundo Uriarte 5/51
Autonomic Computing
Introduction Rafael Brundo Uriarte 6/51
Knowledge for the Self-Management
Policies
Service Definition and Objectives
Status of the Cloud and Services
Specific Knowl...
Challenge and Scope
Introduction Rafael Brundo Uriarte 8/51
Research Questions
Research Question 1
How to describe services and their objectives in the cloud
domain?
Research Questio...
Research Questions
Research Question 4
How to produce a robust measure of similarity for services in
the domain and how ca...
SLA for Clouds
SLA for Clouds Rafael Brundo Uriarte 11/51
Service-Level-Agreement (SLA)
Contract
Service Description
Quality-of-Service
Formalism
Guarantees!
SLA for Clouds Rafael ...
SLA for Cloud Computing - SLAC
Domain Specific
Multi-Party
Deployment Models
Formalism
Ease-of-Use
SLA for Clouds Rafael Br...
Yet Another SLA Definition Language?
Features WSOL WSLA SLAng WSA SLA* SLAC
General
Deployment Models
Broker Support - - - ...
Main Concepts
Predefined Metrics - Involved Parties and Unit
Intervals for Metrics - Template and Variations
Groups - Multi...
Example
SLA for Clouds Rafael Brundo Uriarte 16/51
Business Aspects
Business Actions
Flat and Variable Models
Pricing Schemes - Exchange, Auction, Tender,
Bilateral, Fixed, ...
Implementation
Editor for SLAs (Ecplise-based using Xtext)
SLA Evaluator (Z3 Solver)
Integration with the Monitoring Syste...
Cloud Monitoring
Cloud Monitoring Rafael Brundo Uriarte 19/51
DIKW in the Domain
Data
Information
Knowledge
Wisdom
Cloud Monitoring Rafael Brundo Uriarte 20/51
Cloud Monitoring
The Role of the Monitoring System in Clouds:
Collect data and Provide Information and
Knowledge
No Wisdom...
Related Works
Property PCMONS Monalytics Lattice Wang
Cloud - - -
Autonomic Integration - - - -
Scalability -
Adaptability...
Panoptes
Multi-agent system
Monitoring in different levels
Monitoring Modules - What needs to be
monitored and how to proce...
Architecture
Cloud Monitoring Rafael Brundo Uriarte 24/51
Architecture
Communication:
Publish/Subscribe
Private Message
Adaptativeness:
Priority for Modules
Change of Roles
Cloud M...
Architecture: Autonomic Integration
Urgency Mechanism
Decentralised Architecture
On-the-Fly Configuration
Multiple Abstract...
Experiments
Self-Protection System
Urgency Mechanism
Scalability
Cloud Monitoring Rafael Brundo Uriarte 27/51
Similarity Learning
Similarity Learning Rafael Brundo Uriarte 28/51
Specific Knowledge
Generation of Knowledge for a Specific Purpose, i.e.
not applicable in all clouds. For example, similarit...
Applications of Similarity
Cluster Services:
Group Similar Services
Different Algorithms
(K-Means, PAM, EM)
Applications in...
Domain Requirements
Categorical Characteristics of Services
On-line Prediction
Large Number of Characteristics
Fast Predic...
Random Forest
Clustering with Random Forest
Originally Developed for Classification
Calculate the Similarity
Clustering Alg...
Similarity Using RF: Criteria
Similarity Learning Rafael Brundo Uriarte 33/51
Problems
Similarity Matrix (Big Memory Footprint)
Re-cluster on Every New Observation
Cannot be Used in the Domain
Similar...
Solution: RF+PAM
Similarity Learning Rafael Brundo Uriarte 35/51
Solution: RF+PAM
Similarity Learning Rafael Brundo Uriarte 36/51
Experiments
Compared the performance of our algorithm to
other 2 methodologies
Compared the performance of RF+PAM with
the...
Polus Framework
Polus Framework Rafael Brundo Uriarte 38/51
Polus Framework
Polus Framework Rafael Brundo Uriarte 39/51
Use Case
Polus Framework Rafael Brundo Uriarte 40/51
Use Case
Polus Framework Rafael Brundo Uriarte 41/51
Use Case
Polus Framework Rafael Brundo Uriarte 42/51
Conclusions
Conclusions Rafael Brundo Uriarte 43/51
Summary
Conclusions Rafael Brundo Uriarte 44/51
Research Questions
Research Question 1
How to describe services and their objectives in the cloud
domain?
SLAC
Research Qu...
Research Questions
Research Question 4
How to produce a robust measure of similarity for services in
the domain and how ca...
Limitations
Intelligence of Autonomic Managers
Wide Range of Specific Knowledge
Off-line Training of RF+PAM
Conclusions Rafa...
Contributions
A theoretical and practical framework for the
generation and provision of knowledge for the
autonomic manage...
Publications
1. R. B. Uriarte, S. Tsaftaris and F. Tiezzi. Service Clustering for
Autonomic Clouds Using Random Forest. In...
Future Works
Dynamic SLAs
Negotiation of SLAs
Cloud Case Study
Conclusions Rafael Brundo Uriarte 50/51
Thank you!
Questions?
Rafael Brundo Uriarte
rafael.uriarte@gmail.com
Conclusions Rafael Brundo Uriarte 51/51
SLAC - Expressivity
Core Language
Extensions - Business Aspects
Formal Definition for Extensions
Conclusions Rafael Brundo ...
SLAC - Implementation
Compatibility only with OpenNebula
Toy Implementation
Easily adapted
Conclusions Rafael Brundo Uriar...
SLAC - Cloud Metrics
DTMF Cloud Computing Service Metrics
Description
Recent Document (Still a Draft)
Creation of a Model ...
SLAC Violation
Violation and Penalty are Separated Concepts
“Violation” Concept Flexible
Easy to Understand
Conclusions Ra...
Panoptes - Scalability
Designed to be scalable
Adapt itself
Experiments suggest it is scalable
More experiments for future...
Panoptes - Analysis of Apache Broklyn
Not Focused on Monitoring
Does Not Process the Data
Conclusions Rafael Brundo Uriart...
Panoptes - Analysis with CSPARQL
Data is not Decorated (e.g. RDF)
Impact of Decorated Monitoring Data
(Scalability)
Very I...
Upcoming SlideShare
Loading in …5
×

Supporting Autonomic Management of Clouds: Service-Level-Agreement, Cloud Monitoring and Similarity Learning

Cloud computing has grown rapidly during the past few years and has become a fundamental paradigm in the Information Technology (IT) area. Clouds enable dynamic, scalable and rapid provision of services through a computer network, usually the Internet. However, managing and optimising clouds and their services in the presence of dynamism and heterogeneity is one of the major challenges faced by industry and academia. A prominent solution is resorting to selfmanagement as fostered by autonomic computing. Self-management requires knowledge about the system and the environment to enact the self-* properties. Nevertheless, the characteristics of cloud, such as large-scale and dynamism, hinder the knowledge discovery process. Moreover, cloud systems abstract the complexity of the infrastructure underlying the provided services to their customers, which obfuscates several details of the provided services and, thus, obstructs the effectiveness of autonomic managers. While a large body of work has been devoted to decisionmaking and autonomic management in the cloud domain, there is still a lack of adequate solutions for the provision of knowledge to these processes. In view of the lack of comprehensive solutions for the provision of knowledge to the autonomic management of clouds, we propose a theoretical and practical framework which addresses three major aspects of this process: (i) the definition of services’ provision through the specification of a formal language to define Service-Level-Agreements for the cloud domain; (ii) the collection and processing of information through an extensible knowledge discovery architecture to monitor autonomic clouds with support to the knowledge discovery process; and (iii) the knowledge discovery through a machine learning methodology to calculate the similarity among services, which can be employed for different purposes, e.g. service scheduling and anomalous behaviour detection. Finally, in a case study, we integrate the proposed solutions and show the benefits of this integration in a hybrid cloud test-bed.

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

Supporting Autonomic Management of Clouds: Service-Level-Agreement, Cloud Monitoring and Similarity Learning

  1. 1. Supporting Autonomic Management of Clouds: Service-Level-Agreement, Cloud Monitoring and Similarity Learning by Rafael Brundo Uriarte rafael.uriarte@imtlucca.it Under the Supervision of: Prof. Rocco De Nicola and Prof. Francesco Tiezzi Doctoral Thesis Defense - March 30th, 2015 - Lucca, Italy
  2. 2. Contents 1 Introduction 2 SLA for Clouds 3 Cloud Monitoring 4 Similarity Learning 5 Polus Framework 6 Conclusions Rafael Brundo Uriarte 1/51
  3. 3. Introduction Introduction Rafael Brundo Uriarte 2/51
  4. 4. Cloud Computing Introduction Rafael Brundo Uriarte 3/51
  5. 5. Cloud Characteristics Services Heterogeneity Virtualization Large-Scale Complexity Introduction Rafael Brundo Uriarte 4/51
  6. 6. Autonomic Computing Introduction Rafael Brundo Uriarte 5/51
  7. 7. Autonomic Computing Introduction Rafael Brundo Uriarte 6/51
  8. 8. Knowledge for the Self-Management Policies Service Definition and Objectives Status of the Cloud and Services Specific Knowledge Introduction Rafael Brundo Uriarte 7/51
  9. 9. Challenge and Scope Introduction Rafael Brundo Uriarte 8/51
  10. 10. Research Questions Research Question 1 How to describe services and their objectives in the cloud domain? Research Question 2 What is data, information, knowledge and wisdom in the autonomic cloud domain? Research Question 3 How to collect and transform operational data into useful knowledge without overloading the autonomic cloud? Introduction Rafael Brundo Uriarte 9/51
  11. 11. Research Questions Research Question 4 How to produce a robust measure of similarity for services in the domain and how can this knowledge be used? Research Question 5 How to integrate different sources of knowledge and feed the autonomic managers? Introduction Rafael Brundo Uriarte 10/51
  12. 12. SLA for Clouds SLA for Clouds Rafael Brundo Uriarte 11/51
  13. 13. Service-Level-Agreement (SLA) Contract Service Description Quality-of-Service Formalism Guarantees! SLA for Clouds Rafael Brundo Uriarte 12/51
  14. 14. SLA for Cloud Computing - SLAC Domain Specific Multi-Party Deployment Models Formalism Ease-of-Use SLA for Clouds Rafael Brundo Uriarte 13/51
  15. 15. Yet Another SLA Definition Language? Features WSOL WSLA SLAng WSA SLA* SLAC General Deployment Models Broker Support - - - - - Business Pricing Schemes - Formal Semantics - - - - Verification - - - - “ ” feature covered “ ” feature partially covered “-” no support SLA for Clouds Rafael Brundo Uriarte 14/51
  16. 16. Main Concepts Predefined Metrics - Involved Parties and Unit Intervals for Metrics - Template and Variations Groups - Multiple Service, Community Cloud Constraint Solving Problem SLA for Clouds Rafael Brundo Uriarte 15/51
  17. 17. Example SLA for Clouds Rafael Brundo Uriarte 16/51
  18. 18. Business Aspects Business Actions Flat and Variable Models Pricing Schemes - Exchange, Auction, Tender, Bilateral, Fixed, Posted SLA for Clouds Rafael Brundo Uriarte 17/51
  19. 19. Implementation Editor for SLAs (Ecplise-based using Xtext) SLA Evaluator (Z3 Solver) Integration with the Monitoring System SLA for Clouds Rafael Brundo Uriarte 18/51
  20. 20. Cloud Monitoring Cloud Monitoring Rafael Brundo Uriarte 19/51
  21. 21. DIKW in the Domain Data Information Knowledge Wisdom Cloud Monitoring Rafael Brundo Uriarte 20/51
  22. 22. Cloud Monitoring The Role of the Monitoring System in Clouds: Collect data and Provide Information and Knowledge No Wisdom - Related to Decision-Making Sensor of MAPE-K Loop Cloud Monitoring Rafael Brundo Uriarte 21/51
  23. 23. Related Works Property PCMONS Monalytics Lattice Wang Cloud - - - Autonomic Integration - - - - Scalability - Adaptability - - Resilience - - - - Timeliness - - Extensibility “ ” feature covered “ ” feature partially covered “-” no support Cloud Monitoring Rafael Brundo Uriarte 22/51
  24. 24. Panoptes Multi-agent system Monitoring in different levels Monitoring Modules - What needs to be monitored and how to process the data Cloud Monitoring Rafael Brundo Uriarte 23/51
  25. 25. Architecture Cloud Monitoring Rafael Brundo Uriarte 24/51
  26. 26. Architecture Communication: Publish/Subscribe Private Message Adaptativeness: Priority for Modules Change of Roles Cloud Monitoring Rafael Brundo Uriarte 25/51
  27. 27. Architecture: Autonomic Integration Urgency Mechanism Decentralised Architecture On-the-Fly Configuration Multiple Abstractions Cloud Monitoring Rafael Brundo Uriarte 26/51
  28. 28. Experiments Self-Protection System Urgency Mechanism Scalability Cloud Monitoring Rafael Brundo Uriarte 27/51
  29. 29. Similarity Learning Similarity Learning Rafael Brundo Uriarte 28/51
  30. 30. Specific Knowledge Generation of Knowledge for a Specific Purpose, i.e. not applicable in all clouds. For example, similarity. But what is similarity? How much an object (service) resembles other Similarity Learning Rafael Brundo Uriarte 29/51
  31. 31. Applications of Similarity Cluster Services: Group Similar Services Different Algorithms (K-Means, PAM, EM) Applications in the Domain: Anomalous Behaviour Detections Service Scheduling Application Profiling SLA Risk Assessment Similarity Learning Rafael Brundo Uriarte 30/51
  32. 32. Domain Requirements Categorical Characteristics of Services On-line Prediction Large Number of Characteristics Fast Prediction Similarity Learning Rafael Brundo Uriarte 31/51
  33. 33. Random Forest Clustering with Random Forest Originally Developed for Classification Calculate the Similarity Clustering Algorithm (PAM) Similarity Learning Rafael Brundo Uriarte 32/51
  34. 34. Similarity Using RF: Criteria Similarity Learning Rafael Brundo Uriarte 33/51
  35. 35. Problems Similarity Matrix (Big Memory Footprint) Re-cluster on Every New Observation Cannot be Used in the Domain Similarity Learning Rafael Brundo Uriarte 34/51
  36. 36. Solution: RF+PAM Similarity Learning Rafael Brundo Uriarte 35/51
  37. 37. Solution: RF+PAM Similarity Learning Rafael Brundo Uriarte 36/51
  38. 38. Experiments Compared the performance of our algorithm to other 2 methodologies Compared the performance of RF+PAM with the standard off-line similarity learning Use Case: Scheduler deploys together the most dissimilar services Similarity based on their SLAs Similarity Learning Rafael Brundo Uriarte 37/51
  39. 39. Polus Framework Polus Framework Rafael Brundo Uriarte 38/51
  40. 40. Polus Framework Polus Framework Rafael Brundo Uriarte 39/51
  41. 41. Use Case Polus Framework Rafael Brundo Uriarte 40/51
  42. 42. Use Case Polus Framework Rafael Brundo Uriarte 41/51
  43. 43. Use Case Polus Framework Rafael Brundo Uriarte 42/51
  44. 44. Conclusions Conclusions Rafael Brundo Uriarte 43/51
  45. 45. Summary Conclusions Rafael Brundo Uriarte 44/51
  46. 46. Research Questions Research Question 1 How to describe services and their objectives in the cloud domain? SLAC Research Question 2 What is data, information, knowledge and wisdom in the autonomic cloud domain? DIKW Hierarchy Research Question 3 How to collect and transform operational data into useful knowledge without overloading the autonomic cloud? Panoptes Conclusions Rafael Brundo Uriarte 45/51
  47. 47. Research Questions Research Question 4 How to produce a robust measure of similarity for services in the domain and how can this knowledge be used? RF+PAM Research Question 5 How to integrate different sources of knowledge and feed the autonomic managers? Polus Framework Conclusions Rafael Brundo Uriarte 46/51
  48. 48. Limitations Intelligence of Autonomic Managers Wide Range of Specific Knowledge Off-line Training of RF+PAM Conclusions Rafael Brundo Uriarte 47/51
  49. 49. Contributions A theoretical and practical framework for the generation and provision of knowledge for the autonomic management of clouds (Polus Framework): SLAC - SLA Definition and Evaluation Panoptes - Monitoring RF+PAM - Similarity Learning Conclusions Rafael Brundo Uriarte 48/51
  50. 50. Publications 1. R. B. Uriarte, S. Tsaftaris and F. Tiezzi. Service Clustering for Autonomic Clouds Using Random Forest. In Proc. of the 15th IEEE/ACM CCGrid [In Press], 2015. 2. R.B. Uriarte, F. Tiezzi, R. De Nicola, SLAC: A Formal Service-Level-Agreement Language for Cloud Computing. In IEEE/ACM 7th International Conference on Utility and Cloud Computing (UCC), 2014. 3. R.B. Uriarte, C.B. Westphall, Panoptes: A monitoring architecture and framework for supporting autonomic Clouds, In Proc. of the 16th IEEE/IFIP Network Operations and Management Symposium (NOMS), 2014. 4. R.B. Uriarte, S.A. Chaves, C.B. Westphall, Towards an Architecture for Monitoring Private Clouds. In IEEE Communications Magazine, 49, pages 130-137, 2011. Conclusions Rafael Brundo Uriarte 49/51
  51. 51. Future Works Dynamic SLAs Negotiation of SLAs Cloud Case Study Conclusions Rafael Brundo Uriarte 50/51
  52. 52. Thank you! Questions? Rafael Brundo Uriarte rafael.uriarte@gmail.com Conclusions Rafael Brundo Uriarte 51/51
  53. 53. SLAC - Expressivity Core Language Extensions - Business Aspects Formal Definition for Extensions Conclusions Rafael Brundo Uriarte 51/51
  54. 54. SLAC - Implementation Compatibility only with OpenNebula Toy Implementation Easily adapted Conclusions Rafael Brundo Uriarte 51/51
  55. 55. SLAC - Cloud Metrics DTMF Cloud Computing Service Metrics Description Recent Document (Still a Draft) Creation of a Model for the Definition of Metrics The SLAC Metrics can be Adapted for this Model Conclusions Rafael Brundo Uriarte 51/51
  56. 56. SLAC Violation Violation and Penalty are Separated Concepts “Violation” Concept Flexible Easy to Understand Conclusions Rafael Brundo Uriarte 51/51
  57. 57. Panoptes - Scalability Designed to be scalable Adapt itself Experiments suggest it is scalable More experiments for future works Conclusions Rafael Brundo Uriarte 51/51
  58. 58. Panoptes - Analysis of Apache Broklyn Not Focused on Monitoring Does Not Process the Data Conclusions Rafael Brundo Uriarte 51/51
  59. 59. Panoptes - Analysis with CSPARQL Data is not Decorated (e.g. RDF) Impact of Decorated Monitoring Data (Scalability) Very Interesting Option Conclusions Rafael Brundo Uriarte 51/51

×