Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Supporting Autonomic Management of Clouds: Service-Level-Agreement, Cloud Monitoring and Similarity Learning

545 views

Published on

Cloud computing has grown rapidly during the past few years and has become a fundamental paradigm in the Information Technology (IT) area. Clouds enable dynamic, scalable and rapid provision of services through a computer network, usually the Internet. However, managing and optimising clouds and their services in the presence of dynamism and heterogeneity is one of the major challenges faced by industry and academia. A prominent solution is resorting to selfmanagement as fostered by autonomic computing. Self-management requires knowledge about the system and the environment to enact the self-* properties. Nevertheless, the characteristics of cloud, such as large-scale and dynamism, hinder the knowledge discovery process. Moreover, cloud systems abstract the complexity of the infrastructure underlying the provided services to their customers, which obfuscates several details of the provided services and, thus, obstructs the effectiveness of autonomic managers. While a large body of work has been devoted to decisionmaking and autonomic management in the cloud domain, there is still a lack of adequate solutions for the provision of knowledge to these processes. In view of the lack of comprehensive solutions for the provision of knowledge to the autonomic management of clouds, we propose a theoretical and practical framework which addresses three major aspects of this process: (i) the definition of services’ provision through the specification of a formal language to define Service-Level-Agreements for the cloud domain; (ii) the collection and processing of information through an extensible knowledge discovery architecture to monitor autonomic clouds with support to the knowledge discovery process; and (iii) the knowledge discovery through a machine learning methodology to calculate the similarity among services, which can be employed for different purposes, e.g. service scheduling and anomalous behaviour detection. Finally, in a case study, we integrate the proposed solutions and show the benefits of this integration in a hybrid cloud test-bed.

Published in: Science
  • Be the first to comment

Supporting Autonomic Management of Clouds: Service-Level-Agreement, Cloud Monitoring and Similarity Learning

  1. 1. Supporting Autonomic Management of Clouds: Service-Level-Agreement, Cloud Monitoring and Similarity Learning by Rafael Brundo Uriarte rafael.uriarte@imtlucca.it Under the Supervision of: Prof. Rocco De Nicola and Prof. Francesco Tiezzi Doctoral Thesis Defense - March 30th, 2015 - Lucca, Italy
  2. 2. Contents 1 Introduction 2 SLA for Clouds 3 Cloud Monitoring 4 Similarity Learning 5 Polus Framework 6 Conclusions Rafael Brundo Uriarte 1/51
  3. 3. Introduction Introduction Rafael Brundo Uriarte 2/51
  4. 4. Cloud Computing Introduction Rafael Brundo Uriarte 3/51
  5. 5. Cloud Characteristics Services Heterogeneity Virtualization Large-Scale Complexity Introduction Rafael Brundo Uriarte 4/51
  6. 6. Autonomic Computing Introduction Rafael Brundo Uriarte 5/51
  7. 7. Autonomic Computing Introduction Rafael Brundo Uriarte 6/51
  8. 8. Knowledge for the Self-Management Policies Service Definition and Objectives Status of the Cloud and Services Specific Knowledge Introduction Rafael Brundo Uriarte 7/51
  9. 9. Challenge and Scope Introduction Rafael Brundo Uriarte 8/51
  10. 10. Research Questions Research Question 1 How to describe services and their objectives in the cloud domain? Research Question 2 What is data, information, knowledge and wisdom in the autonomic cloud domain? Research Question 3 How to collect and transform operational data into useful knowledge without overloading the autonomic cloud? Introduction Rafael Brundo Uriarte 9/51
  11. 11. Research Questions Research Question 4 How to produce a robust measure of similarity for services in the domain and how can this knowledge be used? Research Question 5 How to integrate different sources of knowledge and feed the autonomic managers? Introduction Rafael Brundo Uriarte 10/51
  12. 12. SLA for Clouds SLA for Clouds Rafael Brundo Uriarte 11/51
  13. 13. Service-Level-Agreement (SLA) Contract Service Description Quality-of-Service Formalism Guarantees! SLA for Clouds Rafael Brundo Uriarte 12/51
  14. 14. SLA for Cloud Computing - SLAC Domain Specific Multi-Party Deployment Models Formalism Ease-of-Use SLA for Clouds Rafael Brundo Uriarte 13/51
  15. 15. Yet Another SLA Definition Language? Features WSOL WSLA SLAng WSA SLA* SLAC General Deployment Models Broker Support - - - - - Business Pricing Schemes - Formal Semantics - - - - Verification - - - - “ ” feature covered “ ” feature partially covered “-” no support SLA for Clouds Rafael Brundo Uriarte 14/51
  16. 16. Main Concepts Predefined Metrics - Involved Parties and Unit Intervals for Metrics - Template and Variations Groups - Multiple Service, Community Cloud Constraint Solving Problem SLA for Clouds Rafael Brundo Uriarte 15/51
  17. 17. Example SLA for Clouds Rafael Brundo Uriarte 16/51
  18. 18. Business Aspects Business Actions Flat and Variable Models Pricing Schemes - Exchange, Auction, Tender, Bilateral, Fixed, Posted SLA for Clouds Rafael Brundo Uriarte 17/51
  19. 19. Implementation Editor for SLAs (Ecplise-based using Xtext) SLA Evaluator (Z3 Solver) Integration with the Monitoring System SLA for Clouds Rafael Brundo Uriarte 18/51
  20. 20. Cloud Monitoring Cloud Monitoring Rafael Brundo Uriarte 19/51
  21. 21. DIKW in the Domain Data Information Knowledge Wisdom Cloud Monitoring Rafael Brundo Uriarte 20/51
  22. 22. Cloud Monitoring The Role of the Monitoring System in Clouds: Collect data and Provide Information and Knowledge No Wisdom - Related to Decision-Making Sensor of MAPE-K Loop Cloud Monitoring Rafael Brundo Uriarte 21/51
  23. 23. Related Works Property PCMONS Monalytics Lattice Wang Cloud - - - Autonomic Integration - - - - Scalability - Adaptability - - Resilience - - - - Timeliness - - Extensibility “ ” feature covered “ ” feature partially covered “-” no support Cloud Monitoring Rafael Brundo Uriarte 22/51
  24. 24. Panoptes Multi-agent system Monitoring in different levels Monitoring Modules - What needs to be monitored and how to process the data Cloud Monitoring Rafael Brundo Uriarte 23/51
  25. 25. Architecture Cloud Monitoring Rafael Brundo Uriarte 24/51
  26. 26. Architecture Communication: Publish/Subscribe Private Message Adaptativeness: Priority for Modules Change of Roles Cloud Monitoring Rafael Brundo Uriarte 25/51
  27. 27. Architecture: Autonomic Integration Urgency Mechanism Decentralised Architecture On-the-Fly Configuration Multiple Abstractions Cloud Monitoring Rafael Brundo Uriarte 26/51
  28. 28. Experiments Self-Protection System Urgency Mechanism Scalability Cloud Monitoring Rafael Brundo Uriarte 27/51
  29. 29. Similarity Learning Similarity Learning Rafael Brundo Uriarte 28/51
  30. 30. Specific Knowledge Generation of Knowledge for a Specific Purpose, i.e. not applicable in all clouds. For example, similarity. But what is similarity? How much an object (service) resembles other Similarity Learning Rafael Brundo Uriarte 29/51
  31. 31. Applications of Similarity Cluster Services: Group Similar Services Different Algorithms (K-Means, PAM, EM) Applications in the Domain: Anomalous Behaviour Detections Service Scheduling Application Profiling SLA Risk Assessment Similarity Learning Rafael Brundo Uriarte 30/51
  32. 32. Domain Requirements Categorical Characteristics of Services On-line Prediction Large Number of Characteristics Fast Prediction Similarity Learning Rafael Brundo Uriarte 31/51
  33. 33. Random Forest Clustering with Random Forest Originally Developed for Classification Calculate the Similarity Clustering Algorithm (PAM) Similarity Learning Rafael Brundo Uriarte 32/51
  34. 34. Similarity Using RF: Criteria Similarity Learning Rafael Brundo Uriarte 33/51
  35. 35. Problems Similarity Matrix (Big Memory Footprint) Re-cluster on Every New Observation Cannot be Used in the Domain Similarity Learning Rafael Brundo Uriarte 34/51
  36. 36. Solution: RF+PAM Similarity Learning Rafael Brundo Uriarte 35/51
  37. 37. Solution: RF+PAM Similarity Learning Rafael Brundo Uriarte 36/51
  38. 38. Experiments Compared the performance of our algorithm to other 2 methodologies Compared the performance of RF+PAM with the standard off-line similarity learning Use Case: Scheduler deploys together the most dissimilar services Similarity based on their SLAs Similarity Learning Rafael Brundo Uriarte 37/51
  39. 39. Polus Framework Polus Framework Rafael Brundo Uriarte 38/51
  40. 40. Polus Framework Polus Framework Rafael Brundo Uriarte 39/51
  41. 41. Use Case Polus Framework Rafael Brundo Uriarte 40/51
  42. 42. Use Case Polus Framework Rafael Brundo Uriarte 41/51
  43. 43. Use Case Polus Framework Rafael Brundo Uriarte 42/51
  44. 44. Conclusions Conclusions Rafael Brundo Uriarte 43/51
  45. 45. Summary Conclusions Rafael Brundo Uriarte 44/51
  46. 46. Research Questions Research Question 1 How to describe services and their objectives in the cloud domain? SLAC Research Question 2 What is data, information, knowledge and wisdom in the autonomic cloud domain? DIKW Hierarchy Research Question 3 How to collect and transform operational data into useful knowledge without overloading the autonomic cloud? Panoptes Conclusions Rafael Brundo Uriarte 45/51
  47. 47. Research Questions Research Question 4 How to produce a robust measure of similarity for services in the domain and how can this knowledge be used? RF+PAM Research Question 5 How to integrate different sources of knowledge and feed the autonomic managers? Polus Framework Conclusions Rafael Brundo Uriarte 46/51
  48. 48. Limitations Intelligence of Autonomic Managers Wide Range of Specific Knowledge Off-line Training of RF+PAM Conclusions Rafael Brundo Uriarte 47/51
  49. 49. Contributions A theoretical and practical framework for the generation and provision of knowledge for the autonomic management of clouds (Polus Framework): SLAC - SLA Definition and Evaluation Panoptes - Monitoring RF+PAM - Similarity Learning Conclusions Rafael Brundo Uriarte 48/51
  50. 50. Publications 1. R. B. Uriarte, S. Tsaftaris and F. Tiezzi. Service Clustering for Autonomic Clouds Using Random Forest. In Proc. of the 15th IEEE/ACM CCGrid [In Press], 2015. 2. R.B. Uriarte, F. Tiezzi, R. De Nicola, SLAC: A Formal Service-Level-Agreement Language for Cloud Computing. In IEEE/ACM 7th International Conference on Utility and Cloud Computing (UCC), 2014. 3. R.B. Uriarte, C.B. Westphall, Panoptes: A monitoring architecture and framework for supporting autonomic Clouds, In Proc. of the 16th IEEE/IFIP Network Operations and Management Symposium (NOMS), 2014. 4. R.B. Uriarte, S.A. Chaves, C.B. Westphall, Towards an Architecture for Monitoring Private Clouds. In IEEE Communications Magazine, 49, pages 130-137, 2011. Conclusions Rafael Brundo Uriarte 49/51
  51. 51. Future Works Dynamic SLAs Negotiation of SLAs Cloud Case Study Conclusions Rafael Brundo Uriarte 50/51
  52. 52. Thank you! Questions? Rafael Brundo Uriarte rafael.uriarte@gmail.com Conclusions Rafael Brundo Uriarte 51/51
  53. 53. SLAC - Expressivity Core Language Extensions - Business Aspects Formal Definition for Extensions Conclusions Rafael Brundo Uriarte 51/51
  54. 54. SLAC - Implementation Compatibility only with OpenNebula Toy Implementation Easily adapted Conclusions Rafael Brundo Uriarte 51/51
  55. 55. SLAC - Cloud Metrics DTMF Cloud Computing Service Metrics Description Recent Document (Still a Draft) Creation of a Model for the Definition of Metrics The SLAC Metrics can be Adapted for this Model Conclusions Rafael Brundo Uriarte 51/51
  56. 56. SLAC Violation Violation and Penalty are Separated Concepts “Violation” Concept Flexible Easy to Understand Conclusions Rafael Brundo Uriarte 51/51
  57. 57. Panoptes - Scalability Designed to be scalable Adapt itself Experiments suggest it is scalable More experiments for future works Conclusions Rafael Brundo Uriarte 51/51
  58. 58. Panoptes - Analysis of Apache Broklyn Not Focused on Monitoring Does Not Process the Data Conclusions Rafael Brundo Uriarte 51/51
  59. 59. Panoptes - Analysis with CSPARQL Data is not Decorated (e.g. RDF) Impact of Decorated Monitoring Data (Scalability) Very Interesting Option Conclusions Rafael Brundo Uriarte 51/51

×