Successfully reported this slideshow.

S-CUBE LP: Run-time Verification for Preventive Adaptation


Published on

Published in: Technology, Design
  • Be the first to comment

  • Be the first to like this

S-CUBE LP: Run-time Verification for Preventive Adaptation

  1. 1. S-Cube Learning Package Run-time Verification for Preventive Adaptation University of Duisburg-Essen (UniDue), Center forScientific and Technological Research (FBK), Politecnico di Milano (POLIMI) Eric Schmieders, UniDue
  2. 2. Learning Package Categorization S-Cube Quality Definition, Negotiation and Assurance Quality Assurance and Quality Prediction Run-time Verification for Preventive Adaptation © S-Cube
  3. 3. Learning Package Overview Problem Description Preventive Adaptation based on Runtime-Verification Discussion Conclusions © S-Cube
  4. 4. Service-oriented SystemsNeed for Adaptation Highly dynamic changes due to – 3rd party services, multitude of service providers, … – evolution of requirements, user types, … – change in end-user devices, network connectivity, … Difference from traditional software systems – Unprecedented level of change – No guarantee that 3rd party service fulfils its contract (SLA) – Hard to assess behaviour of infrastructure at design time © S-Cube
  5. 5. Let’s Consider a Scenario (1) Assume you are a provider of a composite service Commercial customers order products via a Web service interface Your process checks an internal stock, orders some missing parts, and assembles the product © S-Cube
  6. 6. Let’s Consider a Scenario (2) (Of course) there are contractual obligations – Delivery in time – Availability of your service – Product quality These can be formulated using Service Level Agreements – Note: in this context not necessarily WSLA style machine processable SLAs – can also be just regular contracts © S-Cube
  7. 7. Let’s Consider a Scenario (3) As a provider, you want to receive timely notifications if these obligations are likely to be violated – Usually there are penalties to be paid if contractual obligations are (repeatedly) violated – Even if this is not the case, customer satisfaction will suffer and existing customers may terminate their contracts Based on this notifications … – … countermeasures can be taken (i.e., adaptations can be triggered) – … customers can be notified © S-Cube
  8. 8. Learning Package Overview Problem Description Preventive Adaptation based on Runtime-Verification Discussion Conclusions © S-Cube
  9. 9. Types of Adaptation  Reactive Adaptation Failure! – Repair/compensate for external failure visible to the end-user  Preventive Adaptation – An internal failure/deviation occurs  Will it lead to an external failure? Failure! Failure? – If “yes”: Repair/compensate internal failure/deviation to prevent external failure  Proactive Adaptation  Is internal failure /deviation imminent (but did not occur)? – If “yes”: Modify system before internal Failure? failure/deviation actually occurs
  10. 10. Types of Adaptation For Preventive Adaptation – Data Mining: Extract knowledge from “historical” data - TUW & USTUTT: “Prediction and Prevention of SLA Violations in Composite Services” - SZTAKI: “Historical Data Based Predictions for Resource Allocation” – Run-time Verification: Formally ascertain that properties hold - Paluno, FBK, Polimi: “Assumption-based Run-time Verification” For Proactive Adaptation – Simulation: Execute dynamic models to simulate behavior - UPM & TUW: “Simulation of Provision Resources” - INRIA: “QoS Assurance Using Discrete-Event Simulation” – Static Analysis: Fomally infer properties (without execution) - UCBL & UPM: “Soft Constraints for QoS-Aware Service Selection” - UPM: “Data-aware Resource Analysis for Service Orchestrations” – Online Testing: Actively execute services in parallel to normal use - Paluno & UPC: “Augmenting Monitoring with Online Testing” - SEERC & Paluno : “Just-in-time Testing of Conversational Services”[Deliverable JRA-1.3.5,] © S-Cube
  11. 11. Background:Runtime Verification Runtime Verification is a system analysis approach based on extracting information from a running system  extracted information is used to detect and possibly react to observed behaviors satisfying or violating certain properties. Properties can be functional (certain functionality) or non-functional (e.g. required response time) Runtime verification specifications are typically expressed in regular expressions, context-free patterns, linear temporal logics, etc. Based on formal requirements specifications, monitors are created and applied during the systems runtime Runtime verification can be used for many purposes: security or safety policy monitoring, debugging, testing, verification, validation, profiling, fault protection, etc…[BianculliEtAl2008, GehlertEtAl2010, JRA-1.3.2,] © S-Cube
  12. 12. SPADEAn Example for Runtime Verification The SPADE approach uses runtime verification for preventive adaptation ON THE NEXT SLIDES: SPADE will be used to illustrate preventive adaptation by applying run-time verification Furthermore – (A) an abstract example scenario and – (B) the S-Cube Service Life-Cycle will be used to illustrate the design time and run time activitives of SPADEBackground:S-Cube Service Life-Cycle A life cycle model is a process model that covers the activities related to the entire life cycle of a service, a service-based application, or a software component or system. [Gu2007, http://www.s-cube-] © S-Cube
  13. 13. Runtime Verification with SPADE SPADE (Specification and Assumption based Detection of Adaptation Needs) equips SBAs with adaptation capabilities, empowering them to adapt preventively [MetzgerEtAl2010, SchmiedersEtAl2011] TO ACHIEVE THIS: SPADE uses – run-time verification techniques – execution data of the monitored instances – assumptions concerning the SBAs context, e.g. the response time of constituent services (derived from SLAs: http://www.s-cube- – formalized requirements of the SBA This data is used for performance prediction and preventive adaptationwhich is able to avoid requirement violations of running SBAs © S-Cube
  14. 14. Abstract Example 5900 ms Action 1 :FastWeather Action 3 Action 2 :Google 4000 ms 8400 ms Action 4 :WSAmazonBox 6400 ms Action 5 :HyperlinkExtractor 2700 ms Action 6 :GetJoke 14000 ms Action 7 :CurrencyConverter © S-Cube
  15. 15. Abstract Example Activity Action 1 5900 ms :FastWeather Diagramof Workflow Action 3 Action 2 :Google 4000 ms 3rd Party Service Service Binding Action 4 8400 ms :WSAmazonBox 6400 ms Action 5 :HyperlinkExtractor 2700 ms Action 6 :GetJoke Assumed 14000 ms Action 7 :CurrencyConverter response time © S-Cube
  16. 16. S-Cube Service Life-Cycle Check: S, A‘, M |= R Monitoring Requirements m1 |= a1 R := {r1, r2 …} Identify (vgl. SM) Requirements Adaptation Workflow Operation & Engineering Need Specification S, Management Service Candidates C Identify Design Adaptation Adaptation Evolution StrategySolve: Deployment & Enact Provisioning RealizationC, S |= R AdaptationCreate:Plan p Deployment SLA Negotiation Enact p Check: S, A |= R A:= {a1, a2, …} [MetzgerEtAl2010, SchmiedersEtAl2011] © S-Cube
  17. 17. Requirements EngineeringIn the requirements engineering phase, the functionaland quality requirements for the SBA are elicited and Requirementsdocumented! R := {r1, r2 …} GOAL: enable check, whether application will from its requirements during operation? Requirements Engineering functional and non-functional requirements are formally expressed as input to SPADE SPADE uses the specification language ALBERT to formalize the SBA requirementsFor more information on ALBERT:Bianculli, D., Ghezzi, C., Spoletini, P., Baresi, L., Guinea, S.: A guided tour throughSAVVY-WS: A methodology for specifying and validating web service compositions.In: Borger, E., Cisternino, A. (eds.) Advances in Software Engineering. LNCS, vol.5316. Springer (2008) © S-Cube
  18. 18. Requirements Engineering Example: – Formalize Requirement r: end-to-end response time is at most 55 seconds – ALBERT is used to formalize r: r := onEvent(start, "Action1") -> Within( onEvent( end, "Action7"), 55000) – onEvent operator evaluates to true if the activity specified in its second argument performs the state change denoted in its first argument. – Within operator evaluates to true if its first argument evaluates to true within the amount of milliseconds specified in its second argument © S-Cube
  19. 19. DesignDuring the design phase, the activities andthe control flow of the application are Workflow Specification S,specified! Service Definition of workflow in e.g. BPEL Candidates C Candidate services are identified that can Design provide the required functionality/quality SPADE extends idea of Bianculli et al. [BianculliEtAl2008] by using a model checker during run-time ALBERT expressions can be executed by BOGOR: formalize the workow using BIR, the BOGOR Input Representation. © S-Cube
  20. 20. Design Example: BIR file (excerpt) as input for the BOGOR Model Checker: … © S-Cube
  21. 21. Background:Model Checking With model checking it is automatically testable, whether a model meets a given specification. Concept is general and applies to all kinds of logics and suitable structures Example: verifying whether a given formula in the propositional logic is satisfied by a given structure Typically hardware or software systems are checked The checked specification contains safety requirements, e.g. the absence of deadlocks, critical states that can cause the system to crash, … The model of the system and the specification are formulated in precise mathematical language (e.g. LTL, CTL…) Common model checker:,,, and[GehlertEtAl2011, Bianculli2008,] © S-Cube
  22. 22. RealizationTo achieve the desired end-to-end quality ofSBAs, contracts between service providers Realizationand service consumers on quality aspectsof services are established! SLA Negotiation A:= {a1, a2, …} For each candidate service: negotiate best quality level for the available budget & stipulated in an SLA SPADE: the quality levels are formalized, that have been negotiated and agreed upon with the service providers Quality levels are used as assumptions A about the SBAs context ALBERT is used to formalize AFor more information on negotiating SLAs:Comuzzi, M., Pernici, B.: A framework for qos-based web service contracting. ACMTransactions on web 3(3) (2009) © S-Cube
  23. 23. Realization Example: – For our example we use the assumed response time given in the example SBA. – The assumption for the aFastWeather service bound to Action 1, is formalized as follows1: aFastWeather := onEvent (start, "Action 1")  Within( onEvent(end, "Action 1" ), 5900)Background:Assumptions In the case of service-based applications, assumptions may characterize the constituent services (e. g., their interfaces, QoS parameters, etc.) and/or the context (e. g., infrastructure, business, context, user, etc.). A violation of those assumptions may lead to a situation, in which the software system does not provide the expected quality anymore and, therefore, deviates from its requirements.[GehlertEtAl2010] © S-Cube
  24. 24. DeploymentThe deployment and provisioning phasecomprises all the activities needed to make Deployment &the SBA available to its users! Provisioning SPADE uses BOGOR to check whether the workflow specification (S), under the given Deployment assumptions (A), satisfies the requirements (R), Check: S, A |= R i.e. whether: S, A |= R If requirements are not satisfied  phases of the evolution loop are re- executed, e.g., in order to bind faster services. Example: – The specification of the abstract service composition evaluates to true, i.e. S and A satisfy R. Thus, the SBA is deployed. © S-Cube
  25. 25. Operation and ManagementThis phase comprises the execution ofthe SBA and the monitoring of its Monitoring m1 |= a1constituent services using service (vgl. SM)monitoring techniques! To identify assumption violations Operation & during the operation of the SBA  monitoring Management mechanisms are employed SPADE uses the monitoring facilities of the runtime environment to CHECK: m |= a Example: – Assume: mFastWeath and mGoogle satisfy their related assumptions er – BUT: WSAmazonBox too late  mGoogle |≠ aGoogle. – CONSEQUENCE: Due to the deviation the next phase is entered immediately; a performance violation could be indicated! © S-Cube
  26. 26. Identify Adaptation NeedsIn this phase the need for the adaptationis detected. To this end a runtime Check: S, A‘, M |= Rverification against the SBAs requirements Identifyis performed! Adaptation SPADE checks: are requirements still satisfied? Need For all invoked services: – SPADE replaces the assumption (A) values by concrete monitoring data (M) – only for the not invoked services, assumptions (A’) are used SPADE thus uses a subset A’ (of A), together with the set of monitored data M to check whether: S,M,A’ |= R SPADE utilizes BOGOR to perform this verification during run-time If there is one future path which violates r  adaptation must be triggered ELSE  the workflow execution is continued © S-Cube
  27. 27. Identify Adaptation Needs Example: – Check whether the requirement is still met (S,M,A’ |= R): - workflow specification S - the monitoring data (mFastWeather, mGoogle and mWSAmazonBox) - assumptions of the to be invoked services (aHylinkExtractor, aCurrencyConverter, aGetJoke)  checked against the requirement r. – Predicted end-to-end duration: 56238 ms  exceeds the 55 seconds demanded by r. – CONSEQUENCE: the workflow has to be adapted, otherwise a performance violation seems to be in all probability © S-Cube
  28. 28. Identify Adaptation NeedsResponse Times 5900 ms 5735 ms Action 1 :FastWeather Action 3 4320 ms Action 2 :Google 4000 ms 15083 ms 8400 ms Action 4 :WSAmazonBox 6400 ms Action 5 :HyperlinkExtractor 2700 ms Action 6 :GetJoke 14000 ms Action 7 :CurrencyConverter
  29. 29. Identify Adaptation NeedsResponse Times 5900 ms 5735 ms Action 1 :FastWeather Action 3 4320 ms Action 2 :Google 4000 ms FAILURE 15083 ms 8400 ms Action 4 :WSAmazonBox Assumptions 6400 ms 6400 ms Action 5 :HyperlinkExtractor 2700 ms e2e 6is predicted Action 2700 ms :GetJoke to be violated! 14000ms  Adaptation Action 7 14000 ms :CurrencyConverter necessary! ∑ > e2e_requirement(55 ms)
  30. 30. Identify Adaptation StrategyIn this phase an adaptation strategy is created. IdentifyThe strategy is supposed to avert the menacing AdaptationSLA violation successfully when enacted! Strategy SPADE is equipped with service substitution Solve: capabilities C, S |= R and create SPADE exploits the CHOCO constraint solver Plan p to determine which not-yet-invoked services have to be substituted ( Find a strategy with CHOCO for every future violating path (necessary, as it’s unclear which path will be taken in the further execution) Example: – Two paths are violating, i.e. p1 = (Action5, Action7) and p2 = (Action5, Action6, Action7)  paths will violate r and therefore have to be adapted. – Based on the output of CHOCO: services for Action5 and Action6 are chosen to be substituted by faster services © S-Cube
  31. 31. Enact AdaptationDuring this last adaptation phase the adaptation Enactstrategy is executed. For this purpose, the Adaptationinstructions comprised in the adaptation strategyare dispatched! Enact p Dispatching an adaptation strategy usually utilizes – the facilities provided by the chosen run-time environment – involves additional adaptation mechanisms SPADE: – uses the interception-mechanisms provided by the runtime environment to reroute messages to the new target service – switchs the target service of the invocation to the service identified during the previous phase. © S-Cube
  32. 32. Enact Adaptation Example: – In our example, the two service invocations Action5 and Action6 are redirected to the substituting services. – the message routing table comprising the message destinations is manipulated.  Consequently, the execution of the instance is resumed.. © S-Cube
  33. 33. Learning Package Overview Problem Description Preventive Adaptation based on Runtime-Verification Discussion Conclusions © S-Cube
  34. 34. Is Run-time Verification effective whenadapting preventively? SPADE can be considered as one representative for Run-time Verification techniques We conducted an experiment to access the effectiveness of SPADE Our experimental measurement of SPADEs effectiveness is twofold – false positives: unnecessary adaptations are measured – Impossible adaptations: amount of situations in which SPADE is unable to perform an adaptation is counted © S-Cube
  35. 35. Is Run-time Verification useful foradapting preventively?Determining the Degree of False Positives1. Calculated Response Time: – Calculate the end-to-end response time for each SBA instance execution – Add the monitored response times of the invoked services along the SBA instances path Based on this  determine false positive adaptation triggers: 2. SPADE triggered an adaptation 3. Check: calculated SBA instance response time violates the SBAs end-to-end requirements? False Positive, if: 1, 2, and 3 reveal that the requirement would not have been violated in case the SBA execution continues without adaptation © S-Cube
  36. 36. Is Run-time Verification useful foradapting preventively?Determining the Degree of Impossible Adaptations Situations during experimentation were observed: – service invocations deviated from their stipulated response time, such that the performance requirement is violated – too late to apply SPADE as the performance requirement is already violated determine the percentage of these situations:  relate the number of the service invocations which lead to these SLA violations to the amount of all service invocations © S-Cube
  37. 37. Is Run-time Verification useful foradapting preventively?Experimental Results: SPADE has been applied to 5884 SBA instances (cf. row (a) in Table below). 629 of those SBA instances have been executed without any assumption violations (b), in 5255 SBA instances assumptions have been violated (c), for those instances, SPADE has identified 604 preventive adaptation triggers (d) 72 of the adaptation triggers were false positives (e)  just 1.2% of the workflow instances would have been unnecessarily adapted. With respect to the challenging time constraint of 55 seconds, the very lowpercentage of false positives can be considered as extremely promising © S-Cube
  38. 38. Is Run-time Verification useful foradapting preventively? Amount of situations in which SPADE cannot adapt is very low as well Row (c)  a total of 825 out of 28624 service invocations, i.e. 2.5%, lead to situations where an adaptation is not possible. BUT: this could still mean a threat for applications which are required to meet the requirements in nearly 100%, e.g. in an emergency scenario © S-Cube
  39. 39. Learning Package Overview Problem Description Preventive Adaptation based on Runtime-Verification Discussion Conclusions © S-Cube
  40. 40. Conclusion SPADE is an automated technique to determine adaptation needs to trigger preventive adaptations of SBAs To achieve this high degree of automation SPADE exploits runtime verification techniques The experiments reveal situations in which SPADE cannot perform a preventive adaptation, that again: SPADE was able to avoid SLA violations by a high percentage This indicates the need for further improvement ofSPADE, but also generally demonstrates the effectivenessof applying runtime verification techniques for preventiveadaptation © S-Cube
  41. 41. References[MetzgerEtAl2010] Metzger, A.; Schmieders, E.; Cappiello, C.; Di Nitto, E.; Kazhamiakin,R.; Pernici, B. & Pistore, M.: Towards Proactive Adaptation: A Journey along the S-CubeService Life-Cycle. MESOA: 4th International Workshop on Maintenance and Evolutionof Service-Oriented Systems, 2010[SchmiedersEtAl2011] Schmieders, E. & Metzger, A.: Preventing Performance Violationsof Service Compositions using Assumption-based Run-time Verification. ServiceWave,2011[Gu2007] Q. Gu and P. Lago, A stakeholder-driven service life cycle model for SOA. In 2nd internationalWorkshop on Service Oriented Software Engineering: in Conjunction with the 6th ESEC/FSE Joint Meeting(Dubrovnik, Croatia, September 03 - 03, 2007). IW-SOSWE 07. ACM, New York, NY, 1-7.[BianculliEtAl2008] Bianculli, D.; Ghezzi, C.; Spoletini, P.; Baresi, L. & Guinea, S.: A Guided Tour throughSAVVY-WS: A Methodology for Specifying and Validating Web Service Compositions, Springer-Verlag, 2008,131-160[GehlertEtAl2010] Gehlert, A.; Bucchiarone, A.; Kazhamiakin, R.; Metzger, A.; Pistore, M. & Pohl, K.: Exploitingassumption-based verification for the adaptation of service-based applications. SAC 10: Proceedings of the2010 ACM Symposium on Applied Computing, ACM, 2010, 2430-2437[ComuzziEtAl2009] Comuzzi, M., Pernici, B.: A framework for qos-based web service contracting. ACMTransactions on web 3(3) (2009)[JRA-1.3.2] © S-Cube
  42. 42. Further S-Cube ReadingLeitner, Wetzstein, Rosenberg, Michlmayr, Dustdar, and Leymann. Runtime Prediction of Service LevelAgreement Violations for Composite Services. In Proceedings of the 2009 International conference onService-Oriented Computing (ICSOC/ServiceWave09), Springer-Verlag, Berlin, Heidelberg, 176-186.Leitner, Michlmayr, Rosenberg, and Dustdar. Monitoring, Prediction and Prevention of SLA Violations inComposite Services. In Proceedings of the 2010 IEEE International Conference on Web Services (ICWS 10).IEEE Computer Society, Washington, DC, USA, 369-376.Wetzstein, Leitner, Rosenberg, Brandic, Dustdar, and Leymann. Monitoring and Analyzing Influential Factorsof Business Process Performance. In Proceedings of the 13th IEEE international conference on EnterpriseDistributed Object Computing (EDOC09). IEEE Press, Piscataway, NJ, USA, 118-127. © S-Cube
  43. 43. Acknowledgements The research leading to these results has received funding from the European Community’s Seventh Framework Programme [FP7/2007-2013] under grant agreement 215483 (S-Cube). © S-Cube