simplengineeringService Oriented Architects                                                                            Nov...
simplengineeringService Oriented Architects        Table of contents          1.      SOA Functional Testing          2.  ...
simplengineeringService Oriented Architects        Motivation             In the next 20 years, tens, hundreds, thousands,...
simplengineeringService Oriented Architects        SOA conceptual framework             The collaboration among systems is...
simplengineeringService Oriented Architects        SOA testing conceptual framework             Validation question: "Are ...
simplengineeringService Oriented Architects           Probabilistic Inference Approach                  The ongoing resear...
simplengineeringService Oriented Architects                                                     sd TestSuite_Wiring_003   ...
simplengineeringService Oriented Architects         Grey-box Test             sd TestSuite_Wiring_003                     ...
simplengineeringService Oriented Architects        SOA testing environment                 Testing and Test Control Notati...
simplengineeringService Oriented Architects   deployment BankNet DeploymentTokens_1          TTCN-3                       ...
simplengineeringService Oriented Architects        SOA Testing in WS* platform                 SAUT representation = exten...
simplengineeringService Oriented Architects        BN Inference Approach                 Bayesian network inference is use...
simplengineeringService Oriented Architects        BN4SAT Metamodel            The BN4SAT graph model has six Boolean stoc...
simplengineeringService Oriented Architects        Discussion              A BN for SOA testing can exhibit a very large s...
simplengineeringService Oriented Architects        Thanks            Questions ?   ICSSEA 2011 - Steps towards model-based...
simplengineeringService Oriented Architects        SOA testing environment                 Services Architecture Under Tes...
simplengineeringService Oriented Architects          SOA Test Running                 Each SUT, in a well established SAUT...
simplengineeringService Oriented Architects        SOA Testing Environment                 Test Components: specialized co...
simplengineeringService Oriented Architects        Model-driven Approach                 The starting point for testing a ...
simplengineeringService Oriented Architects        Test Engineering - Negative sample                                     ...
simplengineeringService Oriented Architects        SOA Testing in WS* platform                 The SAUT is translated into...
simplengineeringService Oriented Architects          SAUT Description                                                     ...
simplengineeringService Oriented Architects        Test Case Oracle - Interaction            sd TestSuite_Wiring_002      ...
simplengineeringService Oriented Architects        Test Case Oracle - Envelope                 MessageType Instance       ...
Upcoming SlideShare
Loading in …5
×

Steps towards model-based, inference-driven SOA Testing

386 views

Published on

This presentation reports the progress of a project whose goal is to develop a model-driven, contract-based SOA testing environment in which grey-box testing strategies for large services architectures are driven by stochastic inference. In order to achieve this goal, a Bayesian network is built directly by mapping and model transformation from the services architecture design and test models. Because services architectures are complex and large systems, the generated BN and its associated probability tables may also be very large and the inference process may be excessively slow. Nevertheless, the structures of the services architecture design and test models let techniques of compilation improve the inference task, enabling the use of BN inference as a sustainable tool for driving failure discovering and troubleshooting strategies in large scale services architectures.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Steps towards model-based, inference-driven SOA Testing

  1. 1. simplengineeringService Oriented Architects November 30, 2011 Steps towards model-based, inference-driven SOA Testing Ariele P. Maesano¹ ², Fabio De Rosa ², Libero Maesano ², Perre-Henri Wuillemin¹ {Ariele.Maesano, Pierre-Henri.Wuillemin}@lip6.fr {ariele.maesano, fabio.de-rosa, libero.maesano}@simple-eng.com ¹ Laboratoire d’Informatique de Paris 6 (UPMC), Paris, France ² SIMPLE ENGINEERING, 31-35, rue St. Ambroise, 75011 Paris, France
  2. 2. simplengineeringService Oriented Architects Table of contents 1. SOA Functional Testing 2. Black-box and Grey-box Test 3. SOA Testing Environment 4. Model Driven Test Engineering 5. BN Inference Approach 6. Discussion & Future work ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 2
  3. 3. simplengineeringService Oriented Architects Motivation In the next 20 years, tens, hundreds, thousands, … applications, systems, devices will be connected and will collaborate without human intermediation - that’s the Brian Arthur’s second economy * Distributed enterprise architectures involving companies, government and for-no-profit organizations allow the automation of business processes that support daily activities Service Oriented Architecture (SOA) is a design and implementation style that allows organizations to put in practice dynamic collaborations of loosely coupled systems, in order to achieve flexible, dependable and secure business process automation SOA is the paradigm able to support the second economy * W. Brian Arthur, "The second economy." McKinsey Quarterly, October 2011 ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 3
  4. 4. simplengineeringService Oriented Architects SOA conceptual framework The collaboration among systems is carried out through the exchange of services which are regulated by contracts Service contracts are formal models of the service functions and of the provider and consumer interfaces and external behaviors (including security and quality of service aspects) Service contracts do not include any information about internals of the systems that comply with them - system implementations are black boxes, hidden and private A services architecture is a network of participant systems, playing roles bound by service contracts, that collaborate in order to achieve business goals ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 4
  5. 5. simplengineeringService Oriented Architects SOA testing conceptual framework Validation question: "Are you building the right services architecture?", is translated in the question: "Are you formalizing the right service contracts and participants’ roles (combinations of contract parts)?" Verification question: "Are you implementing participant systems right, i.e. in compliance with the validated service contracts and participants’ roles?“ A model-driven approach simplifies the validation and verification task:  once the service contract model has been validated, the remainder of the overall validation task is the verification of the correct application of the model mapping and transformation, until the implementation The last verification question (about compliance between implementation and contract) “stays open”,  SOA Architects and Integrators cannot employ program static check methods or white-box testing  The only available verification methods are black-box testing of individual SUTs (System Under Test) and grey-box testing of interactions among SUTs of the Services Architecture Under Test (SAUT), where they are observable ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 5
  6. 6. simplengineeringService Oriented Architects Probabilistic Inference Approach The ongoing research on SOA testing spans three main areas:  the automated generation of test cases from contract models (positive, negative, quality of service…)  the automation of the test execution and evaluation environments  inference-based methods and tools in order to help Testers to efficiently manage test campaigns There are not yet applications of probabilistic inference to the management of SOA testing campaigns  it appears to be adequate for discovery of failures and search and identification of faulty participants in a services architecture, where the only available information is the match/mismatch between the actual external behavior and the expected one Bayesian Networks for Services Architecture Testing (BN4SAT) project(*) is centered on the development of a Bayesian network (BN) inference approach and technology to services architecture testing management (*)The project is conducted in cooperation between the Laboratoire dInformatique de Paris VI (LIP6 - Université Pierre et Marie Curie - Paris VI - France), the Centre National de la Recherche Scientifique (CNRS - France) and Simple Engineering, a European group specialized in the design of services architectures. The project is partially funded by the Association Nationale de la Recherche et de la Technologie (ANRT - France) ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 6
  7. 7. simplengineeringService Oriented Architects sd TestSuite_Wiring_003 Black-box Test usbg :BankGateNode usam :AccountMngtNode Black-box tests are run against individual SUTs CheckBalanceResponder/inquiry(m-usbg-usam-001) CheckBalanceRequester/inform(m-usam-usbg-002) whose internals are hidden Connections (channels) with other participants are not observed Only the SUT responses to WithdrawalResponder/invoke(m-usbg-usam-003) WithdrawalRequester/report(m-usam-usbg-004) stimuli are observed CheckBalanceResponder/inquiry(m-usbg-usam-005) CheckBalanceRequester/inform(m-usam-usbg-006) ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 7
  8. 8. simplengineeringService Oriented Architects Grey-box Test sd TestSuite_Wiring_003 usatm usbg usam :ATM_Node :BankGateNode :AccountMngtNode FindAccountResponder/inquiry(m-usatm-usbg-001) Grey-box tests are FindAccountRequester/inform(m-usbg-usatm-002) run against a SAUT CheckBalanceResponder/inquiry(m-usatm-usbg-003) (network of SUTs) CheckBalanceResponder/inquiry(m-usbg-usam-001) CheckBalanceRequester/inform(m-usam-usbg-002) SUTs are involved CheckBalanceRequester/inform(m-usbg-usatm-004) in an end-to-end CheckCredentialsResponder/invoke(m-usatm-usbg-005) CheckCredentialsRequester/report(m-usbg-usatm-006) interaction Channels among WithdrawalResponder/invoke(m-usatm-usbg-007) WithdrawalResponder/invoke(m-usbg-usam-003) SUTs are observed WithdrawalRequester/report(m-usam-usbg-004) WithdrawalRequester/report(m-usbg-usatm-008) CheckBalanceResponder/inquiry(m-usatm-usbg-009) CheckBalanceResponder/inquiry(m-usbg-usam-005) CheckBalanceRequester/inform(m-usam-usbg-006) CheckBalanceRequester/inform(m-usbg-usatm-010) ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 8
  9. 9. simplengineeringService Oriented Architects SOA testing environment Testing and Test Control Notation (V.3)* environment Arbiter, Scheduler, Test components as TTCN-3 scripts Engine as a plug-in (or a “service”) * http://www.ttcn-3.org/ ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 9
  10. 10. simplengineeringService Oriented Architects deployment BankNet DeploymentTokens_1 TTCN-3 «SUT» usbg :BankGateNode usbg4usatminterceptor : usbg4usaminterceptor : BankGate4ATM BankGate4AccountMngt usatminterceptor4usbg: ATM4BankGate usaminterceptor4usbg: AccountMngt4BankGate «executionEnvironment» :TTWorkbench usbginterceptor4usatm: usam4usbginterceptor : «SUT» BankGate4ATM AccountMngt4BankGate «TestComponent» «TestComponent» «SUT» usatm :ATM_Node usbginterceptor : usaminterceptor : usam :AccountMngtNode usatm4usbginterceptor BankGateInterceptorNode usbginterceptor4usam: AccountMngtInterceptorNode :ATM4BankGate BankGate4AccountMngt ushwcstimulator: arbiter :Arbiter HW_Control4ATM usatm4ushwcinterceptor : «TestComponent» ATM4HW_Control usatmstimulator : ATM_StimulatorNode scheduler :Scheduler a4e : s4e : Arbiter4Engine Scheduler4Engine Automated test execution & e4a : e4s : Engine4Arbiter Engine4Scheduler evaluation environment :Engine ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 10
  11. 11. simplengineeringService Oriented Architects SOA Testing in WS* platform SAUT representation = extended UDDI V3 Data Structure + collection of WSDLs UML Interaction (XMI) + collection of SOAP messages ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 11
  12. 12. simplengineeringService Oriented Architects BN Inference Approach Bayesian network inference is used as a  failure discovering driver, by getting the next test case to run in order to reveal a failure. As  troubleshooting driver, by getting the next test to run in order to locate faulty participants BN4SAT engine integrates failure discovering and troubleshooting, by managing a well-adjusted dynamic test sequence (the strategy) that, with a minimum number of test case runs:  reveals a maximum number of failures and  locates a maximum number of faulty participants The BN4SAT graph is built by the BN4SAT loader directly from:  the SAUT deployment UDDI files  the WSDL files for all the involved service contracts and  the test suite files ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 12
  13. 13. simplengineeringService Oriented Architects BN4SAT Metamodel The BN4SAT graph model has six Boolean stochastic variable types: Entity - business organization, {notFaulty, faulty} Participant - participant system, {notFaulty, faulty} Port - participant required interface, {notFaulty, faulty} ActionType - type of communicative action (message, rpc, rpc reply), {notFaulty, faulty} Action (evidence): test case action {pass, fail} Transaction: test case {pass, fail} ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 13
  14. 14. simplengineeringService Oriented Architects Discussion A BN for SOA testing can exhibit a very large size, and can be very dense, therefore with large conditional probability tables (inefficiency in time and space) We have implemented and assessed different classical inference methods such as value elimination, lazy propagation, Shafer-Shenoy and proved that they are not appropriate to cope in a sustainable way with BN4SAT graphs Inference by compilation: Bayesian network can be interpreted as a multi-linear function(MLF) and therefore implemented by an arithmetic circuit (AC) Darwiche “classical” method Coarse Refined deterministic SAUT + test suite Conjunctive Conjunctive Disjunctive Normal Bayesian network Negational Form Arithmetic Circuit specification Normal Form Normal Form (Coarse CNF) (Refined CNF) (d-DNNF) BN4SAT method (Model-driven inference by compilation) - No intermediary generation of BN - Refined CNF in linear time (experimental results) - No CNF optimization step - Push the boundaries with size limitation Refined Conjunctive deterministic Disjunctive SAUT + test suite Normal Negational Form Normal Form (Refined Arithmetic Circuit specification CNF) (d-DNNF) ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 14
  15. 15. simplengineeringService Oriented Architects Thanks Questions ? ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 15
  16. 16. simplengineeringService Oriented Architects SOA testing environment Services Architecture Under Test - SAUT is a network of SUTs (System Under Test) connected by channels conveying communicative actions such as messages, remote procedure calls (rpc), rpc replies Channels may be observable (grey-box stance). Conversely, SUTs internals are never observable (black-box stance) deployment BankNet Deployment BankNet SAUT atm4bg :ATM4BankGate «SUT» «SUT» usatm :ATM_Node usbg :BankGateNode bg4atm :BankGate4ATM usatm4ushwcinterceptor : bg4am :BankGate4AccountMngt ATM4HW_Control hwc4atm :HW_Control4ATM am4bg :AccountMngt4BankGate «SUT» «SUT» ushw c :HW_ControlNode usam :AccountMngtNode ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 16
  17. 17. simplengineeringService Oriented Architects SOA Test Running Each SUT, in a well established SAUT, is initialized with a text context (the SAUT state), that is coherent with the next selected test suite (i.e., a collection of test cases) is going to run A test run takes place when Tester submits one or more stimuli to one or more SUTs Tester compares the SAUT actual response - the exchange of actions between SUTs following the stimuli - with the expected response - the expected action exchange in a specified SAUT state A test matches (mismatches) if the actual response corresponds to (differs from) the expected one Test run has a verdict: pass - the test matches and there is no evidence of failure; fail - there is evidence of a failure; inconclusive - a decision cannot be reached; error - a run-time error has occurred in the test environment ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 17
  18. 18. simplengineeringService Oriented Architects SOA Testing Environment Test Components: specialized components allow reaping SUTs actions, addressing actions to SUTs, comparing SUTs actions with the expected ones, formulating local verdicts and transmitting them to the Arbiter:  Stimulators - send stimuli to one or more SUT, in order to trigger test runs  Interceptors - sit transparently between two connected participants, intercept actions and perform different tasks  Emulators - emulate the behavior of SUTs Arbiter - the Arbiter collects the Test Components local test verdicts and produces final test verdicts Scheduler - the Scheduler drives the execution of the test runs Engine, which is notified by the Arbiter with test verdicts, manages the Scheduler by handling the test strategy on the basis of probabilistic inference on the acquired test verdicts ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 18
  19. 19. simplengineeringService Oriented Architects Model-driven Approach The starting point for testing a SAUT is in the Platform Independent Model (PIM), specifically, the collection of samples A sample is a modeled, schema-compliant occurrence of services interactions, that coordinates either the services deliveries (positive sample), or the services refusals, when the delivery conditions are not satisfied (negative sample) A sample is composed of a:  sample interaction, a UML Interaction together with its MessageTypes instances, and a  sample context, a collection of domain objects (instances) representing the states of the participants resources in which the interactions take place The sample collection is the starting point for the design and the implementation of test cases ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 19
  20. 20. simplengineeringService Oriented Architects Test Engineering - Negative sample class Sample oracle sd TestSuite_Wiring_003 «MessageType» m-usbg-usam-003 :Withdraw alInv okeMessage usbg usam acc# = 123456 :BankGateNode :AccountMngtNode am = 1000,00 curr = USDollar dateTime = 2011-06-15T11:28:38.100+01:00 WithdrawalResponder/invoke(m-usbg-usam-003) «MessageType» WithdrawalRequester/decline(m-usam-usbg-005) m-usam-usbg-005 :Withdraw alDeclineMessage acc# = 123456 am = 1000,00 curr = USDollar dateTime = 2011-06-15T11:28:14.109+01:00 code = NotEnoughBalance Sample context (fact base) CUSTOMER-NAME CUSTOMER CUSTOMER-ACCOUNT ACCOUNT customer# name customer# customer# account# account# currency state balance ABCDEFG Ariele ABCDEFG ABCDEFG 123456 123456 EUR active 900.00 ABCDEFG Paolo ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 20
  21. 21. simplengineeringService Oriented Architects SOA Testing in WS* platform The SAUT is translated into a set of WSDL files and an extended UDDI V3 Data Structure, in order to capture use links (the observable channels in a specific deployment scenario) - each participant system is represented as UDDI Business Service Sample collection is translated in a test suite (a collection of test cases), by the mapping and mechanical transformation of the PIM into the WS* Interoperability PSM A test case includes:  a test case oracle - a XML infoset that represents the interchange of SOAP messages between SUTs (resulting from the transformation of the sample UML Interaction), together with the collection of the corresponding SOAP envelopes (resulting from the transformation of the Message Type instances exchanged in the interaction)  a test case fact base (test context) - a relational database resulting from the object/relation transformation of the sample context ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 21
  22. 22. simplengineeringService Oriented Architects SAUT Description UDDI v3 Business Service deployment BankNet Deployment «SUT» am4bg :AccountMngt4BankGate «SUT» usbg :BankGateNode bg4am :BankGate4AccountMngt usam :AccountMngtNodebg4atm :BankGate4ATM<?xml version="1.0" encoding="UTF-8"?> <uddi:businessService businessKey="urn:My_US_BankNet" serviceKey="urn:My_US_BankNet:BankGate"> <uddi:name>BankGate</uddi:name> <uddi:bindingTemplates> <uddi:bindingTemplate bindingKey="urn:BankGate-AccountMngt:BankGate4AccountMngt:usbg4usam" serviceKey="urn:My_US_BankNet:BankGate"> <uddi:description>urn:BankGateNode:usbg</uddi:description> <uddi:accessPoint useType="endpoint">http://usbg.My_US_BankNet.com/BankGate4AccountMngt/usbg4usam</uddi:accessPoint> <uddi:tModelInstanceDetails> <uddi:tModelInstanceInfo tModelKey="urn:BankGate-AccountMngt"> <uddi:instanceDetails> <uddi:overviewDoc> <uddi:description>required</uddi:description> use links <uddi:description>urn:IAccountMngt</uddi:description> <uddi:description>urn:AccountMngtNode:usam:BankGate-AccountMngt:AccountMngt4BankGate:usam4usbg</uddi:description> </uddi:overviewDoc> </uddi:instanceDetails> </uddi:tModelInstanceInfo> </uddi:tModelInstanceDetails> </uddi:bindingTemplate>...</uddi:businessService> ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 22
  23. 23. simplengineeringService Oriented Architects Test Case Oracle - Interaction sd TestSuite_Wiring_002 usbg usam :BankGateNode :AccountMngtNode XML Infoset describing WithdrawalResponder/invoke(m-usbg-usam-003) interaction among SUT WithdrawalRequester/report(m-usam-usbg-004) <?xml version="1.0" encoding="UTF-8"?> <ss20tc:test-case xmlns:ss20tc="urn:simpleSOAD2.0:test-case" id="urn:InterExchangeNet:Scenario01:Wiring_001"> <ss20tc:sequence> ... <ss20tc:message id="m-usbg-usam-003" sn="m033" from="urn:BankGateNode:usbg" to="urn:AccountMngtNode:usam" mode="WithdrawalResponder/invoke"/> <ss20tc:message id="m-usam-usbg-004" sn="m034" from="urn:AccountMngtNode:usam" to="urn:BankGateNode:usbg" mode="WithdrawalRequester/report"/> ... </ss20tc:sequence> </ss20tc:test-case> ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 23
  24. 24. simplengineeringService Oriented Architects Test Case Oracle - Envelope MessageType Instance class Sample oracle «MessageType» m-usbg-usam-003 :Withdraw alInv okeMessage acc# = 123456 am = 1000,00 SOAP Message curr = USDollar dateTime = 2011-06-15T11:28:38.100+01:00 <?xml version="1.0" encoding="UTF-8"?> <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="urn:ien:bank:services:withdrawal:wsdl:types" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing"> <soapenv:Header> <wsa:MessageID>m-usbg-usam-003</wsa:MessageID> </soapenv:Header> <soapenv:Body> <ns:WithdrawalInvokeMsg xmlns:ns="urn:ien:bank:services:withdrawal:wsdl:types"> <ns:sessionId> <ns:account>123456</ns:account> <ns:dateTime>2011-06-15T11:28:38.100+01:00</ns:dateTime> </ns:sessionId> <ns:currency>USDollar</ns:currency> <ns:amount>10000,00</ns:amount> </ns:WithdrawalInvokeMsg> </soapenv:Body> </soapenv:Envelope> ICSSEA 2011 - Steps towards model-based, inference-driven SOA Testing 30/11/2011 - 24

×