presentation - Henry Muccini


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • As an example, we show two possible views composing the LTS global one. In the first partition we put in evidence the possible flows of messages and the picture is related to the alarm flow. The other perspective is the ComponentBased View were we can understand how a component interact with the others. This view is particularly important if we decide to modify our component and we would like to retest only the real different behavior. In the picture we highlight how Server Component can interact with the others
  • presentation - Henry Muccini

    1. 1. SA-based Code Testing Henry Muccini Computer Science Department Universita’ dell’Aquila – Italy ( [email_address] ) [ http:// ]
    2. 2. My Research <ul><li>Ph.D. Thesis on “ Software Architecture for Testing, Coordination Models and Views Model Checking”, year 2002. </li></ul><ul><li>Software Architecture and Coordination </li></ul><ul><li>Software Architecture and Model Checking </li></ul><ul><li>Software Architecture and Testing </li></ul>
    3. 3. <ul><li>Brief introduction on Software Architecture and Testing </li></ul><ul><li>SA-based Conformance Code Testing </li></ul><ul><li>SA-based Regression Testing </li></ul><ul><li>PLA-based Testing </li></ul><ul><li>Ongoing and future work </li></ul><ul><li>Some references </li></ul>Talk Overview
    4. 4. Software Architecture (SA) <ul><li>1992 and 1993 : </li></ul><ul><ul><li>SA is recognized as an independent area of research; </li></ul></ul><ul><li>1993-1995 : </li></ul><ul><ul><li>SA described through box and line, informal , diagrams; </li></ul></ul><ul><li>1995-2002 : </li></ul><ul><ul><li>Architectural Description Languages (ADLs) are introduced to formally describe SA; </li></ul></ul><ul><li>1997-2003 : </li></ul><ul><ul><li>more interest on dynamic aspects of SA; </li></ul></ul><ul><ul><li>SA used in academia and industry ; </li></ul></ul><ul><ul><li>SA and analysis </li></ul></ul><ul><ul><li>SA recognized as a valid tool to describe complex , concurrent , real systems ; </li></ul></ul>The History: [PhDThesis, Ch2]
    5. 5. In general terms… <ul><li>Describes (in a formal language) how a system is structured into components and connectors… </li></ul><ul><li> </li></ul><ul><ul><li>Components </li></ul></ul><ul><ul><ul><li>computation </li></ul></ul></ul><ul><ul><li>Connectors </li></ul></ul><ul><ul><ul><li>interaction </li></ul></ul></ul><ul><ul><li>Ports, Interfaces, … </li></ul></ul><ul><li>… how these </li></ul><ul><li>components interact </li></ul>SA Structure (topology) SA Dynamics (behavior)
    6. 6. SA dynamics <ul><li>A model of Software Architecture behavior: </li></ul><ul><ul><li>Labeled Transition System (LTS) </li></ul></ul><ul><ul><li>Finite State Machine </li></ul></ul><ul><ul><li>Petri Nets </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><li>Given the SA formal description, the model can be automatically generated; the dynamics is expressed in terms of components interaction via connectors </li></ul>
    7. 7. Motivations on SA-based Analysis <ul><li>For Analysis: [PhDThesis] </li></ul><ul><ul><li>SA management is expensive and time consuming </li></ul></ul><ul><ul><ul><li> maximize the benefits </li></ul></ul></ul><ul><ul><ul><ul><li> Analyze SA as much as possible </li></ul></ul></ul></ul><ul><ul><li>SA and Testing </li></ul></ul><ul><ul><li>SA and Coordination models </li></ul></ul><ul><ul><li>Model checking SA </li></ul></ul><ul><ul><li>SA and Performance Analysis </li></ul></ul><ul><ul><li>Deadlock detection on SA-based specification </li></ul></ul><ul><ul><li>SA and Security </li></ul></ul><ul><ul><li>Refining SA </li></ul></ul><ul><ul><li>SA-based development processes </li></ul></ul><ul><ul><li>ADL- based analysis of SA </li></ul></ul>
    8. 8. Testing <ul><li>Testing is the process of executing a program with the purpose of </li></ul><ul><ul><li>finding errors or defects </li></ul></ul><ul><ul><li>increasing the confidence in the proper functioning of the software (dependability). </li></ul></ul><ul><li>Not exaustive approach… </li></ul><ul><li>… based on the selection of an adeguate set of tests </li></ul>Unit Integration System Structural Functional Code Level Oriented to Syntax Unit Tests Functional properties Formal specifications Module Module Module M M M M
    9. 9. Architecture-based Integration and Conformance Testing <ul><li>Integration Testing: unit tested components are integrated to build complex systems, and </li></ul><ul><ul><li>Integration Testing is aimed at exposing problems in the interfaces and interactions between the system components </li></ul></ul><ul><li>Functional : focus on the functional requirements </li></ul><ul><li>Architectural : information for testing is extracted from the Architectural specification... </li></ul><ul><li>Down to the Code : … and propagated down to the Code... Conformance Testing </li></ul>
    10. 10. The main idea <ul><li>Goal : </li></ul><ul><ul><li>Derive test cases </li></ul></ul><ul><ul><li>using software architectural description </li></ul></ul><ul><ul><li>run test cases on the Code </li></ul></ul>ClientA ClientB ServerMaster Server Slave Recovery ClientC SA Code XClient ClientA ClientB ClientC Class Client ------- ------ ------ ------- Identify SA-Tests Apply the Tests Transform SA-Tests Into Code-level tests
    11. 11. Working Directions <ul><li>1 - SA-based Conformance Code Testing </li></ul><ul><li>(Joined work with Antonia Bertolino and Paola Inverardi) </li></ul><ul><li>2 - SA-based Regression Testing </li></ul><ul><ul><li>Testing C2-style architectures </li></ul></ul><ul><li> (Joined work with Marcio Dias and Debra Richardson) </li></ul><ul><li>3 - PLA-based Testing </li></ul><ul><li> (Joined work with Andre’ van der Hoek) </li></ul>
    12. 12. Step2 Abstraction step3 ALTS Model SA Specification Step1 modeling [ICECCS’97] [ICSE00] 1. SA-based Conformance Code Testing [PhDThesis, ch. 4] [BookChapter03] A B A B D A B C D Test Classes Test Execution Obj1 m1 Obj5 Obj3 Obj2 m6 m3 Obj N Obj k Obj 8 Obj L m1 m6 m3 m1 m2 m2 îe¤ íiÛ ìoö 1 act ï î ï íS ì 3 act { 2 act { 2 act step4 [ICSE01] LTS Model 20 19 6 4 1 14 4 1 12 9 1 13 3 2 20 19 5 3 2 11 7 2 4 12 1 9 6 1 11 3 2 7 5 2 6 4 1 5 3 2 1 0 8 7 6 5 3 2 1 0 4 29 26 2 28 22 20 12 10 13 11 3 9 36 35 33 32 34 15 18 17 16 31 14 13 16 37 29 28 30 24 34 27 33 22 A B C D Components: User, Router, Server, Timer User, Router, Server, Timer Connectors: AlarmUR, AlarmRS, AckSR, AlarmUR, AlarmRS, AckSR, AckRU, Check, Nofunc, Clock AckRU, Check, Nofunc, Clock Components Behavior: … , …, … … , …, … Connectors Behavior: … , …, ... … , …, ...
    13. 13. Step1: Formal SA specification <ul><li>Architectural Description Language (ADL) that produces a Labeled Transition System (LTS) </li></ul><ul><ul><ul><li>Chemical Abstract Machine ( CHAM ) [ICSE00] </li></ul></ul></ul><ul><ul><ul><li>Finite State Process ( FSP ) [ICSE01] </li></ul></ul></ul><ul><li>Dynamics in terms of Component interactions </li></ul>Molecule syntax: Molecule: Process| Operation| M.M Process: User, Router, Server, Timer Channel : check, alarmUR, alarmRS, ackRU, ackSR, nofunc, clock Operation: i(Channel), o(Channel), ... InitialState (S0): Multiset of Molecules: m1, m2, ..., mn Rules: m1, m2, ..., mk m1', m2', ..., ml'
    14. 14. The Architecture-level Behavior 20 19 11 10 6 3 6 16 14 12 6 3 6 3 4 5 3 9 7 5 11 7 6 11 9 6 5 4 3 12 7 5 4 11 7 6 4 9 7 6 5 20 19 6 4 1 14 4 1 12 9 1 13 3 2 20 19 5 3 2 11 7 2 12 5 4 3 11 6 4 3 9 6 5 3 6 5 4 7 4 12 1 9 6 1 11 3 2 7 5 2 6 5 4 3 6 4 1 5 3 2 2 1 0 10 11 8 7 6 5 4 3 2 1 0 9 4 29 26 2 28 22 20 12 10 13 11 3 9 36 35 33 32 34 15 18 17 16 31 14 13 12 16 205 143 90 51 37 22 21 20 29 28 27 26 25 24 23 30 19 25 24 21 23 20 22 19 24 34 27 33 22 44 43 42 48 46 41 40 39 38 43 40 41 42 39 38 45 43 38 49 72 71 159 107 67 107 117 77 69 68 111 113 107 112 109 70 43 23
    15. 15. <ul><li>Architectural Views [Kruchten] </li></ul><ul><ul><li>Flow view </li></ul></ul><ul><ul><li>Component Based view </li></ul></ul><ul><ul><li>Concurrency view </li></ul></ul><ul><ul><li>... </li></ul></ul><ul><li>Obs_Functions to view the system only from a </li></ul><ul><ul><li>perspective that is relevant for testing </li></ul></ul><ul><ul><li>Obs : L  D  {  } </li></ul></ul><ul><li>Abstract LTS (ALTS) </li></ul><ul><li>Minimization and Reduction (trace-equivalence) </li></ul>Step2 : The Abstraction
    16. 16. Flow view: ComponentBased View <ul><ul><li>Alarm Flow </li></ul></ul><ul><ul><li>Check Flow </li></ul></ul><ul><ul><li>Clock Flow </li></ul></ul><ul><ul><li>User Component </li></ul></ul><ul><ul><li>Router Comp. </li></ul></ul><ul><ul><li>Server Comp. </li></ul></ul>
    17. 17. Step3 - Architectural Test Class <ul><li>ALTS Path Coverage </li></ul><ul><li>Each complete path corresponds to an Architectural Test Class </li></ul><ul><ul><li>McCabe path coverage criteria </li></ul></ul><ul><ul><li>FSP hiding operators </li></ul></ul>...
    18. 18. Step4 - Testing the Software Implementation <ul><ul><li>How are the LTS paths implemented by the Code? </li></ul></ul><ul><ul><ul><li>We can test whether the system as-built implements this architectural behavior </li></ul></ul></ul>Comp1 Comp4 Comp3 Comp2 act1 act2 act3 act1 Obj1 m1 Obj5 Obj3 Obj2 m6 m3 Obj N Obj k Obj 8 Obj L m1 m6 m3 SA test Code test
    19. 19. Applications <ul><li>Tele Remote Medical Care System (TRMCS) [ICSE2000,ICSE2001] </li></ul><ul><li>The Whiteboard case study [BookChapter] </li></ul><ul><li>The Elevator case study [Submitted] </li></ul>
    20. 20. <ul><li>LTS can be automatically derived from an Architectural specification ( LTS generator Tool, LTSA Tool); </li></ul><ul><li>ALTS can be semi-automatically generated ( FC2Tools + The Abstractor Tool) </li></ul>Tool Support ALTS LTS generator LTSA tool LTS Abstractor SA formal specification LTS
    21. 21. <ul><li>Step1: </li></ul><ul><ul><li>SA specification and modelization </li></ul></ul><ul><ul><li>State explosion problem … completeness </li></ul></ul><ul><ul><li>Considerations: </li></ul></ul><ul><ul><ul><li>we used Cham and FSP </li></ul></ul></ul><ul><li>Step2: </li></ul><ul><ul><li>Observation and Abstraction </li></ul></ul><ul><ul><li>Considerations: </li></ul></ul><ul><ul><ul><li>It is an empirical task, based on the software architect experience </li></ul></ul></ul><ul><ul><ul><li>Classification of observations could help </li></ul></ul></ul><ul><ul><ul><li>Methods similar to SAAM or SCENT , in which architectural information is empirically captured, could help in this task </li></ul></ul></ul><ul><ul><ul><li>The ALTS construction has been automated using the C AESAR/ALDEBARAN tool </li></ul></ul></ul>Topics of interest and Considerations
    22. 22. <ul><li>Step3: </li></ul><ul><ul><li>Path coverage </li></ul></ul><ul><ul><li>Considerations: </li></ul></ul><ul><ul><ul><li>McCabe's test technique seems a reasonable coverage criterion </li></ul></ul></ul><ul><ul><ul><li>it may be automated </li></ul></ul></ul><ul><li>Step4: </li></ul><ul><ul><li>Traceability and development process </li></ul></ul><ul><ul><li>Deterministic and non deterministic Testing </li></ul></ul><ul><ul><li>Considerations: </li></ul></ul><ul><ul><ul><li>it is the most difficult part </li></ul></ul></ul><ul><ul><ul><li>relating SA specification to code </li></ul></ul></ul><ul><ul><ul><li>key concepts: traceability, development process </li></ul></ul></ul>Topics of interest and Considerations
    23. 23. Step4: Some Considerations <ul><ul><li>Mapping problems: </li></ul></ul><ul><li>It is not so obvious which classes and methods implement an architectural functionality </li></ul><ul><li>More sequences of method calls can implement the desired architectural behavior </li></ul><ul><li>SA description is abstract and hides functionalities and objects defined at the code level </li></ul><ul><li>The SA model usually describes only the expected behaviors, while the code also catches exceptional ones </li></ul><ul><li>Test execution: nondeterministic or deterministic [CarverTai_TSE98] approach </li></ul>
    24. 24. SA behavior Source Code class MasterRouter{ ServerConnection allarm; ServerConnection okFunction; // the services ReceiveUserAllarm serviceReceiveUserAllarm; ReceiveUserOkFunz serviceReceiveUserOkFunz; String name,serverName; static public PrintWriter user_router_ok_funz; static public PrintWriter user_router_alarm; ... ??? drives drives SA (topology and model) Design and Source Code Def. Source Code (abstractions) SA (model) SA (model) Source Code User2 User1 Router Alarm1 Alarm2 Ack2 Ack1
    25. 25. <ul><li>Motivations and Goals: </li></ul><ul><ul><li>To reduce the testing effort during architecture or code maintenance </li></ul></ul><ul><ul><li>To handle the “ architectural drift ” </li></ul></ul><ul><ul><li>To maximize benefits in using Software Architecture </li></ul></ul>2. SA-based Regression Testing [Submitted]
    26. 26. <ul><li>Test modified software and provide a certain confidence that no new errors are introduced into previously tested code . </li></ul><ul><li>Given program P and P’, T is a test suite for P, regression testing techniques: </li></ul><ul><ul><li>provide a certain confidence that P’ is still correct with respect to a set of test T’, which is a subset of T; </li></ul></ul><ul><ul><li>Helps to identify new test cases for T’. </li></ul></ul>Regression Testing
    27. 27. <ul><li>Select T’, subset of T and relevant for P’; </li></ul><ul><li>Test P’ with respect to T’; </li></ul><ul><li>If necessary, create T’’, to test new functionality/structure in P’; </li></ul><ul><li>Test P’ with respect to T’’; </li></ul><ul><li>Create T’’’, a new test suite and test history; </li></ul>Regression Test Selection technique
    28. 28. Project Goal
    29. 29. From Traditional to SA-based Regression Testing
    30. 30. Initial experiment <ul><li>The approach has been applied to the Elevator case study </li></ul><ul><ul><li>Architecture in the C2 style </li></ul></ul><ul><ul><li>Implemented using the C2 framework </li></ul></ul><ul><li>Tool support: </li></ul><ul><ul><li>The SA is formally specified using the C2 specification language and FSP . </li></ul></ul><ul><ul><li>A behavioral model of the system is automatically produced from the FSP specification, using the LTSA tool. </li></ul></ul><ul><ul><li>The abstraction over the LTS behavioral model is realized again in FSP . </li></ul></ul><ul><ul><li>The mapping between architectural tests and code-level tests is provided for by the C2 Framework . </li></ul></ul><ul><ul><li>Test cases are run over the code using the Argus-I environment. </li></ul></ul><ul><ul><li>The selective regression testing step is supported by the DejaVOO tool. </li></ul></ul>
    31. 31. <ul><li>A product line architecture precisely captures, in a single specification, the overall architecture of a suite of closely-related products [Bosch2000] </li></ul><ul><li>A PLA explicitly specifies: </li></ul><ul><ul><li>i ) elements that are present in all products , </li></ul></ul><ul><ul><li>ii ) elements that are optional , and </li></ul></ul><ul><ul><li>iii ) elements which may be incorporated in one of many forms ( variants ) </li></ul></ul><ul><li>Whereas a “regular” architecture defines the structure of a single product, a product line architecture (PLA) defines the common architecture for a set of related products [Bosch2000] </li></ul>3. PLA-based Testing [Tacos 2003]
    32. 32. Testing PLA <ul><li>A new challenge: </li></ul><ul><ul><li>how to deal with optional elements or with the magnitude of products that may be present? </li></ul></ul><ul><li>The goal of this paper is to highlight the challenges and opportunities for software testing of PLAs </li></ul><ul><li>We believe that existing mechanisms with which SAs are tested can be adapted to PLAs </li></ul>
    33. 33. PLA example mandatory optional variant variant In total, twenty-four different product architectures can be formed. foo bar bar goop bar foobar
    34. 34. Testing PLA- Unit Testing <ul><li>SA: </li></ul><ul><ul><li>Each SA component need to be unit tested </li></ul></ul><ul><li>PLA: </li></ul><ul><ul><li>all components should be unit tested as well </li></ul></ul><ul><ul><ul><li>Including each optional component and each variant of a variant component </li></ul></ul></ul><ul><ul><li>However,the order in which they have to be tested can be adjusted based on “ priority ”. </li></ul></ul>
    35. 35. Testing PLA – Integration Testing <ul><li>SA: </li></ul><ul><ul><li>components and connectors are combined together according to the architecture configuration </li></ul></ul><ul><li>PLA: </li></ul><ul><ul><li>no single architectural configuration exists </li></ul></ul><ul><ul><li>Possible solutions: </li></ul></ul><ul><ul><ul><li>Iterative build-up integration approach: powerful but expensive </li></ul></ul></ul><ul><ul><ul><li>big bang : limited but “effortless” </li></ul></ul></ul><ul><ul><li>Core-first + big bang approach </li></ul></ul><ul><ul><ul><li>Integrated with “heuristics” to test only particular combinations </li></ul></ul></ul>
    36. 36. Testing PLA – Conformance Testing <ul><li>SA: </li></ul><ul><ul><li>conformance testing has been used to detect conformance errors between the SA and its implementation . </li></ul></ul><ul><li>PLA: </li></ul><ul><ul><li>an implementation I conforms to its PLA when : </li></ul></ul><ul><ul><ul><li>I conforms to a single PA </li></ul></ul></ul><ul><ul><ul><li>I conforms to all the possible PAs out of the PLA </li></ul></ul></ul><ul><ul><ul><li>I conforms, at least, to all the constraints and functionalities associated to the mandatory, core elements of the PLA </li></ul></ul></ul><ul><ul><li>a product architecture PA conforms to its PLA </li></ul></ul>
    37. 37. Testing PLA – Regression Testing <ul><li>SA: </li></ul><ul><ul><li>If a new version P1 v2 of an implementation P1 v1 is produced, regression testing techniques can be used to test the conformance of P1’ to the initial SA </li></ul></ul><ul><ul><li>If a new version (SA v2 ) of an architecture SA v1 is produced, SA v2 test cases may be selected reusing SA v1 ’s test cases. </li></ul></ul><ul><li>PLA: </li></ul><ul><ul><li>PLA evolution : PLA that becomes PLA’ </li></ul></ul><ul><ul><li>PA becomes PA’ </li></ul></ul><ul><ul><li>Code becomes Code’ </li></ul></ul>
    38. 38. Future Work SA-based Code Testing
    39. 39. <ul><li>SA-based Conformance Code Testing </li></ul><ul><ul><li>Testing and Formal Methods </li></ul></ul><ul><ul><li>Integration with the TGV [TGV] (Test Generation and Validation) environment used to test formal specifications </li></ul></ul><ul><ul><li>Testing C2 style architectures </li></ul></ul><ul><ul><li>C2 framework and Dradel [ongoing work] </li></ul></ul><ul><ul><li>Integration with XADL, Menage, Behavioral model </li></ul></ul><ul><li>SA-based Regression Testing </li></ul><ul><ul><li>Implement the second goal </li></ul></ul><ul><li>PLA-based Testing </li></ul><ul><ul><li>Introduce an approach for PLA-based testing </li></ul></ul><ul><ul><li>Integration with XADL, Menage, Behavioral model </li></ul></ul>
    40. 40. Integration with XADL, Menage, Behavioral model <ul><li>SA specified in XADL </li></ul><ul><ul><li>with an additional behavioral specification </li></ul></ul><ul><li>The XADL architecture is implemented by using the new C2 Framework </li></ul><ul><ul><li>May be something different </li></ul></ul><ul><li>Menage features are used to help the regression testing steps </li></ul><ul><li>The testing capability is embedded into Menage </li></ul>
    41. 41. Some References <ul><li>[ICSE2000] A. Bertolino, F. Corradini, P. Inverardi and H. Muccini. &quot;Deriving Test Plans from Architectural Descriptions&quot;. In ACM Int. Conf. on Software Engineering (ICSE2000) , pp. 220-229, Limerick (Ireland), June 2000. </li></ul><ul><li>[ICSE2001] A. Bertolino, P. Inverardi and H. Muccini. &quot;An Explorative Journey from Architectural Tests Definition downto Code Tests Execution&quot;. In IEEE Int. Conf. on Software Engineering (ICSE2001) , Toronto, May 2001. </li></ul><ul><li>[PhDThesis] H. Muccini. Software Architecture for Testing, Coordination Models and Views Model Checking, PhD thesis, year 2002. </li></ul><ul><li>[Tacos 2003] H. Muccini, A. van der Hoek. &quot;Towards Testing Product Line Architectures“ In Proc. of the ETAPS2003 workshop on &quot;Test and Analysis of Component Based Systems&quot; (Tacos), Warsaw, Poland, April 2003. In Electronic Notes of Theoretical Computer Science, Vol. 82, N. 6 (2003). </li></ul><ul><li>[BookChapter] A. Bertolino, P. Inverardi and H. Muccini. “Formal Methods in Testing Software Architectures”. Chapter in Formal Methods for Software Architectures, SFM-03:SA Lectures, Eds. M. Bernardo, P. Inverardi, LNCS 2804, 2003, p. 124-149. </li></ul><ul><li>[Submitted] H. Muccini, M. Dias and D. Richardson. Software Architecture-based Conformance and Regression Testing. Submitted for publication. </li></ul><ul><li>[ongoing work] H. Muccini, M. Dias. Systematic Testing of C2 style Software Architecture. Under Submission for publication. </li></ul><ul><li>[CarverTai_TSE98] Carver, R.H., Tai, K.-C.Use of Sequencing Constraints for Specification-Based Testing of Concurrent Programs. IEEE Trans. on Software Engineering 24, 6 (June 1998). </li></ul><ul><li>[Kruchten] P. Kruchten. Architectural Blueprints - The “4+1” View Model of Software Architecture. IEEE Software 12 (6), November 1995, pp. 42-50. </li></ul><ul><li>[TGV] Fernandez, J.-C., Jard, C., Jeron, T., Nedelka, L., Viho, C.: An Experiment in Automatic Generation of Test Suites for Protocols with Verification Technology. Special Issue of Science of Computer Programming , Vol. 29, pp. 123-146, 1997. </li></ul>