INTEGRATION AND SYSTEM TESTING  CHAPTER 12 & 13 Csci565 Spring 2009 H.Reza
OBJECTIVES <ul><li>Integration Testing </li></ul><ul><li>Simple ATM (SATM) </li></ul><ul><li>discuss integration testing s...
A SOFTWARE TESTING STRATEGY: SPIRAL MODEL H.Reza System Testing System eng. validation Testing Requirements Integration Te...
INTEGRATION TESTING:1 <ul><li>If all units/components work individually, do they work as a whole when we put them together...
PROBLEMS WITH INTERFACING <ul><li>Integration faults often traceable to incomplete or misunderstood interface specificatio...
INTEGRATION TESTING <ul><li>Tests complete systems or subsystems composed of integrated components </li></ul><ul><li>Integ...
APPROACHES TO INTEGRATION TESTING <ul><li>Two major approaches </li></ul><ul><ul><li>Incremental approaches  </li></ul></u...
INCREMENTAL INTEGRATION TESTING H.Reza
INCREMENTAL APPROACHES: TOP-DOWN <ul><li>Top-down testing </li></ul><ul><ul><li>Start with high-level system  </li></ul></...
STUBS <ul><li>Stubs </li></ul><ul><ul><li>Special  module to simulate some functionality </li></ul></ul><ul><ul><li>Its pr...
TOP-DOWN TESTING H.Reza
TOP-DOWN: COMPLICATIONS <ul><li>The most common complication occurs  when processing at low level hierarchy demands  adequ...
INCREMENTAL TESTING: BOTTOM UP <ul><li>Bottom-up testing </li></ul><ul><ul><li>Integrate individual components in levels u...
BOTTOM-UP APPROACH <ul><li>Starts with construction and testing with atomic modules </li></ul><ul><ul><li>No need for stub...
BOTTOM-UP TESTING H.Reza M
H.Reza A C E B D F J G H I K L
H.Reza A Stub C Stub E B Stub D Stub F second state in the top-down
H.Reza A Stub C Stub E B D F J Stub H I Intermediate state in the top-down
TOP-DOWN <ul><li>Possible Sequences of modules </li></ul><ul><ul><li>A, B, C, D, E, F, G, H, I, J, K, L </li></ul></ul><ul...
GUIDELINE FOR INTEGRATION TESTING <ul><li>Integrate the components that implement the most frequently used functionality <...
NON-INCREMENTAL <ul><li>Big-bang </li></ul><ul><ul><li>Imposes no order  (GOOD) </li></ul></ul><ul><ul><li>Test all the un...
INTEGRATION TEST DOCUMENT <ul><li>Overall plan for integration of the system under construction must be documented in a Te...
INTEGRATION: CRITERIA <ul><li>The following criteria </li></ul><ul><ul><li>Interface integrity  </li></ul></ul><ul><ul><ul...
TOP-DOWN VS. BOTTOM-UP <ul><li>Architectural validation </li></ul><ul><ul><li>Top-down integration testing is better at di...
THE WATERFALL  LIFE CYCLE H.Reza Requirement specifications Preliminary design details design coding Unit testing Integrat...
INTEGRATION TESTING: SOFTWARE ARCHITECTURE <ul><li>Integration testing </li></ul><ul><ul><li>How  software architecture ca...
FIG. 12.2: PRIMARY DESIGN (OR  INFORMAL SOFTWARE ARCHITECTURE)  OF THE ATM USING TREE-BASED DECOMPOSITION H.Reza Requireme...
MORE ON PRIMARY DESIGN <ul><li>How do perform Integration testing for  non-tree  based functional decomposition? </li></ul...
SIMPLE  ATM (SATM) <ul><li>An ATM simple </li></ul><ul><ul><li>Provides  15  screens for interactions </li></ul></ul><ul><...
FIGURE 12.7 H.Reza
FIGURE 12.8 H.Reza
FIGURE 12.9 H.Reza
FIGURE 12.10 H.Reza
FIGURE 12.11 H.Reza
FIGURE 12.12 H.Reza
FIGURE 12.13 H.Reza
DECOMPOSITION BASED STRATEGIES <ul><li>Decomposition based </li></ul><ul><ul><li>Top/down </li></ul></ul><ul><ul><li>Botto...
FIGURE 12.14 H.Reza
FIGURE 13.1 H.Reza
DECOMPOSITION BASED TESTING:1 <ul><li>Discussion revolves around the tree-based decomposition and the order by which units...
TEST SESSIONS <ul><li>A test session refers to  one set of tests  for a specific configuration of actual code and stubs </...
DECOMPOSITION BASED TESTING: 2 <ul><li>For SATM system </li></ul><ul><ul><li>42 integration testing session (i.e., 42 sepa...
DECOMPOSITION BASED STRATEGIES: PROS AND CON <ul><li>Intuitively clear and understandable  </li></ul><ul><ul><li>In case o...
CALL GRAPH BASED INTEGRATION TESTING <ul><li>Call graph </li></ul><ul><ul><li>A directed graph </li></ul></ul><ul><ul><li>...
CALL GRAPH (CG): APPROACHES <ul><li>Two main approaches  based on Call Graph  </li></ul><ul><ul><li>Pair-wise integration ...
FIGURE 13.2: SATM CALL GRAPH H.Reza
TABLE 2: AM H.Reza
PAIR-WISE INTEGRATION <ul><li>The main idea is to eliminate the overhead (i.e., stub/drive) </li></ul><ul><li>Uses actual ...
PAIR-WISE INTEGRATION H.Reza - Uses actual code -one integration test session for each edge -40 edges for SATM
NEIGHBORHOOD INTEGRATION <ul><li>The neighborhood of a node refers to the nodes that are one edge away from the given node...
NEIGHBORHOOD INTEGRATION H.Reza
TABLE 3: SATM NEIGHBORHOODS H.Reza
PROS AND CONS <ul><li>Benefits ( GOOD ) </li></ul><ul><ul><li>Mostly behavioral than structural </li></ul></ul><ul><ul><li...
PATH-BASED INTEGRATION TESTING  <ul><li>The hybrid approach (i.e., structural and behavioral) is an ideal one integration ...
NEW AND EXTENDED CONCEPTS <ul><li>Source node (begin) </li></ul><ul><ul><li>A statement fragment at which program executio...
MORE ON CONCEPTS <ul><li>Module Execution Path (MEP) </li></ul><ul><ul><li>A sequence of statements that  begins with a so...
THE PATH BASED INTEGRATION TESTING (DEFINITION) <ul><li>MM-Path (definition) </li></ul><ul><ul><li>An interleaved sequence...
FIGURE 13.3 H.Reza
H.Reza There are seven module execution paths (MEP):
MM-PATH GRAPH (DEFINITION) <ul><li>Given a set of units, their MM-Path graph is the directed graph in which nodes are modu...
FIGURE 13.4 H.Reza
FIGURE 13.5 H.Reza
PDL DESCRIPTION OF SATM H.Reza
H.Reza
H.Reza
MORE ON  MM-PATH <ul><li>MM-Path issues </li></ul><ul><ul><li>How long is an MM-Path? </li></ul></ul><ul><ul><li>What is t...
MM-PATH GUIDELINES <ul><li>MM-Path Guiltiness </li></ul><ul><ul><li>Points of quiescence are natural endpoints for an MM-p...
PROS AND CONS <ul><li>hybrid approach( GOOD )  </li></ul><ul><li>The approach works equally well for software testing deve...
SYSTEM TESTING <ul><li>Closely related to everyday expertise </li></ul><ul><li>The goal is to Test the quality (not specif...
SOFTWARE  ARCHITECTURE & TESTING <ul><li>Using Traditional approach formalizing and  automating  the integration test stag...
SA-BASED APPROACH <ul><li>SA </li></ul><ul><ul><li>Used for prediction of the system-level quality </li></ul></ul><ul><li>...
THE ADVANTAGES OF USING THE  SATO DERIVE THE AUTOMATON <ul><li>for state transition testing are evident:  </li></ul><ul><u...
ABOUT SURVEY PAPER <ul><li>Example of abstract for survey paper: </li></ul><ul><ul><li>In this report (or paper), we surve...
Upcoming SlideShare
Loading in...5
×

Integration testing(revisited)

1,637

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,637
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
86
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Integration testing(revisited)"

  1. 1. INTEGRATION AND SYSTEM TESTING CHAPTER 12 & 13 Csci565 Spring 2009 H.Reza
  2. 2. OBJECTIVES <ul><li>Integration Testing </li></ul><ul><li>Simple ATM (SATM) </li></ul><ul><li>discuss integration testing strategies </li></ul><ul><ul><li>Top-down </li></ul></ul><ul><ul><li>Bottom-up </li></ul></ul><ul><ul><li>Decomposition Based Integration Testing (DBIT) </li></ul></ul><ul><ul><li>Call Graph Based Integration Testing (CGBIT) </li></ul></ul><ul><ul><li>Path-Based Integration Testing (PBIT) </li></ul></ul>H.Reza
  3. 3. A SOFTWARE TESTING STRATEGY: SPIRAL MODEL H.Reza System Testing System eng. validation Testing Requirements Integration Testing Design Unit Testing Code
  4. 4. INTEGRATION TESTING:1 <ul><li>If all units/components work individually, do they work as a whole when we put them together? </li></ul><ul><ul><li>Not necessarily </li></ul></ul><ul><li>The problem is “ putting them together ” or interfacing them </li></ul>H.Reza
  5. 5. PROBLEMS WITH INTERFACING <ul><li>Integration faults often traceable to incomplete or misunderstood interface specifications </li></ul><ul><ul><li>mismatched assumptions about other components </li></ul></ul><ul><ul><li>Individually acceptable imprecision may be magnified to unacceptable levels </li></ul></ul><ul><ul><li>Global data structures can present problems </li></ul></ul><ul><li>Inconsistent interpretation of parameters or values </li></ul><ul><li>Mixed units (meters/yards) in Martian Lander </li></ul><ul><li>Violations of value domains, capacity, or size limits </li></ul><ul><li>… </li></ul>H.Reza
  6. 6. INTEGRATION TESTING <ul><li>Tests complete systems or subsystems composed of integrated components </li></ul><ul><li>Integration testing should be black-box testing when tests derived from the specification </li></ul><ul><li>Main difficulty is localising errors </li></ul><ul><li>Incremental integration testing reduces this problem </li></ul>H.Reza
  7. 7. APPROACHES TO INTEGRATION TESTING <ul><li>Two major approaches </li></ul><ul><ul><li>Incremental approaches </li></ul></ul><ul><ul><ul><li>The decomposition-based techniques or tree </li></ul></ul></ul><ul><ul><ul><ul><li>Stubs/Drivers </li></ul></ul></ul></ul><ul><ul><ul><li>Call Graph-based techniques </li></ul></ul></ul><ul><ul><ul><ul><li>No stubs or drivers </li></ul></ul></ul></ul><ul><ul><li>Non-incremental approaches </li></ul></ul>H.Reza
  8. 8. INCREMENTAL INTEGRATION TESTING H.Reza
  9. 9. INCREMENTAL APPROACHES: TOP-DOWN <ul><li>Top-down testing </li></ul><ul><ul><li>Start with high-level system </li></ul></ul><ul><ul><li>integrate from the top-down replacing individual components by stubs where appropriate </li></ul></ul><ul><ul><ul><li>Depth-first </li></ul></ul></ul><ul><ul><ul><li>Breadth-first </li></ul></ul></ul><ul><ul><ul><li>No-best order </li></ul></ul></ul><ul><ul><ul><ul><li>Critical sections </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Early skeletal version using I/O modules </li></ul></ul></ul></ul>H.Reza
  10. 10. STUBS <ul><li>Stubs </li></ul><ul><ul><li>Special module to simulate some functionality </li></ul></ul><ul><ul><li>Its production is nontrivial task because the code may simulate a very complicated tasks </li></ul></ul><ul><ul><ul><li>E.g. </li></ul></ul></ul><ul><ul><ul><ul><li>Writing a stub performing a database table search routine </li></ul></ul></ul></ul><ul><ul><li>Creating multiple version of same stub for various reasons </li></ul></ul>H.Reza
  11. 11. TOP-DOWN TESTING H.Reza
  12. 12. TOP-DOWN: COMPLICATIONS <ul><li>The most common complication occurs when processing at low level hierarchy demands adequate testing of upper level </li></ul><ul><li>To overcome: </li></ul><ul><ul><li>Either, delay many tests until stubs are replaced with actual modules (BAD) </li></ul></ul><ul><ul><li>Or, develop stubs that perform limited functions that simulate the actual module ( GOOD) </li></ul></ul><ul><ul><li>Or, Integrate the software using bottom up approach </li></ul></ul><ul><li>Confusion about overlapping with design </li></ul>H.Reza
  13. 13. INCREMENTAL TESTING: BOTTOM UP <ul><li>Bottom-up testing </li></ul><ul><ul><li>Integrate individual components in levels until the complete system is created </li></ul></ul>H.Reza
  14. 14. BOTTOM-UP APPROACH <ul><li>Starts with construction and testing with atomic modules </li></ul><ul><ul><li>No need for stub </li></ul></ul><ul><ul><li>Low level components are combined into cluster (or builds ) to perform a specific sub-function </li></ul></ul><ul><ul><li>A driver (a control program for testing) is written </li></ul></ul><ul><ul><ul><li>Contain hardcoded test input, calls the module being tested, and display the results </li></ul></ul></ul><ul><ul><li>Cluster is tested </li></ul></ul><ul><ul><li>Drivers are removed and clusters are combined moving upward in the program structure </li></ul></ul>H.Reza
  15. 15. BOTTOM-UP TESTING H.Reza M
  16. 16. H.Reza A C E B D F J G H I K L
  17. 17. H.Reza A Stub C Stub E B Stub D Stub F second state in the top-down
  18. 18. H.Reza A Stub C Stub E B D F J Stub H I Intermediate state in the top-down
  19. 19. TOP-DOWN <ul><li>Possible Sequences of modules </li></ul><ul><ul><li>A, B, C, D, E, F, G, H, I, J, K, L </li></ul></ul><ul><ul><li>A, B, E, F, J, C, G, K, D, H, L, I </li></ul></ul><ul><ul><li>A, D, H, I, K, L, C, G, B, F, J, E </li></ul></ul><ul><ul><li>A, B, F, J, D, I, E, C, G,K, H, L </li></ul></ul><ul><li>If parallel testing allowed, then other alternatives are possible </li></ul><ul><ul><li>After A has been tested, one programmer could take A and test the combination A-B, </li></ul></ul><ul><ul><li>Another programmer could test A-C </li></ul></ul>H.Reza
  20. 20. GUIDELINE FOR INTEGRATION TESTING <ul><li>Integrate the components that implement the most frequently used functionality </li></ul><ul><li>Perform regression testing for existing features </li></ul><ul><li>Perform progression testing for new features </li></ul>H.Reza
  21. 21. NON-INCREMENTAL <ul><li>Big-bang </li></ul><ul><ul><li>Imposes no order (GOOD) </li></ul></ul><ul><ul><li>Test all the units (Modules) at once </li></ul></ul><ul><ul><li>Very easy, but difficult to localize the source of errors </li></ul></ul>H.Reza
  22. 22. INTEGRATION TEST DOCUMENT <ul><li>Overall plan for integration of the system under construction must be documented in a Test Specification </li></ul><ul><li>The test plan should describe the overall strategy for integration </li></ul><ul><ul><li>Example of phases </li></ul></ul><ul><ul><ul><li>User interaction (menu, button, forms, display presentation) </li></ul></ul></ul><ul><ul><ul><li>Data Manipulation and analysis </li></ul></ul></ul><ul><ul><ul><li>Display processing and generation (RT-2D, RT-3D, etc) </li></ul></ul></ul><ul><ul><ul><li>Database Mgt </li></ul></ul></ul><ul><ul><ul><li>Logic?? </li></ul></ul></ul>H.Reza
  23. 23. INTEGRATION: CRITERIA <ul><li>The following criteria </li></ul><ul><ul><li>Interface integrity </li></ul></ul><ul><ul><ul><li>(internal and external interfaces are tested as each module (or cluster) is incorporated </li></ul></ul></ul><ul><ul><li>Functional validity </li></ul></ul><ul><ul><ul><li>Testes to uncover functional error </li></ul></ul></ul><ul><ul><li>Information validity </li></ul></ul><ul><ul><ul><li>Tests to uncover error related to local/ global data </li></ul></ul></ul><ul><ul><li>Performance (Quality) </li></ul></ul><ul><ul><ul><li>Tests designed to verify performance bounds during software design </li></ul></ul></ul>H.Reza
  24. 24. TOP-DOWN VS. BOTTOM-UP <ul><li>Architectural validation </li></ul><ul><ul><li>Top-down integration testing is better at discovering errors in the system architecture </li></ul></ul><ul><li>System demonstration </li></ul><ul><ul><li>Top-down integration testing allows a limited demonstration at an early stage in the development </li></ul></ul><ul><li>Test implementation </li></ul><ul><ul><li>Often easier with bottom-up integration testing </li></ul></ul>H.Reza
  25. 25. THE WATERFALL LIFE CYCLE H.Reza Requirement specifications Preliminary design details design coding Unit testing Integration testing System testing
  26. 26. INTEGRATION TESTING: SOFTWARE ARCHITECTURE <ul><li>Integration testing </li></ul><ul><ul><li>How software architecture can be used to conduct tests to uncover errors related to the interface? </li></ul></ul>H.Reza
  27. 27. FIG. 12.2: PRIMARY DESIGN (OR INFORMAL SOFTWARE ARCHITECTURE) OF THE ATM USING TREE-BASED DECOMPOSITION H.Reza Requirement specifications Terminal I/O Mange Session Conduct Transactions Card Entry PIN Entry Select Transaction
  28. 28. MORE ON PRIMARY DESIGN <ul><li>How do perform Integration testing for non-tree based functional decomposition? </li></ul><ul><ul><li>E.g </li></ul></ul><ul><ul><ul><li>integration testing for OO </li></ul></ul></ul><ul><ul><ul><li>Integration testing for Client/server systems </li></ul></ul></ul><ul><ul><ul><li>Integration testing for Layered systems </li></ul></ul></ul><ul><ul><ul><li>… . </li></ul></ul></ul>H.Reza
  29. 29. SIMPLE ATM (SATM) <ul><li>An ATM simple </li></ul><ul><ul><li>Provides 15 screens for interactions </li></ul></ul><ul><ul><li>includes 3 function buttons </li></ul></ul><ul><ul><li>Modeled in structural analysis </li></ul></ul><ul><ul><ul><li>Data Model (ERD) </li></ul></ul></ul><ul><ul><ul><li>Functional Model (DFD) </li></ul></ul></ul><ul><ul><ul><li>Behavioral model (STD) </li></ul></ul></ul>H.Reza
  30. 30. FIGURE 12.7 H.Reza
  31. 31. FIGURE 12.8 H.Reza
  32. 32. FIGURE 12.9 H.Reza
  33. 33. FIGURE 12.10 H.Reza
  34. 34. FIGURE 12.11 H.Reza
  35. 35. FIGURE 12.12 H.Reza
  36. 36. FIGURE 12.13 H.Reza
  37. 37. DECOMPOSITION BASED STRATEGIES <ul><li>Decomposition based </li></ul><ul><ul><li>Top/down </li></ul></ul><ul><ul><li>Bottom up </li></ul></ul><ul><ul><li>Sandwich </li></ul></ul><ul><ul><li>Big bang </li></ul></ul>H.Reza
  38. 38. FIGURE 12.14 H.Reza
  39. 39. FIGURE 13.1 H.Reza
  40. 40. DECOMPOSITION BASED TESTING:1 <ul><li>Discussion revolves around the tree-based decomposition and the order by which units are tested and combined </li></ul><ul><ul><li>Top-to-bottom </li></ul></ul><ul><ul><li>Bottom-to-top </li></ul></ul><ul><ul><li>Sandwich </li></ul></ul><ul><ul><li>Big bang </li></ul></ul><ul><li>The focus is on the structural compatibility among interfaces </li></ul>H.Reza
  41. 41. TEST SESSIONS <ul><li>A test session refers to one set of tests for a specific configuration of actual code and stubs </li></ul><ul><li>The number of integration test sessions using a decomposition tree can be computed </li></ul><ul><ul><li>Sessions=nodes – leaves + edges </li></ul></ul>H.Reza
  42. 42. DECOMPOSITION BASED TESTING: 2 <ul><li>For SATM system </li></ul><ul><ul><li>42 integration testing session (i.e., 42 separate sets of integration test cases) </li></ul></ul><ul><ul><ul><li>top/down </li></ul></ul></ul><ul><ul><ul><ul><li>(Nodes-1) stubs are needed </li></ul></ul></ul></ul><ul><ul><ul><ul><li>32 stub in SATM </li></ul></ul></ul></ul><ul><ul><ul><li>bottom/up </li></ul></ul></ul><ul><ul><ul><ul><li>(Nodes-leaves) of drivers are needed </li></ul></ul></ul></ul><ul><ul><ul><ul><li>10 drivers in SATM </li></ul></ul></ul></ul>H.Reza
  43. 43. DECOMPOSITION BASED STRATEGIES: PROS AND CON <ul><li>Intuitively clear and understandable </li></ul><ul><ul><li>In case of faults, most recently added units are suspected ones </li></ul></ul><ul><li>Can be tracked against decomposition tree </li></ul><ul><li>Suggests breadth-first or depth-first traversals </li></ul><ul><li>Units are merged using the decomposition tree </li></ul><ul><ul><li>Correct behavior follows from individually correct units and interfaces </li></ul></ul><ul><ul><li>Stubs/Drives are major development Overhead </li></ul></ul>H.Reza
  44. 44. CALL GRAPH BASED INTEGRATION TESTING <ul><li>Call graph </li></ul><ul><ul><li>A directed graph </li></ul></ul><ul><ul><li>Nodes corresponds to unit </li></ul></ul><ul><ul><li>Edges corresponds to the call </li></ul></ul><ul><ul><li>E.g. </li></ul></ul><ul><ul><ul><li>A  B (i.e., A is calling B) </li></ul></ul></ul><ul><li>Attempts to overcome the decomposition problem (structural) </li></ul><ul><li>Moves toward behavioral testing </li></ul>H.Reza
  45. 45. CALL GRAPH (CG): APPROACHES <ul><li>Two main approaches based on Call Graph </li></ul><ul><ul><li>Pair-wise integration </li></ul></ul><ul><ul><li>Neighborhood integration </li></ul></ul>H.Reza
  46. 46. FIGURE 13.2: SATM CALL GRAPH H.Reza
  47. 47. TABLE 2: AM H.Reza
  48. 48. PAIR-WISE INTEGRATION <ul><li>The main idea is to eliminate the overhead (i.e., stub/drive) </li></ul><ul><li>Uses actual code by restricting a session testing to a pair of units in the Call Graph </li></ul><ul><ul><li>One integration test for each edge in CG </li></ul></ul><ul><ul><li>40 edges means 40 integration tests for the SATM </li></ul></ul>H.Reza
  49. 49. PAIR-WISE INTEGRATION H.Reza - Uses actual code -one integration test session for each edge -40 edges for SATM
  50. 50. NEIGHBORHOOD INTEGRATION <ul><li>The neighborhood of a node refers to the nodes that are one edge away from the given nodes </li></ul><ul><ul><li>SATM Neighborhoods </li></ul></ul><ul><ul><li>Number of neighborhoods can be computed </li></ul></ul><ul><ul><ul><li>Neighborhoods = nodes – sink nodes </li></ul></ul></ul><ul><ul><ul><li>Or the number of interior-nodes + X ( x=1 if there exists leaf nodes connected directly to the root node otherwise X= 0) </li></ul></ul></ul><ul><li>Results a drastic reduction in the number of integration test session </li></ul><ul><ul><li>In case of SATM (11 vs. 40) </li></ul></ul>H.Reza
  51. 51. NEIGHBORHOOD INTEGRATION H.Reza
  52. 52. TABLE 3: SATM NEIGHBORHOODS H.Reza
  53. 53. PROS AND CONS <ul><li>Benefits ( GOOD ) </li></ul><ul><ul><li>Mostly behavioral than structural </li></ul></ul><ul><ul><li>Eliminates sub/drive overhead </li></ul></ul><ul><ul><li>Works well with incremental development method such as Build and composition </li></ul></ul><ul><li>Liabilities ( BAD ) </li></ul><ul><ul><li>The fault isolation </li></ul></ul><ul><ul><li>E.g., </li></ul></ul><ul><ul><ul><li>Fault in one node appearing in several neighborhood </li></ul></ul></ul>H.Reza
  54. 54. PATH-BASED INTEGRATION TESTING <ul><li>The hybrid approach (i.e., structural and behavioral) is an ideal one integration testing </li></ul><ul><li>The focus is on the interactions among the units </li></ul><ul><ul><li>Interfaces are structural </li></ul></ul><ul><ul><li>Interactions are behavioral </li></ul></ul><ul><li>With unit testing, some path of source statements is traversed </li></ul><ul><ul><li>What happens when there is a call to another unit? </li></ul></ul><ul><ul><ul><li>Ignore the single-entry/single-exit </li></ul></ul></ul><ul><ul><ul><li>Use exist follows by an entry </li></ul></ul></ul><ul><ul><ul><li>Suppress the call statement </li></ul></ul></ul>H.Reza
  55. 55. NEW AND EXTENDED CONCEPTS <ul><li>Source node (begin) </li></ul><ul><ul><li>A statement fragment at which program execution begins or resumes </li></ul></ul><ul><ul><li>E.g., BEGIN </li></ul></ul><ul><li>Sink node (end) </li></ul><ul><ul><li>A statement fragment at which program execution terminates </li></ul></ul><ul><ul><li>E.g., Final END </li></ul></ul>H.Reza
  56. 56. MORE ON CONCEPTS <ul><li>Module Execution Path (MEP) </li></ul><ul><ul><li>A sequence of statements that begins with a source node and ends with a sink node , with no intervening sink nodes </li></ul></ul><ul><li>The implication of this definition is that program graph (PG) may have multiple source/sink nodes </li></ul><ul><li>Message </li></ul><ul><ul><li>A programming language mechanism by which one unit transfers control to another unit </li></ul></ul><ul><ul><ul><li>E.g., </li></ul></ul></ul><ul><ul><ul><ul><li>subroutine invocations </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Procedure calls </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Function references </li></ul></ul></ul></ul>H.Reza
  57. 57. THE PATH BASED INTEGRATION TESTING (DEFINITION) <ul><li>MM-Path (definition) </li></ul><ul><ul><li>An interleaved sequence of module execution paths (MEP) and messages </li></ul></ul><ul><li>MM-Path can be used </li></ul><ul><ul><li>to describe sequences of module execution paths including transfers of control among units using messages </li></ul></ul><ul><ul><li>To represents feasible execution paths that cross unit boundaries </li></ul></ul><ul><ul><li>To extent program graph </li></ul></ul><ul><ul><ul><li>Where </li></ul></ul></ul><ul><ul><ul><ul><li>nodes = execution paths </li></ul></ul></ul></ul><ul><ul><ul><ul><li>edges = messages </li></ul></ul></ul></ul><ul><li>Atomic system function (ASF) </li></ul><ul><ul><li>An action that is observable at the system level in terms of port input and output events </li></ul></ul>H.Reza
  58. 58. FIGURE 13.3 H.Reza
  59. 59. H.Reza There are seven module execution paths (MEP):
  60. 60. MM-PATH GRAPH (DEFINITION) <ul><li>Given a set of units, their MM-Path graph is the directed graph in which nodes are module execution paths (MM-PATHS) and edges represents messages/returns from one unit to another </li></ul><ul><li>Supports composition of units </li></ul>H.Reza
  61. 61. FIGURE 13.4 H.Reza
  62. 62. FIGURE 13.5 H.Reza
  63. 63. PDL DESCRIPTION OF SATM H.Reza
  64. 64. H.Reza
  65. 65. H.Reza
  66. 66. MORE ON MM-PATH <ul><li>MM-Path issues </li></ul><ul><ul><li>How long is an MM-Path? </li></ul></ul><ul><ul><li>What is the endpoint? </li></ul></ul><ul><li>The following observable behavior that can be used as endpoints </li></ul><ul><ul><li>Event quiescence ( event inactivity) </li></ul></ul><ul><ul><ul><li>System level event </li></ul></ul></ul><ul><ul><ul><li>Happens when system is ideal/waiting </li></ul></ul></ul><ul><ul><li>Message quiescence (msg inactivity) </li></ul></ul><ul><ul><ul><li>Unit that sends no messages is reached </li></ul></ul></ul><ul><ul><li>Data quiescence (data inactivity) </li></ul></ul><ul><ul><ul><li>Happens when a sequences of processing generate a stored data that is not immediately used </li></ul></ul></ul><ul><ul><ul><li>E.g. account balance that is not used immediately </li></ul></ul></ul>H.Reza
  67. 67. MM-PATH GUIDELINES <ul><li>MM-Path Guiltiness </li></ul><ul><ul><li>Points of quiescence are natural endpoints for an MM-path </li></ul></ul><ul><ul><li>atomic system functions (system behavior) are considered as an upper limit for MM-Paths </li></ul></ul><ul><ul><ul><li>MM-Paths should not cross ASF boundaries </li></ul></ul></ul>H.Reza
  68. 68. PROS AND CONS <ul><li>hybrid approach( GOOD ) </li></ul><ul><li>The approach works equally well for software testing developed by waterfall model ( GOOD ) </li></ul><ul><li>Testing closely coupled with actual system behavior ( GOOD ) </li></ul><ul><li>Identification of MM-Paths which can be offset by elimination of sub/drivers( BAD ) </li></ul>H.Reza
  69. 69. SYSTEM TESTING <ul><li>Closely related to everyday expertise </li></ul><ul><li>The goal is to Test the quality (not specification) </li></ul><ul><li>Works with notion of threads (or scenarios ) </li></ul><ul><li>A thread is a sequence of events (or ASFs) </li></ul><ul><ul><li>Provides a unifying view of our three level of testing </li></ul></ul><ul><ul><li>e.g., </li></ul></ul><ul><ul><ul><li>A scenario of normal usage </li></ul></ul></ul><ul><ul><ul><li>A stimulus/response pair </li></ul></ul></ul><ul><ul><ul><li>A sequence of machine instructions </li></ul></ul></ul><ul><ul><ul><li>A sequences of ASFs </li></ul></ul></ul><ul><ul><ul><li>… . </li></ul></ul></ul><ul><li>Identifying threads </li></ul><ul><ul><li>FSM (node/edge coverage metrics) </li></ul></ul><ul><ul><ul><li>Top/down </li></ul></ul></ul><ul><ul><ul><li>Bottom/up </li></ul></ul></ul>H.Reza
  70. 70. SOFTWARE ARCHITECTURE & TESTING <ul><li>Using Traditional approach formalizing and automating the integration test stage is difficult </li></ul><ul><ul><li>the selection of the test cases for the subsystems </li></ul></ul><ul><ul><li>stress structure over behavior </li></ul></ul><ul><ul><li>the order in which the components are incrementally combined is entirely dependent on the adopted system </li></ul></ul><ul><li>How can we test if a design and implementation comply using SA? </li></ul><ul><li>How can we specify a SA such that one or more properties can “easily&quot; be tested or verified? </li></ul><ul><li>How integration testing can be planned and controlled based on the SA? </li></ul>H.Reza
  71. 71. SA-BASED APPROACH <ul><li>SA </li></ul><ul><ul><li>Used for prediction of the system-level quality </li></ul></ul><ul><li>the approach would belong to the black box techniques of state transition testing, i.e., </li></ul><ul><ul><li>the system specification is modeled by an automaton, </li></ul></ul><ul><ul><li>the generation of test cases is aimed at covering the arcs (i.e., the transitions) and the nodes (i.e., the states) of it </li></ul></ul><ul><ul><li>Create sub-graphs from SPEC representing specific views of a system </li></ul></ul><ul><ul><li>use these views as a base for the definition of coverage criteria and testing strategies. </li></ul></ul><ul><ul><li>Select test cases to cover these sub-graphs </li></ul></ul>H.Reza
  72. 72. THE ADVANTAGES OF USING THE SATO DERIVE THE AUTOMATON <ul><li>for state transition testing are evident: </li></ul><ul><ul><li>we have a formal description and the automaton can be automatically derived; </li></ul></ul><ul><ul><li>the SA is at a high level of abstraction, thus the number of states in the automaton can be kept manageable; </li></ul></ul><ul><ul><li>we can trace changes in the SA to the corresponding modifications in the automaton, allowing for the testing of new systems obtained after insertion or modification of a component. </li></ul></ul>H.Reza
  73. 73. ABOUT SURVEY PAPER <ul><li>Example of abstract for survey paper: </li></ul><ul><ul><li>In this report (or paper), we survey a number of X-based representations/algorithms/tool support used to generate/identify/execute/etc test cases. </li></ul></ul><ul><ul><li>Many problems with references and citation </li></ul></ul><ul><li>Examples of citation: </li></ul>H.Reza
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×