Your SlideShare is downloading. ×
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Integration testing(revisited)
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Integration testing(revisited)

1,492

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,492
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
81
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. INTEGRATION AND SYSTEM TESTING CHAPTER 12 & 13 Csci565 Spring 2009 H.Reza
  • 2. OBJECTIVES
    • Integration Testing
    • Simple ATM (SATM)
    • discuss integration testing strategies
      • Top-down
      • Bottom-up
      • Decomposition Based Integration Testing (DBIT)
      • Call Graph Based Integration Testing (CGBIT)
      • Path-Based Integration Testing (PBIT)
    H.Reza
  • 3. A SOFTWARE TESTING STRATEGY: SPIRAL MODEL H.Reza System Testing System eng. validation Testing Requirements Integration Testing Design Unit Testing Code
  • 4. INTEGRATION TESTING:1
    • If all units/components work individually, do they work as a whole when we put them together?
      • Not necessarily
    • The problem is “ putting them together ” or interfacing them
    H.Reza
  • 5. PROBLEMS WITH INTERFACING
    • Integration faults often traceable to incomplete or misunderstood interface specifications
      • mismatched assumptions about other components
      • Individually acceptable imprecision may be magnified to unacceptable levels
      • Global data structures can present problems
    • Inconsistent interpretation of parameters or values
    • Mixed units (meters/yards) in Martian Lander
    • Violations of value domains, capacity, or size limits
    H.Reza
  • 6. INTEGRATION TESTING
    • Tests complete systems or subsystems composed of integrated components
    • Integration testing should be black-box testing when tests derived from the specification
    • Main difficulty is localising errors
    • Incremental integration testing reduces this problem
    H.Reza
  • 7. APPROACHES TO INTEGRATION TESTING
    • Two major approaches
      • Incremental approaches
        • The decomposition-based techniques or tree
          • Stubs/Drivers
        • Call Graph-based techniques
          • No stubs or drivers
      • Non-incremental approaches
    H.Reza
  • 8. INCREMENTAL INTEGRATION TESTING H.Reza
  • 9. INCREMENTAL APPROACHES: TOP-DOWN
    • Top-down testing
      • Start with high-level system
      • integrate from the top-down replacing individual components by stubs where appropriate
        • Depth-first
        • Breadth-first
        • No-best order
          • Critical sections
          • Early skeletal version using I/O modules
    H.Reza
  • 10. STUBS
    • Stubs
      • Special module to simulate some functionality
      • Its production is nontrivial task because the code may simulate a very complicated tasks
        • E.g.
          • Writing a stub performing a database table search routine
      • Creating multiple version of same stub for various reasons
    H.Reza
  • 11. TOP-DOWN TESTING H.Reza
  • 12. TOP-DOWN: COMPLICATIONS
    • The most common complication occurs when processing at low level hierarchy demands adequate testing of upper level
    • To overcome:
      • Either, delay many tests until stubs are replaced with actual modules (BAD)
      • Or, develop stubs that perform limited functions that simulate the actual module ( GOOD)
      • Or, Integrate the software using bottom up approach
    • Confusion about overlapping with design
    H.Reza
  • 13. INCREMENTAL TESTING: BOTTOM UP
    • Bottom-up testing
      • Integrate individual components in levels until the complete system is created
    H.Reza
  • 14. BOTTOM-UP APPROACH
    • Starts with construction and testing with atomic modules
      • No need for stub
      • Low level components are combined into cluster (or builds ) to perform a specific sub-function
      • A driver (a control program for testing) is written
        • Contain hardcoded test input, calls the module being tested, and display the results
      • Cluster is tested
      • Drivers are removed and clusters are combined moving upward in the program structure
    H.Reza
  • 15. BOTTOM-UP TESTING H.Reza M
  • 16. H.Reza A C E B D F J G H I K L
  • 17. H.Reza A Stub C Stub E B Stub D Stub F second state in the top-down
  • 18. H.Reza A Stub C Stub E B D F J Stub H I Intermediate state in the top-down
  • 19. TOP-DOWN
    • Possible Sequences of modules
      • A, B, C, D, E, F, G, H, I, J, K, L
      • A, B, E, F, J, C, G, K, D, H, L, I
      • A, D, H, I, K, L, C, G, B, F, J, E
      • A, B, F, J, D, I, E, C, G,K, H, L
    • If parallel testing allowed, then other alternatives are possible
      • After A has been tested, one programmer could take A and test the combination A-B,
      • Another programmer could test A-C
    H.Reza
  • 20. GUIDELINE FOR INTEGRATION TESTING
    • Integrate the components that implement the most frequently used functionality
    • Perform regression testing for existing features
    • Perform progression testing for new features
    H.Reza
  • 21. NON-INCREMENTAL
    • Big-bang
      • Imposes no order (GOOD)
      • Test all the units (Modules) at once
      • Very easy, but difficult to localize the source of errors
    H.Reza
  • 22. INTEGRATION TEST DOCUMENT
    • Overall plan for integration of the system under construction must be documented in a Test Specification
    • The test plan should describe the overall strategy for integration
      • Example of phases
        • User interaction (menu, button, forms, display presentation)
        • Data Manipulation and analysis
        • Display processing and generation (RT-2D, RT-3D, etc)
        • Database Mgt
        • Logic??
    H.Reza
  • 23. INTEGRATION: CRITERIA
    • The following criteria
      • Interface integrity
        • (internal and external interfaces are tested as each module (or cluster) is incorporated
      • Functional validity
        • Testes to uncover functional error
      • Information validity
        • Tests to uncover error related to local/ global data
      • Performance (Quality)
        • Tests designed to verify performance bounds during software design
    H.Reza
  • 24. TOP-DOWN VS. BOTTOM-UP
    • Architectural validation
      • Top-down integration testing is better at discovering errors in the system architecture
    • System demonstration
      • Top-down integration testing allows a limited demonstration at an early stage in the development
    • Test implementation
      • Often easier with bottom-up integration testing
    H.Reza
  • 25. THE WATERFALL LIFE CYCLE H.Reza Requirement specifications Preliminary design details design coding Unit testing Integration testing System testing
  • 26. INTEGRATION TESTING: SOFTWARE ARCHITECTURE
    • Integration testing
      • How software architecture can be used to conduct tests to uncover errors related to the interface?
    H.Reza
  • 27. FIG. 12.2: PRIMARY DESIGN (OR INFORMAL SOFTWARE ARCHITECTURE) OF THE ATM USING TREE-BASED DECOMPOSITION H.Reza Requirement specifications Terminal I/O Mange Session Conduct Transactions Card Entry PIN Entry Select Transaction
  • 28. MORE ON PRIMARY DESIGN
    • How do perform Integration testing for non-tree based functional decomposition?
      • E.g
        • integration testing for OO
        • Integration testing for Client/server systems
        • Integration testing for Layered systems
        • … .
    H.Reza
  • 29. SIMPLE ATM (SATM)
    • An ATM simple
      • Provides 15 screens for interactions
      • includes 3 function buttons
      • Modeled in structural analysis
        • Data Model (ERD)
        • Functional Model (DFD)
        • Behavioral model (STD)
    H.Reza
  • 30. FIGURE 12.7 H.Reza
  • 31. FIGURE 12.8 H.Reza
  • 32. FIGURE 12.9 H.Reza
  • 33. FIGURE 12.10 H.Reza
  • 34. FIGURE 12.11 H.Reza
  • 35. FIGURE 12.12 H.Reza
  • 36. FIGURE 12.13 H.Reza
  • 37. DECOMPOSITION BASED STRATEGIES
    • Decomposition based
      • Top/down
      • Bottom up
      • Sandwich
      • Big bang
    H.Reza
  • 38. FIGURE 12.14 H.Reza
  • 39. FIGURE 13.1 H.Reza
  • 40. DECOMPOSITION BASED TESTING:1
    • Discussion revolves around the tree-based decomposition and the order by which units are tested and combined
      • Top-to-bottom
      • Bottom-to-top
      • Sandwich
      • Big bang
    • The focus is on the structural compatibility among interfaces
    H.Reza
  • 41. TEST SESSIONS
    • A test session refers to one set of tests for a specific configuration of actual code and stubs
    • The number of integration test sessions using a decomposition tree can be computed
      • Sessions=nodes – leaves + edges
    H.Reza
  • 42. DECOMPOSITION BASED TESTING: 2
    • For SATM system
      • 42 integration testing session (i.e., 42 separate sets of integration test cases)
        • top/down
          • (Nodes-1) stubs are needed
          • 32 stub in SATM
        • bottom/up
          • (Nodes-leaves) of drivers are needed
          • 10 drivers in SATM
    H.Reza
  • 43. DECOMPOSITION BASED STRATEGIES: PROS AND CON
    • Intuitively clear and understandable
      • In case of faults, most recently added units are suspected ones
    • Can be tracked against decomposition tree
    • Suggests breadth-first or depth-first traversals
    • Units are merged using the decomposition tree
      • Correct behavior follows from individually correct units and interfaces
      • Stubs/Drives are major development Overhead
    H.Reza
  • 44. CALL GRAPH BASED INTEGRATION TESTING
    • Call graph
      • A directed graph
      • Nodes corresponds to unit
      • Edges corresponds to the call
      • E.g.
        • A  B (i.e., A is calling B)
    • Attempts to overcome the decomposition problem (structural)
    • Moves toward behavioral testing
    H.Reza
  • 45. CALL GRAPH (CG): APPROACHES
    • Two main approaches based on Call Graph
      • Pair-wise integration
      • Neighborhood integration
    H.Reza
  • 46. FIGURE 13.2: SATM CALL GRAPH H.Reza
  • 47. TABLE 2: AM H.Reza
  • 48. PAIR-WISE INTEGRATION
    • The main idea is to eliminate the overhead (i.e., stub/drive)
    • Uses actual code by restricting a session testing to a pair of units in the Call Graph
      • One integration test for each edge in CG
      • 40 edges means 40 integration tests for the SATM
    H.Reza
  • 49. PAIR-WISE INTEGRATION H.Reza - Uses actual code -one integration test session for each edge -40 edges for SATM
  • 50. NEIGHBORHOOD INTEGRATION
    • The neighborhood of a node refers to the nodes that are one edge away from the given nodes
      • SATM Neighborhoods
      • Number of neighborhoods can be computed
        • Neighborhoods = nodes – sink nodes
        • Or the number of interior-nodes + X ( x=1 if there exists leaf nodes connected directly to the root node otherwise X= 0)
    • Results a drastic reduction in the number of integration test session
      • In case of SATM (11 vs. 40)
    H.Reza
  • 51. NEIGHBORHOOD INTEGRATION H.Reza
  • 52. TABLE 3: SATM NEIGHBORHOODS H.Reza
  • 53. PROS AND CONS
    • Benefits ( GOOD )
      • Mostly behavioral than structural
      • Eliminates sub/drive overhead
      • Works well with incremental development method such as Build and composition
    • Liabilities ( BAD )
      • The fault isolation
      • E.g.,
        • Fault in one node appearing in several neighborhood
    H.Reza
  • 54. PATH-BASED INTEGRATION TESTING
    • The hybrid approach (i.e., structural and behavioral) is an ideal one integration testing
    • The focus is on the interactions among the units
      • Interfaces are structural
      • Interactions are behavioral
    • With unit testing, some path of source statements is traversed
      • What happens when there is a call to another unit?
        • Ignore the single-entry/single-exit
        • Use exist follows by an entry
        • Suppress the call statement
    H.Reza
  • 55. NEW AND EXTENDED CONCEPTS
    • Source node (begin)
      • A statement fragment at which program execution begins or resumes
      • E.g., BEGIN
    • Sink node (end)
      • A statement fragment at which program execution terminates
      • E.g., Final END
    H.Reza
  • 56. MORE ON CONCEPTS
    • Module Execution Path (MEP)
      • A sequence of statements that begins with a source node and ends with a sink node , with no intervening sink nodes
    • The implication of this definition is that program graph (PG) may have multiple source/sink nodes
    • Message
      • A programming language mechanism by which one unit transfers control to another unit
        • E.g.,
          • subroutine invocations
          • Procedure calls
          • Function references
    H.Reza
  • 57. THE PATH BASED INTEGRATION TESTING (DEFINITION)
    • MM-Path (definition)
      • An interleaved sequence of module execution paths (MEP) and messages
    • MM-Path can be used
      • to describe sequences of module execution paths including transfers of control among units using messages
      • To represents feasible execution paths that cross unit boundaries
      • To extent program graph
        • Where
          • nodes = execution paths
          • edges = messages
    • Atomic system function (ASF)
      • An action that is observable at the system level in terms of port input and output events
    H.Reza
  • 58. FIGURE 13.3 H.Reza
  • 59. H.Reza There are seven module execution paths (MEP):
  • 60. MM-PATH GRAPH (DEFINITION)
    • Given a set of units, their MM-Path graph is the directed graph in which nodes are module execution paths (MM-PATHS) and edges represents messages/returns from one unit to another
    • Supports composition of units
    H.Reza
  • 61. FIGURE 13.4 H.Reza
  • 62. FIGURE 13.5 H.Reza
  • 63. PDL DESCRIPTION OF SATM H.Reza
  • 64. H.Reza
  • 65. H.Reza
  • 66. MORE ON MM-PATH
    • MM-Path issues
      • How long is an MM-Path?
      • What is the endpoint?
    • The following observable behavior that can be used as endpoints
      • Event quiescence ( event inactivity)
        • System level event
        • Happens when system is ideal/waiting
      • Message quiescence (msg inactivity)
        • Unit that sends no messages is reached
      • Data quiescence (data inactivity)
        • Happens when a sequences of processing generate a stored data that is not immediately used
        • E.g. account balance that is not used immediately
    H.Reza
  • 67. MM-PATH GUIDELINES
    • MM-Path Guiltiness
      • Points of quiescence are natural endpoints for an MM-path
      • atomic system functions (system behavior) are considered as an upper limit for MM-Paths
        • MM-Paths should not cross ASF boundaries
    H.Reza
  • 68. PROS AND CONS
    • hybrid approach( GOOD )
    • The approach works equally well for software testing developed by waterfall model ( GOOD )
    • Testing closely coupled with actual system behavior ( GOOD )
    • Identification of MM-Paths which can be offset by elimination of sub/drivers( BAD )
    H.Reza
  • 69. SYSTEM TESTING
    • Closely related to everyday expertise
    • The goal is to Test the quality (not specification)
    • Works with notion of threads (or scenarios )
    • A thread is a sequence of events (or ASFs)
      • Provides a unifying view of our three level of testing
      • e.g.,
        • A scenario of normal usage
        • A stimulus/response pair
        • A sequence of machine instructions
        • A sequences of ASFs
        • … .
    • Identifying threads
      • FSM (node/edge coverage metrics)
        • Top/down
        • Bottom/up
    H.Reza
  • 70. SOFTWARE ARCHITECTURE & TESTING
    • Using Traditional approach formalizing and automating the integration test stage is difficult
      • the selection of the test cases for the subsystems
      • stress structure over behavior
      • the order in which the components are incrementally combined is entirely dependent on the adopted system
    • How can we test if a design and implementation comply using SA?
    • How can we specify a SA such that one or more properties can “easily" be tested or verified?
    • How integration testing can be planned and controlled based on the SA?
    H.Reza
  • 71. SA-BASED APPROACH
    • SA
      • Used for prediction of the system-level quality
    • the approach would belong to the black box techniques of state transition testing, i.e.,
      • the system specification is modeled by an automaton,
      • the generation of test cases is aimed at covering the arcs (i.e., the transitions) and the nodes (i.e., the states) of it
      • Create sub-graphs from SPEC representing specific views of a system
      • use these views as a base for the definition of coverage criteria and testing strategies.
      • Select test cases to cover these sub-graphs
    H.Reza
  • 72. THE ADVANTAGES OF USING THE SATO DERIVE THE AUTOMATON
    • for state transition testing are evident:
      • we have a formal description and the automaton can be automatically derived;
      • the SA is at a high level of abstraction, thus the number of states in the automaton can be kept manageable;
      • we can trace changes in the SA to the corresponding modifications in the automaton, allowing for the testing of new systems obtained after insertion or modification of a component.
    H.Reza
  • 73. ABOUT SURVEY PAPER
    • Example of abstract for survey paper:
      • In this report (or paper), we survey a number of X-based representations/algorithms/tool support used to generate/identify/execute/etc test cases.
      • Many problems with references and citation
    • Examples of citation:
    H.Reza

×