Sunita Bodigutla
Upcoming SlideShare
Loading in...5
×
 

Sunita Bodigutla

on

  • 1,074 views

 

Statistics

Views

Total Views
1,074
Views on SlideShare
1,073
Embed Views
1

Actions

Likes
0
Downloads
17
Comments
0

1 Embed 1

http://www.slideshare.net 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Sunita Bodigutla Sunita Bodigutla Presentation Transcript

  • Testing: A Roadmap Mary Jean Harrold Sunita Bodigutla CS 589 Fall 2003
  • Definition
    • A verification method that applies a controlled set of conditions and stimuli for the purpose of finding errors.
    • The paper says:
      • It’s a process that is performed to support quality assurance
  • Quality Assurance
    • ISO 9000:2000
      • "Part of quality management focused on providing confidence that quality requirements will be fulfilled“
      • It is a planned program consisting of: all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements; and, activities designed to evaluate the process by which products are developed or manufactured.
      • A product does what it is expected to.
  • Testing
    • Designing test cases
    • executing the software with those test cases
    • examining the results produced by those executions
  • Expensive
    • Testing accounts for 50% of the cost of software development
    • Higher for critical systems
    • Efficient ways to perform testing to control the cost
  • What this paper is about
    • A report written in 2000
    • assesses the state of the art in software testing
    • outlines future directions in software testing
    • gives pointers to software testing resources
  • Testing Component based Systems Testing based on Precode artifacts Demonstrating effectiveness of testing techniques Testing Evolving Software creating effective testing processes Other testing approaches Using testing artifacts Methods and Tools Empirical Studies Practical Testing Methods, Tools, and Processes for Development of High-Quality Software Fundamental Research Status of Testing Methods, Tools, and Processes fig 1. Software Testing Roadmap
  • Testing Component based Systems Testing based on Precode artifacts Demonstrating effectiveness of testing techniques Testing Evolving Software creating effective testing processes Other testing approaches Using testing artifacts Methods and Tools Empirical Studies Practical Testing Methods, Tools, and Processes for Development of High-Quality Software Fundamental Research Status of Testing Methods, Tools, and Processes fig 2
  • Component Based Systems
    • Increased size and complexity of software systems has led to component based systems
    • Component Developer vs. Component user
  • Component User Component Reusable Service Semantics Repository System System Under Assemble Select Composible Executable Understand Assembly Compile fig 3
  • Component Based Systems (contd.)
    • COTS : Commercial off the shelf software
      • e.g. Microsoft Active X, MFC, Java Applets etc
    • Why COTS-
      • saves money
      • saves time
      • component reuse
      • standardization
      • simplified design
  • Problems with Testing COTS
    • Unavailability of source code
    • Technical Heterogeneity
    • Enterprise Heterogeneity
    • Dynamic evolution
    • Component Trust Problem
  • Component Based Software
    • Techniques for testing by Component provider- a number of them
    • Techniques available to component user-very few
    • Theory of test adequacy by Rosenblum
  • Test Adequacy
    • A test adequacy criterion is a systematic criterion that is used to determine whether a test suite provides an adequate amount of testing for a component under test
    • C-adequate-for-P for adequate unit testing of a component
    • C-adequate-on-M for adequate integration testing of a component-based system
    fig 4
  • Future Directions
    • Efficient techniques and tools for
      • testing aspects like security, dependability and safety
      • representing and computing the types of testing information (meta-data) that the component-user needs
        • Java Beans: BeanInfo
      • storing the information
      • using the information
      • A Framework for component deployment [1] called ‘component deployment testing (CDT) framework’
        • component runtime introspection
        • reflection API of Java
  • Testing Component based Systems Testing based on Precode artifacts Demonstrating effectiveness of testing techniques Testing Evolving Software creating effective testing processes Other testing approaches Using testing artifacts Methods and Tools Empirical Studies Practical Testing Methods, Tools, and Processes for Development of High-Quality Software Fundamental Research Status of Testing Methods, Tools, and Processes fig 5
  • Integration test strategies and architectural styles fig 6
  • Fault type and Integration Test Strategy fig 7
  • Testing based on Architecture specifications
    • Formal notation for architecture specification ‘ CHAM ’ : Chemical Abstract Machine.
      • Architecture: a set of components (molecules)
      • states and interactions governed by rules (reactions)
      • CHAM is specified by defining molecules m1,m2,…, solutions s1, s2,….. and transformation rules T, T’,…..
      • Solutions are multisets of molecules interpreted as defining the states of CHAM
      • Transformation rules describe how the system dynamically evolves through reaction steps
  • Testing based on Architecture specifications (contd.)
    • [4] Using the formal description of SA, a transition graph depicting all possible behaviors of the system is arrived at.
    • A suitable set of reduced graphs are identified, each subgraph specific architectural properties of the system.
    • For each subgraph, test cases are derived to cover all the transitions in it.
  • Future Directions
    • Techniques for
      • automatic generation of test cases from formal specification of software architecture
      • evaluating software architectures for testability
  • Testing Component based Systems Testing based on Precode artifacts Demonstrating effectiveness of testing techniques Testing Evolving Software creating effective testing processes Other testing approaches Using testing artifacts Methods and Tools Empirical Studies Practical Testing Methods, Tools, and Processes for Development of High-Quality Software Fundamental Research Status of Testing Methods, Tools, and Processes fig 8
  • Testing Evolving Software
    • Regression Testing: ensuring that no new errors are introduced into previously tested code
    • Expensive
    • Selective Regression Testing: which subset of a test suite must be rerun to test a new version of the system
  • Future Directions
    • Techniques to
      • assist in managing the test suites
      • reduce the size of test suites
      • prioritize test cases
  • Testing Component based Systems Testing based on Precode artifacts Demonstrating effectiveness of testing techniques Testing Evolving Software creating effective testing processes Other testing approaches Using testing artifacts Methods and Tools Empirical Studies Practical Testing Methods, Tools, and Processes for Development of High-Quality Software Fundamental Research Status of Testing Methods, Tools, and Processes fig 9
  • Test Criteria
    • Effective for certain classes of faults
      • data flow
      • complex boolean expressions: Modified Condition/decision coverage
  • Modified Condition/Decision Coverage
    • Definition:
      • Every point of entry and exit in the program has been invoked atleast once
      • every condition in a decision in the program has taken on all possible outcomes atleast once
      • each condition has been shown to independently affect the decisions outcome
  • Modified Condition/Decision Coverage (contd.)
    • Simple example
      • A and B
      • Extension of truth table to pairs table
      • The test set { (T T),
      • (T F), (F T)} is adequate for MC/DC coverage
    fig 10 F F F 4 1 F F T 3 1 F T F 2 2 3 T T T 1 B A Result AB Number
  • Future directions
    • Find ways
      • to determine interaction between the various test- selection criteria and find ways to combine them to perform more effective testing
  • Testing Component based Systems Testing based on Precode artifacts Demonstrating effectiveness of testing techniques Testing Evolving Software creating effective testing processes Other testing approaches Using testing artifacts Methods and Tools Empirical Studies Practical Testing Methods, Tools, and Processes for Development of High-Quality Software Fundamental Research Status of Testing Methods, Tools, and Processes fig 11
  • Effective Processes for Testing
    • Process used for planning and implementing it
    • Test plan
    • Perpetual testing
    • Microsoft uses a different model
      • synchronize and stabilize
    • Future directions
      • validate the existing models
      • integration of various quality techniques and tools
  • Using Testing Artifacts
    • Execution Traces
      • Information about which
        • statements were executed
        • paths were executed
        • values particular variables acquired
    • Artifacts can be stored for use in retesting the software after it is modified
      • dynamic program slices
      • heuristic approaches
      • List of invariants
      • Regression Testing
      • Visualization technique
  • Other topics
    • Methods and Tools
      • Scalability
      • Computational tradeoffs
      • Automatic generation
    • Empirical Studies
    • Testing Resources
  • Conclusion
    • Overview
    • List of techniques
    • No comparison
    • Future directions
    • Testing is a wide area
    • Dynamic field
    • [1] Antonia Bertolino, Andrea Polini, “A Framework for Component Deployment Testing”
    • [2] A.Orso, M.J.Harrold and D.Rosenblum, “Component Metadata for Software Engineering Tasks”, in W.Emmerich and S.Tai(Eds) EDO 2000, LNCS 1999, pp. 129-144
    • [3] J.A.Stafford and A.L.Wolf, “Annotating components to support component based static analysis of software systems “, Proc. the Grace Hopper Celeb. of women in computing 2001
    • [4] Antonia Bertolino, Paola Inverardi, Henry Muccini, Andrea Rosetti, “An Approach to Integration Testing based on Architectural Descriptions”
    References
  • fig 12. Sequential Multiphase Compiler Architecture
  • fig 13.Transition Graph for the Sequential Compile
  • fig 14. Concurrent Reduced Graph
  • Thank You