• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Matada Prashanth

Matada Prashanth






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Matada Prashanth Matada Prashanth Presentation Transcript

    • Testing: A Roadmap Mary Jean Harrold College Of Computing Georgia Institute Of Technology Presented By Prashanth L Anmol N M
    • Introduction
      • Definition
      • Purposes for which testing is performed
      • Key Concepts
      • Advantages
      • Disadvantages
    • Introduction
      • Definition
        • Software testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results.
        • Constitutes more than 50% of cost of software development
    • Introduction
      • Purposes
      • To improve quality
      • For Verification & Validation (V&V)
        • Functionality (Exterior Quality)-Correctness ,Reliability Usability Integrity
        • Engineering (interior quality)-Efficiency ,Testability Documentation Structure
        • Adaptability (future quality )-Flexibility Reusability Maintainability
      • For reliability estimation
    • Introduction
      • Key Concepts
      • Taxonomy
        • correctness testing
        • performance testing
        • reliability testing
        • security testing
      • Testing Automation
      • When to Stop Testing?
    • Introduction
      • Correctness Testing-
        • Minimum Requirements of Software.
        • can be either black box or white box
      • Black Box - treats s/w as blackbox.only the inputs/outputs visible basic functionality testing.
      • White Box-structure & flow of software under test visible.
      • Performance Testing-Design problems in software that cause the system performance to degrade
      • Reliability Testing-robustness of a software component is the degree to
      • which it can function correctly in the presence of exceptional inputs or stressful environmental conditions
      • Security Testing-include identifying and removing software flaws that may potentially lead to security violations, and validating the effectiveness of security measures. Simulated security attacks can be performed to find vulnerabilities
    • Introduction
      • Testing Automation
        • Automation is a good way to cut down time and cost.
        • Software testing tools and techniques usually suffer from a lack of generic applicability and scalability.
      • When to Stop Testing?
        • Testing is a trade-off between budget, time and quality.
        • It is driven by profit models.
        • pessimistic  time, budget, or test cases -- are exhausted.
        • optimistic  reliability meets the requirement, or the benefit from
      • continuing testing cannot justify the testing cost .
      • Advantages
        • Easy generation of test cases & Instrumentation of software.
        • Process automation & execution in expected environment.
    • Roadmap
      • Fundamental Research
        • Testing Component Based Systems
        • Testing Based on Precode Artifacts
        • Testing Evolving Software
        • Demonstrating Effectiveness of Testing Techniques
        • Using Testing Artifacts
        • Other Testing Techniques
        • Methods & Tools
        • Empirical Studies
        • Testing Resources
    • Testing Component Based Systems
      • Issues
        • Component Provider (Developer of the Components)
        • Views Components independently of the context.
        • Component User (Application Developer)
      • Views Components relevant to the application.
      • Limiting Factor
        • Availability of Source Code
      • Roadmap Suggested
        • Types of Testing Information needed.
        • Techniques for representing & computing the types of testing info the user needs.
        • Techniques to use the information provided with the component for testing the application.
    • Testing Based On Precode Artifacts
      • Issues
        • Design
        • Requirements
        • Architectural Specifications
      • Issue Under Spotlight
        • Architecture
      • Roadmap Suggested
        • Use of formal notations used for s/w architecture
        • Develop techniques to be used with architectural specification for test-case development.
        • Develop techniques to evaluate s/w architectures for testability.
    • Testing Evolving Software
      • Regression testing:
      • Validate modified software to ensure no new errors introduced.
      • Some useful techniques
      • * select subset of test suite of previous from previous testing.
      • * techniques to help manage growth in size of test suite.
      • * assess regression testability.
      • Testing techniques needed ….
      • * not only for software but also for architecture and requirements.
      • * manage test suites themselves.
      • * identify parts of modified software for which new test cases are required.
      • * identify test cases no longer needed.
      • * prioritize test cases.
      • * asses the test suite themselves.
    • Demonstrating effectiveness of Testing Techniques
      • How ????
      • * Increase developers confidence.
      • * software behavior
      • * identify classes of faults for a given test criteria.
      • * provide visual interface.
      • * determine interaction among test selection criteria and ways to combine.
      • Research been done In ….
      • * evaluation criteria to determine adequacy of test suites and test cases that inspire confidence.
      • * test cases based on either software’s intended behavior or purely on code.
      • * test cases based on data-flow in a program.
      • * use existing testing techniques to test visual programming languages.
      • * test complex Boolean expressions.
      • * mutation analysis and approximation ( ex: can avoid testing pointer variable in data flow analysis.
    • Establishing Effective Process for Testing
      • Need to develop process for planning and implementation of Testing.
      • Some of the currently used or proposed techniques ….
      • * Develop a test plan during requirements gathering phase and implementation of the test plan after s/w implementation phase. Useful ?
      • * what does Microsoft do ?
      • - frequently synchronize what people are doing.
      • - periodically stabilize the product in increments as project proceeds.
      • - build and test a version every night !!! ( only Microsoft can do…!)
      • * perpetual testing - Build foundation for treating analysis and test ongoing activities for improved quality.
      • * selective regression testing where we test one version and gather testing artifacts such as I/o pairs and coverage information.
      • * explicit progress of regression testing that integrates many key testing techniques into development and maintenance of evolving software.
    • Establishing Effective Process for Testing. Cont’d
      • Some Open questions …
        • Does Microsoft nightly rebuild, minimize testing later ?
        • Does testing show all the software qualities ?
        • Can results obtained from testing be generalized ?
      • Some useful suggestions …
        • Integrate various quality techniques and tools
        • Combine static analysis with testing
    • Using Testing Artifacts
      • Artifacts include :
        • Execution traces of software’s execution with test cases.
        • Results of execution like pass/fail of software for given test cases.
      • Useful : Store results and use for retesting modified software.
      • A whole bunch of research is done on this, some of the proposed ….
      • * use dynamic program slices derived from execution traces along with pass/fail results for execution traces to identify potential faulty code.
      • apply heuristics to find out subset of test suite that passed and failed.
      • * identify program invariants and develop test cases generation.
      • * use coverage information to predict the magnitude of regression testing.
      • * use coverage information also to select test cases from test suite for use in regression testing.
    • Using Testing Artifacts. Cont’d…
      • Proposed research cont’d….
      • * Use artifacts to for test suite reduction and prioritization.
      • * Perform concept analysis on coverage info and compute relationships among executed entities. Helps in uncovering properties of test suites.
      • * Path spectra identifies paths where control diverges in program execution which is helpful in debugging, testing and maintenance. Expensive !
      • * Use branch spectra which is less expensive profiling.
      • * Using visual tools to analyze test info.
      • Additional research Needed ….
      • * use of testing artifacts for software engineering tasks.
      • * identify types of info software engineers and managers need (data mining)
    • Other Testing Techniques
      • Some other techniques helpful in reaching end goal (quality software)
      • * Need for a scalable automatic test data generation
      • * Static analysis required but expensive. Need for a scalable analysis technique that can be used to compute required information.
      • * data-flow analysis expensive and hence need for efficient instrumentation and recording techniques.
    • Method and Tools
      • Goal
        • Develop efficient methods and tools that can be used by practitioners to test their software.
      • Complaint
        • Software Engineering technology requires on average 18 years to be transferred in to practice !!!!!!! We need to reduce this time for technology transfer.
      • Reasons
        • techniques developed demonstrated on contrived or toy system. Not Scalable !
      • What we need
        • Development of tools and methods for industrial setting to demonstrate the usefulness of techniques proposed
        • Techniques proposed should be scalable.
        • Develop robust prototypes and identify the context in which they can function and use them to perform the experiments to demonstrate the techniques.
    • Method and Tools. Cont’d ….
      • What we need. Cont’d …
        • Tools to consider computation trade offs like Precision Vs Efficiency .
        • Automatic development of method and tools on the lines of compilers.
        • Develop tools that are attractive to practitioners.
        • Finally the developed tools require minimal involvement of Software Engineers.
    • Empirical Studies
      • What does it mean ?
        • Studies which will help demonstrate the scalability and usefulness of the techniques in practice in other words feedback for future research.
      • Difficulties in doing Empirical Studies
        • Difficulty in acquiring sufficient robust implementation of these techniques
        • Difficulty in obtaining sufficient experimental subjects (software and test suites)
      • Solutions
        • Collect sets of experimental subjects and make them available for researchers.
        • Create sanitized information that would reveal no proprietary information and still useful for experimentation.
    • Testing Resources
      • Workshops, Conferences, Journals and Useful Links
        • Workshop on Strategic Directions in Software Quality 1996 . (ACM)
        • National Science Foundation & Computational Research Association
        • Workshop on Role of Software Architectures in Testing and Analysis. (INRC)
        • International Conference on Software Engineering Workshop on Testing Distributed Component-based Systems.
        • Middle Tennessee State’s STORM Software Testing
        • http://www.mtsu.edu/~storm/
        • Reliable Software Technology’s Software Assurance Hotlist
        • http://www.rstcorp.com/hotlist
        • Research Institute’s Software Quality Hotlist
        • http://www.soft.com/Institute/HotList/index.html
        • Newsgroup: comp.software.testing
    • Conclusions
        • Relevance To Embedded Systems
        • Emphasizes the basic stages-Sets up the next paper.
        • Talks about evolving systems-ES evolve by the second.
        • Talks about component based testing-COTS.
        • Emphasizes the need for Testing based on Precode Artifacts-
        • Software Architecture.
        • Examining Current Techniques to demonstrate scalability &
        • usefulness of techniques in practice-Empirical.
        • Weakness – Nil