Sw testing


Published on

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Sw testing

  1. 1. SW Testing Copyright © 2012 Embedded Systems Committee
  2. 2. Testing for Quality • What is Quality? – Functionality – Reliability – Usability – Efficiency – Maintainability – Portability Quality Target Testing System Copyright © 2012 Embedded Systems Committee
  3. 3. Source of failure • Humanity • Technology change • System interaction • Environmental conditions – Radiation – Magnetism – Electronic fields – Pollution – Temperature Error Error Bug/Defect Bug/Defect Failure Failure Copyright © 2012 Embedded Systems Committee
  4. 4. Testing in Development Cycle • Common types of V-Model uses four types of test levels: – Component (unit) testing – Integration testing – System testing – Acceptance testing Copyright © 2012 Embedded Systems Committee
  5. 5. Risk vs. Testing Software risk • More testing gives more trust in component under test. Number of passed tests Copyright © 2012 Embedded Systems Committee
  6. 6. Cost of error detection • Early detection saves a lot of effort Copyright © 2012 Embedded Systems Committee
  7. 7. Seven Principles of testing 1. Testing shows only the presence of defects • But cannot prove that there is no defects 2. Exhaustive testing is impossible • Instead of exhaustive testing, risk analysis and priorities should be used to focus testing efforts 3. Early testing • Early defect detection reduces the fixing cost 4. Defect clustering • Small number of modules usually contains most of the defects discovered during pre-release testing 5. Pesticide paradox • SW becomes immune against test cases. Test cases needs to be revised regularly and new ones should be written 6. Testing is context dependent • Testing website is different than safety critical system testing 7. Absence-of-error fallacy • Fixing defects does not help if the system built is unusable for the user Copyright © 2012 Embedded Systems Committee
  8. 8. Psychology of testing • The mindset to be used while testing is different than that used for developing software • With the right mindset, developers can test their own code • A certain degree of independence (avoiding the author bias) often makes the tester more effective at finding defects and failures • If error, defects or failures are communicated in a constructive way, bad feelings between the testers and analysts, designers and developers can be avoided Copyright © 2012 Embedded Systems Committee
  9. 9. Test Levels • • • • Component (unit) Integration System Acceptance Copyright © 2012 Embedded Systems Committee
  10. 10. Test Levels - Component • Also known as unit, module, or program testing • Verifies the functioning of SW modules, programs, objects, classes, ..etc that are separately testable • It may be done in isolation from the rest of the system. Stubs, drivers and simulators may be used • May include testing of functional or non-functional characteristics (such as: searching for memory leaks, …) • Test cases are derived from work products such as a specification of the component, software design or data model • Component testing usually involves the programmer who wrote the code. Defects are fixed as soon as they are found, without formally managing these defects Copyright © 2012 Embedded Systems Committee
  11. 11. Test Levels - Integration • There may be more than one level of integration testing: – Component integration testing: tests the interactions between software components and is done after the component testing – System integration testing: tests the interaction between different systems or between hardware and software and maybe done after system testing • The greater the scope of the integration, the more difficult it becomes to isolate defects to a specific component or system which may lead to increased risk and additional time for troubleshooting • Testing of specific non-functional characteristics (e.g., performance) may be included in integration testing as well as functional testing • Focus is mainly on the communication between modules not the functionality Copyright © 2012 Embedded Systems Committee
  12. 12. Test Levels - System • System testing is concerned with the behavior of the whole system • System testing may include tests based on: – Risks – Requirements specifications – Business processes – Use cases – High level text description or system behavior model – Interaction with the operating system –… • An independent system team often carries out this type of testing Copyright © 2012 Embedded Systems Committee
  13. 13. Test Levels - Acceptance • Acceptance testing is often the responsibility of the customers or users of a system; other stakeholders may be involved as well • The goal is to establish confidence in the system. Finding defects is not the main focus in acceptance testing • Typical forms of acceptance testing include: – – – – User acceptance testing Operational testing Contract and regulation acceptance testing Alpha and beta (or field) testing Copyright © 2012 Embedded Systems Committee
  14. 14. Test Types • A test type is focused on a particular test objective, which could be: – A function to be performed by the software – A non-functional quality characteristics, such as reliability or usability – The structure or architecture of the software or system – Change related (confirmation testing, regression testing) Copyright © 2012 Embedded Systems Committee
  15. 15. Test Types - Functional • Functional testing tests “What” the system does • Functional testing is based on functions and features described in documents or understood by the tester • Specification based techniques may be used to derive test conditions and test cases • Functional testing considers the external behavior of the software (black-box testing) • Security testing, is a type of functional testing related to detection of threats from malicious outsiders • Another type of functional testing, interoperability testing, evaluates the capability of the software product to interact with one or more specified components or systems Copyright © 2012 Embedded Systems Committee
  16. 16. Test Types – Non-Functional • Non-functional testing tests “How” the system works • Non-functional testing can be performed on all test levels • The term non-functional describes the tests required to measure characteristics of system and software that can be quantified on a varying scale, such as response time for performance testing • Non-functional testing considers the external behavior of the software and in most cases uses black-box test design techniques Copyright © 2012 Embedded Systems Committee
  17. 17. Test Types - Structural • Structural (white-box) testing may be performed at all test levels • Structural techniques are best used after specificationbased techniques in order to help measure the thoroughness of testing through assessment of coverage of a type of structure • At all test levels, but especially in component testing and component integration testing, tools can be used to measure the code coverage of elements such as statements or decisions Copyright © 2012 Embedded Systems Committee
  18. 18. Test Types - Changes • After a defect is detected and fixed, the software should be retested to confirm that the original defect has been successfully removed. This is called confirmation • Regression testing is the repeated testing of an already tested program, after modification, to discover any defects introduced or uncovered as a result of the change(s) – The extent of regression testing is based on the risk of not finding defects in software that was working previously – Regression testing may be performed at all test levels, and includes functional, non-functional and structural testing – Regression test suites are run many times and evolve slowly, so regression testing is a strong candidate for automation Copyright © 2012 Embedded Systems Committee
  19. 19. Test Types - Maintenance • Once deployed, the software system is often in service for years or decades. During this time, the system, its configuration data, or its environment are often corrected • Maintenance testing is done on an existing (released/deployed) operational system, and is triggered by modifications, migration, or retirement of the software or the system • Maintenance testing for migration (e.g., from platform to another) should include operational tests of the new environment as well as of the changed software • In addition to testing what has been changed, maintenance testing include regression testing to parts of the system that have not been changed • Depending on the changes, maintenance testing may be done at any or all test levels and for any or all test types Copyright © 2012 Embedded Systems Committee