Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Introduction to Software Testing Chapter 9.2 Challenges in Testing Software – Software Testability Paul Ammann & Jeff Offutt
  2. 2. Chapter 9 Outline <ul><li>Testing for Emergent Properties: Safety and Security </li></ul><ul><li>Software Testability </li></ul><ul><li>Test Criteria and the Future of Software Testing </li></ul>Introduction to Software Testing (Ch 9.2) © Ammann & Offutt Software Testability
  3. 3. Testability Overview <ul><li>Testability is distinct from software testing </li></ul><ul><li>General definition : </li></ul>Introduction to Software Testing (Ch 9.2) © Ammann & Offutt An estimate or measurement of a conditional probability – assuming that a software artifact contains a fault, how likely is it that testing will reveal that fault ? <ul><li>What do we do with testability estimates ? </li></ul><ul><ul><li>Pay more attention to components with low testability – code reviews, formal analysis, stronger test criteria </li></ul></ul><ul><ul><li>Modify low testability components to increase their testability </li></ul></ul><ul><ul><li>We have more confidence in components with high testability </li></ul></ul><ul><ul><li>Risk analysis – low testability components represent risk that management needs to be aware of </li></ul></ul>
  4. 4. Model of Testability Introduction to Software Testing (Ch 9.2) © Ammann & Offutt If a program has a fault, how difficult will it be to find a test that causes a failure ? Impractical to measure Simple model Out failure causing Inputs P Testability = | failure causing | | Input | X 100 %
  5. 5. Approximating Testability Introduction to Software Testing (Ch 9.2) © Ammann & Offutt Testability can be approximated with the RIP model and mutation Sensitivity ( X ) = R * I * P Testability ( P ) = F (Sensitivity ( X )), for all X in P Given a location X in a program P P: entry X exit Induce faults (mutants) R = % inputs from some distribution that reach X I = % inputs that cause a fault to infect (average over N faults) P = % infected states that propagate to output
  6. 6. Issues in Approximating Testability <ul><li>Reasonable input distribution ? </li></ul><ul><li>How to induce faults ? </li></ul><ul><ul><li>What faults ? </li></ul></ul><ul><li>How to measure propagation ? </li></ul><ul><ul><li>Expensive! </li></ul></ul><ul><li>Information hiding reduces propagation </li></ul><ul><li>Assertion checking can be used to increase testability </li></ul>Introduction to Software Testing (Ch 9.2) © Ammann & Offutt
  7. 7. Testability of OO Software <ul><li>Information hiding decreases testability </li></ul><ul><ul><li>State information is saved in instance variables </li></ul></ul><ul><ul><li>No direct access to instance variables </li></ul></ul><ul><li>Inheritance compounds the problem </li></ul><ul><ul><li>Instance variables are defined in ancestor classes – harder to get to </li></ul></ul><ul><li>These are primarily issues of observability </li></ul><ul><li>Increasing observability in OO software : </li></ul><ul><ul><li>Require additional get ( ) methods – must be done during development </li></ul></ul><ul><ul><li>Use Java reflection to access internal variables – hard to interpret the data values </li></ul></ul><ul><li>This is an area of ongoing research </li></ul>Introduction to Software Testing (Ch 9.2) © Ammann & Offutt
  8. 8. Testability of Web Applications <ul><li>Both controllability and observability are very low </li></ul><ul><li>User interface and most software components distributed on different computers </li></ul><ul><ul><li>Server software may be distributed even further </li></ul></ul><ul><li>Most communication is through message passing </li></ul><ul><li>Much of the inter- component communication goes through the client </li></ul><ul><ul><li>Stateless HTTP messages </li></ul></ul><ul><li>State is kept in an unusual combination of technologies </li></ul><ul><ul><li>Cookies and session objects </li></ul></ul>Introduction to Software Testing (Ch 9.2) © Ammann & Offutt Testability in web applications is still very much an open research area
  9. 9. Testability Summary <ul><li>Testability can give valuable information to testers, managers and developers </li></ul><ul><li>Testability is often thought of as combining two characteristics of software </li></ul><ul><ul><li>Controllability and observability </li></ul></ul><ul><li>Measuring testability is a very technical task </li></ul>Introduction to Software Testing (Ch 9.2) © Ammann & Offutt How best to measure testability is still an open research problem