Successfully reported this slideshow.

Software Testing

365 views

Published on

  • Be the first to comment

  • Be the first to like this

Software Testing

  1. 1. Figure 1. Basic Software Testing Words <ul><li>Error An error is a mistake of commission or omission that a person makes. An error causes a defect. In software development one error may cause one or more defects in requirements, designs, programs, or tests. </li></ul><ul><li>Defect Also called a fault or a bug , a defect is an incorrect part of code that is caused by an error. An error of commission causes a defect of wrong or extra code. An error of omission results in a defect of missing code. A defect may cause one or more failures. </li></ul><ul><li>Error of omission: missing design components OR design omission </li></ul><ul><li>Failure A failure is a deviation from expected behaviour exhibited by software and observed as a set of symptoms by a tester or user. A failure is caused by one or more defects. </li></ul><ul><li>The Causal Trail A person makes an error that causes a defect that causes a failure. </li></ul><ul><li>Root Cause Analysis needs traceability mechanism and pref. A TOOL </li></ul><ul><li>Sources of errors: FIXES </li></ul><ul><li>REQTS </li></ul><ul><li>DESIGN </li></ul><ul><li>CODE </li></ul><ul><li>CHANGES </li></ul><ul><li>Based on V&V FAULT MODEL </li></ul>
  2. 2. Process Management <ul><li>SEI Capability Maturity Model Level </li></ul><ul><li>1 2 3 4 </li></ul><ul><li>Initial SE Repeatable Defined SE Managed SE </li></ul><ul><li>Process Process Process </li></ul><ul><li>Chaotic, Measurements Metrics </li></ul><ul><li>over budget (compilers, TOOLS Testing </li></ul><ul><li>late word processors, time (real) Tools & </li></ul><ul><li>Powerpoint) (elapsed) Traceability </li></ul><ul><li># of people Tool </li></ul><ul><li>LOCC </li></ul><ul><li>Excel, Micro Planner </li></ul><ul><li>5 </li></ul><ul><li>Optimized SE Process </li></ul><ul><li>Continuous Improvement (modify process to achieve </li></ul><ul><li>quality or productivity gains) </li></ul><ul><li>REQTS & DESIGN capturing, </li></ul><ul><li>checking, analyzing TOOLS </li></ul>
  3. 3. Figure 3. Sample STL Sentence <ul><li>Action A01 </li></ul><ul><li>is allowed in state normal </li></ul><ul><li>receives eventitem warning_interrupt; </li></ul><ul><li>transmits event safety_action1; </li></ul><ul><li>uses dataitem water_temp_reading, </li></ul><ul><li>water_temp_base; </li></ul><ul><li>produces dataitem safety_action_report; </li></ul><ul><li>has duration time maximum 3 ; </li></ul><ul><li>has duration time unit “second”. </li></ul><ul><li>Tool can analyze REQTS. in Formal notation for </li></ul><ul><li>consistency (pairwise) </li></ul><ul><li> ij [if Ri is satisfied, Rj can also be satisfied] </li></ul><ul><li>Criticism: Too low level (“states”) </li></ul><ul><li>e.g. SDL too low level in TAU? For REQTS., try use cases or MSCs. </li></ul>
  4. 4. Verification & Validation  SQE <ul><li>SQE contains </li></ul><ul><li>management strategies </li></ul><ul><ul><li>milestones </li></ul></ul><ul><ul><li>measurements </li></ul></ul><ul><li>techniques </li></ul><ul><li>REQTS-based (Black Box) </li></ul><ul><li>DESIGN-based (Grey Box) </li></ul><ul><li>Code-based (White Box) </li></ul><ul><li>Behaviour-based (Reliability & Robustness) </li></ul><ul><li>Fix-based (Regression) </li></ul><ul><li>Tools </li></ul>Progress quality
  5. 5. <ul><li>Benefit - MSC’s have - tool support </li></ul><ul><li> - precise syntax </li></ul><ul><li> - some checking for consistency and ambiguity </li></ul><ul><li> - customer friendly </li></ul><ul><li> - graphical (with abstraction) </li></ul><ul><li>Weakness - graphical (?) </li></ul><ul><li>- finite # of MSC’s </li></ul><ul><li>(same as reqts. List - completeness issue) </li></ul><ul><li>- tend to leave original reqts. behind </li></ul>RTM MSCs REQTS M1 M2 M3 M4 M5 M6 R1 R2 · · · ·
  6. 6. SQE ideally with tools, REQTS gathering, representation, validation <ul><li>REQTS. Engineering tools (RTM) </li></ul><ul><li>Arrival of New REQT. </li></ul><ul><li>How does this affect integrity of R1 - R29? [not many tools for checking consistency or completeness of requirements] </li></ul><ul><li>need for check that set of requirements is </li></ul><ul><li>(worked phrases requirements) </li></ul><ul><li>“ must” </li></ul><ul><li>“ must not” </li></ul>RTM Test REQTS Cases T1 T2 T3 T4 R1 C 0 R11 0 C R12 C R2 ; R30 unambiguous consistent complete
  7. 7. Problems with RTM: <ul><li>Updating RTM to code and to test cases </li></ul><ul><li>needed for functional tests · regression tests </li></ul><ul><li>· need for new or revised tests </li></ul><ul><li>Ideally </li></ul><ul><li>hyper-links REQTS. </li></ul><ul><li>DESIGN TESTS </li></ul><ul><li>CODE </li></ul><ul><li>Allows change documentation to flow from REQTS. </li></ul><ul><li>A RTM tool can be very useful in keeping REQTS, DES., CODE, TESTS unambiguous, consistent </li></ul> 2  1  1 ?  1  1  2  2 not recommended
  8. 8. Requirements-based Testing <ul><li>Mapping traceability (REQTS, Tests) </li></ul><ul><ul><li>test plans </li></ul></ul><ul><ul><li>scenarios or MSCs </li></ul></ul><ul><li>generate reqts.-based tests </li></ul><ul><ul><li>test oracle [whether a test result is a PASS or FAIL or INCONCLUSIVE] </li></ul></ul><ul><ul><li>avoid obsolete tests when reqts. change, but </li></ul></ul><ul><ul><li>incur additional testing cost </li></ul></ul><ul><li>automate test execution </li></ul><ul><li>measure test quality (coverage) </li></ul><ul><li>optimize testing cost-benefit </li></ul>
  9. 9. Specification Languages <ul><li>ASN.1 Neufeld, G. and S. Vuong, “An Overview of ASN.1,” Comput. Networks ISDN Syst ., Vol. 23, 1992, pp. 319-415. </li></ul><ul><li>Estelle “A Formal Description Technique Based on an Extended State Transition Model,” Intl. Org. Standard , IS 9074, 1988. </li></ul><ul><li>Larch Guttag, J., J. Horning, and J. Wing, “The Larch Family of Specification Languages,” IEEE Software , Vol. 2, No. 5, 1985, pp. 24-36. </li></ul><ul><li>LOTOS “A Formal Description Technique Based on Temporal Ordering of Observed Behavior,” Intl. Org. Standard , IS 8807, 1988. </li></ul><ul><li>SDL Saracco, R. and P. Tilanus, “CCITT SDL: Overview of the (Specification description) Language and Its Applications,” Comput. Networks ISDN Syst ., Vol. 13, No. 1, 1987, pp. 65-74. </li></ul><ul><li>STL “The Semantic Transfer Language,” in IEEE Standard 1175-1994, Standard Reference Model for Computing System Tool Interconnections . IEEE Press, New York, pp. 37-117, 1994. </li></ul><ul><li>Z Wordsworth, J., Software Development with Z: A Practical Approach to Formal Methods in Software Engineering , Addison-Wesley, Workingham, England, 1992. </li></ul><ul><li>VDM Jones, C.B., Systematic Software Development Using VDM , Prentice-Hall, Englewood Cliffs, NJ, 1986. </li></ul>

×