Verification and validation


Published on

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Verification and validation

  1. 1. CS 425/625 Software Engineering Verification and Validation <ul><li>Based on Chapter 19 of the textbook [SE-6] Ian Sommerville, </li></ul><ul><li>Software Engineering, 6 th Ed., Addison-Wesley, 2000 and on the </li></ul><ul><li>Ch19 PowerPoint presentation available at the book’s web-site: </li></ul><ul><li> </li></ul><ul><li>November 10, 2003 </li></ul>
  2. 2. Outline <ul><li>Introduction </li></ul><ul><li>Testing and Debugging </li></ul><ul><li>Verification and Validation Planning </li></ul><ul><li>Software Inspections </li></ul><ul><li>Program Inspections (forms of Software Inspections) </li></ul><ul><li>Automatic Static Analysis </li></ul>
  3. 3. Introduction….. <ul><li>Verification and validation (V&V) : the checking and analysis processes that ensure the software satisfies its specification and meets the needs of the clients who are paying for it </li></ul><ul><li>Validation: “Are we building the right product?” </li></ul><ul><li>Verification: “Are we building the product right?” </li></ul><ul><li>Verification involves checking the software conforms with its specification while the more general process of validation ensures the software meets the needs of the clients </li></ul><ul><li>V&V is a whole life-cycle process, encompassing requirements reviews, design reviews, code inspections, and program testing </li></ul>
  4. 4. .Introduction.… <ul><li>V & V techniques: </li></ul><ul><ul><li>Software inspections </li></ul></ul><ul><ul><li>Software testing </li></ul></ul><ul><li>Software inspections </li></ul><ul><ul><li>Are static V&V techniques as they do not require the software to be executed </li></ul></ul><ul><ul><li>Consist of inspections, automated static analyses, and formal verifications of source code or system models </li></ul></ul><ul><ul><li>Can only check the correspondence between the software and its specification </li></ul></ul><ul><ul><li>Cannot demonstrate the system is operationally useful </li></ul></ul><ul><ul><li>Cannot check non-functional requirements of the software </li></ul></ul>
  5. 5. ..Introduction... <ul><li>Software testing </li></ul><ul><ul><li>It is a dynamic V&V technique as it needs an executable version of the software system </li></ul></ul><ul><ul><li>The system is executed with test data and its operational behaviour is assessed </li></ul></ul><ul><ul><li>Can reveal the presence of errors, not their absence </li></ul></ul><ul><ul><li>A successful test is a test which discovers one or more errors </li></ul></ul><ul><ul><li>The V&V technique for checking non-functional requirements and for verifying system integration </li></ul></ul><ul><ul><li>Should be used in conjunction with static techniques to provide full V&V coverage </li></ul></ul>
  6. 6. … Introduction.. <ul><li>Types of software testing </li></ul><ul><ul><li>Defect testing </li></ul></ul><ul><ul><ul><li>Intended to find inconsistencies between a program and its specification </li></ul></ul></ul><ul><ul><ul><li>Tests designed to discover program faults and defects </li></ul></ul></ul><ul><ul><ul><li>A successful defect test is one which reveals the presence of defects in a system </li></ul></ul></ul><ul><ul><li>Statistical testing </li></ul></ul><ul><ul><ul><li>Designed for software’s performance and reliability estimation </li></ul></ul></ul><ul><ul><ul><li>By running tests that reflect actual user inputs and their frequency, an estimate of operational reliability can be made </li></ul></ul></ul>
  7. 7. … .Introduction. <ul><li>Static and dynamic V&V [Fig 19.1, Somm00] </li></ul>
  8. 8. … ..Introduction <ul><li>Verification and validation should establish confidence that the software is fit for purpose </li></ul><ul><li>This does not mean the software is completely free of defects; rather, it must be good enough for its intended use </li></ul><ul><li>The required level of confidence depends on: </li></ul><ul><ul><li>Software’s purpose: the level of confidence depends on how critical the software is to an organisation </li></ul></ul><ul><ul><li>User expectations: users may have lower expectations of certain types of software </li></ul></ul><ul><ul><li>Marketing environment: getting a product to market early may have higher priority than finding its defects </li></ul></ul>
  9. 9. Testing and debugging. <ul><li>Defect testing and debugging are distinct processes </li></ul><ul><li>V&V is concerned with establishing the existence of defects in a program </li></ul><ul><li>Debugging is concerned with locating and repairing these errors </li></ul><ul><li>Debugging involves formulating hypotheses about program behaviour then testing these hypotheses to find the errors </li></ul>
  10. 10. .Testing and debugging <ul><li>The debugging process [Fig. 19.2, Somm00] </li></ul>
  11. 11. <ul><li>The planning of V&V should start early in the development process </li></ul><ul><li>The plan should balance static verification and testing, specify testing standards and procedures, establish checklists for inspections, and define the software test plan </li></ul><ul><li>Test planning breaks down V&V into a number of stages, often organized according to the V-model (shown on next page) </li></ul><ul><li>The focus is on setting standards and procedures for inspections and testing, not on describing product tests </li></ul>V & V Planning..
  12. 12. .V&V Planning. <ul><li>The V-model of development [Fig. 19.3, Somm00] </li></ul>
  13. 13. ..V&V Planning <ul><li>The structure of a software test plan </li></ul><ul><ul><li>The testing process </li></ul></ul><ul><ul><li>Requirements traceability </li></ul></ul><ul><ul><li>Tested items </li></ul></ul><ul><ul><li>Testing schedule </li></ul></ul><ul><ul><li>Test recording procedures </li></ul></ul><ul><ul><li>Hardware and software requirements </li></ul></ul><ul><ul><li>Constraints </li></ul></ul>
  14. 14. Software Inspections. <ul><li>Involve people examining software code or models (representations) with the aim of discovering defects </li></ul><ul><li>Do not require execution of a system thus can be used throughout the development process </li></ul><ul><li>May be applied to any representation of the system: requirements, design, test data, etc. </li></ul><ul><li>Very effective technique for discovering errors </li></ul>
  15. 15. .Software Inspections <ul><li>In software inspections many different defects can be discovered in a single review of the source code or software model </li></ul><ul><li>In testing, one defect may mask another hence several executions are required </li></ul><ul><li>Software inspections reuse domain and programming knowledge so reviewers are likely to have seen the types of error that commonly occur </li></ul><ul><li>Software inspections and software testing are complementary, not competing techniques (see also slides 4 and 5) </li></ul>
  16. 16. Program Inspections……. <ul><li>Program inspections are a type of software inspections </li></ul><ul><li>Consist of formal reviews conducted by teams and intended for program defect detection </li></ul><ul><li>Defects may be logical errors, anomalies in the code that might indicate an erroneous condition (e.g., an un-initialized variable), or non-compliances with standards </li></ul>
  17. 17. .Program Inspections…… <ul><li>Inspection pre-conditions: </li></ul><ul><ul><li>A precise specification must be available </li></ul></ul><ul><ul><li>Team members must be familiar with the organisation standards </li></ul></ul><ul><ul><li>Syntactically correct code must be available </li></ul></ul><ul><ul><li>An error checklist should be prepared </li></ul></ul><ul><ul><li>Management must accept that inspection will increase costs early in the software process </li></ul></ul><ul><ul><li>Management must not use inspections for staff appraisal </li></ul></ul>
  18. 18. ..Program Inspections….. The inspection process [Fig. 19.6, Somm00]
  19. 19. … Program Inspections…. <ul><li>Program inspection procedure: </li></ul><ul><ul><li>The inspection is planned by the moderator (or chairman ) </li></ul></ul><ul><ul><li>The system overview is presented to the inspection team by the program author (or owner ) </li></ul></ul><ul><ul><li>The code and associated documents are distributed to the inspectors (inspection team) in advance for individual preparation </li></ul></ul><ul><ul><li>The inspection meeting takes place and errors are noted (the inspection also involves a reader and, possibly, a scribe ) </li></ul></ul><ul><ul><li>During re-work modifications are made by the author to repair discovered errors </li></ul></ul><ul><ul><li>Re-inspection may or may not be required, based on moderator’s decision </li></ul></ul>
  20. 20. … .Program Inspections… <ul><li>Inspection teams are made up of at least 4 members: </li></ul><ul><ul><li>Author of the code being inspected </li></ul></ul><ul><ul><li>Inspector who finds errors, omissions and inconsistencies </li></ul></ul><ul><ul><li>Reader who reads the code to the team </li></ul></ul><ul><ul><li>Moderator who chairs the meeting and notes discovered errors </li></ul></ul><ul><ul><li>Other roles are chief moderator and scribe </li></ul></ul>
  21. 21. … ..Program Inspections.. <ul><li>Inspection checklist </li></ul><ul><ul><li>A checklist of common errors should be used during the inspection </li></ul></ul><ul><ul><li>This error checklist is programming language dependent </li></ul></ul><ul><ul><li>The weaker the type-checking of the language, the larger the checklist </li></ul></ul><ul><ul><li>Examples of possible errors: initialisation, constant naming, loop termination, array bounds, etc. </li></ul></ul>
  22. 22. Inspection checks (fault classes) [Fig. 19.7, Somm00]
  23. 23. …… .Program Inspections <ul><li>Inspection rates: </li></ul><ul><ul><li>500 statements/hour during overview </li></ul></ul><ul><ul><li>125 source statement/hour during individual preparation </li></ul></ul><ul><ul><li>90-125 statements/hour can be inspected </li></ul></ul><ul><ul><li>Inspection is therefore an expensive process </li></ul></ul><ul><ul><li>Inspecting 500 lines costs about 40 man/hours effort = £2800 </li></ul></ul>
  24. 24. Automated Static Analysis…. <ul><li>Static analyzers are software tools for source code processing </li></ul><ul><li>They parse the program text to discover erroneous conditions </li></ul><ul><li>Very effective as an aid to inspections; supplement but cannot replace inspections </li></ul><ul><li>Particularly valuable for languages such as C that have weak typing (many errors can remain undetected by the compiler) </li></ul><ul><li>Less cost-effective for languages such as Java that have strong type checking (many errors can be detected during compilation) </li></ul>
  25. 25. .Automated Static Analysis… Automatic static analysis checks [Fig. 19.8, Somm00]
  26. 26. ..Automated Static Analysis.. <ul><li>Stages of static analysis : </li></ul><ul><ul><li>Control flow analysis . Checks for loops with multiple exit or entry points, finds unreachable code, etc. </li></ul></ul><ul><ul><li>Data use analysis . Detects un-initialised variables, variables written twice without an intervening assignment, variables which are declared but never used, etc. </li></ul></ul><ul><ul><li>Interface analysis . Checks the consistency of routine and procedure declarations and their use </li></ul></ul>
  27. 27. … Automated Static Analysis. <ul><li>Stages of static analysis [cont’d]: </li></ul><ul><ul><li>Information flow analysis . Identifies the dependencies of output variables. Does not detect anomalies itself but highlights information for code review (inspection) </li></ul></ul><ul><ul><li>Path analysis . Identifies paths through the program and sets out the statements executed in that path. Again, potentially useful in the review process </li></ul></ul>
  28. 28. LINT static analysis [Fig 19.9, Somm00] 138% more lint_ex.c #include <stdio.h> printarray (Anarray) int Anarray; { printf(“%d”,Anarray); } main () { int Anarray[5]; int i; char c; printarray (Anarray, i, c); printarray (Anarray) ; } 139% cc lint_ex.c 140% lint lint_ex.c lint_ex.c(10): warning: c may be used before set lint_ex.c(10): warning: i may be used before set printarray: variable # of args. lint_ex.c(4) :: lint_ex.c(10) printarray, arg. 1 used inconsistently lint_ex.c(4) :: lint_ex.c(10) printarray, arg. 1 used inconsistently lint_ex.c(4) :: lint_ex.c(11) printf returns value which is always ignored