Software Quality Assurance CIS 375 Bruce R. Maxim UM-Dearborn
Quality Concepts - 1 <ul><li>Variation control is the heart of quality control </li></ul><ul><li>Software engineers strive...
Quality Concepts - 2 <ul><li>Quality of conformance </li></ul><ul><ul><li>degree to which design specifications are follow...
Quality Costs <ul><li>Prevention costs </li></ul><ul><ul><li>quality planning, formal technical reviews, test equipment, t...
Total Quality Management - 1 <ul><li>Kaizen </li></ul><ul><ul><li>develop a process that is visible, repeatable, and measu...
Total Quality Management - 2 <ul><li>Kanse </li></ul><ul><ul><li>examine the way the product is used by the customer with ...
Software Quality Assurance <ul><li>Conformance to software requirements is the foundation from which software quality is m...
SQA Group Activities - 1 <ul><li>Prepare SQA plan for the project. </li></ul><ul><li>Participate in the development of the...
SQA Group Activities - 2 <ul><li>Audit designated software work products to verify compliance with those defined as part o...
Software Reviews <ul><li>Purpose is to find defects (errors) before they are passed on to another software engineering act...
Review Roles <ul><li>Presenter (designer/producer). </li></ul><ul><li>Coordinator (not person who hires/fires). </li></ul>...
Formal Technical Reviews - 1 <ul><li>Involves 3 to 5 people (including reviewers) </li></ul><ul><li>Advance preparation (n...
Formal Technical Reviews - 2 <ul><li>Reviewers ask questions that enable the producer to discover his or her own error (th...
Why do peer reviews? <ul><li>To improve quality. </li></ul><ul><li>Catches 80% of all errors if done properly. </li></ul><...
Formality and Timing <ul><li>Formal review presentations </li></ul><ul><ul><li>resemble conference presentations. </li></u...
Formality and Timing <ul><li>Analysis is complete. </li></ul><ul><li>Design is complete. </li></ul><ul><li>After first com...
Review Guidelines <ul><li>Keep it short (< 30 minutes). </li></ul><ul><li>Don’t schedule two in a row. </li></ul><ul><li>D...
Formal SQA Approaches <ul><li>Proof of correctness. </li></ul><ul><li>Statistical quality assurance. </li></ul><ul><li>Cle...
Statistical Quality Assurance <ul><li>Information about software defects is collected and categorized </li></ul><ul><li>Ea...
Software Reliability <ul><li>Defined as the probability of failure free operation of a computer program in a specified env...
Software Reliability Metrics <ul><li>Reliability metrics are units of measure for system reliability </li></ul><ul><li>Sys...
Reliability Metrics - part 1 <ul><li>Probability of Failure on Demand (POFOD) </li></ul><ul><ul><li>POFOD = 0.001 </li></u...
Reliability Metrics - part 2 <ul><li>Mean Time to Failure (MTTF)  </li></ul><ul><ul><li>average time between observed fail...
Time Units <ul><li>Raw Execution Time </li></ul><ul><ul><li>non-stop system </li></ul></ul><ul><li>Calendar Time </li></ul...
Software Safety <ul><li>SQA activity that focuses on identifying potential hazards that may cause a software system to fai...
Validation Perspectives <ul><li>Reliability validation </li></ul><ul><ul><li>Does measured system reliability meet its spe...
Validation Techniques <ul><li>Static techniques </li></ul><ul><ul><li>design reviews and program inspections </li></ul></u...
Static Validation Techniques <ul><li>Concerned with analysis of documentation </li></ul><ul><li>Focus is on finding system...
Static Safety Validation Techniques <ul><li>Demonstrating safety by testing is difficult </li></ul><ul><li>Testing all pos...
Safety Reviews <ul><li>Intended system functions correct? </li></ul><ul><li>Is structure maintainable and understandable? ...
Hazard-Driven Analysis <ul><li>Effective safety assurance relies on hazard identification </li></ul><ul><li>Safety can be ...
System Safety Case <ul><li>The normal practice for a formal safety case to be required for all safety-critical computer-ba...
Poka-Yoke Devices <ul><li>Mechanisms that lead to the prevention of a potential quality problem before it occurs or to the...
SQA Plan – 1 <ul><li>Management section </li></ul><ul><ul><li>describes the place of SQA in the structure of the organizat...
SQA Plan - 2 <ul><li>Reviews and audits section </li></ul><ul><ul><li>provides an overview of the approach used in the rev...
SQA Plan - 3 <ul><li>Problem reporting and corrective action section </li></ul><ul><ul><li>defines procedures for reportin...
Upcoming SlideShare
Loading in...5
×

Software Quality Assurance

1,117

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,117
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
39
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Software Quality Assurance

  1. 1. Software Quality Assurance CIS 375 Bruce R. Maxim UM-Dearborn
  2. 2. Quality Concepts - 1 <ul><li>Variation control is the heart of quality control </li></ul><ul><li>Software engineers strive to control the </li></ul><ul><ul><li>process applied </li></ul></ul><ul><ul><li>resources expended </li></ul></ul><ul><ul><li>end product quality attributes </li></ul></ul><ul><li>Quality of design </li></ul><ul><ul><li>refers to characteristics designers specify for the end product to be constructed </li></ul></ul>
  3. 3. Quality Concepts - 2 <ul><li>Quality of conformance </li></ul><ul><ul><li>degree to which design specifications are followed in manufacturing the product </li></ul></ul><ul><li>Quality control </li></ul><ul><ul><li>series of inspections, reviews, and tests used to ensure conformance of a work product to its specifications </li></ul></ul><ul><li>Quality assurance </li></ul><ul><ul><li>auditing and reporting procedures used to provide management with data needed to make proactive decisions </li></ul></ul>
  4. 4. Quality Costs <ul><li>Prevention costs </li></ul><ul><ul><li>quality planning, formal technical reviews, test equipment, training </li></ul></ul><ul><li>Appraisal costs </li></ul><ul><ul><li>in-process and inter-process inspection, equipment calibration and maintenance, testing </li></ul></ul><ul><li>Failure costs </li></ul><ul><ul><li>rework, repair, failure mode analysis </li></ul></ul><ul><li>External failure costs </li></ul><ul><ul><li>complaint resolution, product return and replacement, help line support, warranty work </li></ul></ul>
  5. 5. Total Quality Management - 1 <ul><li>Kaizen </li></ul><ul><ul><li>develop a process that is visible, repeatable, and measurable </li></ul></ul><ul><li>Atarimae hinshitsu </li></ul><ul><ul><li>examine the intangibles that affect the process and work to optimize their impact on the process </li></ul></ul>
  6. 6. Total Quality Management - 2 <ul><li>Kanse </li></ul><ul><ul><li>examine the way the product is used by the customer with an eye to improving both the product and the development process </li></ul></ul><ul><li>Miryokuteki hinshitsu </li></ul><ul><ul><li>observe product use in the market place to uncover new product applications and identify new products to develop </li></ul></ul>
  7. 7. Software Quality Assurance <ul><li>Conformance to software requirements is the foundation from which software quality is measured. </li></ul><ul><li>Specified standards are used to define the development criteria that are used to guide the manner in which software is engineered. </li></ul><ul><li>Software must conform to implicit requirements (ease of use, maintainability, reliability, etc.) as well as its explicit requirements. </li></ul>
  8. 8. SQA Group Activities - 1 <ul><li>Prepare SQA plan for the project. </li></ul><ul><li>Participate in the development of the project's software process description. </li></ul><ul><li>Review software engineering activities to verify compliance with the defined software process. </li></ul>
  9. 9. SQA Group Activities - 2 <ul><li>Audit designated software work products to verify compliance with those defined as part of the software process. </li></ul><ul><li>Ensure that any deviations in software or work products are documented and handled according to a documented procedure. </li></ul><ul><li>Record any evidence of noncompliance and reports them to management. </li></ul>
  10. 10. Software Reviews <ul><li>Purpose is to find defects (errors) before they are passed on to another software engineering activity or released to the customer. </li></ul><ul><li>Software engineers (and others) conduct formal technical reviews (FTR) for software engineers. </li></ul><ul><li>Using formal technical reviews (walkthroughs or inspections) is an effective means for improving software quality. </li></ul>
  11. 11. Review Roles <ul><li>Presenter (designer/producer). </li></ul><ul><li>Coordinator (not person who hires/fires). </li></ul><ul><li>Recorder </li></ul><ul><ul><li>records events of meeting </li></ul></ul><ul><ul><li>builds paper trail </li></ul></ul><ul><li>Reviewers </li></ul><ul><ul><li>maintenance oracle </li></ul></ul><ul><ul><li>standards bearer </li></ul></ul><ul><ul><li>user representative </li></ul></ul><ul><ul><li>others </li></ul></ul>
  12. 12. Formal Technical Reviews - 1 <ul><li>Involves 3 to 5 people (including reviewers) </li></ul><ul><li>Advance preparation (no more than 2 hours per person) required </li></ul><ul><li>Duration of review meeting should be less than 2 hours </li></ul><ul><li>Focus of review is on a discrete work product </li></ul><ul><li>Review leader organizes the review meeting at the producer's request. </li></ul>
  13. 13. Formal Technical Reviews - 2 <ul><li>Reviewers ask questions that enable the producer to discover his or her own error (the product is under review not the producer) </li></ul><ul><li>Producer of the work product walks the reviewers through the product </li></ul><ul><li>Recorder writes down any significant issues raised during the review </li></ul><ul><li>Reviewers decide to accept or reject the work product and whether to require additional reviews of product or not. </li></ul>
  14. 14. Why do peer reviews? <ul><li>To improve quality. </li></ul><ul><li>Catches 80% of all errors if done properly. </li></ul><ul><li>Catches both coding errors and design errors. </li></ul><ul><li>Enforce the spirit of any organization standards. </li></ul><ul><li>Training and insurance. </li></ul>
  15. 15. Formality and Timing <ul><li>Formal review presentations </li></ul><ul><ul><li>resemble conference presentations. </li></ul></ul><ul><li>Informal presentations </li></ul><ul><ul><li>less detailed, but equally correct. </li></ul></ul><ul><li>Early </li></ul><ul><ul><li>tend to be informal </li></ul></ul><ul><ul><li>may not have enough information </li></ul></ul><ul><li>Later </li></ul><ul><ul><li>tend to be more formal </li></ul></ul><ul><ul><li>Feedback may come too late to avoid rework </li></ul></ul>
  16. 16. Formality and Timing <ul><li>Analysis is complete. </li></ul><ul><li>Design is complete. </li></ul><ul><li>After first compilation. </li></ul><ul><li>After first test run. </li></ul><ul><li>After all test runs. </li></ul><ul><li>Any time you complete an activity that produce a complete work product. </li></ul>
  17. 17. Review Guidelines <ul><li>Keep it short (< 30 minutes). </li></ul><ul><li>Don’t schedule two in a row. </li></ul><ul><li>Don’t review product fragments. </li></ul><ul><li>Use standards to avoid style disagreements. </li></ul><ul><li>Let the coordinator run the meeting and maintain order. </li></ul>
  18. 18. Formal SQA Approaches <ul><li>Proof of correctness. </li></ul><ul><li>Statistical quality assurance. </li></ul><ul><li>Cleanroom process combines items 1 & 2. </li></ul>
  19. 19. Statistical Quality Assurance <ul><li>Information about software defects is collected and categorized </li></ul><ul><li>Each defect is traced back to its cause </li></ul><ul><li>Using the Pareto principle (80% of the defects can be traced to 20% of the causes) isolate the &quot;vital few&quot; defect causes </li></ul><ul><li>Move to correct the problems that caused the defects </li></ul>
  20. 20. Software Reliability <ul><li>Defined as the probability of failure free operation of a computer program in a specified environment for a specified time period </li></ul><ul><li>Can be measured directly and estimated using historical and developmental data (unlike many other software quality factors) </li></ul><ul><li>Software reliability problems can usually be traced back to errors in design or implementation. </li></ul>
  21. 21. Software Reliability Metrics <ul><li>Reliability metrics are units of measure for system reliability </li></ul><ul><li>System reliability is measured by counting the number of operational failures and relating these to demands made on the system at the time of failure </li></ul><ul><li>A long-term measurement program is required to assess the reliability of critical systems </li></ul>
  22. 22. Reliability Metrics - part 1 <ul><li>Probability of Failure on Demand (POFOD) </li></ul><ul><ul><li>POFOD = 0.001 </li></ul></ul><ul><ul><li>For one in every 1000 requests the service fails per time unit </li></ul></ul><ul><li>Rate of Fault Occurrence (ROCOF) </li></ul><ul><ul><li>ROCOF = 0.02 </li></ul></ul><ul><ul><li>Two failures for each 100 operational time units of operation </li></ul></ul>
  23. 23. Reliability Metrics - part 2 <ul><li>Mean Time to Failure (MTTF) </li></ul><ul><ul><li>average time between observed failures (aka MTBF) </li></ul></ul><ul><li>Availability = MTBF / (MTBF+MTTR) </li></ul><ul><ul><li>MTBF = Mean Time Between Failure </li></ul></ul><ul><ul><li>MTTR = Mean Time to Repair </li></ul></ul><ul><li>Reliability = MTBF / (1+MTBF) </li></ul>
  24. 24. Time Units <ul><li>Raw Execution Time </li></ul><ul><ul><li>non-stop system </li></ul></ul><ul><li>Calendar Time </li></ul><ul><ul><li>If the system has regular usage patterns </li></ul></ul><ul><li>Number of Transactions </li></ul><ul><ul><li>demand type transaction systems </li></ul></ul>
  25. 25. Software Safety <ul><li>SQA activity that focuses on identifying potential hazards that may cause a software system to fail. </li></ul><ul><li>Early identification of software hazards allows developers to specify design features to can eliminate or at least control the impact of potential hazards. </li></ul><ul><li>Software reliability involves determining the likelihood that a failure will occur without regard to consequences of failures. </li></ul>
  26. 26. Validation Perspectives <ul><li>Reliability validation </li></ul><ul><ul><li>Does measured system reliability meet its specification? </li></ul></ul><ul><ul><li>Is system reliability good enough to satisfy users? </li></ul></ul><ul><li>Safety validation </li></ul><ul><ul><li>Does system operate so that accidents do not occur? </li></ul></ul><ul><ul><li>Are accident consequences minimized? </li></ul></ul><ul><li>Security validation </li></ul><ul><ul><li>Is system secure against external attack? </li></ul></ul>
  27. 27. Validation Techniques <ul><li>Static techniques </li></ul><ul><ul><li>design reviews and program inspections </li></ul></ul><ul><ul><li>mathematical arguments and proof </li></ul></ul><ul><li>Dynamic techniques </li></ul><ul><ul><li>statistical testing </li></ul></ul><ul><ul><li>scenario-based testing </li></ul></ul><ul><ul><li>run-time checking </li></ul></ul><ul><li>Process validation </li></ul><ul><ul><li>SE processes should minimize the chances of introducing system defects </li></ul></ul>
  28. 28. Static Validation Techniques <ul><li>Concerned with analysis of documentation </li></ul><ul><li>Focus is on finding system errors and identifying potential problems that may arise during system operation </li></ul><ul><li>Documents may be prepared to support static validation </li></ul><ul><ul><li>structured arguments </li></ul></ul><ul><ul><li>mathematical proofs </li></ul></ul>
  29. 29. Static Safety Validation Techniques <ul><li>Demonstrating safety by testing is difficult </li></ul><ul><li>Testing all possible operational situations is impossible </li></ul><ul><li>Normal reviews for correctness may be supplemented by specific techniques intended to make sure unsafe situations never arise </li></ul>
  30. 30. Safety Reviews <ul><li>Intended system functions correct? </li></ul><ul><li>Is structure maintainable and understandable? </li></ul><ul><li>Verify algorithm and data structure design against specification </li></ul><ul><li>Check code consistency with algorithm and data structure design </li></ul><ul><li>Review adequacy of system testing </li></ul>
  31. 31. Hazard-Driven Analysis <ul><li>Effective safety assurance relies on hazard identification </li></ul><ul><li>Safety can be assured by </li></ul><ul><ul><li>hazard avoidance </li></ul></ul><ul><ul><li>accident avoidance </li></ul></ul><ul><ul><li>protection systems </li></ul></ul><ul><li>Safety reviews should demonstrate that one or more of these techniques have been applied to all identified hazards </li></ul>
  32. 32. System Safety Case <ul><li>The normal practice for a formal safety case to be required for all safety-critical computer-based systems </li></ul><ul><li>A safety case presents a list of arguments, based on identified hazards, as to why there is an acceptably low probability that these hazards will not result in an accident </li></ul><ul><li>Arguments can be based on formal proof, design rationale, safety proofs, and process factors </li></ul>
  33. 33. Poka-Yoke Devices <ul><li>Mechanisms that lead to the prevention of a potential quality problem before it occurs or to the rapid detection of a quality problem if one is introduced </li></ul><ul><li>Are a simple, cheap, part of the engineering process, and are located near the process task where the mistakes are likely to occur </li></ul>
  34. 34. SQA Plan – 1 <ul><li>Management section </li></ul><ul><ul><li>describes the place of SQA in the structure of the organization </li></ul></ul><ul><li>Documentation section </li></ul><ul><ul><li>describes each work product produced as part of the software process </li></ul></ul><ul><li>Standards, practices, and conventions section </li></ul><ul><ul><li>lists all applicable standards/practices applied during the software process and any metrics to be collected as part of the software engineering work </li></ul></ul>
  35. 35. SQA Plan - 2 <ul><li>Reviews and audits section </li></ul><ul><ul><li>provides an overview of the approach used in the reviews and audits to be conducted during the project </li></ul></ul><ul><li>Test section </li></ul><ul><ul><li>references the test plan and procedure document and defines test record keeping requirements </li></ul></ul>
  36. 36. SQA Plan - 3 <ul><li>Problem reporting and corrective action section </li></ul><ul><ul><li>defines procedures for reporting, tracking, and resolving errors or defects, identifies organizational responsibilities for these activities </li></ul></ul><ul><li>Other </li></ul><ul><ul><li>tools, SQA methods, change control, record keeping, training, and risk management </li></ul></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×