Soft quality & standards


Published on

Describes the detail of software quality, tradeoffs, quality with testing, quality with inspection, need of standards, standards organizations & different type of software standards.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Humphrey (2002)
  • This is why cleanrooms are so clean.
  • Prevents idiosyncrasy: e.g. Standards for primitives in programming languages
  • EIAfor applying systems engineering techniques/process to the development of all kinds of systems.
  • IEC: International Electrotechnical Commission AIAA: American Institute of Aeronautics and Astronautics
  • IEC: International Electrotechnical Commission
  • AIAA: American Institute of Aeronautics and Astronautics
  • IEC: International Electrotechnical Commission
  • Soft quality & standards

    1. 1. Prince Bhanwra<br />801031024<br />Thapar University<br />Software Quality & Standards<br />
    2. 2. What isSoftware Quality ??<br />1<br />3<br />2<br />High quality software meets the needs of users while being reliable, well supported, maintainable, portable and easily integrated with other tools.<br />Is higher quality better? Is it more expensive?<br />- Not always, on both counts.<br />We will look at how to achievequality, the tradeoffs involved, modeling quality improvement and standards designed to ensure quality.<br />
    3. 3. Making changes to improve software quality requires time and money to:<br />• Spot the problem<br />• Isolate its source<br />• Fix the requirements, design, and code<br />• Test the fix for this problem<br />• Test the fix has not caused new problems<br />• Change the documentation<br /> <br />Cost/BenefitTradeoff<br />
    4. 4. Meeting the needs of users (not to mention marketing) requires adding features.<br /> <br />• However, given a fixed amount of development time and money, adding features adds bugs and reduces time for testing.<br />• Do the features increase user productivity more than the bugs decrease it..?<br /><ul><li>Difficult to answer this question, because data on users is sparse, and other factors like reputation usually take precedence.</li></ul> <br />Feature/Bug Tradeoff<br />
    5. 5. But is increasing quality always more expensive, in terms of total cost of production and maintenance? NO<br /> <br />In fact, if you focus on quality from the scratch, then:<br />You tend to produce components with fewer defects, so<br /> <br />• You spend less time debugging, thus<br /> <br />• You have more time in your schedule for improving other aspects of quality, like usability etc.<br />Quality for free?<br />
    6. 6. If you don’t focus on product quality then:<br /> • You tend to produce components with more (hidden) defects, so <br /> • You have to spend more time fixing these (late), so<br /> <br /> • You have little time for anything else, so<br /> <br /> • You produce poor quality software even though<br />you put huge amounts of effort into defect checking<br /> <br />Thus quality is something that has to be considered throughout the product lifecycle; it cannot be added in later.<br />Skimp Now, Pay Later<br />
    7. 7. Thus it makes sense to focus on improving component quality before testing, to catch difficult defects early.<br />QualityDelaysareExpensive<br />
    8. 8. A study estimates that experienced software engineers normally inject 100 or more defects per KLOC.<br /> <br />Perhaps half of these are detected automatically (e.g. by the compiler).<br /> <br />So a 50 KLOC program probably contains around 2500 defects to find (semi-)manually.<br /> <br />Suppose we need about five hours to find each of these defects by testing.<br />That’s over 20000 hours for the whole program - bad news.<br />BetterQuality Through Testing?<br />
    9. 9. Code inspection may be able to find up to (say) 70% of these defects in 0.5 hours per defect.<br /> <br />So the first 1750 defects could take 875 hours; then we only have 750 to find in testing at (say) 8 hours each. That’s less than 7000 hours in total - better news.<br />BetterQuality ThroughInspection?<br />
    10. 10.   <br />where:<br /> • y(N ) is fraction of defects removed in step N<br /> <br />• r(N ) is the number of defects removed at step N .<br /> <br />• e(N ) is the number of defects escaping at step N . The difficulty with this equation is that we can only<br />estimate e(N ) as a function of e(1), . . . , e(N − 1).<br />  <br />Notice that e(N ) can increase when a change injects defects.<br />ModelingQuality Improvement<br />y(N ) = r(N ) _<br />r(N )+e(N )<br />
    11. 11. Suppose you have 1000 KLOC with an average of 100 defects per KLOC. That’s 100 000 defects to find.<br /> <br />Scenario 1:<br /> <br />• You have an inspection process which finds 75% of these, leaving 25000 to find in test.<br /> <br />• You then use 4 levels of test, each trapping 50% of remaining defects. That leaves 1562 defects in the final code.<br /> <br /> <br />Sounds good so far...<br />Sensitivityto Inspection Yield (1)<br />
    12. 12. Scenario 2:<br /> <br />• Your inspection process only finds 50% of defects, leaving 50000 to find in test.<br /> <br />• The same 4 levels of test each trap 50% of remaining defects. That leaves 3125 defects in the final code.<br /> <br /> <br />So a 33% drop in yield in inspection caused a doubling in the number of remaining defects. Thus the effectiveness of your inspection process is crucial.<br />Sensitivity to Inspection Yield (2)<br />
    13. 13. Assuming we start with no defects, Pi = (1 − p)i , where:<br />• p is the probability of injecting a defect at a stage.<br />• i is the number of stages.<br />• P is the probability of a defect-free product at stage i.<br /> <br /> <br /> <br />A high probability of fault injection in one step radically drops the overall probability of freedom from defects:<br />(1 − 0.01)10 = 0.904<br />Sensitivityto Defect Injection<br />(1 − 0.01)9 ∗ (1 − 0.5)1 = 0.4567<br />
    14. 14. Ri = N ∗ (1 − y)i , where:<br />• N is the initial number of defects.<br />• y the fraction of defects removed per stage.<br />• i is the number of stages.<br />• Ri is the number of defects remaining at stage i.<br /> <br /> <br />Dropping a lot lower on one stage of a high quality defect removal process has a small effect on overall yield.<br />100000 ∗ (1 − 0.8)5 = 32<br /> <br /> <br />Thus being defect-free is better than relying on fixes.<br />Sensitivityto Defect Removal<br />100000 ∗ (1 − 0.8)4 ∗ (1 − 0.4) = 96<br />
    15. 15. If we had no resource limitations then an 80-40 test-inspection yield is no different from a 40-80 yield.<br /> <br /> <br />But test defect correction typically involves more labor than inspection defect correction, so it costs more and the extra labor means . . . more opportunities for defect injection.<br /> <br />So manage for maximum return for minimum cost and, if in doubt, attempt to maximize on early design stages.<br />Yield Management<br />
    16. 16. Most products have safety standards, and many have usability standards, but computer software rarely has such standards.<br /> <br />Can quality be improved by enforcing standards? Unclear:<br /> <br />• It is very difficult to enforce standards on actual program behavior<br /> <br />• Standardizing the process can help make sure that no steps are skipped, but<br />  • Standardizing to an inappropriate process can reduce productivity, and thus leave less time for quality<br />BetterQuality via Standards?<br />
    17. 17. According to the IEEE Comp. Soc. Software Engineering<br /> <br />Standards Committee a standard can be:<br />• An object or measure of comparison that defines or represents the magnitude of a unit<br /> <br />• A characterization that establishes allowable tolerances or constraints for categories of items,<br /> <br />• A degree (or level) of required excellence (or attainment)<br />Software EngineeringStandards<br />
    18. 18.  <br />Repeatability: e.g. Repeating complex inspection processes<br />Consensus wisdom: e.g. Software metrics<br />Cross-specialisation: e.g. Software safety standards<br />Customer protection: e.g. Quality assurance standards<br />Professional discipline: e.g. V & V standards<br />Badging: e.g. Capability Maturity Model levels<br />WhyBother with Standards?<br />
    19. 19. Comparatively few software products are forced by law to comply with specific standards, and most have comprehensive non-warranty disclaimers. However:<br /> <br />• For particularly sensitive applications (e.g. safety critical) your software will have to meet certain standards before purchase<br /> <br />• US courts have used voluntary standards to establish a supplier’s “duty of care”<br />Legal Implications (1)<br />
    20. 20. Adherence to standards is a strong defense against negligence claims (admissible in court in most US states)<br /> <br /> <br />There are instances of faults in products being traced back to faults in standards, so<br /> <br /> <br />Standards writers must themselves be vigilant against malpractice suits<br />Legal Implications (2)<br />
    21. 21. ANSI: American National Standards Institute. Does not itself make standards but approves them<br />AIAA: American Institute of Aeronautics and Astronautics (e.g. AIAA R-013-1992 Recommended Practice for Software Reliability).<br />EIA: Electronic Industries Association (e.g. EIA/IS-632 Systems Engineering)<br />  <br />IEC: International Electrotechnical Commission (e.g. IEC 61508 Functional Safety - Safety-Related Systems)<br /> <br />IEEE: Institute of Electrical and Electronics Engineers Computer Society Software Engineering Standards Committee (e.g.IEEE Std 1228-1994 Standard for Software Safety Plans)<br /> <br />ISO: International Organization for Standardization (e.g. ISO 9000 Standard for SQM & SQA)<br />Some Standards Organizations<br />
    22. 22. ComputerScience Standards<br />Surprisingly few CS standards exist, although one could argue this is because CS is pervasive in others.<br /> <br />Examples:<br /> <br />Terminology: IEEE Std 610.12:1990 Standard Glossary of Software Engineering Terminology<br /> <br /> <br />Techniques: ISO/IEC 8631:1989 Program Constructs and Conventions for their Representation<br />
    23. 23. QualityAssurance Standards<br />Differing views of quality standards: <br /><ul><li>Systems view (that good management systems yield high quality)
    24. 24. Analytical view (that good measurement frameworks yield high quality). </li></ul>Examples:<br /> <br />Quality management: ISO 9000-3 Quality Management and Quality Assurance Standards - Part 3: Guidelines for the application of 9001 to the development, supply, installation and maintenance of computer software<br /> <br />Quality measurement: IEEE Std 1061-1992 Standard for Software Quality Metrics Methodology<br />
    25. 25. Dependability Standards (1)<br /><ul><li>As hardware dependability has improved, software has received more attention as a dependability risk.
    26. 26. Dependability of software isn’t just a question of internal measures (e.g. availability, reliability) but also broader issues (e.g. maintainability)</li></ul> <br />Dependability standards often set integrity levels necessary to maintain system risks within acceptable limits.<br />
    27. 27. Dependability Standards (2)<br />Examples:<br /> <br />Dependability management: IEC 300-1(1993) Dependability management Part 1: Dependability programemanagement<br />Risk analysis: IEC 1025(1990) Fault Tree Analysis<br /> <br />Reliability: AIAA R-013-1992 Recommended Practice for Software Reliability<br />
    28. 28. Safety Standards<br />These traditionally come out of specific industrial sectors (e.g. Nuclear Society, Ministry of Defense), since safety requires deep analysis of the domain as well as the technology. <br />Examples:<br /> <br />Safety plans: IEEE Std 1228-1994 Standard for Software Safety Plans<br /> <br />Functional safety: IEC 61508 Functional Safety - Safety-Related Systems<br /> <br />Nuclear domain: IEE 603 Criteria for Safety Systems of Nuclear Plants<br />
    29. 29. Resources Standards<br />Although software engineering is in flux, it is possible to standardize on some forms of resources which are used widely across applications. <br />Examples:<br />Terminology: IEEE 610,12-1990 Standard Glossary of Software Engineering terminology<br /> <br />Semantics: IEEE P1320.1 Standard Syntax and Semantics for IDEF0<br /> <br />Re-use libraries: AIAA G-010-1993 Guide for Reusable Software: Assessment Criteria for Aerospace Application<br /> <br />Tools: ISO/IEC 14102:1995 Guideline for the Evaluation and Selection of ASE tools<br />
    30. 30. Product Standards<br />These focus on the products of software engineering, rather than on the processes used to obtain them. Perhaps surprisingly, product standards seem difficult to obtain. Examples:<br /> <br /> <br />Product evaluation: ISO/IEC 14598 Software product evaluation<br /> <br /> <br />Packaging: ISO/IEC 12119:1994 Software Packages - Quality Requirements and Testing<br />
    31. 31. Process Standards<br />A popular focus of standardization, partly because product standardization is elusive and partly because much has been gained by refining process. Much of software engineering is in fact the study of process. <br />Examples:<br />Life cycle: ISO/IEC 12207:1995 Information Technology - Software Life Cycle Processes<br /> <br />Acquisition: ISO/IEC 15026 System and software Integrity Levels<br /> <br />Maintenance: IEEE Std 1219-1992 Standard for Software Maintenance<br /> <br />Productivity: IEE Std 1045-1992 Standard for Software Productivity Metrics<br />
    32. 32. Company Guidelines<br />Specific companies may develop their own guidelines for system/software design. These define good practice within a company. They often conform to more general standards. Example:<br /> <br />Shell UK Code of Practice: Fire and Gas Detection and Alarm Systems for Offshore Installations. Describes what a fire and gas alarm system must do; prescribes properties of that system; sets goals for achieving those properties; gives examples of typical design solutions.<br />
    33. 33. Trends<br />• Concern about absence of scientific foundation for standards<br /> <br />• Recognition that standards usually aren’t isolated<br /> <br />• Questioning of software (non)warranty agreements<br />
    34. 34. Some Facts<br /><ul><li>Over 350,000 companies world are registered to ISO 9001.
    35. 35. 85% of registered firms report external benefits
    36. 36. Higher perceived quality
    37. 37. Greater customer demand
    38. 38. 95% report internal benefits
    39. 39. Greater employee awareness
    40. 40. Increased operational efficiency
    41. 41. Reduced scrap expense</li></li></ul><li>A Case Study<br /><ul><li>Published in the Dallas Business Journal, the study credited ISO 9001 for:
    42. 42. Increased Productivity
    43. 43. Reduced scrap and waste
    44. 44. Record sales level
    45. 45. Catching inferior raw materials before they were used
    46. 46. A contract with Romania’s state owned oil company.</li></li></ul><li>Other Case Studies showed<br />ISO Registration resulted in:<br /><ul><li>30% Reduction in customer claims
    47. 47. 95% improvement in delivery time
    48. 48. Reduced defects from 3% to 0.5%
    49. 49. 40% reduction in product cycle time.
    50. 50. 20% increase in on-time delivery
    51. 51. International acceptance and recognition
    52. 52. Facilitated trade in international markets
    53. 53. Promoting of safety, reliability and quality in food products.</li></li></ul><li>CIRAS News Says:<br /><ul><li>Cutting quality cost in half can increase profit more than a 10 percent increase in sales.
    54. 54. ISO 9001 is the most promising method for cutting the cost of quality.</li></li></ul><li>Benefits of ISO 9001 Registration as:<br />Improved management<br /><ul><li>86% of respondents</li></ul>Better customer service<br /><ul><li>73% of respondents</li></ul>Improved efficiency <br /><ul><li>69% of respondents</li></ul>Reduced waste<br /><ul><li>53% of respondents</li></ul>Improved staff motivation<br /><ul><li>50% of respondents </li></ul>Reduced costs<br /><ul><li>40% of respondents</li></li></ul><li>Irwin Professional Publishing survey results:<br /><ul><li>30% of registered companies experience increased customer demand.
    55. 55. 50% of registered companies have decreased number of customer audits.
    56. 56. 69% of registered companies have a competitive advantage in their market.
    57. 57. 83% of registered companies products have higher perceived quality in the marketplace.</li></li></ul><li>Survey of NYSE traded companies<br /><ul><li>US publicly held companies traded on the NYSE that received registration under the ISO 9001 quality standard show significant improvement in financial performance compared to those companies that have not pursued the standard.
    58. 58. The firms that failed to seek registration experienced substantial deterioration in return on assets, productivity and sales. Registered companies avoided such declines.</li></ul>Source:<br />
    59. 59. How to implement ISO 9000 in your organization?<br /><ul><li>One of the fastest and most effective ways to implement ISO 9001 is to use The 9000 Store Documentation and Training Package
    60. 60. By using this package you will implement a professionally designed Quality Management System in the most cost effective manner available. </li></li></ul><li>Start your project with years of expertise on your side:<br /><ul><li>Use our professionally designed Quality Manual, Procedures and Forms.
    61. 61. Save countless hours of your time by editing these proven procedures instead of starting from scratch.
    62. 62. These procedures document processes to meet the requirements of the standard, and are ready for you to tailor for your organization.
    63. 63. Professionals at The 9000 Store are available to answer your questions.</li></li></ul><li>Quality Manual, Procedures and Forms<br /><ul><li>All documented in Microsoft Word and Excel, they are easy to edit and use
    64. 64. These documents have been used by many different types of industries in many different countries for successful registration to ISO 9001</li></li></ul><li>Train your employees<br /><ul><li>We offer the most convenient, cost effective training available
    65. 65. Computer based training provides each employee with an introduction to the standard and what it means to them </li></li></ul><li>Documentation and Training Packages are available<br />This package includes:<br /><ul><li>Project manager’s guide
    66. 66. Project manager online training
    67. 67. Gap Analysis Checklist
    68. 68. Quality Manual, Procedures and Forms
    69. 69. Internal Auditor Training Materials package
    70. 70. Employee Training
    71. 71. Online support for answers to your questions</li></li></ul><li>Some Facts<br /><ul><li>Over 350,000 companies world are registered to ISO 9001.
    72. 72. 85% of registered firms report external benefits
    73. 73. Higher perceived quality
    74. 74. Greater customer demand
    75. 75. 95% report internal benefits
    76. 76. Greater employee awareness
    77. 77. Increased operational efficiency
    78. 78. Reduced scrap expense</li></li></ul><li>Summary<br />• It is crucial to think about quality when you start the project<br /> <br />• More quality is not always better, but it is usually is<br /> <br />• Correcting defects is very different at different stages<br /> <br />• Standards can help ensure consistent quality, but primarily for process, not product<br />
    79. 79. It’s not just about the visuals, but strengthening<br />MESSAGE.<br />
    80. 80. Software Quality & Standards<br />Thank you !!<br />Query..?<br />