Controlling Project during Development with a Defect Model, Ben Linders, ICSTest Conference 2004

1,494 views

Published on

Wouldn’t it be nice if you could have more insight into the quality of a product, while it is developed, and not afterwards? Would you like to be able to estimate how many defects are inserted in the product in a certain phase, and how effective a (test) phase is in capturing these defects? To optimize your test phases regarding focus and effort in relation to how many defects they will find? This presentation will show a simple but very effective model that makes it possible: The Project Defect Model.

The aim of the Project Defect Model is to track product quality, take corrective actions and reduce quality risks. To get more insight into the quality of the product during development, it is needed to measure the software development processes with two views: Introduction and detection of defects. Introduction is done during the specification, design and coding phases; defects are either introduced into documents or into the actual product. Detection of defects is done via inspections and test during all the phases of the project.

A tool was developed using a spreadsheet. The purpose of the tool was to estimate the number of defects per phase, and to track all defects discovered in inspections and tests against these estimates. The tool supported analysis of the data with both calculated values and graphs comparing actuals to estimates in terms of current status and trends over time.

The Project Defect Model has been beneficial to projects. It has helped estimating, planning, and tracking quality during the project, including an estimate of the the number of defects left in the released product. The quality data has been used in the project together with time and cost data, to take better decisions on test, review and inspections, and design. Also it has identified quality risks at an early stage, helping the project take corrective actions and decisions on product release and maintenance capacity planning. Finally the model provided insight into the effectiveness of the verification activities, supporting effective process improvement.



Paper:

The presentation is on a defect planning/tracking tool and approach. Focus will be upon:
• Goals: What was the purpose of the model, why developed, what did we want to reach?
• How: Show the definition of the model and its implementation and application.
• Tools: The tool that was developed to implement the model, how it works, strengths.
• Results: How did the model and tool help the project? Did it live up to its purpose?
• Success factors: What were key issues that we have dealt successfully with?
• Future: How is this model used in future projects, what could further increase its benefits?

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,494
On SlideShare
0
From Embeds
0
Number of Embeds
648
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Controlling Project during Development with a Defect Model, Ben Linders, ICSTest Conference 2004

  1. 1. Controlling Product Quality during Development with a Defect Model 1 February 9, 2004 Ben Linders Controlling Product Quality during Development with a Defect Model ICSTest 2004 Conference, Düsseldorf, April 22 Ben Linders Operational Development & Quality Ericsson R&D, The Netherlands ben.linders@ericsson.com, +31 161 24 9885
  2. 2. Controlling Product Quality during Development with a Defect Model 2 February 9, 2004 Ben Linders Overview • Why a defect model? • How does it work? • Experiences from projects • Conclusions Measurements for product quality and process effectiveness
  3. 3. Controlling Product Quality during Development with a Defect Model 3 February 9, 2004 Ben Linders Ericsson, The Netherlands • Main R&D Design Center • Full product responsibility for Intelligent Networks – Strategic Product Management – Provisioning & total project management – Development & maintenance – Supply & support • 1400 employees, of which 350 in R&D Projects: Quality next to Lead-time and Costs
  4. 4. Controlling Product Quality during Development with a Defect Model 4 February 9, 2004 Ben Linders Purpose Project Defect Model Why? – to control quality of the product during development – and improve development/inspection/test processes Business Benefit: ➨ Higher test efficiency ➨ Better planning & tracking ➨ Early risks signals ➨ Better focus on important issues ➨ Save time and costs ➨ Happy customers!
  5. 5. Controlling Product Quality during Development with a Defect Model 5 February 9, 2004 Ben Linders Test Efficiency • How much testing is needed? – Too little: Product quality risk – Too much: Money/time wasted • Aim/Focus for each test phase? • Budget/time/resource constraints? • Incremental deliveries: What can be tested? Plan Testing & Track/Adjust on defects found.
  6. 6. Controlling Product Quality during Development with a Defect Model 6 February 9, 2004 Ben Linders Defect Flow • Prevent defect insertion • Detect & remove defects where most economical • Track inspection/test progress
  7. 7. Controlling Product Quality during Development with a Defect Model 7 February 9, 2004 Ben Linders Planning & Tracking of Quality • Plan Quality Up Front – Documents/code (# defects made) – Inspection & Test effectiveness (% detection rate) Quality consequence of project decisions • Track Quality during project – Actual # defects found (inspection/test) – Estimate remaining defects: to be found / delivered 'Real-time' prediction of product quality possible Quality view of design/test progress Quicker escalation of quality risks
  8. 8. Controlling Product Quality during Development with a Defect Model 8 February 9, 2004 Ben Linders Measurements: Defect Insertion Defect insertion Target Defect Density: Max 1 major/page per document! Phase Expec- ted #def Expec- ted size Expec- ted DD Act. Size Fnd #def DD Act Not found yet % Foun d % Exp of total Specification 4 10 0.4 10 4 0.40 0 100% 4% High Level Design 12 107 0.112 107 10 0.09 2 83% 12% Detailed Design 12 47 0.255 47 10 0.21 2 83% 12% Im plem entation 70 15000 4.667 13000 18 0.00 52 26% 71% Total 98 42 56 43% 100% • Input: Design team estimates # of defects inserted & expected size • Gathered data during project: Actual defects & size • For each delivery, # of defects in product is calculated: – Indication of the delivered product quality – Input to the test planning: How many defects can be found? – Input to project management to staff/dimension test teams accordingly
  9. 9. Controlling Product Quality during Development with a Defect Model 9 February 9, 2004 Ben Linders Measurements: Defect Detection Defect detection Target detection rate: 70% document, 60% code, 50% test! Avail- Phase Def. Det # Goal %Det % Left Det # Det % Cum % Specification 4 2 70% 50% 2 2 50% 50% High Level Design 14 11 70% 79% 3 11 79% 69% Detailed Design 15 6 70% 40% 9 6 40% 61% Implementation 79 40 60% 51% 39 6 8% 23% Unit test 39 8 20% 21% 31 6 15% 30% Function test 31 15 50% 48% 16 3 10% 33% System Test 16 9 50% 56% 7 3 19% 36% Network Test 7 2 40% 29% 5 2 29% 14% Installation 5 1 15% 20% 4 2 40% 12% First Customer 4 1 10% 25% 3 1 25% 10% Average/Total: 95 42% 42 31% Actual totalExpected in phase • Input: Nr of defects expected to detect & detection rate goal • Gathered data during project: Actual defects & detection rate • Track: Defects found = Product Quality, Test progress
  10. 10. Controlling Product Quality during Development with a Defect Model 10 February 9, 2004 Ben Linders Implementation • Tool: Excel based defect data base & estimation • Frequent estimation & analysis sessions – Ones per week/2 weeks, per project – Duration: 10 minutes – 1 hour, usually ½ hour – Attending: Design, test, quality • Weekly tracking & reporting of product quality • Includes proven techniques: ODC, requirement coverage, test matrices Tailored per project, flexible, result oriented Overall data based on all projects: Planning constants Quality data, additional to time & costs!
  11. 11. Controlling Product Quality during Development with a Defect Model 11 February 9, 2004 Ben Linders Experiences in Pilot Project • Quality tracked during the project: – Specification defects slip through: Clarified requirements in feasibility – Design defects (inspection): Re-enforced design rules – Code quality (inspection/test): Base Product risk, design rules – Test efficiency, defect slip though: Better inspection/Unit Test – Release Quality per requirement: Test focus, risk management • Prediction nr of defects at First Customer Delivery and Release: – Decisions on delivery/release, design follow up and maintenance planning – Actual defects: Expected 21, actual 20 (in 6 months operation) Pilot Project Defect Detection rate: 95% (best in class)!
  12. 12. Controlling Product Quality during Development with a Defect Model 12 February 9, 2004 Ben Linders Experiences from ongoing projects • Classification/analysis of defect with Design & Test Leaders provides very valuable information. • Feedback sessions with Project Management Group are essential for conclusions, and taking actions. • Estimated latent defects supported release decisions. • Defect data improved time & costs planning, and risk management. Also historical data of project support planning of new projects. Though some conclusions from the model are not unique, they would have been missed or discovered too late without the model.
  13. 13. Controlling Product Quality during Development with a Defect Model 13 February 9, 2004 Ben Linders Conclusions Project Defect Model helped the project to: – Estimate/track defects: Improve product release quality, save time/cost – Design/test progress: Better planning, risk management, decisions – Setup Organization: Dimension project teams/maintenance teams Future – Internal & Industry data: Improve estimates – Extend model with cost & planning data – Exchange experiences with similar models? Questions?

×