Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The sequel: Another year using the Project Defect Model 1 May 4, 2004 Ben Linders The sequel: another year using the Proje...
The sequel: Another year using the Project Defect Model 2 May 4, 2004 Ben Linders Overview • Why a defect model? • How doe...
The sequel: Another year using the Project Defect Model 3 May 4, 2004 Ben Linders Ericsson, The Netherlands • Benelux Mark...
The sequel: Another year using the Project Defect Model 4 May 4, 2004 Ben Linders Purpose Project Defect Model Why? – to c...
The sequel: Another year using the Project Defect Model 5 May 4, 2004 Ben Linders History of the Model • 2001 – Defined, i...
The sequel: Another year using the Project Defect Model 6 May 4, 2004 Ben Linders Modeling Defect Flow Insertion: Where ar...
The sequel: Another year using the Project Defect Model 7 May 4, 2004 Ben Linders Planning & Tracking of Quality • Plan Qu...
The sequel: Another year using the Project Defect Model 8 May 4, 2004 Ben Linders Implementation • Tool: Excel based defec...
The sequel: Another year using the Project Defect Model 9 May 4, 2004 Ben Linders Results • Data from the projects • Feedb...
The sequel: Another year using the Project Defect Model 10 May 4, 2004 Ben Linders Detection rates projects • Big projects...
The sequel: Another year using the Project Defect Model 11 May 4, 2004 Ben Linders Injection rates phases • Very elaborate...
The sequel: Another year using the Project Defect Model 12 May 4, 2004 Ben Linders Detection rates phases • Lower detectio...
The sequel: Another year using the Project Defect Model 13 May 4, 2004 Ben Linders Feedback sessions • Frequent, short • A...
The sequel: Another year using the Project Defect Model 14 May 4, 2004 Ben Linders Conclusions Project Defect Model helps ...
The sequel: Another year using the Project Defect Model 15 May 4, 2004 Ben Linders Further reading Papers – Controlling Pr...
Upcoming SlideShare
Loading in …5
×

The sequel: another year using the Project Defect Model, Ben Linders, European SEPG 2004

1,911 views

Published on

Last year I presented the Project Defect Model, a defect measurement model piloted in one large project. Meanwhile the model has been used in a multiple of projects, varying from small to large, and from one delivery to multiple increments with various in between deliveries. This presentation shows how the model evolved, the benefits, and what we have learned on defect prevention.
To able to use the model in a broad set of projects, a template model was made, including parts that are applicable in most projects in a configurable way. Also a user guide, with industry reference data, and overview presentation has been made.
The model was used in several smaller projects with a single team. Main benefit was comprehensive defect information, used in release decisions. Also the model was used in several larger projects, some still ongoing. There the model provides more insight in the defect flows and process performance, supporting early quality risk detection and process improvement. Finally the model is used in several subprojects from one total project, making interdependencies clearer, enabling better planning and more reliable product release decisions.
The organization learned a lot while using the model. Initially focus was on earlier defect detection, e.g. function- iso system test and inspection iso test. With the increase of data, more insight is acquired in design and coding activities, providing means for defect prevention. Now that data is available from a larger set of projects, processes can be compared enabling decisions about best practices from one project that can be spread towards future projects. Frequent estimation & feedback sessions with the design & test teams show that it is initially difficult to provide reliable estimates, but defect data that is acquired during the project enables early action taking, and better defect estimates resulting in early release quality predictions.

Published in: Business, Technology
no profile picture user

  • Be the first to comment

  • Be the first to like this

The sequel: another year using the Project Defect Model, Ben Linders, European SEPG 2004

  1. 1. The sequel: Another year using the Project Defect Model 1 May 4, 2004 Ben Linders The sequel: another year using the Project Defect Model ESEPG 2004 Conference, London, June 14 Ben Linders Operational Development & Quality Ericsson R&D, The Netherlands ben.linders@ericsson.com, +31 161 24 9885
  2. 2. The sequel: Another year using the Project Defect Model 2 May 4, 2004 Ben Linders Overview • Why a defect model? • How does it work? • Experiences from projects • Conclusions Measurements for product quality and process effectiveness
  3. 3. The sequel: Another year using the Project Defect Model 3 May 4, 2004 Ben Linders Ericsson, The Netherlands • Benelux Market Unit & Main R&D Design Center • R&D: Intelligent Networks – Strategic Product Management – Product marketing & technical sales support – Provisioning & total project management – Development & maintenance – Customization – Supply & support • 1300 employees, of which 350 in R&D Projects: Quality next to Lead-time and Costs
  4. 4. The sequel: Another year using the Project Defect Model 4 May 4, 2004 Ben Linders Purpose Project Defect Model Why? – to control quality of the product during development – and improve development/inspection/test processes Business Benefit: Better planning & tracking Early risks signals Save time and costs Happy customers!
  5. 5. The sequel: Another year using the Project Defect Model 5 May 4, 2004 Ben Linders History of the Model • 2001 – Defined, introduced in first project • 2002 – Used in 2 projects, improved along the way – First release predictions • 2003 – Industrialize model/tool – First results presented at ESEPG 2003 – Used in all (5) major projects • 2004 – Target defined (Balanced ScoreCard) – New applications: Total projects, defect flows
  6. 6. The sequel: Another year using the Project Defect Model 6 May 4, 2004 Ben Linders Modeling Defect Flow Insertion: Where are defects made? How to prevent? Detection: Where are defects found? Early/economic removal?
  7. 7. The sequel: Another year using the Project Defect Model 7 May 4, 2004 Ben Linders Planning & Tracking of Quality • Plan Quality Up Front – Documents/code (# defects made) – Inspection & Test effectiveness (% detection rate) Quality consequence of project decisions • Track Quality during project – Actual # defects found (inspection/test) – Estimate remaining defects: to be found / delivered 'Real-time' prediction of product quality possible Quality view of design/test progress Quicker escalation of quality risks
  8. 8. The sequel: Another year using the Project Defect Model 8 May 4, 2004 Ben Linders Implementation • Tool: Excel based defect data base & estimation • Frequent estimation & analysis/feedback sessions • Weekly tracking & reporting of product quality • Includes proven techniques: ODC, requirement coverage, test matrices Tailored per project, flexible, result oriented Overall data based on all projects: Planning constants Quality data, additional to time & costs!
  9. 9. The sequel: Another year using the Project Defect Model 9 May 4, 2004 Ben Linders Results • Data from the projects • Feedback sessions • Conclusions 7 projects, of which 3 ongoing Incremental development, team based Different size/length: size factor used. RUP based process
  10. 10. The sequel: Another year using the Project Defect Model 10 May 4, 2004 Ben Linders Detection rates projects • Big projects have a better detection rate: – More extensive test phases – Interdependencies/risks between projects clear, quicker actions – Incremental development, learning from first increments brings benefits • Average detection rate in line with industry figures: – DACS: Typical software projects 15% slip though (85% detection) – Jones: Average 85%, most efficient 95% Analyze/track projects that go below the target performance of 90% * Project still ongoing at time of measurement Project detection rates (inspections & test) Proj A Proj B Proj C Proj D Proj E* Proj F* Proj G* Average Rate 95% 95% 90% 59% 94% 86% 89% 90% Size 1 4 1 1 5 3 1
  11. 11. The sequel: Another year using the Project Defect Model 11 May 4, 2004 Ben Linders Injection rates phases • Very elaborated architecture (feasibility phase). Many defects made, most of them are found in the architecture reviews. • Lean design. • Most defects made during coding “Normal” defect pattern, with sufficient focus in all phases on defect prevention. Phase injection rates Requirements Architecture Design Code Rate 6% 21% 15% 58%
  12. 12. The sequel: Another year using the Project Defect Model 12 May 4, 2004 Ben Linders Detection rates phases • Lower detection rate in requirements phase: incremental development, start when only part of requirements is stable • High architecture/design: effective inspections, good architecture skills • Lower code detection: one project just starting with code inspections (when excluded from measurement: 50% code detection rate) • Function & system test: Acceptable rates • Network test, low rate, but defects that are found would give major problems to customers: Good cost/benefit of the test phase Focus on inspection improvement & test focus, capture defects earlier Phase detection rates Requirements Architecture Design Code Function Test System Test Netw ork Test Total Rate 30% 67% 66% 40% 48% 48% 27% 47%
  13. 13. The sequel: Another year using the Project Defect Model 13 May 4, 2004 Ben Linders Feedback sessions • Frequent, short • At the workplace • All data available (Excel) • Design/test leaders Show data ask questions form conclusions take needed actions Feedback sessions enabled earlier conclusions, better acceptance of results, and quick and focused corrective/preventive actions. Feedback: Collected data delivered to the people that have been doing the work, in order to support their understanding of the situation at hand and help them to take needed actions
  14. 14. The sequel: Another year using the Project Defect Model 14 May 4, 2004 Ben Linders Conclusions Project Defect Model helps projects to: – Estimate/track defects: Improve product release quality, save time/cost – Design/test progress: Better planning, risk management, decisions Benefits for R&D – Project portfolio: Dimension project teams/maintenance teams – Product quality: Less maintenance, satisfied customers – Employees: More involved, empowered, motivated
  15. 15. The sequel: Another year using the Project Defect Model 15 May 4, 2004 Ben Linders Further reading Papers – Controlling Product Quality During Development with a Defect Model, Proceedings ESEPG 2003 – Make what’s counted count, Better Software magazine march 2004 References – Managing the software process. Watts Humphrey. – Metrics and models in Software Quality Engineering. Stephen H. Kan. Ben Linders, Ericsson R&D, The Netherlands ben.linders@ericsson.com, +31 161 24 9885

×