The sequel: another year using the Project Defect Model, Ben Linders, European SEPG 2004
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

The sequel: another year using the Project Defect Model, Ben Linders, European SEPG 2004

  • 597 views
Uploaded on

Last year I presented the Project Defect Model, a defect measurement model piloted in one large project. Meanwhile the model has been used in a multiple of projects, varying from small to large,......

Last year I presented the Project Defect Model, a defect measurement model piloted in one large project. Meanwhile the model has been used in a multiple of projects, varying from small to large, and from one delivery to multiple increments with various in between deliveries. This presentation shows how the model evolved, the benefits, and what we have learned on defect prevention.
To able to use the model in a broad set of projects, a template model was made, including parts that are applicable in most projects in a configurable way. Also a user guide, with industry reference data, and overview presentation has been made.
The model was used in several smaller projects with a single team. Main benefit was comprehensive defect information, used in release decisions. Also the model was used in several larger projects, some still ongoing. There the model provides more insight in the defect flows and process performance, supporting early quality risk detection and process improvement. Finally the model is used in several subprojects from one total project, making interdependencies clearer, enabling better planning and more reliable product release decisions.
The organization learned a lot while using the model. Initially focus was on earlier defect detection, e.g. function- iso system test and inspection iso test. With the increase of data, more insight is acquired in design and coding activities, providing means for defect prevention. Now that data is available from a larger set of projects, processes can be compared enabling decisions about best practices from one project that can be spread towards future projects. Frequent estimation & feedback sessions with the design & test teams show that it is initially difficult to provide reliable estimates, but defect data that is acquired during the project enables early action taking, and better defect estimates resulting in early release quality predictions.

More in: Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
597
On Slideshare
597
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
5
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The sequel: another year using the Project Defect Model ESEPG 2004 Conference, London, June 14 Ben Linders Operational Development & Quality Ericsson R&D, The Netherlands ben.linders@ericsson.com, +31 161 24 9885The sequel: Another year using the Project Defect Model 1 May 4, 2004 Ben Linders
  • 2. Overview• Why a defect model?• How does it work?• Experiences from projects• Conclusions Measurements for product quality and process effectiveness The sequel: Another year using the Project Defect Model 2 May 4, 2004 Ben Linders
  • 3. Ericsson, The Netherlands• Benelux Market Unit & Main R&D Design Center• R&D: Intelligent Networks – Strategic Product Management – Product marketing & technical sales support – Provisioning & total project management – Development & maintenance – Customization – Supply & support• 1300 employees, of which 350 in R&D Projects: Quality next to Lead-time and Costs The sequel: Another year using the Project Defect Model 3 May 4, 2004 Ben Linders
  • 4. Purpose Project Defect Model Why? – to control quality of the product during development – and improve development/inspection/test processes Business Benefit: Better planning & tracking Early risks signals Save time and costs Happy customers!The sequel: Another year using the Project Defect Model 4 May 4, 2004 Ben Linders
  • 5. History of the Model• 2001 – Defined, introduced in first project• 2002 – Used in 2 projects, improved along the way – First release predictions• 2003 – Industrialize model/tool – First results presented at ESEPG 2003 – Used in all (5) major projects• 2004 – Target defined (Balanced ScoreCard) – New applications: Total projects, defect flows The sequel: Another year using the Project Defect Model 5 May 4, 2004 Ben Linders
  • 6. Modeling Defect FlowInsertion: Where are defects made? How to prevent?Detection: Where are defects found? Early/economic removal?The sequel: Another year using the Project Defect Model 6 May 4, 2004 Ben Linders
  • 7. Planning & Tracking of Quality• Plan Quality Up Front – Documents/code (# defects made) – Inspection & Test effectiveness (% detection rate) Quality consequence of project decisions• Track Quality during project – Actual # defects found (inspection/test) – Estimate remaining defects: to be found / delivered Real-time prediction of product quality possible Quality view of design/test progress Quicker escalation of quality risks The sequel: Another year using the Project Defect Model 7 May 4, 2004 Ben Linders
  • 8. Implementation• Tool: Excel based defect data base & estimation• Frequent estimation & analysis/feedback sessions• Weekly tracking & reporting of product quality• Includes proven techniques: ODC, requirement coverage, test matrices Tailored per project, flexible, result oriented Overall data based on all projects: Planning constants Quality data, additional to time & costs! The sequel: Another year using the Project Defect Model 8 May 4, 2004 Ben Linders
  • 9. Results• Data from the projects• Feedback sessions• Conclusions 7 projects, of which 3 ongoing Incremental development, team based Different size/length: size factor used. RUP based process The sequel: Another year using the Project Defect Model 9 May 4, 2004 Ben Linders
  • 10. Detection rates projectsProject detection rates (inspections & test) Proj A Proj B Proj C Proj D Proj E* Proj F* Proj G* AverageRate 95% 95% 90% 59% 94% 86% 89% 90%Size 1 4 1 1 5 3 1 * Project still ongoing at time of measurement• Big projects have a better detection rate: – More extensive test phases – Interdependencies/risks between projects clear, quicker actions – Incremental development, learning from first increments brings benefits• Average detection rate in line with industry figures: – DACS: Typical software projects 15% slip though (85% detection) – Jones: Average 85%, most efficient 95% Analyze/track projects that go below the target performance of 90%The sequel: Another year using the Project Defect Model 10 May 4, 2004 Ben Linders
  • 11. Injection rates phasesPhase injection rates Requirements Architecture Design CodeRate 6% 21% 15% 58%• Very elaborated architecture (feasibility phase). Many defects made, most of them are found in the architecture reviews.• Lean design.• Most defects made during coding “Normal” defect pattern, with sufficient focus in all phases on defect prevention.The sequel: Another year using the Project Defect Model 11 May 4, 2004 Ben Linders
  • 12. Detection rates phasesPhase detection rates Requirements Architecture Design Code Function Test System Test Netw ork Test TotalRate 30% 67% 66% 40% 48% 48% 27% 47%• Lower detection rate in requirements phase: incremental development, start when only part of requirements is stable• High architecture/design: effective inspections, good architecture skills• Lower code detection: one project just starting with code inspections (when excluded from measurement: 50% code detection rate)• Function & system test: Acceptable rates• Network test, low rate, but defects that are found would give major problems to customers: Good cost/benefit of the test phase Focus on inspection improvement & test focus, capture defects earlierThe sequel: Another year using the Project Defect Model 12 May 4, 2004 Ben Linders
  • 13. Feedback sessions• Frequent, short Feedback: Collected data delivered to the• At the workplace people that have been doing the work, in order• All data available (Excel) to support their understanding of the situation at• Design/test leaders hand and help them to take needed actions Show data ask questions form conclusions take needed actions Feedback sessions enabled earlier conclusions, better acceptance of results, and quick and focused corrective/preventive actions. The sequel: Another year using the Project Defect Model 13 May 4, 2004 Ben Linders
  • 14. ConclusionsProject Defect Model helps projects to:– Estimate/track defects: Improve product release quality, save time/cost– Design/test progress: Better planning, risk management, decisionsBenefits for R&D– Project portfolio: Dimension project teams/maintenance teams– Product quality: Less maintenance, satisfied customers– Employees: More involved, empowered, motivatedThe sequel: Another year using the Project Defect Model 14 May 4, 2004 Ben Linders
  • 15. Further readingReferences – Managing the software process. Watts Humphrey. – Metrics and models in Software Quality Engineering. Stephen H. Kan.Papers – Controlling Product Quality During Development with a Defect Model, Proceedings ESEPG 2003 – Make what’s counted count, Better Software magazine march 2004 Ben Linders, Ericsson R&D, The Netherlands ben.linders@ericsson.com, +31 161 24 9885The sequel: Another year using the Project Defect Model 15 May 4, 2004 Ben Linders