Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Controlling Project Performance using PDM - PSQT2005 - Ben Linders

1,765 views

Published on

• A hands-on model for control of product and process quality.
• Support of release risk decisions based on defect data.
• ODC and Test Matrices applied in different test phases.
• Usage of feedback to analyze data and come to actions.
• Using project data for a business case for improvement.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Controlling Project Performance using PDM - PSQT2005 - Ben Linders

  1. 1. Controlling Project Performance Using the Project Defect Model 1 March 18, 2005 Ben Linders Controlling Project Performance Using the Project Defect Model PSQT 2005 Conference, Las Vegas, May 3 Ben Linders Operational Development & Quality Ericsson R&D, The Netherlands ben.linders@ericsson.com, +31 161 24 9885
  2. 2. Controlling Project Performance Using the Project Defect Model 2 March 18, 2005 Ben Linders Overview • Why a defect model? • How does it work? • Experiences from projects • Conclusions Measurements for product quality and process effectiveness
  3. 3. Controlling Project Performance Using the Project Defect Model 3 March 18, 2005 Ben Linders Ericsson, The Netherlands • Market Unit Northern Europe & Main R&D Design Center • R&D: Intelligent Networks – Strategic Product Management – Product marketing & technical sales support – Provisioning & total project management – Development & maintenance – Customization – Supply & support • 1300 employees, of which 350 in R&D Projects: Quality next to Lead-time and Costs
  4. 4. Controlling Project Performance Using the Project Defect Model 4 March 18, 2005 Ben Linders Purpose Project Defect Model Why? – to control quality of the product during development – and improve development/inspection/test processes Business Benefit: ➨ Better planning & tracking ➨ Early risks signals ➨ Save time and costs ➨ Happy customers!
  5. 5. Controlling Project Performance Using the Project Defect Model 5 March 18, 2005 Ben Linders History of the Model • 2001 – Defined, introduced in first project • 2002 – Used in 2 projects, improved along the way – First release predictions • 2003 – Industrialize model/tool – Used in all (5) major projects • 2004 – Management decisions based on model – New applications: Solution/Total projects, defect flows • 2005 – Extension with Cost of Quality
  6. 6. Controlling Project Performance Using the Project Defect Model 6 March 18, 2005 Ben Linders Modeling Defect Flow Insertion: Where are defects made? How to prevent? Detection: Where are defects found? Early/economic removal?
  7. 7. Controlling Project Performance Using the Project Defect Model 7 March 18, 2005 Ben Linders Process View Process Inputs and outputs Influencing factors Measurement DefectsInserted (documentation, code) DefectsDetected (Inspection, test) (Un)happy customers Design Process Competence, skills Tools, environment Test Process Competence, skills Test Capacity Tools, environment Resident Defectsin Delivered Product Resident Defectsin Design Base Detection Rate Defect Density Fault Slip Through Defect Level Defect Classification
  8. 8. Controlling Project Performance Using the Project Defect Model 8 March 18, 2005 Ben Linders Planning & Tracking of Quality • Plan Quality Up Front – Documents/code (# defects made) – Inspection & Test effectiveness (% detection rate) Quality consequence of project approach • Track Quality during project – Actual # defects found (inspection/test) – Estimate remaining defects: to be found / delivered Quality view of design/test, quicker escalation • Decide based upon Quality Status – Toll Gates (go/no go) and Product Release Product Quality figures
  9. 9. Controlling Project Performance Using the Project Defect Model 9 March 18, 2005 Ben Linders Implementation • Tool: Excel based defect data base & estimation • Frequent estimation & analysis/feedback sessions • Weekly tracking & reporting of product quality • Includes proven techniques: ODC, requirement coverage, test matrices Tailored per project, flexible, result oriented Overall data based on all projects: Planning constants Quality data, additional to time & costs!
  10. 10. Controlling Project Performance Using the Project Defect Model 10 March 18, 2005 Ben Linders Results • Data from the projects • Feedback sessions • Conclusions 11 projects, of which 2 ongoing Incremental development, team based Different size/length: size factor used. RUP based process
  11. 11. Controlling Project Performance Using the Project Defect Model 11 March 18, 2005 Ben Linders Detection rates projects Project detection rates Q1 2005 (PSQT Conference) Proj A Proj B Proj C Proj D Proj E Proj F* Proj G Proj H* Proj J Proj K Proj L Average Rate 95% 95% 90% 59% 97% 86% 93% 88% 91% 94% 93% 91% Size 1 4 1 1 5 3 1 4 1 2 3 * Project still ongoing at time of measurement • Limited variance – Project D different: Integration of products (no design) – Range (excluding D) from 86%-97% (projects F and H still ongoing) • Average detection rate in line with industry figures: – DACS: Typical software projects 15% slip though (85% detection) – Jones: Average 85%, most efficient 95% Analyze/track projects that go below the target performance of 90%
  12. 12. Controlling Project Performance Using the Project Defect Model 12 March 18, 2005 Ben Linders Injection rates phases Phase injection rates, Q1 2005 (PSQT Conference) R e quire m e n A rc hite c tureD e s ig n C o de D o c wa re Rate 7% 18% 12% 58% 4% • Very elaborated architecture (feasibility phase). Many defects made, but most of them are found in the architecture reviews. • Lean design, few defects made. • Most defects made during coding “Normal” defect pattern, sufficient focus on defect prevention.
  13. 13. Controlling Project Performance Using the Project Defect Model 13 March 18, 2005 Ben Linders Detection rates phases, averages Phase detection rates, Q1 2005 (PSQT Conference) R e quire m e n A rc hite c tureD e s ig n C o de D o c wa re F unc tio n Te S ys te m Te s N e two rk Te A v e ra g e Det. Rate 56% 64% 51% 36% 70% 56% 49% 23% 51% • High requirements/architecture/design: effective inspections, good architecture skills • Low code detection: improvement program ongoing • Function & system test: Acceptable rates • Network test, low rate, but defects that are found would give significant problems to customers: Good cost/benefit of the test phase Focus on inspection improvement, capture defects earlier
  14. 14. Controlling Project Performance Using the Project Defect Model 14 March 18, 2005 Ben Linders Detection rates phases, variance 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% R equirem entsArchitecture D esign C ode D ocw areFunction TestSystem TestN etw ork Test Total Det. Rate • Large variance, except for docware & total (excl proj D) Process alignment, standardize & re-use best practices
  15. 15. Controlling Project Performance Using the Project Defect Model 15 March 18, 2005 Ben Linders Feedback sessions • Frequent, short • At the workplace • All data available (Excel) • Design/test leaders Show data ask questions form conclusions take needed actions Feedback sessions enabled earlier conclusions, better acceptance of results, and quick and focused corrective/preventive actions. Feedback: Collected data delivered to the people that have been doing the work, in order to support their understanding of the situation at hand and help them to take needed actions
  16. 16. Controlling Project Performance Using the Project Defect Model 16 March 18, 2005 Ben Linders Conclusions Project Defect Model helps projects to: – Estimate/track defects: Improve product release quality, save time/cost – Design/test progress: Better planning, risk management, decisions Benefits for R&D – Project portfolio: Dimension project teams/maintenance teams – Product quality: Less maintenance, satisfied customers – Employees: More involved, empowered, motivated
  17. 17. Controlling Project Performance Using the Project Defect Model 17 March 18, 2005 Ben Linders Further reading References – Managing the software process. Watts Humphrey. – Metrics and models in Software Quality Engineering. Stephen H. Kan. Papers (see also PSQT conference paper!) – Controlling Product Quality During Development with a Defect Model, in Proceedings ESEPG 2003 – Make what’s counted count, in Better Software magazine march 2004 – Measuring Defects to Control product Quality, in Measure! Knowledge! Action! The NESMA anniversary book. Oct 2004. ISBN: 90-76258-18-X – A Proactive Attitude Towards Quality: The Project Defect Model, Software Quality Professional Dec 2004 (with Hans Sassenburg) Ben Linders, Ericsson R&D, The Netherlands ben.linders@ericsson.com, +31 161 24 9885

×