Deployment of DFSS in software – experience within  Philips Consumer Lifestyle Guy Van Hooveld CTO Office
Objectives of this presentation <ul><li>Give insight in two different approaches to apply DFSS in a software environment w...
The TV experience in Bruges
CMM model <ul><li>  ______  KPA  _____   ( Key Process Area ) </li></ul><ul><li>CMM Level 1 </li></ul><ul><li>CMM Level 2 ...
CMM  model <ul><li>Level 4   SQM  :  definition </li></ul><ul><li>The purpose of Software Quality Management is to develop...
Quality goals to satisfy the needs and  desires of the customer and end user  for high quality products. <ul><li>The refer...
What is SW Quality ? (choices made for a specific project) <ul><ul><li>Stability </li></ul></ul><ul><ul><li>Critical resou...
SQM in a TV project <ul><li>Step 1.  -  create plan  :  SW_Quality_attribute_baseline   </li></ul><ul><li>Step 2.  -  depl...
Plan for a TV project :  overview of CTQ’s x x Critical res. : Memory Budget tracking x Defect densities x NFR conformity ...
Example  (zoom-in)  :  Performance (prod),  QAC (subsys)
TV project :  quality parameters Reporting  Example :
Right design vs design right <ul><li>Right design (what does the consumer want ?) </li></ul><ul><ul><li>DFSS in requiremen...
Lessons learned <ul><li>The CMM L4/DFSS approach has worked to improve quality </li></ul><ul><li>Approach made non-functio...
Experience in Bangalore
Approach in Bangalore <ul><li>More classical DFSS deployment (cf Bruges example) </li></ul><ul><li>Alignment between DFSS ...
DFSS Approach in Software Life cycle CONQ reduction Listen to Consumer needs  (Voice of consumer) Translate needs into CTQ...
Requirements Architecture / Top Level Design Detail Design Implementation Module testing Integration Testing Alpha testing...
Quality process - Software Mapping  <ul><li>Quality process based on 4 main principles </li></ul><ul><ul><li>Identify the ...
Quality - Software Mapping  1- Identify the Voice of customer (VOC) Kano Risk Benefit VPH CRS CTQs , CFs HECE, UIS, FRS
Quality - Software Mapping  2- Manage Critical Parameters CTQ flow down CTQ – Targets & Limits CTQ flow-up & predictions C...
Quality - Software Mapping  3- Establish robust & reliable design Arch/Des choices CTQ Trade-offs CTQ optimisation,  Noise...
Quality - Software Mapping  4 – Manage Risks & Problems User/CEC tests Test coverage Stress Tests Field Tests FMEA PMI, PR...
Gains <ul><li>Software engineering (addressing the pain areas) </li></ul><ul><ul><li>Better Requirements Management in ter...
Issues <ul><ul><li>Everything is linked to CTQs so there is a chance of completely missing important ones </li></ul></ul><...
 
Upcoming SlideShare
Loading in …5
×

Software Reliability CMM-DFSS

2,541 views

Published on

Presentation given during a software reliability seminar organized by Holland Innovative BV - subject was the combination of CMM and DFSS within Philips Consumer Lifestyle

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,541
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
121
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Software Reliability CMM-DFSS

  1. 1. Deployment of DFSS in software – experience within Philips Consumer Lifestyle Guy Van Hooveld CTO Office
  2. 2. Objectives of this presentation <ul><li>Give insight in two different approaches to apply DFSS in a software environment with Philips Consumer Lifestyle </li></ul><ul><ul><li>Bruges TV experience – link between DFSS and CMM L4 </li></ul></ul><ul><ul><li>Bangalore experience – link between DFSS and milestone driven software project management </li></ul></ul><ul><li>Conclude on the gains and limitations of those approaches based on the current experience. </li></ul>
  3. 3. The TV experience in Bruges
  4. 4. CMM model <ul><li> ______ KPA _____ ( Key Process Area ) </li></ul><ul><li>CMM Level 1 </li></ul><ul><li>CMM Level 2 RM Requirements Management </li></ul><ul><li>SLC Software lifecycle </li></ul><ul><li>PP Project Planning </li></ul><ul><li>PTO Project Tracking and Oversight </li></ul><ul><li>SM Subcontract Management </li></ul><ul><li>QA/SCV Quality Assurance/Software Compliance Verification </li></ul><ul><li>SCM Software Configuration Management </li></ul><ul><li>CMM Level 3 OPF Organisation Process Focus </li></ul><ul><li>OPD Organisation Process Definition </li></ul><ul><li>TP Training Program </li></ul><ul><li>IM Integrated Management </li></ul><ul><li>PE Product Engineering </li></ul><ul><li>IC Intergroup Coordination </li></ul><ul><li>PR Peer Reviews </li></ul><ul><li>CMM Level 4 QPM Quantitative Process Management </li></ul><ul><li>SQM Software Quality Management </li></ul><ul><li>CMM Level 5 DP Defect Prevention </li></ul><ul><li>TCM Technology Change Management </li></ul><ul><li>PCM Process Change Management </li></ul>
  5. 5. CMM model <ul><li>Level 4 SQM : definition </li></ul><ul><li>The purpose of Software Quality Management is to develop a quantitative understanding of the quality of the project's software products and achieve specific quality goals. </li></ul><ul><li>Software Quality Management involves defining quality goals for the software products, establishing plans to achieve these goals, and monitoring and adjusting the software plans, software work products, activities, and quality goals to satisfy the needs and desires of the customer and end user for high quality products. </li></ul>
  6. 6. Quality goals to satisfy the needs and desires of the customer and end user for high quality products. <ul><li>The reference framework in the software is CMM (L4) </li></ul><ul><li>Theoretically no actual DFSS deployment as such in the software, but: </li></ul><ul><li>Use of VOC to derive requirements </li></ul><ul><li>Use of derived CTQs to measure software quality </li></ul><ul><li>Focus on non-functional requirements </li></ul><ul><li>CMM philosophy applied + classical software reliability engineering applied </li></ul><ul><ul><li>Fault prevention </li></ul></ul><ul><ul><ul><li>Reviews </li></ul></ul></ul><ul><ul><ul><li>Requirements management </li></ul></ul></ul><ul><ul><ul><li>FMEAs / risk management </li></ul></ul></ul><ul><ul><ul><li>Architecture </li></ul></ul></ul><ul><ul><li>Fault tolerance </li></ul></ul><ul><ul><ul><li>Reboots / watchdogs </li></ul></ul></ul><ul><ul><ul><li>… </li></ul></ul></ul><ul><ul><li>Fault detection/correction </li></ul></ul><ul><ul><ul><li>Verification/validation </li></ul></ul></ul><ul><ul><ul><li>Root cause analysis with feedback loop etc… </li></ul></ul></ul><ul><li>In fact a lot of commonality with DFSS, close to an actual DFSS deployment. </li></ul>
  7. 7. What is SW Quality ? (choices made for a specific project) <ul><ul><li>Stability </li></ul></ul><ul><ul><li>Critical resources : e.g. </li></ul></ul><ul><ul><ul><li>Memory Budget tracking </li></ul></ul></ul><ul><ul><ul><li>CPU load, </li></ul></ul></ul><ul><ul><ul><li>latency, </li></ul></ul></ul><ul><ul><ul><li>bandwidth </li></ul></ul></ul><ul><ul><li>Performance : e.g. startup time, zapping time, UI </li></ul></ul><ul><ul><li>SDE warnings : Koala, compiler </li></ul></ul><ul><ul><li>QAC warnings </li></ul></ul><ul><ul><li>Asserts </li></ul></ul><ul><ul><li>Debugging </li></ul></ul><ul><ul><li>NFR conformity (validation) </li></ul></ul><ul><ul><li>Reviews : Defect densities </li></ul></ul>
  8. 8. SQM in a TV project <ul><li>Step 1. - create plan : SW_Quality_attribute_baseline </li></ul><ul><li>Step 2. - deploy the plan measurements & reporting ( by product, subsystems, suppliers) </li></ul><ul><li>Step 3. - corrective actions on deviations </li></ul>
  9. 9. Plan for a TV project : overview of CTQ’s x x Critical res. : Memory Budget tracking x Defect densities x NFR conformity (validation) x Debugger warnings x x Asserts x x QAC warnings x x SDE warnings : Koala, compiler x x Performance : startup, zapping, UI x x Critical res.: CPU load, latency, bandwidth x Stability Subsystem Platform Product CTQ metrics on the Plan
  10. 10. Example (zoom-in) : Performance (prod), QAC (subsys)
  11. 11. TV project : quality parameters Reporting Example :
  12. 12. Right design vs design right <ul><li>Right design (what does the consumer want ?) </li></ul><ul><ul><li>DFSS in requirements process </li></ul></ul><ul><ul><li>Requirements process – functionality vs quality </li></ul></ul><ul><ul><li>Support from development to marketing </li></ul></ul><ul><li>Design right (how do we implement it correctly ?) </li></ul><ul><ul><li>Software processes </li></ul></ul><ul><ul><li>DFSS in development </li></ul></ul>
  13. 13. Lessons learned <ul><li>The CMM L4/DFSS approach has worked to improve quality </li></ul><ul><li>Approach made non-functional requirements more visible </li></ul><ul><li>When marketing is not capable of providing exact requirements (logical for technical constraints for instance), development can help suggesting CTQs to start discussion. </li></ul><ul><li>FMEAs not efficient enough in the software (risks can be listed but more difficult to make things measurable/w tolerance cf mechanical/electrical) </li></ul><ul><li>A number of measurements reflect mainly process quality, not product quality (defects density) </li></ul><ul><li>Supplier management: </li></ul><ul><ul><li>improve control over critical parameters from our suppliers : Stability, NFR, robustness. </li></ul></ul><ul><ul><li>for some SW-suppliers this is handled as well as possible through provisions in requirements (NFR) and work agreements (measurement & reporting) </li></ul></ul><ul><ul><li>for others : there are still examples where this is difficult to achieve. </li></ul></ul><ul><li>SW CTQ's (and NFR's) are twofold : </li></ul><ul><ul><li>those that contribute mainly to end-product (Stability, performance) </li></ul></ul><ul><ul><li>those that contribute mainly to development trajectory (warnings…) </li></ul></ul><ul><ul><li>Outside SW, only CTQ's contributing to the end-product are considered </li></ul></ul>
  14. 14. Experience in Bangalore
  15. 15. Approach in Bangalore <ul><li>More classical DFSS deployment (cf Bruges example) </li></ul><ul><li>Alignment between DFSS DIDOV phases and project milestones </li></ul><ul><li>Translation of DFSS to software tasks </li></ul><ul><li>This has been applied in numerous projects (wireless audio, DVD-recorder, …) </li></ul><ul><li>Clear results with respect to customer satisfaction, reduced cost of non-quality, first time right design . </li></ul>
  16. 16. DFSS Approach in Software Life cycle CONQ reduction Listen to Consumer needs (Voice of consumer) Translate needs into CTQs with targets and specification limits Flow down the top-level CTQ (Y) into lower level inputs Xs Select Best design to meet the CTQ Predict Capability on “Y” based on current Xs Optimize gap between predicted and desired Verify CTQs on target, Factory Monitor CTQs in Field FMEA & Mistake proofing Req. Mgt Arch, Des & Imp Int. Test Maint Sw Life-cycle D I D O v M
  17. 17. Requirements Architecture / Top Level Design Detail Design Implementation Module testing Integration Testing Alpha testing DFSS mapping to V-model and SPEED Feasibility Beta testing Maintenance VPH CS PRS DR CR MPR Programming Conception Development Maturity Maintenance Define Identify Design Optimize Verify Monitor
  18. 18. Quality process - Software Mapping <ul><li>Quality process based on 4 main principles </li></ul><ul><ul><li>Identify the Voice of the Customer (VOC) </li></ul></ul><ul><ul><ul><li>Obtain & prioritize customer requirements, Derive CTQs </li></ul></ul></ul><ul><ul><li>Manage Critical parameters </li></ul></ul><ul><ul><ul><li>Define targets for CTQs, Flow-down CTQs </li></ul></ul></ul><ul><ul><ul><li>Flow-up CTQs through predictions, Establish control plan </li></ul></ul></ul><ul><ul><li>Establish Robust & Reliable Design </li></ul></ul><ul><ul><ul><li>Identify sources of variation, de-sensitize design </li></ul></ul></ul><ul><ul><ul><li>Optimize Performance, Verify & Validate robustness </li></ul></ul></ul><ul><ul><li>Manage risks & problems </li></ul></ul><ul><ul><ul><li>Anticipate risk, counter or mitigate them </li></ul></ul></ul><ul><ul><ul><li>Detect and solve problems </li></ul></ul></ul>
  19. 19. Quality - Software Mapping 1- Identify the Voice of customer (VOC) Kano Risk Benefit VPH CRS CTQs , CFs HECE, UIS, FRS
  20. 20. Quality - Software Mapping 2- Manage Critical Parameters CTQ flow down CTQ – Targets & Limits CTQ flow-up & predictions Control Plan - Dashboard
  21. 21. Quality - Software Mapping 3- Establish robust & reliable design Arch/Des choices CTQ Trade-offs CTQ optimisation, Noise de-sensitization Transfer functions
  22. 22. Quality - Software Mapping 4 – Manage Risks & Problems User/CEC tests Test coverage Stress Tests Field Tests FMEA PMI, PR/CR tracking Verify CTQs
  23. 23. Gains <ul><li>Software engineering (addressing the pain areas) </li></ul><ul><ul><li>Better Requirements Management in terms of specifications, prioritization, traceability </li></ul></ul><ul><ul><li>Focus on non-functional aspects of usability, interoperability, performance, robustness </li></ul></ul><ul><ul><li>Importance of Execution architecture (State-transitions, CPU loading, Memory) </li></ul></ul><ul><ul><li>Increase in Design effort, upfront design discussions automatically reduce rework </li></ul></ul><ul><ul><li>Early involvement of test team – CTQ measurement method </li></ul></ul><ul><ul><li>CTQs as leading indicators of product quality as against PMI </li></ul></ul><ul><li>Cross-functional approach </li></ul><ul><ul><li>Reduction in communication barrier – way of working with marketing </li></ul></ul><ul><ul><ul><li>e.g. challenging specifications, Kano, risk-benefit analysis </li></ul></ul></ul><ul><ul><li>Common language of CTQs with other disciplines for synergy e.g. hardware, electrical </li></ul></ul><ul><li>End-user perspective </li></ul><ul><ul><li>Development community sensitive to VOC E.g. needs versus wants </li></ul></ul><ul><ul><li>Measurement focus (variation rather than average), best case-worst case spectrum </li></ul></ul><ul><li>System understanding (Product perspective) </li></ul><ul><ul><li>Transfer functions triggers to understand the system better (may expose competency) </li></ul></ul><ul><ul><li>Understanding of noise and its effect on the system (usage environment) </li></ul></ul><ul><ul><li>Trigger on “what could fail” (FMEA, mistake-proofing) improves robustness </li></ul></ul><ul><li>Soft Aspects </li></ul><ul><ul><li>Mindset of “first time right” instead of “build-test-fix” </li></ul></ul>
  24. 24. Issues <ul><ul><li>Everything is linked to CTQs so there is a chance of completely missing important ones </li></ul></ul><ul><ul><li>Does not compensate for domain competency (CTQ flow down, FMEA etc) </li></ul></ul><ul><ul><li>Many requirements in software fall under Yes/No, Pass/Fail category so limit setting limits, target becomes fuzzy (Critical factors rather than CTQs) </li></ul></ul><ul><ul><li>Out of limits need not strictly mean defective as in case of hardware e.g. start-up time, hence predicting DPMO (defects per million opportunities) may not reflect reality </li></ul></ul><ul><ul><li>Random failures due to only software are rare due to which concept like MTBF for software alone is questionable (however makes sense at product level) </li></ul></ul><ul><ul><li>No concept of samples – the same piece of code is corrected and used, so hard-core statistical concepts cannot be applied at software level only (however can be done at product level) </li></ul></ul><ul><ul><li>Configuration Management has to be addressed in the traditional way </li></ul></ul><ul><ul><li>No direct tools available to control regression and side-effects in software </li></ul></ul><ul><ul><li>Additional effort needs to be budgeted in the initial phases (confidence of Project) </li></ul></ul><ul><ul><li>With more and more projects going the ODM way (Black-box model) – not clear on how to select/manage suppliers on software CTQs </li></ul></ul><ul><ul><li>Marketing community still needs to be educated to be able to specify CTQs in a quantifiable way and as part of the requirements specification itself </li></ul></ul>

×