Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

TSP SYMPOSIUM Schneider, Henry.pptx

230 views

Published on

  • Be the first to comment

  • Be the first to like this

TSP SYMPOSIUM Schneider, Henry.pptx

  1. 1. TSP Symposium Mexico City February 23 – 24, 2016
  2. 2. ®2016 All Rights Reserved - PPQC 2 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Core Concepts •  Iden-fying Sta-s-cal Stability •  Selec-ng Processes and Performance Measures •  Ques-ons and Answers •  Some Common Measures (reference only)
  3. 3. ®2016 All Rights Reserved - PPQC 3 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G
  4. 4. ®2016 All Rights Reserved - PPQC 4 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Performed Process o  A process that accomplishes the needed work to produce work products •  Managed Process o  A performed process that is planned and executed in accordance with policy; employs skilled people having adequate resources to produce controlled outputs; involves relevant stakeholders; is monitored, controlled, and reviewed; and is evaluated for adherence to its process descripLon •  Defined Process o  A managed process that is tailored from the organizaLon’s set of standard processes according to the organizaLon’s tailoring guidelines; has a maintained process descripLon; and contributes process related experiences to the organizaLonal process assets
  5. 5. ®2016 All Rights Reserved - PPQC 5 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Special Cause of Varia-on o  A cause of a defect that is specific to some transient circumstance and is not an inherent part of a process •  Common Cause of Varia-on o  The variaLon of a process that exists because of normal and expected interacLons among components of a process •  Stable Process o  The state in which all special causes of process variaLon have been removed and prevented from recurring so that only common causes of process variaLon of the process remain
  6. 6. ®2016 All Rights Reserved - PPQC 6 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Quality and Process Performance Objec-ves (QPPOs) o  QuanLtaLve objecLves and requirements for product quality, service quality, and process performance •  Capable Process o  A process that can saLsfy its specified product quality, service quality, and process-performance objecLves •  Process Performance o  A measure of actual results achieved by following a process. Process performance is characterized by both process measures (e.g., effort, cycle Lme, and defect removal efficiency) and product or service measures (e.g., reliability, defect density, and response Lme)
  7. 7. ®2016 All Rights Reserved - PPQC 7 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Process Performance Baseline (PPB) o  A documented characterizaLon of process performance, which can include central tendency and variaLon. A process performance baseline can be used as a benchmark for comparing actual process performance against expected process performance
  8. 8. ®2016 All Rights Reserved - PPQC 8 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Process Performance Model (PPM) o  A descripLon of the relaLonships among the measurable aZributes of one or more processes or work products that is developed from historical process performance data and is used to predict future performance o  One or more of the measureable aZributes represent controllable inputs Led to a sub-process to enable performance of “what-if” analyses for planning dynamic re-planning, and problem resoluLon o  PPMs include staLsLcal, probabilisLc, and simulaLon based models that predict interim or final results by connecLng past performance with future outcomes o  PPMs model the variaLon of the factors and provide insight into the expected range and variaLon of predicted results
  9. 9. ®2016 All Rights Reserved - PPQC 9 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Compose the Defined Process o  Composing the Project’s Defined Process (PDP) involves idenLfying alternaLves to one or more processes or sub-processes, performing quanLtaLve analysis of performance and selecLng the alternaLves that are best able to help the project achieve its QPPOs •  Sta-s-cal Process Control (SPC) o  StaLsLcally based analysis of a process and measures of process performance, which will idenLfy common and special causes of variaLon in process performance and maintain process performance within limits
  10. 10. ®2016 All Rights Reserved - PPQC 10 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Sta-s-cal Techniques o  Techniques adapted from the field of mathemaLcal staLsLcs used for acLviLes such as characterizing process performance, understanding process variaLon, and predicLng outcomes. o  Examples of staLsLcal techniques include sampling techniques, analysis of variance (ANOVA), chi-squared tests, and process control charts •  Causal Analysis o  The analysis of outcomes to determine their causes o  Used to address selected issues and diagnose deficiencies in achieving the project’s QPPOs
  11. 11. ®2016 All Rights Reserved - PPQC 11 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G
  12. 12. ®2016 All Rights Reserved - PPQC 12 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Before an organiza-on can perform High Maturity ac-vi-es, it must have the following in place: o  Capability to gather and use data at all organizaLonal levels, i.e., project members who •  Gather data on their own work •  Understand and use the data in planning and performing their work o  Processes that specify how and when data are collected o  Consistent execuLon of the defined process, where tailoring is handled in a controlled and disciplined fashion
  13. 13. ®2016 All Rights Reserved - PPQC 13 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Everything we do in life, including product development and service delivery, is a series of interconnected processes •  All processes are variable •  People performing the process is the largest source of varia-on •  Understanding varia-on is the basis for management by fact and systema-c improvement: o  Understand the past – quan-ta-vely o  Control the present – quan-ta-vely o  Predict the future – quan-ta-vely
  14. 14. ®2016 All Rights Reserved - PPQC 14 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  ScaOer Diagrams •  Run Charts •  Cause-and-Effect Diagrams •  Histograms •  Bar Charts •  Pareto Charts •  Control Charts
  15. 15. ®2016 All Rights Reserved - PPQC 15 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Real process behavior must be understood before drawing reasonable conclusions about where performance needs to be improved •  Ask these ques-ons to find out about real process behavior: o  What is the normal or inherent process variaLon? o  What differenLates inherent from anomalous variaLon? o  What is causing the anomalous variaLon? •  Sta-s-cal Process Control provides the methods and tools needed to measure and analyze process behavior, draw conclusions, and drive improvement
  16. 16. ®2016 All Rights Reserved - PPQC 16 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  A summary or characteriza-on of a distribu-on (i.e., a set of numbers) •  A characteriza-on of a central tendency (e.g., mean, median, and mode) •  A characteriza-on of dispersion (e.g., variance, standard devia-on, interquar-le range, and range)
  17. 17. ®2016 All Rights Reserved - PPQC 17 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Statistics (specifically hypothesis testing) enable you to place a confidence interval on the Central Tendency and Dispersion of the population and on future samples A data population consists of the total possible observations with which you are concerned but to which you do not necessarily have access (X1 thru X15). X5 X15 X14 X13 X12 X11 X6 X10X8 X7 X9 X3 X2 X1 X4 A sample is a set of observations selected from a data population that you can access.
  18. 18. ®2016 All Rights Reserved - PPQC 18 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Hypothesis tes-ng uses sta-s-cal analysis as a formal way of making a comparison and deciding whether or not the difference is sta-s-cally significant •  Hypothesis tes-ng consists of comparing a null and an alterna-ve hypothesis: o  The null hypothesis states that the members of the comparison are staLsLcally equal; there is no difference (a concrete, default posiLon) o  The alternaLve hypothesis states that there is a staLsLcally significant difference; it is supported when the null hypothesis is rejected •  The conclusion either rejects or fails to reject the null hypothesis.
  19. 19. ®2016 All Rights Reserved - PPQC 19 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G When the p is low, the null must go. When the p is high, the null must fly. Note: The p-value is the key output in staLsLcal analysis that students are taught to idenLfy and use to draw a conclusion regarding the hypothesis test comparison or regarding the significance of a staLsLcal model
  20. 20. ®2016 All Rights Reserved - PPQC 20 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G
  21. 21. ®2016 All Rights Reserved - PPQC 21 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  “While every process displays varia1on, some processes display controlled varia1on, while others display uncontrolled varia1on” •  Rou-ne varia-on: o  A stable paZern of random variaLon over Lme o  Typical observaLon within a common cause system •  A process is out of sta-s-cal control if it is opera-ng in the presence of assignable causes o  A factor that is not inherent in the process is affecLng its performance o  VariaLon in the process can be “assigned” to a specific cause •  A process under sta-s-cal control is predictable
  22. 22. ®2016 All Rights Reserved - PPQC 22 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Wheeler’s Empirical Rules: o  60% ≤ ±1σ ≤ 75% o  90% ≤ ±2σ ≤ 98% o  99% ≤ ±3σ ≤ 100% Wheeler’s Empirical Rules are robust with regard to a remarkable number of distribu-ons
  23. 23. ®2016 All Rights Reserved - PPQC 23 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  SPC is a technique that can be used to tell you how much variability there is in a process o  How much variability is due to rouLne variaLon o  How much variability is due to some unique event or factor not inherent in the process (assignable causes) o  A control chart is a tool used in SPC to help do this •  Control charts are generated from two data sets o  PPB data – used to staLsLcally derive a process average, Upper Control Limit (UCL), and Lower Control Limit (LCL) (±3σ) o  Actual performance data – ploZed like a Run Chart and compared against baseline data
  24. 24. ®2016 All Rights Reserved - PPQC 24 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Which of these 20 projects should be acted on? 0 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Number of Non-compliances Project ID
  25. 25. ®2016 All Rights Reserved - PPQC 25 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Different techniques can be used to understand process varia-on o  Simple graphs, such as frequency histograms and bar charts o  StaLsLcal formulae, such as standard deviaLon and range o  StaLsLcal process control tools, such as XmR charts and u-charts o  Analysis of variance techniques, such as ANOVA and hypothesis test
  26. 26. ®2016 All Rights Reserved - PPQC 26 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G ExcepLonal (unexpected) variaLon RouLne (expected) variaLon ExcepLonal (unexpected) variaLon
  27. 27. ®2016 All Rights Reserved - PPQC 27 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Number of Non-compliances Project ID Upper Control Limit = 11.7 Mean = 5 Lower Control Limit = 0
  28. 28. ®2016 All Rights Reserved - PPQC 28 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Upper Control Limit Lower Control Limit Process Average ExcepLonal process variaLon ExcepLonal process variaLon Actual Performance Data from Project XYZ
  29. 29. ®2016 All Rights Reserved - PPQC 29 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 1σ 2σ 3σ 3σ 2σ 1σ ZONE A ZONE A ZONE B ZONE B ZONE C ZONE C UCL LCL Process Average
  30. 30. ®2016 All Rights Reserved - PPQC 30 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  1 single point outside 3σ •  2 of 3 successive values on same side of centerline and outside of 2σ •  8 successive values on same side of centerline
  31. 31. ®2016 All Rights Reserved - PPQC 31 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G “… in so:ware development it is difficult to use control charts in the formal SPC manner. It is a formidable task, if not impossible, to aDempt to define the process capability of a so:ware development process.” - Stephen H. Kan (1995, p. 143) •  Cri-cal assump-on for interpre-ng control charts – data are affected by homogeneous sources of varia-on o  Same populaLon o  Same behavioral implementaLon of the process o  Same common cause system
  32. 32. ®2016 All Rights Reserved - PPQC 32 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G “Despite these issues, control charts are useful for so:ware quality improvement – when they are used in a relaxed manner … control charts use in so:ware is not in terms of formal sta1s1cal process control and process capability … used as a tool for improving consistency and stability … more appropriately called pseudo-control charts.” - Stephen H. Kan (1995, p. 144)
  33. 33. ®2016 All Rights Reserved - PPQC 33 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  System development processes rou-nely operate within the presence of mul-ple common cause systems – PPBs mix natural varia-on with: o  VariaLon in developer skills o  VariaLon in aZributes of work products (e.g. complexity) o  VariaLon in situaLons •  The voice of the process is a cacophony of many voices from many behavioral implementa-ons of the process: o  Confounded by variaLon from many different sources o  Control limits are excessively wide – occasionally 10X range o  Out of control events are difficult to detect o  ‘StaLsLcal stability’ may not offer control or predictability o  A single PPB may provide poor characterizaLon
  34. 34. ®2016 All Rights Reserved - PPQC 34 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Experienced Inexperienced Method & Support Tools No Method & Support Tools Each cell represents a unique common cause system with different mean results
  35. 35. ®2016 All Rights Reserved - PPQC 35 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  In the example, methods and experience levels are: o  Sources of process variaLon that guide the idenLficaLon of common cause systems o  Factors whose variaLon must be controlled in establishing separate common cause systems o  Likely to appear as assignable causes of process variaLon if data from several common cause systems are combined into a single PPB •  Each cell is a different common cause system o  Different PPBs o  Different behavioral implementaLons of the process •  The typical uses of control charts in system development regularly mix data from mul-ple common cause systems
  36. 36. ®2016 All Rights Reserved - PPQC 36 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Control charts are appropriate when: o  The ploZed results characterize process performance within a single common cause system o  Data describe a single behavioral implementaLon of the process o  Process fidelity is high (the process is well-defined and performed in compliance with how it is defined) •  When condi-ons are appropriate for control charts; o  Process stability is a meaningful concept o  AggregaLng PPBs provides a good basis for understanding performance
  37. 37. ®2016 All Rights Reserved - PPQC 37 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Individual Aggregate OR
  38. 38. ®2016 All Rights Reserved - PPQC 38 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Typical ML 4 prac-ces result in plokng aggregated inspec-on results o  Across different developers o  Across different reviewers o  Across components or work products with different aZributes
  39. 39. ®2016 All Rights Reserved - PPQC 39 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Since people are the greatest source of varia-on in system development a more Personal Solware Process (PSP)-like approach: o  Treats individuals as separate common cause systems rather than as assignable causes o  Decreases the width of control limits o  Makes unlikely or unexpected events easier to detect
  40. 40. ®2016 All Rights Reserved - PPQC 40 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G “A:er product size, people factors have the strongest influence in determining the amount of effort required to develop a so:ware product.” - Barry Boehm, et. al. (2000)
  41. 41. ®2016 All Rights Reserved - PPQC 41 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G
  42. 42. ®2016 All Rights Reserved - PPQC 42 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G
  43. 43. ®2016 All Rights Reserved - PPQC 43 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Establish and maintain the organization’s QPPOs, which are traceable to business objectives o  The QPPOs are derived from the organizaLon’s business objecLves and will typically be specific to the organizaLon, group, or funcLon o  The QPPOs take into account what is realisLcally achievable based upon a quanLtaLve understanding (knowledge of variaLon) of the organizaLon’s historic quality and process performance
  44. 44. ®2016 All Rights Reserved - PPQC 44 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  A simple task that many -mes does not specify enough informa-on for use at Maturity Level 4 or 5 •  QPPOs need to be SMART o Specific o Measurable o Achievable o RealisLc o Time-based
  45. 45. ®2016 All Rights Reserved - PPQC 45 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Select measures and processes that provide insight into the organiza-on’s ability to meet its QPPOs o  Create/update clear unambiguous operaLonal definiLons for the selected measures o  Revise and update the set of measures, analyses, and procedures as warranted o  In usage, be sensiLve to measurement error o  The set of measures may provide coverage of the enLre lifecycle and be controllable
  46. 46. ®2016 All Rights Reserved - PPQC 46 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G
  47. 47. ®2016 All Rights Reserved - PPQC 47 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Each project and organiza-on has unique informa-on needs •  When iden-fying goals and objec-ves, consider sources such as o  business and customer needs o  constraints and assumpLons o  project or organizaLon characterisLcs •  domain, technology •  people and skills •  product features and acceptance criteria •  cost, schedule, and milestones o  risks o  past experience of the team
  48. 48. ®2016 All Rights Reserved - PPQC 48 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Enterprise How do we perform, compared to others? Which improvements should we fund? In what training or tools must we invest? Are our customers sa-sfied? Process How consistent is our performance? Which processes require improvement? What is the ROI of improvement efforts? How are improvement efforts progressing? Project How big is this project? Is the project plan feasible? How is the project progressing? When will we be ready to deliver? Product How big is the applica-on? Which modules are have the most defects? What is the cost to maintain? What are the quality and growth trends? Goals Data
  49. 49. ®2016 All Rights Reserved - PPQC 49 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Term Defini-on Data A collecLon of actual values assigned to a measure (example: data for a source lines of code base measure might be 100, 115, 121). Measurement A set of operaLons having the object of determining a value of a measure. Measurement process The process for establishing, planning, performing, and evaluaLng measurement within a project or organizaLon. Measure (noun) The rule for assigning a quanLtaLve or categorical value to one or more enLty/ object/aZribute (e.g. process, project, resource). Informa-on Need Reason for which data or measures are needed; generally phrased as “<who> needs to know <what>, in order to <do what>”
  50. 50. ®2016 All Rights Reserved - PPQC 50 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G • Process Compliance •  Process Efficiency • Process Effec-veness Process Perform- ance • Milestone CompleLon • CriLcal Path Performance • Work Unit Progress • Incremental Capability Schedule and Progress • Physical Size and Stability • FuncLonal Size and Stability Product Size and Stability Source: Prac=cal So>ware and Systems Measurement • FuncLonal Correctness • Supportability – Maintainability • Efficiency • Portability • Usability • Dependability - Reliability Product Quality • Personnel Effort • Financial Performance • Environment and Support Resources Resources and Cost • Technology Suitability • Technology VolaLlity Technology Effec-ve- ness • Customer Feedback • Customer Support Customer Sa-sfac-on
  51. 51. ®2016 All Rights Reserved - PPQC 51 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Balanced Scorecards provide a mechanism to balance o  internal goals and external goals o  past performance and drivers for future performance o  short term and long term performance o  People, Process, and Technology investment •  Categories generally used Financial Outcomes Customer Sa-sfac-on Internal Business Processes Learning and Growth Balanced Scorecard
  52. 52. ®2016 All Rights Reserved - PPQC 52 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  First, establish and communicate a strategy o  set financial objecLves o  idenLfy key needs of customers and market segments; link those to financial goals o  focus on internal business processes and efficiencies in meeLng customer needs o  invest in learning and growth to excel with the processes •  Within the scorecard o  idenLfy how to see financial goals are met o  set targets for meeLng customer needs o  idenLfy process changes or innovaLons needed o  match learning and improvement objecLves to these
  53. 53. ®2016 All Rights Reserved - PPQC 53 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Financial Increase profitability Increase revenue Increase margins Customer Serve more levels of business Use social networking services Internal Processes Increase range of services Provide social networking services Maintain social networking site Learning and Growth Build capability Increase social networking skills Business Goals QPPOs
  54. 54. ®2016 All Rights Reserved - PPQC 54 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Area Measure Target Financial Profit Revenue Growth Margin 25% before tax +30% over last year +5% over last year Customer Mean # business levels served % using social networking services +10% over last year 15% this year Internal Processes (and efficiency) increase range of services Minimize social networking downLme new PMO sales by July web site up 99.9% Learning and Growth # of PMPs # of months of total social networking experience +3 over last year +24 over last year
  55. 55. ®2016 All Rights Reserved - PPQC 55 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  In general, process improvement directly affects the lower half of the Balanced Scorecard o  process development and/or maintenance o  process performance o  learning and growth of process users o  a way of idenLfying and tracing QPPOs •  Changes in these areas can link to o  increased customer saLsfacLon o  increase in customer count or transacLon value o  reducLon in cost to meet customer needs •  These, in turn, translate to financial results
  56. 56. ®2016 All Rights Reserved - PPQC 56 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Financial Develop new business Reduce costs Customer Increase Customer Contact Increase customer value Internal Processes Increase process compliance Improve process quality Improve product quality Learning and Growth Increase skills Retain more employees Raise employee saLsfacLon Overall Goal: Controlled Growth and Sa-sfied Return Business Goals QPPOs
  57. 57. ®2016 All Rights Reserved - PPQC 57 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Areas Objec-ves Measures Financial •  Develop New Business •  Reduce Cost Total revenue Revenue in new business segment ProducLvity (LOC/project hr.) Customer •  Increase Customer Contact •  Increase Customer Value % customers with annual interview % employees visiLng customers customer saLsfacLon % with maintenance contracts Internal Processes •  Increase Process Compliance •  Improve Process Quality •  Improve Product Quality 1-5 on key pracLces % rework % milestones on schedule # warranty hours needed Learning and Growth •  Increase Skills •  Retain More Employees •  Raise Employee SaLsfacLon # cerLfied in key areas % retenLon employee saLsfacLon score
  58. 58. ®2016 All Rights Reserved - PPQC 58 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N GStrategic Goal supporLng goal supporLng goal supporLng goal supporLng goal supporLng goal measure measure measure measure measure performance performance measure measure measure measure Corporate Business Goals and Objec-ves Quality and Process Performance Objec-ves
  59. 59. ®2016 All Rights Reserved - PPQC 59 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N GImproved Profitability OpLmize ROI Improve customer saLsfacLon Improve posiLoning in current market Minimize cost of business Improve operaLons efficiency Overhead rates Product- ivity Customer SaLsfacLon Market Share Revenue Corporate Business Goals and Objec-ves Minimize development Lme Improve review process to remove defects early COQ # CRs # defects Cycle Lme Quality and Process Performance Objec-ves
  60. 60. ®2016 All Rights Reserved - PPQC 60 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Improvement Goals QPPO For small features requested by customers, provide a response next year in half the Lme we do today by using an incremental release model •  Reduce cycle Lme for a release by 50% Reduce negaLve impact of change management on customer saLsfacLon •  Cut response Lme on defect handling by 25% •  Cut response Lme on change requests by 10% each quarter Use peer reviews for key parts of requirements work products on all projects •  Reduce cost of rework from requirements defects by 50% next year
  61. 61. ®2016 All Rights Reserved - PPQC 61 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G
  62. 62. ®2016 All Rights Reserved - PPQC 62 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G
  63. 63. ®2016 All Rights Reserved - PPQC 63 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Core Base and Derived Measures o  Requirements Measures o  So{ware Size Measures o  Cost and Effort Measures o  Schedule and Progress Measures o  Defect Measures •  Other Key Derived Measures o  Earned Value o  ProducLvity/Unit Costs •  Summary Measures for a Program or Organiza-on
  64. 64. ®2016 All Rights Reserved - PPQC 64 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  How used: o  requirements baselined and tracked throughout project o  provides insight into growth and stability; also used to status progress •  Useful measures: o  number of requirements (growth over Lme) o  number of new requirements, changed or deleted requirements (stability/ volaLlity) o  requirements status (approved, tested, implemented) •  Typical coun-ng issues: o  how to count a “requirement” consistently, since size and difficulty vary o  stated vs. derived requirements o  traceability/lack of requirements tool use o  in maintenance, more likely to see change requests
  65. 65. ®2016 All Rights Reserved - PPQC 65 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 100 200 300 400 500 600 700 800 Number of Requirements Project Status Date Requirements Stability Plan Actual Project: OLDMS
  66. 66. ®2016 All Rights Reserved - PPQC 66 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 10 20 30 40 50 60 Feb Mar Apr May Jun Jul Aug Person Hours of Work or Rework Add Requirements Change Requirements
  67. 67. ®2016 All Rights Reserved - PPQC 67 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 70% 73% 73% 76% 79% 80% 81% 82% 82% 83% 8% 10% 10% 13% 13% 15% 14% 13% 13% 12% 4% 6% 6% 6% 6% 5% 4% 4% 4% 2% 18% 11% 11% 5% 2% 0% 1% 1% 1% 2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 6/3/2009 6/10/2009 6/17/2009 6/24/2009 7/1/2009 7/8/2009 7/15/2009 7/22/2009 7/29/2009 8/5/2009 Passed Failed Blocked Unverified
  68. 68. ®2016 All Rights Reserved - PPQC 68 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Size is a cri-cal component of measurement and management o  basis for esLmaLng effort, and quanLfying and managing scope o  basis for producLvity measurement (effort to build a unit/size) o  allows normalizaLon and comparison across systems and projects (defects/size) •  Size Measures o  # of modules, screens/reports o  # of calls o  # of boards o  # of requirements
  69. 69. ®2016 All Rights Reserved - PPQC 69 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 50 100 150 200 250 Plan 1 Plan 2 Plan 3 Actual Number of Components Reused New/Modified
  70. 70. ®2016 All Rights Reserved - PPQC 70 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0.1 1 10 100 1000 10000 0.1 1 10 100 1000 10000 Staff Months Source Lines of Code Thousands Size – Effort Es-ma-ng Rela-onship Project Under ConsideraLon Average -1 Standard DeviaLon +1 Standard DeviaLon
  71. 71. ®2016 All Rights Reserved - PPQC 71 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Effort is es-mated, staffing planned, and actual effort tracked to plan o  overall project level o  by Lme period, by work unit/task, build, phase •  Variance monitored for consistent over or under-run o  underesLmated by size or effort o  unexpectedly complex or defect-prone •  Cost usually derived from effort (plus other costs) •  Useful measures: o  #of labor hrs/FTEs (by event, Lme period) •  Typical coun-ng issues: o  no Lme reporLng tool; tool not used or enforced o  contractors vs. employees; mgt, admin, others not captured o  lack of granularity of plan and actual effort data 0 5 10 15 20 25 30 Q1 Q2 Q3 Q4 Financial Summary Plan Actual
  72. 72. ®2016 All Rights Reserved - PPQC 72 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 2 4 6 8 10 12 14 1 2 3 4 5 6 7 8 9 10 11 12 Staff Hours (thousands) Repor-ng Periods Plan Actual
  73. 73. ®2016 All Rights Reserved - PPQC 73 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 1 2 3 4 5 6 7 8 9 10 1-Aug 1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May 1-Jun 1-Jul 1-Aug 1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May Effort Alloca-on Subcontractor Actual Internal Actual
  74. 74. ®2016 All Rights Reserved - PPQC 74 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Ac-vity % Plan Hours Project Management 12% 9654 ConfiguraLon Management 8% 6436 Quality Assurance 3% 2414 Training 4% 3218 Requirements 9% 7241 Design 12% 9654 Code and Unit Test 20% 16090 IntegraLon Test 16% 12872 User Acceptance Test 9% 7241 Maintenance 5% 4023 Customer Support 2% 1609 100% 80450
  75. 75. ®2016 All Rights Reserved - PPQC 75 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 10 20 30 40 50 60 70 80 90 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Time to Close (days) Ticket #
  76. 76. ®2016 All Rights Reserved - PPQC 76 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  How used: o  examine planned schedule and milestones and actual aZainment of each o  monitor task compleLon at a level of detail to be able to recover from problems •  Useful measures: o  milestone dates (schedule) o  days late o  work unit progress (requirements, design, code, test, etc.) •  # of units planned to be complete, # completed o  schedule accuracy/performance •  Typical Coun-ng Issues: o  plan data not baselined; no CM control o  work not broken down into manageable work units o  work units of different size, complexity, and importance
  77. 77. ®2016 All Rights Reserved - PPQC 77 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Milestone Original Revised Actual Proposal Complete 14 January 17 January 20 January Approval to Proceed 1 03 February 03 February 03 February Reqmts Signed Off 08 April 15 April 15 April Approval to Proceed 2 02 May 02 May 05 May Design Signed Off 10 June 14 July 15 July System Test Complete 17 July 10 August Beta Test Complete 03 October 07 November Approval to Proceed 3 14 October 10 November Release to Customer 10 November 16 December
  78. 78. ®2016 All Rights Reserved - PPQC 78 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G
  79. 79. ®2016 All Rights Reserved - PPQC 79 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 20 40 60 80 100 120 140 # of Units Comple-ng Unit Test Project Status Date Unit Test Status Plan Actual
  80. 80. ®2016 All Rights Reserved - PPQC 80 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  How used: o  idenLfies the number of defects (problem reports) in the so{ware product o  provides insight into test progress and both pre- and post-producLon quality o  provides insight into rework remaining •  Useful measures: o  defect trends (open, closed, severity, age of open, types) o  defect densiLes (by system, subsystem, module) o  defect origins, detecLon effecLveness, and leakage •  Typical Coun-ng Issues: o  defects versus effort; rework not always 1:1 o  definiLon of a defect/classificaLon inconsistencies o  cultural/behavioral issues associated with collecLng
  81. 81. ®2016 All Rights Reserved - PPQC 81 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 5 5 14 4 8 18 10 10 4 18 8 4 9 9 10 72 66 74 67 60 60 57 57 48 52 45 40 38 39 37 11 6 14 15 18 13 5 12 11 10 10 11 6 8 0 10 20 30 40 50 60 70 80 Number of Defects Status of Severity 1 Defects New Open (Including New) Worked off During the Week Open Last Week = Closed This Week = Downgraded = New This Week = Upgraded = Open This Week = 39 8 4 ------ 27 10 0 ------ 37
  82. 82. ®2016 All Rights Reserved - PPQC 82 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 1017 1028 1034 1039 1048 1063 1077 1078 1096 1097 1111 1118 1129 1143 1147 1165 1179 1191 71 72 108 124 125 379 454 454 484 509 510 522 605 653 726 776 751 782 0 200 400 600 800 1000 1200 1400 Number of Defects Severity 2 Defects Severity 2 DRs Total Discovered Severity 2 DRs Total Closed
  83. 83. ®2016 All Rights Reserved - PPQC 83 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 10 20 30 40 50 60 70 80 90 <1 1 - 2 3 - 4 5 - 8 9 - 18 >18 Number of Problem Reports Weeks Open Problem Report Aging Open Problem Reports Data as of 15 June 2009 Average = 5.7 weeks Target < 8 weeks
  84. 84. ®2016 All Rights Reserved - PPQC 84 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 4 9 8 10 6 0 2 4 6 8 10 12 # of Severity 1 Defects Status of 37 Open Severity 1 Defects Pending Valida-on Defect Under Analysis Fix Started – Work in Progress Fix Made – Pending Delivery Fix Delivered – Pending Test
  85. 85. ®2016 All Rights Reserved - PPQC 85 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 5 10 15 20 25 30 35 40 A B C D E F # of Problem Reports Module ID Problem Report Status Open Severity 1 and 2 by Module Data as of 15 June 2009
  86. 86. ®2016 All Rights Reserved - PPQC 86 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Module Size (KSLOC) Defects Defect Density (Defects/KSLOC) A 44 48 1.1 B 32 60 1.9 C 36 36 1.0 D 28 33 1.2 E 34 42 1.2 F 15 46 3.1 G 9 30 3.3 Total 198 295 1.5 Data as of 15 June 2009
  87. 87. ®2016 All Rights Reserved - PPQC 87 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 20 40 60 80 100 120 140 160 180 Reqmts Design Code Unit Test Integ Test Sys Test Del + 90 days Defects/KSLOC Life Cycle Phase Defect Detec-on Profile All Projects New Process
  88. 88. ®2016 All Rights Reserved - PPQC 88 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Phase Discovered Phase Inserted Reqmts Analysis Design Code Integ. Test Sys Test Accept. Test Ops Requirements 50 Analysis 65 65 Design 70 80 65 Code & Unit Test 74 83 85 70 Integra-on Test 76 89 94 88 55 System Test 85 93 95 93 70 50 Acceptance Test 94 95 96 94 75 70 70 1 year Opera-ons 98 97 98 96 88 90 90 76 Columns present cumula=ve % of defects removed
  89. 89. ®2016 All Rights Reserved - PPQC 89 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Core Base and Derived Measures o  Requirements Measures o  So{ware Size Measures o  Cost and Effort Measures o  Schedule and Progress Measures o  Defect Measures •  Other Key Derived Measures o  Earned Value o  ProducLvity/Unit Costs •  Summary Measures for a Program or Organiza-on
  90. 90. ®2016 All Rights Reserved - PPQC 90 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 20 40 60 80 100 120 140 160 3 6 9 12 15 18 21 Cost ($K) Months ACWP BCWP Now Cost Variance Actual Cost of Work Performed (ACWP) Budgeted Cost of Work Performed (BCWP) Cost Variance (CV) is the difference between actual and budgeted costs (BCWP - ACWP) CPI = .63 (<1 over budget)
  91. 91. ®2016 All Rights Reserved - PPQC 91 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 20 40 60 80 100 120 140 160 3 6 9 12 15 18 21 Cost ($K) Months BCWS BCWP Now Schedule Varianc Budgeted Cost of Work Scheduled (BCWS) Budgeted Cost of Work Performed (BCWP) Schedule Variance (SV) is the difference between budgeted cost for work performed and work scheduled (BCWP- BCWS) SPI = .52 (<1 behind schedule)
  92. 92. ®2016 All Rights Reserved - PPQC 92 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 20 40 60 80 100 120 140 160 180 Proposal Actuals to Date Re-Plan 1 (2 builds) Re-Plan 2 (3 builds) SLOCs/Person Month Build Produc-vity Build 1 Build 2 Build 3 15 June 2009
  93. 93. ®2016 All Rights Reserved - PPQC 93 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Core Base and Derived Measures o  Requirements Measures o  So{ware Size Measures o  Cost and Effort Measures o  Schedule and Progress Measures o  Defect Measures •  Other Key Derived Measures o  Earned Value o  ProducLvity/Unit Costs •  Summary Measures for a Program or an Organiza-on
  94. 94. ®2016 All Rights Reserved - PPQC 94 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Performance to Schedule milestone baseline forecast actual xxx 1/03 1/03 1/02 yyy 6/16 6/30 zzz 7/04 8/05 Risks •  Descrip-ons •  Ac-on Plans Overall Status (Red/Yellow/Green): Project Alpha as of April 0 10 20 30 40 1st Qtr 2nd Qtr 3rd Qtr 4th Qtr Financial Summary Plan Actual 0 200 400 600 1st Qtr 2nd Qtr 3rd Qtr 4th Qtr Func-onality Delivered Actual Plan
  95. 95. ®2016 All Rights Reserved - PPQC 95 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G 0 5 10 15 20 25 30 35 40 45 Monthly Effort (FTE Hours) Actual Projected Budget 0.72 0.72 0.17 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 Cumula-ve Project Costs ($100K) Projected Budget Actual Cost Situa-on Current Forecast Month Total Month Total DeviaLon more than 30% DeviaLon more than 15% MeeLng budget Milestone Envision Start Vision/Scope Planning Ends Devel Complete V&V Compete Beta Start Beta Live GA Project Closed Delay more than 60 days Delay more than 30 days On Target or Early 0 5 10 15 20 25 30 35 40 45 Monthly Effort (FTE Hours) Actual Projected Budget 0.72 0.72 0.17 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 Cumula-ve Project Costs ($100K) Projected Budget Actual Schedule Situa-on Milestone Curr Last Envision Start Vision/Scope Planning Ends Devel Complete V&V Compete Beta Start Beta Live GA Project Closed Delay more than 60 days Delay more than 30 days On Target or Early
  96. 96. ®2016 All Rights Reserved - PPQC 96 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Size Cost and Effort Schedule Quality Process Perfor- mance Customer Sa-sfac- -on Risk Quick Overview Suppor-ng Comments <Project Name> Monthly Status Report Technology Effec-veness Size Cost and Effort Schedule Quality and Process Performance Risk and Customer Sa-sfac-on Tech. Effec-ve- ness
  97. 97. ®2016 All Rights Reserved - PPQC 97 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Suite of indicators for each area of reporLng for review by project managers, team members, and business managers Total Physical Size -10 0 10 20 30 40 50 60 08-jun 22-jun 06-jul 20-jul 03-aug 17-aug 31-aug KLOC EsLmated Actual Milestone Trend Module Size 0 5 10 15 20 Module A Module B Module C KLOC Current esLmate Actual Milestone Tracking 19-may 29-may 08-jun 18-jun 28-jun 08-jul MST1 P MST1 A MST2 P MST2 A Milestones actual/planned Milestone Status Completed on Lme 20% Completed Late 10% Not yet completed 70% Cost and Schedule Performance -40 -35 -30 -25 -20 -15 -10 -5 0 5 10 Apr-08 Jul-08 Oct-08 Jan-09 Apr-09 Jul-09 Oct-09 Jan-10 Apr-10 Jul-10 Oct-10 Percent variance Schedule Performance Variance Cost Performance Variance Cri-cal Ra-o 0,40 0,60 0,80 1,00 1,20 1,40 Apr-08 Jul-08 Oct-08 Jan-09 Apr-09 Jul-09 Oct-09 Jan-10 Apr-10 Jul-10 Oct-10 CriLcal RaLo Threshold (red) Threshold (yellow)
  98. 98. ®2016 All Rights Reserved - PPQC 98 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G •  Issues in solware projects generally require use of common core measures to handle o  requirements growth and stability o  monitoring of product size o  management of cost and effort to budget o  schedule and other progress indicators o  monitoring of defect levels, to see trends •  Derived measures provide insight with combina-ons of base measures
  99. 99. ®2016 All Rights Reserved - PPQC 99 4/18/16 PPQC PROCESS AND PRODUCT Q U A L I T Y C O N S U LT I N G Address: Henry Schneider President/Senior Principal Consultant Process and Product Quality Consul-ng, LLC 2009 Simond Avenue, Suite B Aus-n, TX 78723 Phone: 832-628-2486 E-mail: henry@ppqc.net Web Site: www.ppqc.net

×