Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Project Metrics & Measures


Published on

Roadmap to gain agreement on key metrics vital to measure project success

Published in: Business, Technology
  • Login to see the comments

Project Metrics & Measures

  1. 1. Project Metrics & Measures Roadmap to gain agreement on key metrics vital to measure project success Anand Subramaniam
  2. 2. <ul><li>“If Columbus had turned back, no one would have blamed him. Of course, no one would have remembered him either.” </li></ul><ul><ul><li>- Source Unknown </li></ul></ul>
  3. 3. Highlights <ul><li>Project Metric & Measures - Overview </li></ul><ul><li>Project Metric & Measures - Roadmap </li></ul><ul><li>Project Process Metric - Model & Guidance </li></ul><ul><li>Project Metrics </li></ul><ul><li>Project Measures </li></ul><ul><li>Monitor & Control </li></ul><ul><li>Earned Value </li></ul>
  4. 4. Project Metric & Measures - Overview
  5. 5. Definition Metrics Provides measuring units to depict values, thresholds, constraints, scope, duration, maximums and minimums, averages, etc. Measures Represent information used to establish a common understanding of status, condition, and position of something
  6. 6. Definition Metrics A precise description of what is measured including: numerator, denominator, inclusions and exclusions Targets The agreed upon metric value the organisation desires to achieve
  7. 7. Metric - Challenges Metric Challenges No defined approach to measure return on investment No data validation for accuracy Lack of resources to perform function No formal approach to establish measures and metrics (program & projects) Lack data for reviewing metrics pre & post-live reporting No top management support or participation for metric definition & management No timeline for refining and reporting metrics in line with implementation activities Lack of organisation buy-in towards a single methodology and standardisation of data elements Lack continuous reinforcement, correlation of metrics to project outcomes Lack training / buy in
  8. 8. Metrics – Key Success Factors Key Success Factors Clear charter, scope, roles & responsibilities and direction vital to measure project success Consistent reinforcement of the terminology and metric definition / cal Gain agreement on key metrics for projects & program Governance process defined for metric management Metrics & measures clear and concise Support and ownership from business Dashboard” and summary reports developed pre & post implementation Gain agreement on reporting and ongoing monitoring Develop metric program early in the project Defining metrics at the beginning of the project
  9. 9. Project Metric & Measures - Roadmap
  10. 10. Metrics & Measures - Roadmap #1 Scope & Governance #2 Metric Identification #3 Baseline Measurement #4 Industry Benchmarks #5 Targets #6 Establish Reporting Process #7 Implement #8 (Post) Implement Review / Audit
  11. 11. #1 Governance & Scope <ul><li>Process </li></ul><ul><li>Define scope – In / Out </li></ul><ul><li>Define governance process </li></ul><ul><li>Identify key individuals </li></ul><ul><li>Define authority and accountability levels </li></ul><ul><li>Define participants and reporting structure </li></ul><ul><li>Risks </li></ul><ul><li>Lack organisational understanding of scope </li></ul><ul><li>Lack participation </li></ul><ul><li>Lack commitment from steering committee team </li></ul><ul><li>Lack clear definition of authority (who can make decisions) </li></ul>
  12. 12. Metrics – Scope Validation Metrics – Identification & Validation <ul><li>Involve stakeholders </li></ul><ul><li>Correlate to business goals and objectives </li></ul><ul><li>Develop a manageable list of metrics </li></ul><ul><li>Draft calculations and definitions </li></ul><ul><li>Perform industry research and recommend improvement </li></ul><ul><li>Set appropriate targets (lag & lead) </li></ul><ul><li>Create plans to achieve goals </li></ul><ul><li>Monitor and communicate results </li></ul><ul><li>Periodic review of improvement ranges </li></ul><ul><li>Recommend metric owners </li></ul><ul><li>Collect baseline data </li></ul><ul><li>Develop tools & techniques </li></ul><ul><li>Prepare comparative reporting tool </li></ul><ul><li>Generate & distribute report / changes </li></ul>Operating Processes Management Process
  13. 13. #2 Metric Identification <ul><li>Process </li></ul><ul><li>Define terminology </li></ul><ul><li>Basic definitions </li></ul><ul><li>Identify guidelines </li></ul><ul><li>Define what to consider </li></ul><ul><li>Ensure guidelines support project’s objectives </li></ul><ul><li>Define approach, metrics and calculations </li></ul><ul><li>Define metric criteria </li></ul><ul><li>Identify anticipated project impacts </li></ul><ul><li>Risks </li></ul><ul><li>Not gaining agreement on terminology and definitions </li></ul><ul><li>Not knowing what to measure </li></ul><ul><li>Lack standardisation on the approach </li></ul><ul><li>Lack well defined guidelines </li></ul>
  14. 14. Metric Identification - Criteria <ul><li>Effort and cost required to obtain the metric target </li></ul><ul><li>Financial quantification associated with the metric </li></ul><ul><li>Availability of industry information to benchmark the metric </li></ul><ul><li>Whether or not the metric would have a negative impact on employees and culture </li></ul><ul><li>Reliability (believability) of the baseline data supporting the metric </li></ul>
  15. 15. #3 Baseline Measurements <ul><li>Process </li></ul><ul><li>Measurement Criteria </li></ul><ul><li>Identify required sample size </li></ul><ul><li>Measurement period </li></ul><ul><li>Audit review process </li></ul><ul><li>Statistically validity </li></ul><ul><li>Identify how to obtain </li></ul><ul><li>Ensure consistency pre- and post-system implementation </li></ul><ul><li>Risks </li></ul><ul><li>No resources or man hours to collect baselines </li></ul><ul><li>No standardisation of data definition, source and / or the collection process </li></ul><ul><li>Measurement period not defined </li></ul><ul><li>No audit review process defined </li></ul>
  16. 16. #4 & 5 Industry Benchmarks & Targets <ul><li>Process </li></ul><ul><li>Review terminology </li></ul><ul><li>Perform industry research </li></ul><ul><li>Obtain benchmark data </li></ul><ul><li>Define improvement ranges and tolerances </li></ul><ul><li>Establish targets </li></ul><ul><li>Identify Metric Owners </li></ul><ul><li>Gain approval </li></ul><ul><li>Risks </li></ul><ul><li>No time commitment for research </li></ul><ul><li>Improvement ranges lack clear definition </li></ul><ul><li>Tolerances not defined </li></ul><ul><li>No target defined and / or lack understanding </li></ul><ul><li>Targets not approved </li></ul>
  17. 17. #6 Establish Metric Reporting Process <ul><li>Process </li></ul><ul><li>Define reporting approach </li></ul><ul><li>Define responsibilities as to who will collect, compile and validate data </li></ul><ul><li>Define executive reporting needs, training, frequency and distribution </li></ul><ul><li>Define data repository required </li></ul><ul><li>Define what will be done with the results </li></ul><ul><li>Risks </li></ul><ul><li>Lack ownership nor time commitment </li></ul><ul><li>Lack agreement on data sources </li></ul><ul><li>Executive reporting not defined and / or does not make sense </li></ul><ul><li>No training or education given on the reports </li></ul><ul><li>No data repository in place </li></ul>
  18. 18. #7 & 8 (Post)Implementation Review <ul><li>Process </li></ul><ul><li>Implement metrics and compare target to actual metric reports </li></ul><ul><li>Review for consistency </li></ul><ul><li>Review for executive leader ownership and action plans </li></ul><ul><li>Activate internal audit validation </li></ul><ul><li>Risks </li></ul><ul><li>Lack agreement on source of variance and action plans </li></ul><ul><li>Lack resources for monitoring and reporting </li></ul><ul><li>Not a metric driven implementation project </li></ul>
  19. 19. Project Process Metric - Model & Guidance
  20. 20. Project Process Metrics Model Output Input Process Project Metrics Productivity Rate Test Defect Cost / Schedule Variance Change request <ul><li>Attributes </li></ul><ul><li>People </li></ul><ul><li>Environment </li></ul><ul><li>Tools </li></ul><ul><li>Techniques </li></ul><ul><li>Time </li></ul><ul><li>Cost </li></ul><ul><li>Size </li></ul><ul><li>Defects </li></ul>Resources & People Deliverables Time - Schedule Risk Risk Risk Risk Initiate Plan Design Build Test Control Close Operate
  21. 21. Guidance on Metrics <ul><li>Set clearly defined goals </li></ul><ul><li>Train and allocate adequate resources </li></ul><ul><li>Focus on project issues (risk management) </li></ul><ul><li>Provide timely feedback on metrics analysis </li></ul><ul><li>Begin by only collecting a few key measurements </li></ul><ul><li>Obtain top management commitment and “buy-in” </li></ul><ul><li>Automate the collection process, as much as possible </li></ul><ul><li>Understand that metrics are not an end to themselves </li></ul><ul><ul><li>Metrics cannot identify, explain or predict everything </li></ul></ul><ul><ul><li>Most issues / problems require more than one measurement to characterise and understand </li></ul></ul><ul><ul><li>Metrics should augment, not replace, good management judgment </li></ul></ul>
  22. 22. Guidance on Measurement Program <ul><li>Encourage feedback from participants </li></ul><ul><li>Must be institutionalised and repeatable </li></ul><ul><li>Develop a “Project / Program Measurement Plan” </li></ul><ul><li>Reward participation in the measurement program </li></ul><ul><li>Measurements - not analysed & acted upon is useless </li></ul><ul><li>Should be part of an overall process improvement effort </li></ul><ul><li>Plan for changes on the measurements types collected </li></ul><ul><li>Measurement results should remain anonymous </li></ul><ul><ul><li>Do not measure individuals </li></ul></ul><ul><ul><li>Focus on the process and the product </li></ul></ul>
  23. 23. Project Dashboard 20/07/2009 27/07/2009
  24. 24. Project Metrics
  25. 25. Metrics – Process Steps 1. Identify Metrics Customers 3. Ask Questions 4. Select Metrics 5. Standar-dise Definitions 6. Choose a Model 2. Target Goals 7. Establish Counting Criteria 9. Define Reporting Mechanisms 10. Determine Additional Qualifiers 11. Collect Data 12. Consider Human Factors 8. Decide On Decision Criteria
  26. 26. Focus - Project Metrics & Measures Schedule Quality Risk Cost / Budget Contracts Resources Focus - Metrics & Measure Status Stakeholder Involvement Scope Change Manage-ment
  27. 27. Example – Project Metrics <ul><li>Cost </li></ul><ul><ul><li>EV, AC, PV, </li></ul></ul><ul><ul><li>CPI, SPI, EAC, VAC </li></ul></ul><ul><li>Schedule </li></ul><ul><ul><li>Activities </li></ul></ul><ul><ul><li>Milestones </li></ul></ul><ul><ul><li>Reviews </li></ul></ul><ul><ul><li>Commitments </li></ul></ul><ul><ul><li>Completion dates </li></ul></ul><ul><li>Resources </li></ul><ul><ul><li>Response times </li></ul></ul><ul><ul><li>Capacity / load </li></ul></ul><ul><ul><li>Forecast / Actual Usage </li></ul></ul><ul><li>Risk </li></ul><ul><ul><li>Schedule, Cost, Resources, Deliverables </li></ul></ul><ul><li>Quality </li></ul><ul><ul><li>Issue </li></ul></ul><ul><ul><li>Defects </li></ul></ul><ul><ul><li>Re-work </li></ul></ul>
  28. 28. Example – Project Metrics Description Metric # of training hours per project team member. Training Hours # of project team members who quit or terminated. Turnover # of resources assigned to more than one task. Over allocated Resources Budgeted Cost of Work Performed (BCWP) Earned Value # of tasks that have cost more to complete than expected Over budgeted tasks # of task that should have started but have been delayed Tasks # of tasks that were started but not finished on time Overdue tasks # of scope changes that were approved Scope Change Approvals # of scope changes requested by the client or sponsor Scope Change Requests
  29. 29. Metrics – Do & Don’t <ul><li>Do </li></ul><ul><li>Obtain “buy-in” </li></ul><ul><li>Provide feedback </li></ul><ul><li>Select metrics based on goals </li></ul><ul><li>Focus on processes, products & services </li></ul><ul><li>Don’t </li></ul><ul><li>Ignore the data </li></ul><ul><li>Use only one metric </li></ul><ul><li>Measure individuals </li></ul><ul><li>Use metrics as a “carrot” & “stick” </li></ul>
  30. 30. Project Measures
  31. 31. Measure - Purpose Processes, Products & Services Understand Predict Control Evaluate
  32. 32. Measurement Process Tailor Measures Apply Measures Implement Process Stakeholder Needs Issues, Process Characteristics Measurement Plan External Constraints Measurement Needs Data Actions Project Boundary
  33. 33. Monitor & Control
  34. 34. Monitor & Control - Purpose <ul><li>To track progress </li></ul><ul><li>To take corrective action </li></ul><ul><li>To detect variance from plan </li></ul>
  35. 35. Monitor & Control – Tracking <ul><li>Periodic project status meetings & progress reports </li></ul><ul><li>Formal project milestones review </li></ul><ul><li>Compare actual vs. planned start date for each project task </li></ul><ul><li>Earned Value Management </li></ul>
  36. 36. Pitfalls to Avoid <ul><li>Not assessing significant variances or unfavorable trends </li></ul><ul><li>Not monitoring status frequently and / or all aspects of the project </li></ul><ul><li>Not monitoring the effects of corrective actions applied </li></ul><ul><li>Not addressing the cause of a variance, but addressing the effects </li></ul><ul><li>Not enough interim milestones or measurement points to objectively monitor status and when to apply backup plans </li></ul><ul><li>Not considering potential impacts of corrective actions – for example, actions taken to address schedule variances will generally affect cost </li></ul>
  37. 37. Earned Value
  38. 38. Definition <ul><li>Planned Value (PV) or Budgeted Cost of Work Scheduled (BCWS) </li></ul><ul><ul><li>Planned expenditure cash flows based on the completion of tasks in accordance with the project’s budget and schedule </li></ul></ul><ul><li>Actual Cost (AC) or Actual Cost of Work Performed (ACWP) </li></ul><ul><ul><li>Actual Project Expense based on completed tasks </li></ul></ul><ul><li>Earned Value (EV) or Budgeted Cost of Work Performed (BCWP) </li></ul><ul><ul><li>The amount of the budget that we should have spent for a given amount of work completed </li></ul></ul>
  39. 39. Example - Budget, Schedule, Tasks <ul><li>Project Budget = $40,000 </li></ul><ul><li>Schedule = 4 months </li></ul><ul><li>Tasks = 20 Tasks (evenly divided over 4 months) </li></ul><ul><ul><li>$2,000 per task </li></ul></ul><ul><ul><li>5 tasks per month </li></ul></ul><ul><li>Planned Value or BCWS = $10,000 / month </li></ul>
  40. 40. Example - Budget, Schedule, Tasks <ul><li>Invoice – 1 st Month Payment = $8,000 Actual Cost or ACWP </li></ul><ul><li>Work Completed – 1 st Month, 3 tasks costing= $8,000 as against $6,000 ($2,000 x 3) </li></ul><ul><li>Earned Value or BCWP = $6,000 </li></ul>
  41. 41. Comparison – PV, AC & EV
  42. 42. Cost Performance Indicators 1 month $1000 Scheduled / Budgeted to do $10,000 work over 5 tasks in a month window BCWS = $10,000 PV / BCWS 1 month $6000 $4000 EV / BCWP Schedule slippage permits only 3 tasks/$6,000 work to be performed BCWP = $6,000 Schedule variance = $4,000 1 month $8000 $2000 AC / ACWP Actual cost of work performed = $8,000 ACWP = $8,000 Actual cost variance = $2,000
  43. 43. Cost Metrics <ul><li>Cost Variance (CV) - difference between a task’s estimated cost and its actual cost </li></ul><ul><ul><li>CV = EV – AC or BCWP - ACWP </li></ul></ul><ul><ul><li>Negative Value = over budget and / or behind schedule </li></ul></ul><ul><ul><li>Positive Value = under budget and / or ahead of schedule </li></ul></ul><ul><li>Cost Performance Index (CPI) - percentage of work completed per dollar spent </li></ul><ul><ul><li>CPI = EV  AC or BCWP  ACWP </li></ul></ul><ul><ul><li>ratio > 1 = ahead of schedule and / or under budget </li></ul></ul><ul><ul><li>ratio < 1 = behind schedule and / or over budget </li></ul></ul>
  44. 44. Schedule Metrics <ul><li>Schedule Variance (SV) – the difference in terms of cost between the current progress and our originally scheduled progress </li></ul><ul><ul><li>SV = EV – PV or BCWP – BCWS </li></ul></ul><ul><li>Schedule Performance Index (SPI) – a ratio of the work performed to the work scheduled </li></ul><ul><ul><li>SPI = EV  PL or BCWP  BCWS </li></ul></ul><ul><ul><li>ratio > 1 = ahead of schedule and / or under budget </li></ul></ul><ul><ul><li>ratio < 1 = behind schedule and / or over budget </li></ul></ul>
  45. 45. Earned Value Metrics <ul><li>Minimum Funds Needed if things do not get worse </li></ul><ul><ul><li>Minimum funds = Original total budget  CPI </li></ul></ul><ul><li>Funds Needed if things continue to get worse at the same level of slippage </li></ul><ul><ul><li>Funds Needed = Original total budget  (CPI x SPI) </li></ul></ul>
  46. 46. <ul><li>“The fellow who does things that count, doesn't usually stop to count them.” </li></ul><ul><ul><li>- Source Unknown </li></ul></ul>
  47. 47. <ul><li>Good Luck </li></ul><ul><li> </li></ul>