Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
0
Metrics Based
Management
1
Contents
Key AD / AM Metrics
Who are the Key personnel
Why Metrics
When and how to use Metrics
How to identify Data Qual...
METRICS
Effort Metrics Formula
Effort Variance ((Actual Effort – Estimated Effort) / Estimated
Effort)*100
Load Factor (Actual Eff...
AD - Basic Process Metrics and Formula – Cont.
Defect
Metrics Formula
Defect Removal
Efficiency
(Total number of Pre-shipm...
AVM - Basic Process Metrics, Definition and Formula
Metrics Formula
Acknowledging
Severity 1….5 Incident
(No. of Sev 1/2/3...
Other set of Product Quality Metrics
Metric /
UOM
Formula Operational Definition
Tools Usage
.Net Java
Code Review
Coverag...
Metrics – Testing
Project
Metrics
Intent Definition
Reporting
Frequency
Test
Effectiveness
Indicates ability to unearth an...
METRICS
9
Metrics Work Flow
Developer / TL
enters/updates
the data
PL Reviews the
data
PM Approves the
Metrics
Metrics review
by D...
Some proactive approaches for reviewing the Metrics data
 Metrics Submission date to be done by end of every month.
 Rev...
METRICS
12
Customer’s Expectations (i.e. Why Metrics)
Improved
Business Value
&
New Revenue
Generation
On time delivery &
Improvem...
13
Senior Management Expectations (i.e Why Metrics)
Are we fixing
more Tickets
over a period
Are we building
more LoC over...
14
Where are we - Now
Follow-up
on Project
planning &
tracking
Data Quality Issues
No Data porting
from external tool
Expe...
15
Metrics Based Project Management – Work flow
Process Performance Objective ; Process
Performance models
Benchmarking
Gu...
METRICS
Scenario - 1
Effort Variation Effort Over Run
Schedule Variation On Schedule
Defects (Internal)
Defects (Customer)
Inferen...
Scenario - 2
Effort Variation Effort Over Run
Schedule Variation Schedule Over Run
Defects (Internal)
Defects (Customer)
I...
METRICS
Data Quality Issues- What is wrong here
A Project with a –ve Effort variation cannot have a +ve Schedule overrun and a hig...
Data Quality Issues- What is wrong here
Failure Cost is zero in spite of Defects recorded and having a very High
Appraisal...
Thank you
22
Upcoming SlideShare
Loading in …5
×

Metrics based Management

473 views

Published on

This presentation explains the metrics based management for Software Quality, its workflow, expectations of management and customers

Published in: Software
  • Be the first to comment

  • Be the first to like this

Metrics based Management

  1. 1. 0 Metrics Based Management
  2. 2. 1 Contents Key AD / AM Metrics Who are the Key personnel Why Metrics When and how to use Metrics How to identify Data Quality issues Some Enablers for Improvement
  3. 3. METRICS
  4. 4. Effort Metrics Formula Effort Variance ((Actual Effort – Estimated Effort) / Estimated Effort)*100 Load Factor (Actual Effort / Effort available)) % Review Effort (Total Effort expended on Reviews across all stages)/ (Actual Overall Project Effort) *100 % Cost of Quality (Effort spent on Prevention + Effort spent on Appraisal + Effort spent on Failure) / (Effort spent on Prevention + Effort spent on Appraisal + Effort spent on Failure + Effort spent on Production)) *100 Schedule Metrics Formula Schedule Variation ((Actual End date – Planned End date) / (Planned End date - Planned Start date))*100 Duration Variation ((Actual End Date – Actual Start Date) – (Planned End Date – Planned Start Date)) / (Planned End Date – Planned Start Date) * 100 AD - Basic Process Metrics and Formula Schedule Effort
  5. 5. AD - Basic Process Metrics and Formula – Cont. Defect Metrics Formula Defect Removal Efficiency (Total number of Pre-shipment Defects)/ (Total number of Pre-shipment Defects + Total number of post-shipment Defects + Total number of Post production Defects) *100 Defect Detection Efficiency (Number of Pre-shipment defects / Appraisal Effort) Defect Density by Effort Total no of Defects Detected/Total overall actual effort spent. Defect Leakage Sum((Number of defects attributed to a stage but only captured in subsequent stages) / (Total number of defects captured in that stage + Total Number of defects attributed to a stage but only captured in subsequent stages)) *100 Metrics Formula Size Variation ((Actual Overall Size – Planned Overall Size) / (Planned Overall Size))* 100 Productivity Overall Productivity = (Overall Project Size) / (Total Effort for the Project) Size
  6. 6. AVM - Basic Process Metrics, Definition and Formula Metrics Formula Acknowledging Severity 1….5 Incident (No. of Sev 1/2/3/4/5 incidents acknowledged within the applicable Acknowledgement Time / Total No. of Sev 1/2/3/4/5 Incidents) * 100 Severity 1…..5 Incidents resolved within the allotted time (Number of Sev 1/2/3/4/5 incidents resolved within the allotted time / number of Sev 1/2/3/4/5 incidents resolved) * 100. Avoidable Problems / Unforced Errors for Severity level 1 and 2 (No. of incidents/problems caused by the Supplier's Actions / Total No. of incidents/problems) * 100 Metrics Formula Function Points per $1K Spent FP count / Total amount spend in ($1K ) Defect Injection Rate - Release or Project (Total number of defects injected in the Release or Project / size of product) % of SLAs met % of fixes without escalation
  7. 7. Other set of Product Quality Metrics Metric / UOM Formula Operational Definition Tools Usage .Net Java Code Review Coverage Number of impacted programs reviewed / Total number of programs * 100 A measure of the review coverage on the number of programs This is higher the better metric VSTS-Code Analysis, FxCop, Sonar SONAR, JCAP, PMD, Checkstyle, FindBugs Unit Test Coverage Based on Unit Test Coverage Tools such as Junits/JCoverage The Code coverage metric identifies the sections of the source code that were either tested/ not tested as part of white box testing This is higher the better metric NUnit, VSTS- Unit Testing NCover Junit, Test NG, Code pro analytix, Cobertura, EMMA Code Quality - Cyclomatic complexity Cyclomatic Complexity at class level (Highest method CC) This metric estimates the complexity of the individual functions, modules, methods or classes within a program so as to measure the program's structural complexity. Lower the better metric IDE,Sonar, VSTS-Code Analysis, FxCop SONAR, JCAP, PMD, Checkstyle, FindBugs Requirements to Test Case coverage % of requirements linked to Test cases An indication of how extensively the requirements are covered by Test Cases This is higher the better metric VSTS- Unit Testing, NCover Junit, Test NG, Code pro analytix, Cobertura, EMMA
  8. 8. Metrics – Testing Project Metrics Intent Definition Reporting Frequency Test Effectiveness Indicates ability to unearth and fix defects before they reach UAT and Production (Number of accepted defects in SIT / (Number of accepted defects in SIT + UAT + Post UAT)) * 100 Monthly Test Design coverage How much requirements are covered by test cases ? (Total number of baselined testable requirements mapped to test cases / Total number of baselined testable requirements)*100 Monthly Test Case Preparation Productivity Test case creation productivity of the team ((No of Test Cases or Test Case points (TCP) prepared)/ (Effort spent for Test Case Preparation) Monthly Test Case Execution Productivity Indicates test execution productivity of the team ((No of Test Cases or TCP Executed)/ (Effort spent for Test Execution) Monthly
  9. 9. METRICS
  10. 10. 9 Metrics Work Flow Developer / TL enters/updates the data PL Reviews the data PM Approves the Metrics Metrics review by DD / DM
  11. 11. Some proactive approaches for reviewing the Metrics data  Metrics Submission date to be done by end of every month.  Review of Metrics by PM/Delivery Manger to be completed subsequently.  Monthly Metrics Review scheduled with Delivery Director by 1st week of subsequent month for critical projects.
  12. 12. METRICS
  13. 13. 12 Customer’s Expectations (i.e. Why Metrics) Improved Business Value & New Revenue Generation On time delivery & Improvement in Time to Market Zero defects High Quality Business / Technology Solutions Reduction in IT Operating Cost Am I getting more work for lesser $ spent over a period
  14. 14. 13 Senior Management Expectations (i.e Why Metrics) Are we fixing more Tickets over a period Are we building more LoC over a period Are we delivering Zero Defects Software Are we making expected Profitability Are we adopting Best Practices and Reusable Are we getting accurate Productivity while submitting RFPs Are we getting repeat business from this engagement
  15. 15. 14 Where are we - Now Follow-up on Project planning & tracking Data Quality Issues No Data porting from external tool Expending our energy in process compliance Follow up mailers for Metrics data submission Project Health Scorecards JUST THINK Are these parameters helping you to meet Customer Expectations ? Senior Management expectations ?? Delivery Managers Expectations ??? Are we using these data in a true sense for the success of the project ????
  16. 16. 15 Metrics Based Project Management – Work flow Process Performance Objective ; Process Performance models Benchmarking Guidelines for Metrics and QPM, Statistical Techniques Data Validation, Analysis and reporting ; Facilitation for usage of statistical tools like control charts Facilitation Data Trend analysis and validation of trends using hypothesis testing Metrics Based Project Management
  17. 17. METRICS
  18. 18. Scenario - 1 Effort Variation Effort Over Run Schedule Variation On Schedule Defects (Internal) Defects (Customer) Inference • Is there a problem in Estimation • Is the team over burdened? • Is the team possess right skills to carry out the tasks in a stipulated time period? Or takes longer time to find a solution & fix • Is there any scope scale down? Bring back to Track • Revisit the Estimation • Additional Trainings to Team
  19. 19. Scenario - 2 Effort Variation Effort Over Run Schedule Variation Schedule Over Run Defects (Internal) Defects (Customer) Inference • Is there a problem in Estimation • Is the team over burdened? • Is the team possess right skills to carry out the tasks in a stipulated time period? Or takes longer time to find a solution & fix • Is there any scope creep ? • Are requirements changed frequently (stability of requirement is low) Bring back to Track • Revisit the Estimation • Additional Trainings to Team • Reach out to customer, if Scope creep observed, Requirements changed
  20. 20. METRICS
  21. 21. Data Quality Issues- What is wrong here A Project with a –ve Effort variation cannot have a +ve Schedule overrun and a high LF
  22. 22. Data Quality Issues- What is wrong here Failure Cost is zero in spite of Defects recorded and having a very High Appraisal cost. Production Cost shows zero
  23. 23. Thank you 22

×